finally 266-276 paper 7 (40219) seraceddin levent zorluoğlu 27 a © 2019 indonesian society for science educator 1 j.sci.learn.2019.3(1).1-6 received: 10 july 2019 revised: 4 october 2019 published: 27 november 2019 development of interactive e-module chemistry magazine based on kvisoft flipbook maker for thermochemistry materials at second grade senior high school sri saraswati1, roza linda1*, herdini1 1chemistry education study program, mathematics and sciences education, teachers training and education faculty, riau university, pekanbaru, indonesia *corresponding author. roza.linda@lecturer.unri.ac.id abstract this investigation aims to develop an interactive e-module chemistry magazine-based teaching material based on kvisoft flipbook maker on thermochemistry for senior high school grade xi/ma level. the research design is a development research (research and development) with plomp’s model. the object of the research is the e-module inter-addictive chemistry magazine based on kvisoft flipbook maker. the data instrument collection is a validation sheet form that is given to four validators and user response sheets to two chemistry subject teachers and 30 students of senior high school grade xi focusing on science class. the data analysis technique is by calculated the percentage score of the validation assessment and user response. the results revealed that the developed e-module interactive chemistry magazine based on kvisoft flipbook maker filled out the valid criteria by the material validators based on content substance assessment aspect with a percentage 88.89% and learning design 97.22% and by the media validators based on display (visual communication) with a percentage96.88% and software utilization 100%. while the user responses based on the teachers' and students’ responses were in very good criteria with each score 92.61% and 86.80% keywords interactive e-module, kvisoft flipbook maker, chemistry magazine, plomp development model, thermochemistry 1. introduction in industrial revolution 4.0, technology has become the basis of human life. this era affected many sets of life, such as economics, politics, arts, cultures and including education sides. technology in the education side is called e-learning. according to wahyuningsih & makmur (2017), the use of e-learning in learning according to the latest research has a positive impact on processes and target learning. e-learning will enable students to interact with anyone and access information whenever needed. in addition, the learning process will be more effective and efficient in terms of time and energy. technology development can be used in the learning process. students are positioned as learners who hold the main role; students are required to have full activities, even individuals who learn teaching materials (sanjaya, 2014). the learning resources used are usually books, student worksheets and articles. the learning source is still in printed form, therefore it is necessary to develop technology and develop learning resources in the form of improved print into computer-based or e-learning (asyhar, 2012). one of the learning resources that can be developed based on computers is the learning module. electronic modules are expected to attract students' interest in learning and can illustrate abstract material, can be accessed easily by students using computers and various types of gadgets anywhere and anytime, thus allowing students to get immediate feedback and understand the subject matter completely. based on interviews with chemistry teachers, the learning module has been used as a learning resource in sman 8 pekanbaru and sma cendana pekanbaru. information was obtained that the module is still not interactive because it only contains a description of the materials, set of formulas and some images related to the material that causes unpleasant learning. supported by the questionnaire, the data obtained 71.38% of students find it difficult to understand thermochemistry topics because mailto:roza.linda@lecturer.unri.ac.id journal of science learning article doi: 10.17509/jsl.v3i1.18166 2 j.sci.learn.2019.3(1).1-6 most of them do not have many references for the material. according to keenan, thermochemistry is a basic knowledge that needs to be understood not only to find out how much energy needs to be given or that can be obtained from chemical reactions, but also as a basic knowledge for the study of the theory of chemical bonds and structures. according to linda, herdini, and putra (2018), one of the materials in chemistry that is difficult is the chemical reaction and energy. therefore, it needs a teaching material that can make students happy and easy to understand about thermochemistry. lesson materials need to be developed in interactive modules. this electronic-based module (emodule) is a magazine form called e-magazine. the interactive chemistry magazine electronic module will be developed in the form of html format software that can be accessed via pc, android, usb and so on. the application used for making the e-module interactive chemistry magazine is kvisoft flipbook maker. kvisoft flipbook maker is software used to make the display of books or other teaching materials into a flipbook-shaped digital electronic book. the software can be downloaded freely through internet access (sugianto, abdullah, elvyanti, & muladi, 2013). related to those explanations, it is needed to do research about the development of interactive teaching materials in chemistry learning in thermochemistry materials. the development of interactive teaching materials proposed in the form of interactive e-module chemistry magazine based on kvisoft flipbook maker for thermochemistry material at second-grade senior high school. 2. method this study was conducted in the chemistry education study program of fkip riau university pekanbaru with the test in class xi science 2 of sman 8 pekanbaru and xi science 1 of sma cendana pekanbaru started from november 2018 until april 2019. the development of the thermochemistry interactive e-module chemistry magazine was designed using research and development (r & d) with the plomp model. r & d is a research design used to produce specific products and to test the effectiveness of it. the design of this study uses the plomp model, which according to rochmad (2012), is a development model which consists of the first investigation phase (preliminary investigation), design phase (design), realization/construction phase (realization/ construction), and validation phase, trial and revision (evaluation, test and revision) and implementation phase. it can be seen in figure 1. the instrument which is used as a data collecting technique is the validation sheet and user response sheet. the validation sheet is used to get information about the validity of teaching materials and other instruments based on expert judgment. information obtained through this instrument can be used as a material for consideration in revising the thermochemistry interactive e-module chemistry magazine. this validation sheet was assessed by 2 material validators and 2 media validators regarding the assessment of substance content, learning design, display (visual communication), and software utilization. the user response questionnaire was used to collect the data on user responses (teachers and students) to the thermochemistry e-module interactive chemistry magazine in sman 8 pekanbaru and sma cendana pekanbaru. the data collection techniques were a literature review and field studies. the literature review was conducted to obtain theoretical relevant information, while field studies were conducted to obtain data related to the validity and user response to the thermochemistry interactive e-module chemistry magazine. the data analysis techniques used in this study are the validity and analysis of user responses. validity analysis is based on scores that are taken from the validator for each aspect of the assessment. this is how to calculate the percentage score with the equation formula: 𝑃 = 𝑛 𝑁 × 100% explanation: p = percentage score (%) n = number of scores obtained n = maximum of score obtained the results obtained were converted into qualitative values based on the assessment criteria of the attitude measurement scale with a score of 1-4. the validity criteria can be referred to in table 1. analysis of the user responses (teachers and students) is determined based on the percentage of alternative scores positive statements by using this formula: 𝑅 = 𝑓 𝑛 × 100% explanation: r = percentage of alternative scores statement of positive attitude (%) f = number of scores obtained n = total maximum score table 1 the validity criteria percentage (%) the validity criteria 75,00-100 valid 50,00-74,99 quite valid 25,00-49,99 less valid 0,00-24,99 invalid table 2 criteria response of users percentage (%) criteria response of users 75,00-100 very good 50,00-74,99 good 25,00-49,99 lessbaik 0,00-24,99 not good journal of science learning article doi: 10.17509/jsl.v3i1.18166 3 j.sci.learn.2019.3(1).1-6 the results obtained were converted into qualitative values according to the assessment criteria for attitude measurement scale with a score of 1-4. criteria for the response of users (teachers and students) can be seen in table 2. 3. result and discussion this development research produced the product of an interactive e-module chemistry magazine form based on kvisoft flipbook maker for thermochemistry material. interactive e-module chemistry magazine thermochemistry was developed within 6 months. this product can be accessed online and offline by teachers and students, it can be used during the learning process in the classroom and as an independent teaching material outside school hours using computers, laptops, notebooks, and various gadgets with android and ios operating systems. the gadget used must be equipped with supporting applications with html, html mobile, and exe / app formats such as gom and mozilla firefox. development research uses the research and development (r & d) method with the plomp model. these are the following explanation of this research finding results for each development stage. 3.1 preliminary investigation phase this phase consists of several stages: front end analysis, student analysis, competency analysis, and material analysis. in the front end analysis, the results were related to the teaching materials used. the teaching materials commonly used in the learning process are printed teaching materials like textbooks and printed modules at sma cendana pekanbaru. the module only contains a description of the material, a set of formulas and some exercises so that the module is not in accordance with the structure of teaching materials that have been settled by the directorate of education staff. ideally, a module has a compilation component consisting of titles, learning instructions, basic competencies, learning materials, supporting information, training, assignments, and assessments. on student analysis, the information gained is related to the abilities and attitudes of students towards thermochemistry material and analysis of the students’ needs. data distribution of questionnaires shows that 90.67% of students need alternative teaching materials such as modules that can be used to study thermochemistry material more easily and interestingly. in addition, 88.54% of students often use a pc or gadget as a learning resource for chemistry, at school and at home. this is in accordance with the 21st-century learning process that requires students to be able to know information and communication in technology as one of the media to improve students' understanding in a particular subject, especially in thermochemistry. in the competency analysis, the results are related to competency attitudes, knowledge and skills that are expected to be possessed by the students after the following thermochemistry learning on the syllabus of high school chemistry subjects by the ministry of education and culture. thermochemistry materials are found in essential competencies 3.4, 4.4, 3.5 and 4.5 with the main subject and learning activities. in the material analysis, the information obtained is related to the concepts of thermochemistry material, which are systematically compiled and detailed. it also formulated learning objectives that must be achieved by students. the material is arranged according to the senior high school chemistry syllabus and appropriate with the lesson plan on the thermochemistry topic. the number of meetings for thermochemistry material consists of 4 meetings. 3.2 design phase in this phase, the prototype design and assessment instruments were produced. the prototype project is in the form of an interactive e-module chemistry magazine thermochemistry as the results from various analyses that have been conducted to obtain a prototype which contains some contents suited with the ict-based learning materials development guide by the ministry of national education directorate general of primary and secondary education management in 2010. the design of the figure 1 plomp development model flow adapted and modified from azhar (2011) journal of science learning article doi: 10.17509/jsl.v3i1.18166 4 j.sci.learn.2019.3(1).1-6 instrument is in the form of a validation sheet by the material and media validators and the response questionnaire user (teachers and students). it refers to the guidelines for the development of ict-based teaching materials by the ministry of national education directorate general of primary and secondary education in 2010. the questionnaire responses for users (teachers and students) were obtained according to the research needs with reference to the questionnaire response of users (teachers and students) from relevant research. the e-module is designed in the form of a magazine so that it looks more attractive and contains magazine contents in the form of scientific articles, profile personalities, tips and galleries that can add students' insights into thermochemical material. besides, the designed e-module is also equipped with audio, video, animation, flash and links. 3.3 realization/construction phase this phase is the realization of the prototype design and assessment of the instruments. the realization of the prototype design was the interactive e-module chemistry magazine based on kvisoft flipbook maker on thermochemistry material. then the realization of the assessment instrument is in the form of validation sheets and user response questionnaires (teachers and students). (a) (b) figure 2 revised result by the material validators (a) before revision and (b) after revision (a) (b) figure 3 revised result by the media validators (a) before revision and (b) after revision journal of science learning article doi: 10.17509/jsl.v3i1.18166 5 j.sci.learn.2019.3(1).1-6 3.4 realization/construction phase the results were obtained from the validation phase, test and revision in comments and suggestions form, which also obtained the final assessment of the prototype that has been made. the followings are the results of each activity in each phase. validation validation aims to determine the feasibility of the product to be used in the learning process based on the substance content aspect, learning design for material validator and display (visual communication) and software utilization for media validators in accordance with the ict based learning material development guidelines by the directorate of kementerian pendidikan dan kebudayaan (2016). suggestions and improvements are gotten during validation are used as a reference for revisions to get a better e-module. some examples of revisions made during the validation process by media and material validators can be seen in figure 2 and figure 3. overall, the validation results show the validity criteria with an average validation score of 95.74%. it means the emodule that has been developed has fulfilled all aspects set by the kementerian pendidikan dan kebudayaan (2016). the results of the assessment of each aspect can be referred to in table 3 and figure 4 for the diagram. test tests were carried out to obtain comments, suggestions, and user-evaluations of the thermochemistry e-module interactive chemistry magazine using the user table 4 the results of the assessment of the response of users (teachers and students) respondent result (%) category teachers 92,61% very good students 86,80% very good average 89,70% very good table 3 the results of the assessment of each aspect validation no assessment aspect percentage score by validator 1 contentsubtance 88,89% 2 learning design 97,20% 3 display (visual communication) 96,88% 4 software utilization 100% average score percentage 95,74% criteria for the overall validity of all aspects valid figure 4 percentage diagram of the score validation of each aspect by the material validators and media validators 88.89 97.2 96.88 100 80 90 100 c o n te n t s u b st an ce l e ar n in g d e si g n a p p e ar an ce ( v is u al c o m m u n ic at io n ) s o ft w ar e u ti li za ti o n p e rc e n ta g e ( % ) aspect (a) (b) figure 5 the results of a revision made based on test with teachers (a) before revision and (b) after revision journal of science learning article doi: 10.17509/jsl.v3i1.18166 6 j.sci.learn.2019.3(1).1-6 response questionnaire for teachers and students. overall, the score percentage from the tests of the response of users (teachers and students) to the thermochemistry interactive e-module chemistry magazine was 89.70% with very good criteria. this is because according to the user, the e-module has an attractive appearance and is in accordance with the characteristics of high school level students, contains material clearly and easily understood, and can be accessed anytime and anywhere. the result is in accordance with research conducted by putri, muhaimin & sayhri. (2017) with the title "development of e-magazine on material acid and base solutions for class xi mipa students in sman 1 jambi city" with the results of research in the form of e-magazine assessments according to media experts, material experts, teachers and students' responses each obtained a mean score of 4.53 (very good); 4.86 (very good); 4.1 (good) and 91.64% (very good). the results of the assessment of the response of users (teachers and students) can be seen in table 4. revision the results of the test obtained comments, suggestions, and improvements from the teacher and students on the emodule. examples of the results of revisions made based on trials with teachers can be seen in figure 4 and students can be seen in figure 5. 4. conclusion based on the results of the study, it can be concluded that the development of interactive e-module chemistry magazine based on kvisoft flipbook maker on thermochemistry materials for senior high school grade xi/islamic senior high school was declared valid by the material validator based on aspects of material substance with 88.89% and 97.20% learning design, and by the media validator based on the display aspect (visual communication) 96.88% and the utilization of software 100%. while the tests to teachers and students the results obtained are in very good criteria with an average 92.61% by teachers and 86.80% by students. the author hopes that the development of interactive e-module chemistry magazine in thermochemistry can be continued in the next research, the implementation phase to test its effectiveness in the learning process by applying interactive e-module chemistry magazine thermochemistry into the learning process directly. references asyhar, r. (2012). kreatif mengembangkan media pembelajaran. jakarta, indonesia: gp press. ayyildiz, y., & tarhan, l. (2012). the effective concepts on students' understanding of chemical reactions and energy. hacettepe university journal of education, 42, 72-83. azhar, e. (2011). pengembangan perangkat pembelajaran teori peluang berbasis rme untuk meningkatkan pemahaman, penalaran, dan komunikasi matematik siswa slta. seminar nasional matematika dan pendidikan matematika yogyakarta 3 desember 2011, 213-222. keenan, c. w. (1984). kimia untuk universitas. jakarta, indonesia: erlangga. kementerian pendidikan dan kebudayaan. (2016). silabus mata pelajaran kimia sekolah menengah atas/madrasah aliyah (sma/ma). jakarta, indonesia: kementerian pendidikan dan kebudayaan. linda, r., herdini, h., & putra, t. p. (2018). interactive e-module development through chemistry magazine on kvisoft flipbook maker application for chemistry learning in second semester at second grade senior high school. journal of science learning, 2(1), 21-25. putri, r. r., muhaimin & sayhri, w. (2017). pengembangan e-magazine pada materi larutan asam dan basa untuk siswa kelas xi mipa di sman 1 kota jambi. artikel ilmiah. program studi pendidikan kimia fkip universitas jambi. rochmad, r. (2012). desain model pengembangan perangkat pembelajaran matematika. kreano, jurnal matematika kreatif-inovatif, 3(1), 59-72. sanjaya, w. (2014). strategi pembelajaran berorientasi standar proses pendidikan. jakarta, indonesia: prenada media group. sugianto, d., abdullah, a. g., elvyanti, s., & muladi, y. (2013). modul virtual: multimedia flipbook dasar teknik digital. innovation of vocational technology education, 9(2). wahyuningsih, d., & makmur, r. (2017). e-learning teori dan aplikasi. jakarta, indonesia: informatika. microsoft word publish 16204-35293-1-ce wawan setiawan.docx a © 2019 indonesian society for science educator 65 j.sci.learn.2019.2(2).65-70 received: 27 march 2019 revised: 15 may 2019 published: 18 may 2019 development of smart content model-based augmented reality to support smart learning siti fatimah1, wawan setiawan2*, enjun junaeti2, ahmad syukron surur2 1department of mathematics education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia 2department of computer science education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia *corresponding author. wawans@upi.edu abstract augmented reality (ar) is an optical technology that combines virtual objects or worlds into real worlds like in real time and increases user perceptions and interactions with the real world. information conveyed by virtual objects helps users carry out activities (tasks) in the real world. the convenience offered makes ar technology can be used for various fields, including education, such as the development of materials or learning media. augmented reality in its development is more comfortable, cheaper, and can be widely implemented in various multimedia needs. this research is a study of the development of multimedia based on augmented reality to produce dynamic learning materials or media in supporting the concept of smart learning. this research provides multimedia models of mathematics for circles, ellipses, parabola, and hyperbole based on augmented reality to create more dynamic learning as smart learning. multimedia is produced according to operational standards and meets content standards based on media experts, content, and users. multimedia-based augmented reality math is easy to operate, helps, and increases understanding and increases student motivation. keywords augmented reality, multimedia learning, smart learning 1. introduction along with the current technological developments, the learning media always follow the progress of existing technology (reisse, heider, giersich, & kirste, 2008). the oldest technology utilized in the learning process is printing that works on the basis of mechanical principles, then audio-visual technology that combines mechanical and electronic inventions for teaching purposes, the latest emerging technology is the microprocessor technology that spawned the use of computers and interactive activities (reisse, heider, giersich, & kirste, 2008). based on these technological developments, the teaching media is divided into four parts, namely: (1) media resulting from printing technology, (2) media resulting from audio-visual technology, (3) media result of computer-based technology, (4) media combined print technology and computers (dekdouk, 2012). referring to the classification of learning media was born a combined learning media of print and computer technology known as ar (augmented reality) an optical technology that combines the object or virtual world into the real-world view in real time. also, augmented reality improves perception and interaction of users with the real world. the virtual object displays information that the user can not directly detect with his senses. the information presented by the virtual object helps the user perform realworld activities/tasks (azuma, 1997). ar is one of the most exciting technologies of interest; ar presents an immersive level in which none of the virtual tools can do it. the convenience offered makes ar technology usable to various fields, such as military, medicine, education, industrial engineering, to entertainment. there are many learning models, based on smartphone technology that need to be supported with more real and dynamic content, especially the content aspect which is one of the characteristics of smart learning (di, gang, & juhong, 2008). one of the dynamic content can be developed with ar technology, making learning more dynamic and exciting. the tendency of learning that is less attractive is one of them due to the use of even static media using technology. the use of information and communication technology journal of science learning article doi: 10.17509/jsl.v2i2.16204 66 j.sci.learn.2019.2(2).65-70 based multimedia learning is quite lively, but the application of ar technology is still low (dekdouk, 2012). for content that involves multiple dimensional objects, ar much helps build object abstractions for students' understanding. the idea of the concept of area and space has severe problems in mathematical content such as circles, ellipses, parabola, and hyperbole. based on the above problems, this research examines the development of learning media using augmented reality technology for learning mathematical material circles, ellipses, parabola, and hyperbole. the main problem of research is: "how to develop multimedia based on augmented reality to support smart learning." from the formulation of the main problem, the researcher divides the research question into small points as follows: (a) how to design and develop learning multimedia based on augmented reality? (b) how is the feasibility of learning multimedia based on augmented reality developed for a limited trial? (c) the primary purpose of this research is the development of multimedia-based augmented reality for student learning activities. the specific objectives are: (a) the acquisition of multimedia learning model based augmented reality? (b) knowing the feasibility of multimedia learning model based augmented reality developed before being tested in a limited way. 2. method the main objective of this research is to develop augmented reality-based media to support smart learning for students of junior high school. in general, the development of learning media, based on augmented reality to support quick learning consists of 3 (three) significant steps, namely preliminary studies, product development, and testing as seen in figure 1 (dekdouk, 2012). this research conducted in the department of computer science education, faculty of mathematics and natural sciences education, universitas pendidikan indonesia. the research steps are as follows: 2.1 preliminary study at this stage set the goal of software development, both for students, teachers, and the environment. for this purpose, the analysis is done in cooperation with the teacher and still refers to the curriculum used. in addition to the objective analysis, analysis of software development needs is also required. needs analysis is the first stage that becomes the basis of the next software development process. the smoothness of the entire software creation process and the completeness of the resulting software features are highly dependent on the results of this needs analysis. to obtain information about the needs in making this interactive learning media, researchers through explorative studies and literature study (azuma, 1997). 2.2 developing this stage includes the determination of the elements that need to be loaded in the multimedia learning that will be developed based on the design of learning or often referred to as the id model (instructional design). the results of this stage include a storyboard (storyboard), which is how this multimedia is displayed (interfacing). how to present materials, 3d models for learning, animation, evaluation, and more. also, the result of this step is the interactive multimedia learning system flowchart from start to program until the end of the plan (ardhianto, hadikurniawati, winarno, 2012; kemp & dayton, 2003). this stage is the stage of multimedia learning development based on a storyboard that has been made, making multimedia such as 3d model and animation until evaluation, storyline creation, integration among all these aspects, and program design. after that, judgment is made to the expert. this assessment covers the assessment of interfaces, text, 3d models, interactivity and the content of learning (ardhianto, hadikurniawati, & winarno, 2012, kemp & dayton, 2003). 2.3 assessment stage to measure the results of the expert judgment, the scale rating scale is used. rating scale or scale is a subjective measure made scale (kemp & dayton, 2003). the rating scale is not limited to the measurement of attitudes alone, but to measure respondents' perceptions of other phenomena, such as scales for socioeconomic status, institutional status, knowledge, skills, the process of activities and others (kemp & dayton, 2003). figure 1 research procedure figure 2 the criterion score journal of science learning article doi: 10.17509/jsl.v2i2.16204 67 j.sci.learn.2019.2(2).65-70 regarding aspects assessed at the expert validation stage, the adaptation of the multimedia learning development criteria. these aspects are common aspects, aspects of media, aspects of learning, and visual communication aspects (cawood, fiala, & steinberg, 2007). the data that has been collected in the validation questionnaire is qualitative data because each statement item is divided into a very bad, bad, useful, and excellent category. to calculate the data first into quantitative data by the weight of the score of one, two, three, and four. after the data is transformed, then the calculation of the rating scale can be done with the following formula (azuma, 1997). p = 100% note: p = percentage the interpretation scale is made by dividing the criterion score into four continua then the continuum result is made as the following categories in figure 2 (azuma, 1997). qualitative data, such as comments and suggestions, serve as a basis for revising interactive multimedia learning. 3. result and discussion 3.1 multimedia products the resulting multimedia is -based augmented reality for learning conic sections consisting of circular, parabolic, elliptical, and hyperbola materials. learning tools consist of material books, object cards, and a camera or mobile phone. the map includes all the image objects present in the book for ease of use and is named matsemat as a mathematic smart extension. the image code is adjusted between the paper and the card based on the material affairs set out in the book. figure 3 and 4 is an example of augmented based multimedia card (santoso & gook, 2012; geroimenko, 2012). the front page of the card contains the logo of universitas pendidikan indonesia; the back includes an image of the object, the number of the picture according to the book, the name of the drawing, and the short description. 3.2 products judgment by experts and users 3.2.1 multimedia aspect judgment  using 3 (three) essential parameters of an electronic media, namely general aspects, software engineering, and visual communication, media experts provide an assessment as in table 1. from table 1. it can be shown, according to media experts that the multimedia developed has an outstanding category and in a contingent basis with average feasibility of 87.61% which is categorized very high and in the continuum as shown in figure 5. figure 3 matsemat card model figure 4 illustration of cone slice table 1 assessment of media aspect aspec sum expert sum comp. ideal score actua l score % g 2 3 30 26 86.67 se 2 9 90 80 88.89 vc 2 11 110 96 87.27 average 87.61 note: g: general; se: software enginnering; vc: visual communication journal of science learning article doi: 10.17509/jsl.v2i2.16204 68 j.sci.learn.2019.2(2).65-70 multimedia-based augmented reality generated is declared appropriate for use in learning circles, parabola, ellipses, and hyperbole. multimedia produced contributes to learning, including the following (kemp and dayton, 2003). (1) submission of learning messages can be more standardized. (2) learning can be more enjoyable. (3) learning becomes more interactive by applying learning theory. (4) the time for implementing learning can be shortened. (5) quality of learning can be improved. (6) the learning process can take place whenever and wherever needed. (7) the positive attitude of students towards learning materials and the learning process can be improved. (8) the role of the teacher experiences changes in a positive direction. 3.2.2 content aspect judgment  assessment of material aspects uses 3 (three) essential parameters of an electronic media, namely general elements, content subject; and learning. material experts provide an assessment, as stated in table 2. from table 2., it can be shown that the multimedia assessment by material experts found an average percentage of the feasibility of 87.22%, which is categorized very high and in contour, as shown in figure 6. multimedia-based augmented reality generated was declared feasible by material experts for use in learning circles, parabola, ellipses, and hyperbole. material aspects show feasibility above average, while the other two points are below average. this indicates that multimedia based on augmented reality can extract material richer, more varied, detailed, and dynamic. such dynamic learning media are by the characteristics of students as some millennia, including fun, multitasking, and random access (trilling & fadel, 2009). 3.2.3 assessment by users  user assessment is an operational aspect of multimedia that is more precisely testimony. the operational element uses 4 (four) essential parameters of an electronic media, namely completeness, and clarity of flow, compatibility/suitability of appearance, ease of operation, and interconnection, interactivity. the users provide an assessment, as stated in table 3. from table 3, it can be shown that according to prospective users, the average percentage of feasibility is 81.67%, which is categorized very high, and on a continuum, as shown in figure 7. multimedia-based augmented reality generated is expressed according to operational standards and is appropriate for users to use in learning circles, ellipses, parabola, and hyperbole. navigation and interactivity aspects are below average, while the appearance and convenience are above average. in this context, prospective users, namely students, pay more attention to the presentation, and ease of operation. assessment of potential users is lower than media experts. this shows that the millennial generation has been waiting to interact with the media like that, especially those that smell like games that feature appearance (trilling & fadel, 2009). figure 5 critical scale assessment of media experts table 2 assessment of ascpect matter aspect num expert num comp. ideal score actual score quality (%) g 2 3 30 26 86.67 cs 2 4 40 34 85,00 l 2 12 120 108 90,00 average 87.22 note: g: general; cs: content subject; l: learning figure 6 critical scale material assessment table 3 user assessment no. aspect quality (%) 1 navigation key in multimedia 80,00 2 multimedia view 83,36 3 ease of use multimedia 83,33 4 multimedia interactivity 8 0,00 average 81,67 figure 7 user critical scale table 4 target test no. results 1 tracking works well, but the distance between the camera and the marker is not too close. 2 the display looks ideal on the tracking process with a viewing angle of 45o, and the target is seen entirely. 3 the target looks too small with a 60o viewpoint, and in this case, the tracking process works long enough. 4 the target looks too small, with a 70o point of view, and in this case, the tracking process cannot work. table 5 distance test elevation dintance (cm) 0 10, 15, 25, 50 30 10, 15, 50 45 10, 15, 25, 50 journal of science learning article doi: 10.17509/jsl.v2i2.16204 69 j.sci.learn.2019.2(2).65-70 3.3 multimedia visibility to determine media visibility, black box testing is carried out by looking at the input, treatment, response, sensitivity, output, or event changes. for this reason, alpha and beta testing techniques are used. 3.3.1 alpha testing  alpha testing is done to see the initial condition of the media. the initial term of the media is the reading or introduction of marker objects by the camera until the final output of the object. the following are the results of testing the target object, as shown in table 4. the next test is the distance between the camera and marker, the reflection of light and the success of the camera's tilt angle at a minimum illumination of bright lights or cloudy sunlight and the results are read well at some angles of distance funds as seen in table 5. the alpha test results show the success of excellent and visible viewing of augmented reality objects. with bright levels of natural sunlight, augmented objects can be observed. the tilt angle 0o-45o is a standard limit for smartphone users, as well as a distance of 10-50 cm is an average distance that is naturally done or occurs (geroimenko, 2012). the following are the results of the multimedia product journey test as to where tables 6, 7, 8, and 9. the module starts running successfully, which is the initial gateway that checks all the readiness of multimedia devices to run (geroimenko, 2012). the control module runs successfully, which is a guarantee of the running of the rules that have been set according to the set and expected flow (geroimenko, 2012). the instruction module runs successfully, which is a guarantee of the running of the rules that have been set according to the predetermined and expected path (geroimenko, 2012). the exit module works successfully, which is the finalization of the process of running the media. the output module creates an aggregate or dashboard of all processes that have been run electronically (geroimenko, 2012). 3.3.2 beta testing  the beta test is the level of use or use of multimedia by the user which includes aspects of navigation, attraction, satisfaction, and perceived or acquired impacts as shown in tables 10, 11, 12, 13, 14, and 15. in general, 87% of users stated that they were used to dynamic media, and only 13% said they were unfamiliar or familiar. the users mostly understand the flow that occurs in running a progressive media (santoso & gook, 2012). in general, 100% of users state that dynamic media is enjoyable to use in learning. all users agree that media augmented reality makes it happy in its implementation and matches the fun character of the millennial generation (santoso & gook, 2012). in general, 87% of users expressed satisfaction with dynamic media, and only 13% said they were less or dissatisfied. there are still things that need to be improved, especially the availability of existing or owned infrastructure (geroimenko, v. 2012). in general, 100% of users say there are no difficulties and are accustomed to running dynamic media, and no users have trouble. this is in line with the digital age and users as millennials (geroimenko, v. 2012). in general, 100% stated that they understood the material thoroughly from dynamic media, and no one indicated that they did not understand. content that is presented table 6 starting menu test input expected observation conclusion click the "star" menu displays notification information whether you want to leave the media or not displays a confirmation notification accepted table 7 control test input expected observation conclusion click the "control" menu displays notification information whether you want to leave the media or not displays a confirmation notification accepted table 8 navigation test input expected observation conclusion click the "help" menu view information and how to play objects show game instructions shown table 9 quit test input expected observation conclusion click the "exit" menu display notification information whether to get out of media or not displays a confirmation notification accepted table 10 navigation test answer percentage (%) very helpful 27% help 60% doubt 13% table 11 level of interest answer percentage (%) interested 53% doubtful 13% not interested 0% table 12 level of satisfaction answer percentage (%) very helpful 27% help 60% doubt 13% journal of science learning article doi: 10.17509/jsl.v2i2.16204 70 j.sci.learn.2019.2(2).65-70 dynamically is more acceptable and appreciated because it is easier to abstract in the minds of users or students (geroimenko, v. 2012). in general, 87% of users stated that magnetic media motivated them, and only 13% said it was mediocre. augmented reality media can increase learning motivation of most users (geroimenko, v. 2012). 4. conclusion this study produced a multimedia learning model of mathematical material for circles, ellipses, parabola, and hyperbole based on augmented reality. some things that can be concluded from this research activity are: first, as smart-based technology, smart content is needed to support quick learning. second, multimedia-based augmented reality is part of smart content capable of making learning more dynamic. third, multimedia-based augmented the reality of mathematical material for circles, ellipses, parabola, and hyperbole according to the standards of electronic media and dynamic learning. fourth, multimedia-based augmented the reality of mathematical material for circles, ellipses, parabola, and hyperbole according to the character of millennial generations. fifth, multimedia-based augmented reality of mathematical material for circles, ellipses, parabola, and hyperbole are generally attractive, easy to operate, facilitate understanding, increase motivation, and challenge. acknowledgment this research was supported by institutional funds of universitas pendidikan indonesia in 2017. we thank our colleagues at the artificial intelligence laboratory of the computer science, and multimedia studio of the information and communication technology directorate who helped in this research. references azuma, r. t. (1997). a survey of augmented reality. presence: teleoperators & virtual environments, 6(4), 355-385. cawood, s., fiala, m., & steinberg, d. h. (2007). augmented reality: a practical guide. raleigh, nc: pragmatic bookshelf. dekdouk, a. (2012). integrating mobile and ubiquitous computing in a smart classroom to increase learning effectiveness. in international conference on education and e-learning innovations (pp. 1-5). ieee. di, c., gang, z., & juhong, x. (2008). an introduction to the technology of blending-reality smart classroom. in 2008 international symposium on knowledge acquisition and modeling (pp. 516-519). ieee. geroimenko, v. (2012). augmented reality technology and art: the analysis and visualization of evolving conceptual models. in 2012 16th international conference on information visualisation (pp. 445-453). ieee. kemp, j. e., & dayton, d. k. (2003). planning and procing instructional media (fifth edition). new york: harper & row. santoso, m., & gook, l. b. (2012). arkanoid: development of 3d game and handheld augmented reality. international journal of computational engineering research, 2(4), 1053-1059. trilling, b., & fadel, c. (2009). 21st century skills: learning for life in our times. john wiley & sons. table 13 level of convenience answer percentage (%) easy 83% ordinary 17% difficult 0% table 14 level of material understanding answer percentage (%) yes 93% doubt 7% no 0% table 15 level of interest/motivation answer percentage (%) yes 57% doubt 30% no 13% a © 2019 indonesian society for science educator 92 j.sci.learn.2019.2(3).92-96 received: 22 april 2019 revised: 21 may 2019 published: 22 july 2019 the effect of the science web module integrated on batik’s local potential towards students’ critical thinking and problem solving (thinking skill) agnesi sekarsari putri1*, nurfina aznam2 1postgraduate school of science education, yogyakarta state university, yogyakarta, indonesia 2department of chemistry education, faculty of mathematics and science education, yogyakarta state university, yogyakarta, indonesia *corresponding author. agnesi.sekar@gmail.com abstract the 21st-century learning paradigm requires teachers to provide teaching materials that can develop students' thinking skills. this research aims to determine the effect of the web module science integrated local batik potential toward the thinking ability of seventh-grade students of junior high school. the research method used was quasi-experimental design with a posttest-only design. the instrument used is about thinking skill. the data analysis technique is a kruskal wallis test and effect size. the results of the research showed that there were differences in students' thinking skill between the experimental class and the control class, as indicated by the kruskal wallis test, gives significant results. web science module integrated with local batik potential has a significant influence on students' thinking skill as indicated by cohen's effect size score of 0.8. the results of this research can be used to provide insights to science teachers create innovative learning materials to make students more interested in learning science and practicing thinking skills of students. keywords science web module, local batik potential, thinking skill 1. introduction the 21st century is known as the century of the development of science and technology, which has developed rapidly in all aspects of human life. the problems that occur in this century can only be solved by improving and mastering science. science can be enhanced through education. the 21st-century learning paradigm demands a change in school to produce excellent human resources. therefore, a school must be able to develop all the potential in students so that students have adequate provision to overcome global challenges and competition. partnership for 21st century learning (p21) developed 21st-century educational frameworks throughout the world. one of the 21st-century skills in the context is learning and innovation skills. learning and innovation skills include critical thinking skill and problem-solving as thinking skills. these skills are keys in learning to generate superior human resources and essential competency required to overcome global competition. thinking skill is an essential ability that trains the brain to think critically and logically in understanding information, using analytical ability, solving the problem, and appropriately increasing decision-making ability (prajapati, sharma, & sharma, 2017; salonen, hartikainenahia, hense, & keinonen, 2017; butterworth, & thwaites, 2013; prima, & kaniawati, 2011). according to salih (2010), smit (2015), roekel (2011), thinking skill is essential so that students can optimize thinking skill so that they can improve problem-solving ability in daily life, analyze thinking to ensure that students have made the right decision, and can be a provision to compete in a global world. efforts to develop students' thinking skill can be made through learning activities, one of which is learning science. science is a systematic and holistic knowledge associated with natural phenomena obtained through scientific methods. through science learning, humans not only gain understanding regarding natural phenomena but also their interaction with technology and society (wisudawati, & sulistyowati, 2014; susilowati, 2015). mailto:agnesi.sekar@gmail.com journal of science learning article doi: 10.17509/jsl.v2i3.16843 93 j.sci.learn.2019.2(3).92-96 science learning in schools mostly explains about the development of science in the west so that knowledge is not contextual because students cannot see directly and very far, even though science learning should be taught contextually by integrating the local potential of the area where students live. ibrohim, afiat, nurdiana, estiningsih, & martiana (2014); novana, sajidan, & maridi (2014); mulyanto, masykuri, & sarwanto (2017) suggested the integration of local potential in the region so that learning can improve thinking skill and process skill, be more contextual and meaningful, and knowledge can also improve care and responsible for the environment. one of the local potentials that develop in yogyakarta is batik. parmono (2013); wijayanti (2014) explained that batik is a cultural heritage that contains local values; each motif has symbolic and philosophical meaning. batik initially may only be used by the royal family, but now it can be used by all people. batik until now is used for performances of dance, ceremonies, and kraton rituals of yogyakarta. batik is made through several processes; in that process, science content can be analyzed that can be raised in learning. batik making is using chemicals if batik industrial waste is disposed of into the environment without being processed, it will cause environmental pollution. this environmental pollution is found in the necessary competency in curriculum 2013 in kd 3.8, namely 'analyzing the occurrence of environmental pollution and its impact on the ecosystem' and kd 4.8, which is writing about the idea of solving pollution problems in the environment based on observation. one of the efforts to develop students' thinking skill, learning activities are carried out by actively involving students. the implementation of learning requires appropriate learning tools, one of which is teaching the material. teaching material is used to support the learning process and achieve basic competency. based on the observation in junior high school of 2 gamping, teaching material used by teachers are in the form of conventional teaching material in the form of printed media which include government-sponsored textbook, lesson textbooks and student worksheets purchased through school suppliers. teachers have not maximized computer and internet facilities in the school, especially in the learning process and for creating website-based teaching material, even though now they have entered the era of developing technology and information. the use of ict in learning has the benefit of making it easier for students to find information, concretize abstract messages, and improve efficiency and effectiveness in learning activities to enhance the quality of learning (rahmayanti, 2015). for this reason, it is necessary to provide innovative teaching materials to improve the quality of learning. kusuman, mukhidin, & hasan (2016) stated that the use of innovative teaching materials creates interesting learning, fosters interest, motivation, reduces dependency and gets ease in learning each indicator contained in the learning tool compiled by the teacher. teaching material that can be used is the science module web. the module is chosen because it has the characteristics of self-instruction and self-contained, so students are trained independently to manage their learning time and understand the subject (kaur, singh, & singh, 2017). in addition, the module is made in the form of a web module so that students can learn independently by accessing the module at school or at home (weni, & isnaini, 2016; linda, herdini, & putra, 2018). based on the explanation above, this research was conducted to determine the effect of the science web module integrated on batik's local potential towards students' thinking skill. 2. method this research is classified as quasi-experimental research with posttest only design, according to creswell (2012) that can be seen in table 1. this study has two variables, namely independent variables and dependent variables. the independent variable in this study is a science web module integratedpotential local batik (x), while the dependent variable is thinking skill (y2). the population of this study was 192 students of grade vii of junior high school of 2 gamping in the academic year of 2018/2019 divided into six classes; those are class vii a-vii f. the sampling technique used was cluster random sampling. each class is assumed to have the same ability because class grouping has been determined by the school based on knowledge and gender that has been spread equally. the sample in this study were students from two classes used, namely vii e as the experimental class table 1 postest control group design group treatment posttest control class xa o2 experiment class xb o2 table 2 thinking skill questions blueprint aspect indicator number of items comprehend the information identify problems from the information obtained from many resources. 1 using thinking analyzing information obtained in full/complex. 2 make consideration and decision. formulate the strategy to solve the problem. 1 reflecting the decision and process critically choosing the best solution for solving problems. 1 (adapted and modified: trilling, b., & fadel, c., 2009 & pacific policy research center, 2010) journal of science learning article doi: 10.17509/jsl.v2i3.16843 94 j.sci.learn.2019.2(3).92-96 and vii f as the control class. each class consists of 32 students. the instrument used to collect data in the form of questions about thinking skill. thinking skill questions are used to measure students' thinking skill after learning using the science web module integrated with potential local batik. thinking skill questions are arranged based on the thinking skill blueprint that can be seen in table 2. the data analysis technique used is the k-independent sample test / kruskal wallis test and effect size. the kindependent test / kruskal wallis test was used to find out there were no differences in students' thinking skills using the web science module with students' thinking skills using teaching materials used by the teacher, while the effect size was used to determine the effect of the science module on students' thinking skills 3. result and discussion the thinking skill of students is known from the results of the students' scores after carrying out on thinking ability. data on the thinking skills of experimental class and control class students can be seen in table 3. table 3 shows that the average thinking skill value of experimental class is 80.16, and the average thinking skills value of control class is 75.31. based on these results, the average of the thinking skills value of the experimental course is higher than the average value of thinking skills in the control class. the value of students' thinking skill can be seen in more detail by looking at the comparison of the data thinking skills value of the experimental class students and the control class of each indicator that can be seen in figure 1. figure 1 shows that the average thinking skill of each indicator in the experimental class and control class is different. the trial class has an average of each thinking skill indicator that is higher than the average for each control class thinking skill indicator. this shows that there are differences in thinking skill of students who use the science module web with thinking capability of students who use teaching material used by teachers. the difference is supported by the results of the k-independent sample test / kruskal wallis test with spss 22 using the 0.05 significance level presented in table 4. table 4 shows that the result of the kruskal wallis test obtained the asymp value. sig. amount of 0,000 so asymp. sig. < α with α of 0.05, h0 is rejected. therefore, it can be concluded that there is a difference in thinking skill of students who use the science web module integrated with potential local batik by thinking ability of students who use natural science teaching material commonly used by teachers. the subsequent analysis is to find out the magnitude of the effect of the science module web integrated local potential batik towards thinking skill of students using cohen's effect size. based on the results of the calculation of the thinking skill value using the cohen effect size equation, it obtained a score of 0.8 with a high category. it can be said that science web module integrated local potential batik has a high influence on students' thinking skill. the results showed that learning to use the science web module can train students ' thinking skills. in this study, it was found that there was the difference in thinking skills learners who use the science web module with thinking skills learners use learning materials commonly used science teachers, as well as the web module science, integrated potential local batik has a high influence against the thinking skills learners. science learning using the science web module not only displays a series of material but contains learning models, images, videos, practice / task questions, downloads, other website link related to the content contained in a web page that can be accessed online so students can learn independently with or without the help of the teacher in order to achieve the specified competencies (fitri, kurniawan, & ngazizah, 2013; setiyadi, ismail, & gani, 017; tambunan, 2013). according to kaur, singh, & singh, (2017), mangesa, & dirawan, (2016); johar, risdianto, fera, & indiyati, (2014) explained that the science module web has the characteristic of being able to teach in a language that is easily understood by students. for learners, science module web can help students to think holistically table 3 score data thinking skill description experimental class control class n 32 32 maximum score 100 80 minimum score 75 40 average 80,16 75,31 table 4 kruskal wallis thinking skill test results kruskal wallis results chi-square 13.199 df 1 asymp. sig. .000 figure 1 average comparison of thinking skill for each indicator journal of science learning article doi: 10.17509/jsl.v2i3.16843 95 j.sci.learn.2019.2(3).92-96 and systematically related to the learning material and can be used for independent learning. for teachers, it can make more accessible to design learning because the content is presented in one complete competency which includes learning objectives, material, learning activities, assessment, and assessment feedback. science module web learning is learning that refers to constructivism theory that helps students gain the learning experience and knowledge to find and formulate knowledge through the exploration of material-related information together with other students so that they can train students' thinking skills (noel, 2014; smit, 2015; pattiwael, 2016). thinking skills are the necessary capabilities that train the brain to think critically and logically in understanding information, using the analytical capabilities, solve issues, as well as improved decision-making ability (trilling & fadel, 2009). science web module integrated potential local batik is made with joining the material, and the concept of content had known by the students toward their environment. hence, the pupils can relate the knowledge that already gained with the new experience easier. the local potential of batik integrated into science web module can be seen by students in the process of making batik using chemicals that have heavy metal content. batik waste containing heavy metals if not treated properly can cause pollution of the environment. learning activities with the science web module asks learners to discuss problems of the identification of the related environmental pollution arising out of batik waste, analyzing the causes of the occurrence of environmental pollution due to debris the results of activities batik making, interpreting the results of a laboratory test related waste batik, formulate the best strategy in providing solutions that can be done to solve the problem of environmental pollution due to waste batik, as well as suggested solutions best solve the problem of environmental pollution due to waste batik. therefore, the learners thinking skills can be adequately trained. science web module learning integrated the local potential of batik encourages teachers to connect material taught with real/contextual world situation so that they can prepare to identify concept related to potential local batik, practice logical thinking skill, critical, and problem-solving, are interested in science, and grow concern for the environment (lia, udaibah, & mulyatun., 2016; agung, 2015; ilmiyah, wasino, & utomo, 2019; syabandari, firman, & rusyati, 2017). the results of this research are consistent with the research that has been done anafidah, sarwanto, & masykuri (2017) conduct research-based ctl module development to improve thinking skills. the module used contains material application law of newton in life to improve thinking skills. indicators used include thinking skills provide an explanation, the skills of solving problems is appropriate content, concluded, and organize problemsolving strategies and tactics. the results showed that the learning modules could improve thinking skills based on n-gain of 0.36 categorized are, and there is a difference in the average mastery of physics students before and after using the module based on paired t-test earned asymp. sig (2-tailed) less than 0.05 i.e. 0.000. other studies conducted pistanty & mingle (2015) related science module to improve thinking skills. the module used contains material pollution and its impacts on humans and the environment to train thinking skills. indicators used include thinking skills to understand the problem, analyze the problem, make the right decisions, logical, and systematic, and consider the most appropriate choices from various viewpoints. the results showed that the module could improve thinking ability based on normalized n-gain categorized are by a score of 0.62. 4. conclusion based on the formulation of the problem, exposure to data, the result of research, and discussion, this study concludes that there are differences in thinking skill of students between the experimental class and the control class as indicated by the result of the kruskal wallis test with the asymp value. sig. of 0,000. the science web module integrated with potential local batik, has lots of influences on students' thinking skill as indicated by the cohen effect size score of 0,8. for the researchers who will conduct similar research, the advice given to further researchers is expected to examine more about the effect of the science module web that is integrated with different local potential and the influence of the natural science web module on different variables with different background setting. acknowledgment thanks to drs. agus dwiyono, s. ip., m.h. as head of school in junior high school 2 gamping who have given permission to conduct research as well as wanita hendarwesti, s.pd as the science teacher middle school 2 that have helped in the implementation of data retrieval during the study. references agung, l. s. (2015). the development of local wisdom-based social science learning model with bengawan solo as the learning source. american international journal of social science, 4(4), 51-57. anafidah, a., sarwanto, & masykuri, m. (2017). pengembangan modul fisika berbasis ctl (contextual teaching and learning) pada materi dinamika partikel untuk meningkatkan keterampilan berpikir kritis siswa kelas x sman 1 ngawi. jurnal inkuiri, 6(3), 2940. butterworth, j., & thwaites, g. (2013). thinking skill: critical thinking and problem solving. new york: cambridge university press. creswell, j.w. (2012). educational research: planning, conducting and evaluating quantitative and qualitative research. lincoln: university of nebraska. fitri, l. a., kurniawan, e. s., & ngazizah, n. (2013). pengembangan modul fisika pada pokok bahasan listrik dinamis berbasis domain pengetahuan sains untuk mengoptimalkan minds on siswa sma journal of science learning article doi: 10.17509/jsl.v2i3.16843 96 j.sci.learn.2019.2(3).92-96 negeri 2 purworejo kelas x tahun pelajaran 2012/2013. jurnal radiasi, 3(1), 1-5. ibrohim, i., afiat, d., nurdiana, f. r., estiningsih, y., & martiana, c. (2014). pengembangan perangkat pembelajaran ipa-biologi berbasis diskoveri-inkuiri dengan sumber belajar potensi lingkungan lokal kabupaten pasuruan. in proceeding biology education conference: biology, science, enviromental, and learning, 11(1), 1050-1059. ilmiyah, r., wasino, utomo, u. (2019). the development of local wisdom-based contextual social science teaching materials with the theme of indahnya kebersamaan (the beauty of togetherness) for the fourth grade level of elementary school. journal of primary education, 8(3), 296-297. johar, a., risdianto, e., fera, d. a., & indiyati. (2014). perancangan dan implementasi media pembelajaran berbasis web pada bidang studi bahasa inggris di kelas vii smp negeri 1 kota bengkulu dengan menggunakan php dan mysql. jurnal rekursif, 2(1), 8-12. kusuman, a., mukhidin., & hasan, b. (2016). pengembangan bahan ajar mata pelajaran dasar dan pengukuran listrik untuk sekolah menengah kejuruan. jurnal pendidikan teknologi dan kejuruan, 23(1), 39-45. kaur, r., singh, g., & singh, s. (2017). effect of self-learning modules and constructivist approach on academic performance of secondary school students: a comparative study. international journal of humanities and social science research, 3(1), 61-63. lia, r. m., udaibah, w., & mulyatun. (2016). pengembangan modul pembelajaran kimia berorientasi etnosains dengan mengangkat budaya batik pekalongan. unnes science education journal, 5(3), 14191421. linda, r., herdini, h., & putra, t. p. (2018). interactive e-module development through chemistry magazine on kvisoft flipbook maker application for chemistry learning in second semester at second grade senior high school. journal of science learning, 2(1), 21-25. mangesa, r. t and dirawan, g. d. (2016). development of learning module work competence integrated character value of electricity in vocational high school. international journal of applied engineering research, 11(10), 6947-6955. mulyanto, masykuri, m., & sarwanto. (2017). pengembangan modul ipa terpadu smp/mts kelas vii dengan model discovery learning tema air limbah industri batik untuk meningkatkan keterampilan proses sains. jurnal inkuiri, 6(2), 58-60. noel, l. (2014). using blogs to create a constructivist learning environment. journal social and behaviour science, 174, 618-619. novana, t., sajidan, maridi. (2014). pengembangan modul inkuiri terbimbing berbasis potensi lokal pada materi tumbuhan lumut dan tumbuhan paku. jurnal inkuiri, 3 (2), 108-122. pacific policy research center. (2010). 21st century skills for students and teachers. journal kamehameha schools research & evaluation division, 1(1), 5-8. parmono, k. (2013). nilai kearifan lokal dalam batik tradisional kawung. jurnal filsafat, 23(2), 23-29. pistanty, m.a., sunarno, w., & maridi. (2015). pengembangan modul ipa berbasis problem based learning untuk meningkatkan kemampuan memecahkan masalah pada materi polusi serta dampaknya pada manusia dan lingkungan siswa kelas xi smk pancasila purwodadi. jurnal inkuiri, 4(2), 26-29. pattiwael, a. s. (2016). addressing 21st century communication skills: some emerging issues from eil pedagody & intercultural communicative competence. indonesian journal of english education, 3(2), 46-52. prajapati, r., sharma, b., & sharma, d. (2017). significance of life skills education. journal contemporary issues in education research-first quarter 2017, 10(1), 1-6. prima, e. c., & kaniawati, i. (2011). penerapan model pembelajaran problem based learning dengan pendekatan inkuiri untuk meningkatkan keterampilan proses sains dan penguasaan konsep elastisitas pada siswa sma. jurnal pengajaran mipa, 16(1), 179-184. roekel, d. v. (2011). preparing 21st century students for a global society. usa: national education association, 8. rahmayanti. (2015). penggunaan media it dalam pembelajaran. jurnal ilmiah circuit, 1(1), 93-94. salih, m. (2010). developing thinking skills in malaysian science via an analogical task. journal of science and mathematic, 33(1), 110-120. salonen, a., hartikainen-ahia, a., hense, j., & keinonen, t. (2017). secondary school students’ perceptions of working life skills in science-related careers. international journal of science education, 39(10), 2-7. setiyadi, m.w., ismail, & gani, h. m. (2017). pengembangan modul pembelajaran biologi berbasis pendekatan saintifik untuk meningkatkan hasil belajar siswa. journal of education science and technology, 3(2), 102-112. smit, l. s. (2015). a better understanding of 21st century skills in mathematics education and a view on these skills in current practice. journal the mathematics enthusiast, 36, 4-7. susilowati. (2015). ipa dan pembelajarannya. yogyakarta: uny press. syabandari, y., firman, h., & rusyati, l. (2017). the development and validation of science virtual test to assess 7th grade students' critical thinking on matter and heat topic. journal of science learning, 1(1), 17-27. trilling, b., & fadel, c. (2009). 21st century skills: learning for life in our times. san francisco: jossey-bass. tambunan, h. (2013). pengembangan pembelajaran berbasis website dalam matakuliah pengaturan mesin listrik. jurnal cakrawala pendidikan, 31(1), 75-82. wisudawati, a. w., & sulistyowati, e. (2014). metodologi pembelajaran ipa: disesuaikan dengan pembelajaran kurikulum 2013. jakarta: bumi aksara. wijayanti, e. e. y. (2014). pelestarian motif batik batang-pekalongan untuk meningkatkan kesejahteraan masyarakat dalam kurun 5 tahun (2009-2014). jurnal ilmiah pendidikan sejarah ikip veteran semarang, 2(1), 32-40. weni, d. m., & isnaini, g. (2016). meningkatkan hasil belajar siswa dengan pengembangan media pembelajaran e-learning berbasis blog. jurnal pendidikan bisnis dan manajemen, 2(2), 117-121. a © 2019 indonesian society for science educator 29 j.sci.learn.2019.3(1).29-35 received: 11 june 2019 revised: 15 november 2019 published: 28 november 2019 enhancing students’ science achievement through jigsaw ii strategy james l. tabiolo1, danilo jr. villar rogayan2* 1schools division of zambales, department of education, philippines 2college of education, arts & sciences, president ramon magsaysay state university, san marcelino, philippines *corresponding author. danrogayan@gmail.com abstract the science education curriculum in the philippines has shifted from inputs-based to outcomes-based education, putting the learners at the core of the instruction. hence, educators continue to innovate ways on how to engage the learners into relevant and responsive science instruction. further, the implementation of the k to 12 curricula brings a paradigm shift in education in terms of pedagogy, assessment, and outcomes. within-group quasi-experimental research attempts to test the effects of the jigsaw ii strategy on the students’ science achievement. a total of 51 grade 9 students in a government-run secondary school in zambales, philippines, participated in the study. results revealed that the class improved from “developing” to “proficient” level in their science achievement after the implementation of the strategy. it found out that the jigsaw ii strategy had a significant effect on the science achievement of the learners. the study recommends the use of the instructional strategy in enhancing students’ performance. the strategy may be applied in other science topics to see its effectiveness further. this paper likewise contributes to the literature on the effectiveness of the jigsaw ii learning strategy in science teaching in the philippine context. keywords jigsaw ii teaching strategy, k to 12 science curriculum, quasi-experimental research, science teaching 1. introduction globally, science education has been facing a multitude of challenges in today’s digital era in terms of pedagogy, assessment, and outcomes. the challenge for educators of science is to continually think of innovative ways to make science more responsive and relevant. in the philippines, rogayan (2019) reiterated that the science education confronts a myriad of changes in terms of curricular approach brought about by globalization, the industry 4.0, association of southeast asian nations (asean) integration, and the full implementation of the k to 12 curricula. the k to 12 science curriculum stresses that science and innovation should put in common human issues (rogayan & bautista, 2019). the curriculum requires active student participation and dynamic engagement in the learning process. the country lags behind other countries in terms of quality of education, particularly in science education (rogayan & dollete, 2019). the world economic forum in 2018 reports that the philippines ranked 55th out of 137 participating countries in terms of higher education and ranked 76th out of 137 countries in the quality of math and science education. science teachers are challenged to be more innovative and creative for higher student achievement and favourable attitudes at the same time (gernale, duad, & arañes, 2015). it observed that some teachers in science still stagnated in the traditional way of teaching the subject, making them less effective teachers (candrasekaran, 2014). teachers see that the conventional method to be the only best pedagogy in science. the teachers in the traditional method tend to be the sole purveyor of knowledge and ask students to work individually. results in boredom because there are no engaging tasks, challenging activities, and creative works to be accomplished by the learners. teachers need to reexamine how they teach science and move from a traditional method to a more productive method (candrasekaran, 2014). gernale, arañes, & duad. (2015) stressed that a science teacher must be responsible for the device and provide the necessary materials for use in science classes and use practical teaching approaches to bridge the difficulties of students. mailto:danrogayan@gmail.com journal of science learning article doi: 10.17509/jsl.v3i1.17680 30 j.sci.learn.2019.3(1).29-35 one of the instructional approaches implemented in the classroom is cooperative learning, which considered a useful technique in enhancing student achievement in the learning process and has been used more often in a typical classroom environment (siegel, 2005). one type of cooperative learning pedagogical technique is the jigsaw strategy. jigsaw is a cooperative learning model that involves small groups of 5–6 students teaching each other subject matter with success dependent upon student cooperation (gömleksiz, 2007). it is a technique that is very flexible, can be applied in the classroom, and can be customized according to the needs of the learners (hedeen, 2003; doymus, 2007). in this study, a variation of jigsaw called jigsaw ii was used. according to aronson (2000), jigsaw learning strategy allows the students to partake in the challenging and engaging tasks in their respective expert groups fuelled with dynamism since they know they are the only ones with that piece of information when they go back to their respective groups. each group member becomes an expert on the various concepts or methodologies and is tasked to instruct these back to the group (panitz, 1996). just like a jigsaw puzzle, each piece is essential for the completion, and full understanding of the whole concept taught. as time passes by, robert slavin adapted elliot aronson’s work on the jigsaw technique and developed the jigsaw ii technique that allows competition among groups (aronson & patnoe, 2011). this healthy competition brings out the eagerness to participate more in the groups for its improvement. because of the gaps presented, the researchers were prompted to conduct the study. this study looked into the effects of the jigsaw ii teaching strategy in enhancing students’ achievement. innovative, student-centred, and engaging teaching strategies will increase students’ performance in science. these instructional strategies should be utilized in science teaching to make the students more engaged, active, and curious, leading to increased science achievement. 1.1 the framework of the study the present study is quasi-experimental research that determined the effects of the jigsaw ii strategy on the students’ science achievement. the study anchored on the cooperative learning approach. cooperative learning is the instructional practice in which students help each other to learn in small groups towards a common goal (johnson & johnson, 1987). in jigsaw ii cooperative instructional strategy, students are assigned to three-member teams to work on academic materials. initially, all students are assigned to study and understand the basic concepts of the materials. later, each student gives a section/topic on which to become an expert. students with the same section/topic meet in expert groups to discuss their topic, after which they return to their original teams to teach what they have learned to their teammates. then students take group and individual quizzes that result in a team score based on the improvement score system (slavin, 1986). as slavin (2006) points out, teachers cannot only impart knowledge to the learners. the learners must be able to construct their knowledge with the guidance of the teachers, being the facilitators of learning. the use of the jigsaw ii teaching strategy exemplifies a student-dominated learning process (figure 1). figure 1 shows the level of students’ science achievement before the study as the pre-intervention. the intervention used is the jigsaw ii strategy, which measured the students’ achievement in science in terms of knowledge, process/skill, understanding, and performance/ product. the level of students’ achievement in science after the implementation of the strategy served as the post-intervention. 1.2 purpose of the study this quasi-experimental study aimed to test the effects of the jigsaw ii teaching strategy in improving the science achievement of grade 9 students in a secondary school in zambales, philippines. specifically, it sought answers to the following research questions: rq1. what is the level of scientific achievement of the grade 9 students before the intervention? rq2. how does the jigsaw ii strategy improve the science achievement of grade 9 students? rq3. what is the level of scientific achievement of the grade 9 students after the application of strategy? rq4. is there a significant difference in the science achievement of the students before and after the application of the jigsaw ii teaching strategy? figure 1 the conceptual paradigm of the study journal of science learning article doi: 10.17509/jsl.v3i1.17680 31 j.sci.learn.2019.3(1).29-35 2. method 2.1 research design this within-group quasi-experimental research attempted to determine the effects of the jigsaw ii teachings strategy on the students’ science achievement. 2.2 research setting and participants this study was conducted in a government-run secondary school in zambales, philippines. it involved one different class composed of 51 grade 9 students. the class was divided into 17 boys and 34 girls with ages ranging from 14 to 17. 2.3 research instruments to gather reliable and valid data, the researcher used a teacher-made pretest/posttest as the primary gathering tool. the pretest/post-test used is composed of 30 items based on the topics covered for the third grading quarter in grade 9 science-based from the science learners’ module prescribed by the department of education (deped). the conceptual test is composed of 30 items that cover topics of earth and space, such as volcanoes and the interior of the earth, climate, and constellations. the test is divided into knowledge (6 items), process (8 items), understanding (4 items), and performance/product (12 items). to see the improvement of the students’ science achievement during the application of the strategy, they were also evaluated based on their quizzes, laboratory activities, and team quizzes. 2.4 data collection the pretest was conducted at the start of the lesson to measure the science achievement of the class before the application of the technique. on the other hand, the posttest was administered toward the end of the study to determine how much the said technique helped in improving the science achievement of the students. the researcher utilized the jigsaw ii as an intervention for a total of 4 weeks. this teaching strategy is a cognitive and collaborative strategy that recognizes the efforts made by each student in every activity and skill in studying different science concepts and ideas. the steps in infusing the strategy are modified based on slavin (1986), as shown in table 1. 2.5 data analysis data were analyzed using item analysis, frequency count and percent, weighted mean, standard deviation, and t-test for paired samples. the item analysis was used to measure the proficiency level of the students in the four domains of achievement, namely, knowledge, process, understanding, and performance/product. the frequency counts and percentages used for the tabular presentation of the raw scores of the students during the pre-test and post-test. the mean was used to determine the average scores of the students in the pretest/posttest, quizzes, laboratory activities, and team quizzes. using the way, the researchers can identify the level of students’ science achievement before and after the application of the intervention. the score interpretations, based on the deped order no. 31, s. 2012 are as follows (table 2). table 1 modified steps of the jigsaw ii strategy step title description 1 reading each participant of the expert group gives an identical set of materials relevant to the topic, as well as an expert sheet. each student had a designated sub-topic to study. the researcher let them study first the different issues before discussing it properly. 2 expert group discussion participants working on the same topic share what they had to learn based on the reading 3 homegroup reporting participants in the working group go back to their original homegroup to teach others the things they have discussed. 4 testing after the mastering of the testing materials, a quiz bee-like evaluation happened. it was composed of eight groups with six to seven members. it was like a quiz bowl type of assessment. 5 group recognition each member of the winning group had a reward because their efforts exerted to perform successfully. in every assessment, there was a group champion whereby the challenge was to maintain their throne as a champion and for the other groups to replace the winning group. table 2 score interpretation verbal description (vd) score range pretest/ posttest quiz lab activity team quiz advanced 25-30 9-10 21-25 5 proficient 19-24 7-8 16-20 4 approaching proficiency 13-18 5-6 11-15 3 developing 7-12 3-4 6-10 2 beginning 1-6 1-2 1-5 1 table 3 distribution of students’ scores in pretest score frequency percent 19-24 4 7.84 13-18 36 70.59 7-12 11 21.57 total 51 100.0 weighted mean 14.96 (approaching proficiency) journal of science learning article doi: 10.17509/jsl.v3i1.17680 32 j.sci.learn.2019.3(1).29-35 3. result and discussion 3.1 level of science achievement of the students before the intervention a 30-item diagnostic test was administered to assess the science achievement of the students and to determine their proficiency in the different domains of learning. the pretest results were tabulated to determine the science achievement level of the student before the application of the technique (table 3). the results of the pretest showed that only 7.84% of the class belonged to the proficient level, most of the students belonged to the approaching proficiency level (70.59%) and developing level (21.57%). none of the students belonged to the advanced and beginning levels. with a calculated mean of 14.96 in the pretest, the performance of the class considered as approaching proficiency. it means that most of the students in the class have the fundamental knowledge and skills and core understandings and with little guidance from the teacher. table 4 shows the level of students’ proficiency in the pretest, which is computed based on the total number of students who performed it correctly. the percentage of the students classified according to the level of performance of the class was 50.98% in knowledge, 56.86% in process/skills, 49.02% in understanding, and 47.06% in performance/product. the result of the pretest showed that 26 (50.98%) students performed well in all of the four domains of students of academic performance. the table displays that the least performed domain is the performance/product (47.06%), which implies that only 24 out of 56 students can answer the questions that need an application to the real situations demonstrated through products and performances. the process skill was the most performed domain (56.86%). it only means that 29 students had no difficulties in questions based on the student’s ability to process and make sense of information. it shows the understanding of the content of students and develops their critical thinking. it can be observed from the results that the students can hardly answer the items on the performance/product and understanding domains. it shows that students were good at performing different skills or doing activities, but their understanding of their activities and applying to the real situation were impoverished. it led the researcher to improve these domains through the use of the teaching strategy. generally, the mean scores of the four domains were on the average level. it can be seen that the class was performing well before the application of the teaching strategy. 3.2 level of science achievement of the students during the intervention formative assessment tools such as quizzes, laboratory activities, and team quizzes were gathered to determine the development of the science achievement of the students. table 5 shows that the students are at the advanced level based on the quiz overall mean score of 8.84 (sd=0.91), implying the effectiveness of the strategy during its application. results of the respondents’ quiz mean scores had a favourable increase during the implementation of the strategy. the strategy also helped much in improving the class science achievement seeing the results of the students’ laboratory activities. the mean score of the students’ four laboratory activities was 22.09 (sd=2.19), which is interpreted as advanced. that increased from 21.76 to 22.53 which implies that the strategy is effective in improving student achievement in terms of laboratory activities. the strategy requires the participation of each student to make a successful outcome of their group. each effort of the students is precious in this kind of strategy. for this reason, the researcher conducted a series of team quizzes that will also determine the development of the students. the results of their team quizzes showed that there increased their academic performance with regards to their participation in the evaluation process. table 4 distribution of proficient students in pretest per domain domain frequency percent knowledge 26 50.98 process 29 56.86 understanding 25 49.02 performance/product 24 47.06 average 26 50.98 note: n=56 table 5 students’ mean scores in the formative assessments formative assessment mean score sd verbal description quiz 8.84 0.91 advanced lab activity 22.09 2.19 advanced team quiz 4.09 0.70 proficient table 6 distribution of students’ scores in post-test score frequency percent 25-30 9 17.65 19-24 27 52.94 13-18 15 29.41 total 51 100.0 weighted mean 21.02 (proficient) table 7 distribution of proficient students in post-test per domain domain frequency percent knowledge 37 72.55 process 33 64.71 understanding 40 78.43 performance/product 35 68.63 average 36 70.59 note: n=56 journal of science learning article doi: 10.17509/jsl.v3i1.17680 33 j.sci.learn.2019.3(1).29-35 the overall mean score (4.09) in the team quizzes interpreted as proficient implies that the students have a favorable performance with the use of the intervention. the increase in the mean scores from 3.45 to 4.69 indicated that the strategy improved the science achievement of students in terms of the team quiz. jigsaw cooperative learning strategy is one that mostly emphasizes facilitating learners with the opportunity to help each other in building and understanding the tasks assigned in the classroom (abed, sameer, kasim, & othman, 2019). 3.3 level of science achievement of the students after the intervention to assess the effectiveness of the jigsaw ii teaching strategy in improving the science achievement of the students, a post-test that has the same questions as the pretest was administered (table 6). the results have shown that none of the class belonged to the beginning and developing level, 29.41% of the class belonged to the approaching proficiency, 52.94% for proficient, and 17.65% of the class are in the advanced level. with the calculated mean of 21.02 in the post-test, the class belonged to the proficient level. the frequency of correct answers in the post-test revealed that the science achievement of the class has improved (table 7). the results showed that more than half of the class got correct answers in each of the domains used. that shows significant improvement in the science achievement of the students after the application of the jigsaw ii teaching strategy. based on the table, it can be observed that the students’ level of scientific achievement marked significant improvement after the application of the technique. more than half of the class, 36 students (70.59%), performed well in the four domains. in the domain of knowledge, students got an average of 37 (72.55%). that means that students conquered their difficulties in answering questions that test the information they acquired and information to firm up and deepen understanding. process skill had an average of 33 (64.71%). that only means that students had lessened their difficulties in questions based on the student’s ability to process and make sense of information. it shows the understanding of the content of students and develops their critical thinking. the domain of understanding had a total of 40 (78.43%). that shows that 39.86 students can answer the questions that were expressed using explanation, application, empathy, perspective, and self-knowledge, or any other discipline-based manifestation or indicator of understanding. in performance/product, an average of 35 (68.63%) students. it shows that the students can only answer the questions that need an application to the real situations demonstrated through products and performances. the results of the study is parallel with previous studies on the effectiveness of jigsaw learning strategy in improving students’ achievement (abed et al., 2019; azmin, 2016; gömleksiz, 2007; kam-wing, 2004), developing students’ conceptual knowledge and understanding (yimer & feza, 2019), enhancing students’ attitude (kam-wing, 2004; yimer & feza, 2019). 3.4 difference in the science achievement of the students before and after application of the jigsaw ii results of the respondents’ pretest and post-test scores were compared (figure 2). it can be observed that there was an improvement in the students’ science achievement. there was a positive change of 17.65% in the percentage table 8 t-test of pretest and post-test scores test mean sd mean gain tvalue df pvalue pretest 14.96 2.99 6.06 19.881 50 0.000 posttest 21.02 3.11 note: significant at p<0.05 figure 2 comparison of students’ pretest and post-test scores figure 3 comparison of students’ proficiency level pretest and post-test journal of science learning article doi: 10.17509/jsl.v3i1.17680 34 j.sci.learn.2019.3(1).29-35 of students that belong to the advanced level and 45.10% in the level of proficient. also, there was a negative difference of 41.18% in the student percentage who belong to approaching proficiency level and -21.57% on the developing level. although this marked a positive change, no one belongs to the beginning and developing standards after the application of the strategy. on the other hand, figure 3 presents the comparison of the students’ proficiency level in the pretest and posttest. it shows that after the application of the strategy, there was an increase of 21.57% in the students’ knowledge domain, a 7.85% increase in their process/skills domain, 29.41% difference in their understanding and 21.57% increase in their performance/product domain. an increase of 19.61% on the calculated mean that determines the effectiveness of the strategy. the data show that there was a significant improvement in the overall science achievement of the grade 9 students after the application of the jigsaw ii teaching strategy. jigsaw would be a significant strategy to cooperate with the students in the classroom with an intimate atmosphere (fry, ketteridge & marshall, 2008). to further determine the change in the science achievement of the class before and after the application, the t-test of the pretest and post-test scores are presented (table 8). the table shows that the pretest means score of the class is 14.96 (sd=2.99), and the post-test mean score is 21.02 (sd=3.11). a gain score of 6.06 was obtained after the application of the strategy, which indicates that there was an improvement in the science achievement of the students with the use of the strategy. to determine if there was a significant difference before and after the application of the strategy, the t-test for paired samples computed. the t-value obtained was 19.881, and the p-value (p=0.000) is less than the 0.05 level of significance. hence, the null hypothesis is rejected. it means that there was a significant difference in the science achievement of grade 9 students after the application of the jigsaw ii teaching strategy. the results corroborated several studies (baron, 2019; barrett, 2005; chukwu & dike, 2019; evcim & ipek, 2012; maden, 2011; mbacho & changeiywo, 2013; mohammed & hamied, 2019; oliveira, vailati, luiz, boll, & mendes, 2019; sudrajat, iasha, & femayati, 2019; suroto, 2017; ward & lee, 2005; yoshida, 2018) which provide empirical evidences in the effectiveness of jigsaw in enhancing student achievement in general. 4. conclusion the study determined the effects of the jigsaw ii strategy in enhancing students’ science achievement in the philippine setting. the use of the jigsaw ii teaching strategy marked a significant impact on the science achievement of the grade 9 students based on the results of the study. the purpose of the intervention has improved the science achievement of the students. it was evident in the results of their pretest/post-test, quizzes, laboratory activities, and team quizzes. that further validated with the teacher’s observation of the students’ active participation in the class discussion and the different learning tasks. furthermore, the pedagogical strategy employed by the teacher likewise enhanced the students’ sub-skills in the four learning dimensions, the knowledge, process, understanding, and performance/product. in the application of the instructional strategy, the teacher may prepare a set of sub-topics for students of each group before the lesson proper. the teacher may sternly reinforce their teaching authority to implement the technique in the learning process effectively. classroom management may take into consideration to maximize the learning space. teachers may employ the jigsaw ii teaching strategy in improving the achievement of the students in science. they may further localize and contextualize the learning tasks to suit the needs of their students. a followup study may be done to validate the effects of the intervention in improving the achievement of the students in science. since the present study only involved one group, further studies may involve two groups, which will serve as experimental and control groups for better comparison. acknowledgment the authors would like to acknowledge dr. ma. ester dlr marinas and ms. lea f. dollete of the president ramon magsaysay state university (prmsu) for the constructive criticisms in the improvement of the paper; to the college of education, arts & sciences of the university and the schools division of zambales for the support in this action research project. references abed, a. z., sameer, s. a., kasim, m. a., & othman, a. t. (2019). predicting effect implementing the jigsaw strategy on the academic achievement of students in mathematics classes. international electronic journal of mathematics education, 15(1), em0558. aronson, e. (2000). nobody left to hate — the humanist, 60(3), 17. aronson, e., & patnoe, s. (2011). cooperation in the classroom: the jigsaw method. london. uk: pinter & martin. azmin, n. h. (2016). effect of the jigsaw-based cooperative learning method on student performance in the general certificate of education advanced-level psychology: an exploratory brunei case study. international education studies, 9(1), 91-106. baron, r. (2019). employing jigsaw in english academic writing: an action research in an efl class in indonesia. research and innovation in language learning, 2(3), 228-236. barrett, t. (2005). effects of cooperative learning on performance of sixth-grade physical education students. journal of teaching in physical education, 24(1), 88-102. candrasekaran, s. (2014). productive methods of teaching middle school science. international journal of humanities and social science invention, 3(7), 15-25. chukwu, j. c., & dike, j. w. (2019). effects of jigsaw-puzzle and graphic organizer instructional strategies on biology students’ performance in abia state. archives of current research international, 1-6. journal of science learning article doi: 10.17509/jsl.v3i1.17680 35 j.sci.learn.2019.3(1).29-35 doymus, k. (2007). effects of a cooperative learning strategy on teaching and learning phases of matter and one-component phase diagrams. journal of chemical education, 84(11), 1857. evcim, h., & i̇pek, ö. f. (2013). effects of jigsaw ii on academic achievement in english prep classes. procedia-social and behavioral sciences, 70, 1651-1659. fry, heather, ketteridge, steve, & marshall, stephanie. a handbook for teaching and learning in higher education: enhancing academic practice. routledge, 2008. gernale, j. p., arañes, f. q., & duad, v. (2015). the effects of predictobserve-explain (poe) approach on students’ achievement and attitudes towards science. the normal lights, 9(2). gömleksiz, m. n. (2007). effectiveness of cooperative learning (jigsaw ii) method in teaching english as a foreign language to engineering students (case of firat university, turkey). european journal of engineering education, 32(5), 613-625. hedeen, t. (2003). the reverse jigsaw: a process of cooperative learning and discussion. teaching sociology, 31(3), 325-332. johnson, d. w., & johnson, r. t. (1987). learning together and alone: cooperative, competitive, and individualistic learning. prentice-hall, inc. kam-wing, c. (2004). using jigsaw ii in teacher education programmes. hong kong teachers’ centre journal, 3, 91-97. maden, s. (2011). effect of jigsaw i technique on achievement in written expression skill. educational sciences: theory and practice, 11(2), 911-917. mbacho, n. w., & changweiywo, j. m. (2013). effects of jigsaw cooperative learning strategy on students’ achievement by gender differences in secondary school mathematics in laikipia east district, kenya. journal of education and practice, 4(16), 55-63. mohammed, i. j., & hamied, d. t. (2019). using jigsaw cooperative learning strategy to improve the efl iraqi students in writing skill. journal of al-frahedis arts, 11(38), 504-524. oliveira, b. r., vailati, a. l., luiz, e., böll, f. g., & mendes, s. r. (2019). jigsaw: using cooperative learning in teaching organic functions. journal of chemical education. panitz, t. (1996). getting students ready for learning. cooperative learning and college teaching, 6(2), 10-30. rogayan jr, d. v., & bautista, j. r. (2019). filipino students’ preferred motivational strategies in science: a cross-sectional survey. irje (indonesian research journal in education), 358-372. rogayan, d.v., jr. & dollete, l.f. (2019). development and validation of physical science workbook for senior high school. science education international, 30(4). rogayan jr, d. v. (2019). biology learning station strategy (bliss): its effects on science achievement and attitude towards biology. international journal on social and education sciences, 1(2), 78-89. siegel, c. (2005). implementing a research-based model of cooperative learning. the journal of educational research, 98(6), 339-349. slavin, r. e. (1986). using student team learning (3rd ed.). baltimore, md: the johns hopkins university. slavin, r. e., & davis, n. (2006). educational psychology: theory and practice. sudrajat, a., iasha, v., & femayati, f. (2019). the influence of the use of cooperative learning model jigsaw & two stay two stray and the learning interest result on 5th grade social science. iceap 2019, 2(2), 28-33. suroto, m. (2017). the effectiveness of jigsaw ii model in improving students' understanding of citizenship education. in 5th sea-dr (south east asia development research) international conference 2017 (seadric 2017). atlantis press. ward, p., & lee, m. a. (2005). peer-assisted learning in physical education: a review of theory and research. journal of teaching in physical education, 24(3), 205-225. yimer, s. t., & feza, n. n. (2019). learners’ conceptual knowledge development and attitudinal change towards calculus using jigsaw co-operative learning strategy integrated with geogebra. international electronic journal of mathematics education, 15(1), em0554. yoshida, m. (2018). communication jigsaw: a teaching method that promotes scholarly communication. international journal of emerging technologies in learning (ijet), 13(10), 208-224. a © 2019 indonesian society for science educator 12 j.sci.learn.2019.3(1).12-18 received: 25 july 2019 revised: 11 november 2019 published: 28 november 2019 the development of science comic in human digestive system topic for junior high school students nelah roswati1*, nuryani y. rustaman2, ikmanda nugraha1 1international program on science education, faculty of mathematics and science education, universitas pendidikan indonesia, bandung, indonesia 2department of biology education, faculty of mathematics and science education, universitas pendidikan indonesia, bandung, indonesia *corresponding author. nelahroswati96@gmail.com abstract the science education approach has covered by using printed teaching media. one of the popular printed press that most accessible and may used in science education is a comic book. however, it is sometimes difficult to find the ideal and the appropriate comic books that can be used as the instructional tool of science education, because most of them are inappropriate for learning science and did not fit with the readers’ culture. the purpose of this study is to develop and evaluate the science comic to contribute to science learning about human digestive system topic (hdst) concepts. in this study, a science comic book was created and implemented to 92 students of year eight from three different junior high schools and three science teachers as a subject implementation development. students’ responses through the questionnaire and students-teachers’ implementation test sheets evaluated through qualitative content analysis. the model used for this study is design and development. the result shows, most of the students agreed that science comic book helps them to learn through simplifying science concept and understanding the topic more accessible. the ideal science comic should also follow the right steps, appropriate aspects consisted, gain more science experience for both students and teachers to enjoy the learning process. keywords instructional media, science comic book, human digestive system topic 1. introduction in many countries, the engagement of science education and communication has been emphasized using teaching media. teaching media that is used usually in the form of printed media such as books, magazine, and newspaper (falk, storksdieck, &dierking, 2007). the use of teaching media in science education is an essential issue because science education mostly appears through public media such as newspapers, magazines, radio, and television that people learn primarily about what they know about science (detjen, 1995). in 2014, a survey from media studies centre or roper centre found that 68% of the public get most of their science information in an environment from television, 59% from the newspaper, 32% from the magazine, and 30% from the radio. the result of the us government is even farther back at 24%, schools are 19%, and local businesses also corporations are far down the list at 11% and 9% (detjen, 1995). among the popular media, printed media has a strong capability to communicate science education because they are available to all various locations such as public or private (gilbert & lin, 2013). although the availability of printed teaching media is useful and useful to use in science education, the content of teaching media itself should be covered the need to communicate the fascination, joy, and utility of science (hosler & boomer, 2011). the reality says differently; most of the printed teaching media cannot bring these needs; one of the common examples is a traditional science textbook. among a variety of printed media, comics believed as a possible means of conveying scientific information that could cover fascination, joy and the utility of science because the contamination of humour as the specific feature of comics may attract many people’s attention, interest, and enjoyment (tatalovic, 2009). moreover, the production of science comics nowadays is not always to be proper by its way. most of science comic mailto:nelahroswati96@gmail.com journal of science learning article doi: 10.17509/jsl.v3i1.18120 13 j.sci.learn.2019.3(1).12-18 production focus on one aspect only without considering the sense of comic itself, and this condition aggravated by the fact that science comic for science teacher is sometimes challenging to find (ozdemir, 2017). therefore, the present science comic that is ideal for science teaching should develop along with this study. this study focused on developing and evaluating some comic books that contribute to learning about the human digestive system topic. the topic is for 8th grade junior high school students in the first semester. this topic chapter will emphasize the concept of the human digestive system process and organ's roles, also added by the effort to maintain the health of the social digestive system concept. this development focused on several aspects such as science content, sense of artwork that includes humour and uniqueness, and the language aspect to produce a better science comic book. the validation is conducted by the experts to check the suitability of science comics before science comics implemented to the science teachers and students for the pilot study. the difference between comic strip and comic book in presentation aspect support that science comic book is more available to deliver the science information because both frames and words can explain the story. this statement supported by the fact that comic books and animated cartoons can use as media communication of education, especially in science education. many common comics have put the reference about the scientific facts and ideas. besides, comics are a popular art form, especially for children, and they provide a potential medium for science education and communication (tatalovic, 2009). the development of science comics has been improved based on three aspects (lin, lin, lee, & yore) 2015) those aspects that have been considered consist of humour, contextualize learning, and visualize learning. discipline-specific knowledge of biology has developed in the fields of developmental physiology and cognitive physiology. the investigation of children's knowledge about the concept in biology, especially in the human digestive system topic, formed has gathered momentum in pre-school education research in the last 30 years (ahi, 2017). through the amount of research, many researchers found that the biological knowledge viewpoint by the students is always increasing. there is a paucity of research that biological knowledge is ever-changing over time, and depending on the developmental process (geerdts, van de walle, & lobue, 2015). on the contrary, istikomayanti and mitasari (2017) found that there are overcoming misconceptions in the learning process conducted by three teachers, neither certified nor uncertified. the result of students’ misconception test mostly reached only level 3 or medium; thus, the study of the misconception of the digestive system material needs to get the attention of the teacher and educational practitioner. with that explanation above, it can state that comic books may use as teaching media in the science classroom. however, how an instructional comic book should be designed and develop with scientific methods is not answered well. the purpose of this study is to develop a science comic book about the human digestive system and to get a review from the experts during the development. after the construction of a comic book has developed, the teachers’ feedback through the comic also analyzed together with the students’ questionnaire and implementation test sheets. the results will use as feedback for improving a better science comic book. 2. method 2.1 research methodology the research design used in this study is design and development. the development model used the following data input from several subject implementation. this type of research aiming to develop an instructional finding and solution to achieve a goal related to education or learning. this research design working based on the existing theory and evidence, testing individual components, and get feedback in the development process. this study had two main stages: development and evaluation of science comic book about human digestive system concept that started from identification of problem, formulating research objective, curriculum and content analysis, aspect measurement analysis, the construction of science comic which included the creative process of writing the first outline & storyboard and drawing the comic with expert reviews. the evaluation stage consists of the implementation and analysis of participants’ responses. the participants’ responses investigated through qualitative content analysis. 2.2 material this study has been passed several stages and reviews to develop the science comic book in the human digestive system. the study begins with the curriculum analysis as science content development. the standard curriculum used in this study following the indonesian 2013 curriculum and focused on the process, organs, and how to maintain the health of the human digestive system. after the science content analysis has constructed, the next stage is the analysis of science comic aspects as the measurement of this study. in brief, there are four aspects of science comics that are sought by this research. the aspects are consist of artwork, science content, language, and uniqueness. those aspects would be used as the main aspects to help the reviewers and the students focus on giving feedback and suggestion through science comic development. the feedback and score collected from the participant they give. the way how participants give the score is by providing an assessment gradient. table 1 below shows the journal of science learning article doi: 10.17509/jsl.v3i1.18120 14 j.sci.learn.2019.3(1).12-18 assessment gradient of each indicator measurement provided by the participants. after the analysis of the science content aspect done, the creative process of science comic construction can start. in general, the stage of the creative process consists of: writing the first outline of the comic, making the panels and storyboard, drawing the comic, reviewing by an expert, and revising the comic. the first outline of the story in this comic tells that there are the two-twin sibling that lived with their grandfather, who becomes a professor. they live happily with one robot named payjo. every science phenomenon that happens in their daily life always be asked by two sibling twins and professor will ever explain it to their child. in this series of comic books, they were talking and told about the human digestive system so that the topic explanation that they are discussing is about human digestive system and its been added by some humour, culture, unique, and drawing style to make a proper sequence of comic. after finishing drawing, some experts reviewed the comic by using the instrument rubric that consists of several indicators from the aspects (artwork, science content, languages, and uniqueness). the comic book revised according to the feedback and suggestion from the experts, to develop the final of the science comic book for the implementation test, two examples of comic pages developed shown in figure 1. 2.3 implementation test on the implementation test, participants are consists of three science teachers and three groups in 92 total of students in grade eight chosen from three different junior high schools in bandung city area, indonesia. after getting permission for implementation, the researcher starts to conduct the test at a different time because the location of the school is far from each other. the implementation process is the same for three of the classroom. after the science comic book printed on papers given to the students, the students will be guided to read all the comic and directed to fill the questionnaire and implementation rubric test, the questionnaire and implementation rubric about the comic of the human digestive system listed in table 2. after answering the questionnaire, the students are directed to fill the implementation rubric test that consists of several aspects of measurement. the gradient measurement can see in table 1. the indicator of each table 3 students’ implementation rubric no aspect indicator 1. legibility i can understand the story, characters, and image easily 2. science content the story of each organ can be easy to understand. 3. characters the characters of the comic give a strong personality to explain the story 4. image and text i can see the image and text balloon in the comic is related to each other 5. learning outcomes i think comic gives learning more easy to understand 6. science terms the science term that used in the comics is understandable for me. 7 image to content the comic gives me a natural picture to explain science concept more easily 8. ambiguous words there are no difficulties for me to comprehend this comic (there are no ambiguous words) 9. expression character to the story the expression and the text of the characters are related to the story 10. uniqueness this comic is interesting and unique to read table 1 assessment gradient on the rubric for each indicator measurement assessment (√) gradient scale very poor poor enough good very good 1 2 4 5 6 table 2 students’ questionnaire about human digestive system no question aspect 1. what is the story of the comic? can you explain it using your own words? general information about science comic 2. who are the characters of the comics? can you call it one by one? 3. how is the process of digestion happen in the mouth? science content understanding 4. during swallowing, the food that will transfer to the esophagus from the mouth will shape like a food ball. what is it called? 5. why does the condition of the stomach should be acid? 6. why should the small intestine be longer than the large intestine? 7. please mention three accessory organs based on comics you have read! recalling knowledge 8. what is the function of the appendix in the small intestine? 9 why should we clean our hands after going back from the toilet? 10. mention three good eating habits for humans, especially for children! journal of science learning article doi: 10.17509/jsl.v3i1.18120 15 j.sci.learn.2019.3(1).12-18 aspect can see in table 3. in addition to this study, the researcher also added the data of observation in class and the students’ interview after implementation. the result data will analyze in the result session. 2.4 data analysis in this research, there are several sources of data: judgment review from the experts, students and teacher implementation result, the written answer of students’ questionnaire, observation during the implementation and students’ interview. because these data are in verbal form and patterns that may be qualitative feedback, they analyzed through qualitative content analysis. all data that have obtained, such as the review from the expert judgments, teachers-students implementation result, and written answer to the questionnaire, will be analyzed. the participants, such as experts, teachers, and students, as the primary data of this research, were coded for some categories to draw a meaningful pattern and to obtain useful feedback for the development process of instructional science comics. the analysis of all data will be presented in percentage form before it shows in the table bar form for comparison and discussion results. the other data, such as researchers' observation and interviews, which are supplementary data of the study, were used to ensure the pattern in data from the questionnaire in the implementation process. in other words, the objective was to establish the validity of the results by using three different sources of the data. the descriptive statistic will use as the analysis of data technique. in this study, quantitative analysis avoided because numeric results would not provide in-depth and detailed feedback for the development process of instructional science comics. 3. result and discussion 3.1 experts’ review to science comic aspects the result of experts’ reviews divided into four aspects result. each result was review and scored by the experts. the first is about the artwork aspect; this aspect consists of ten indicators related to the artwork things of the science comic. experts review through the artwork aspect shows the positive value because each expert (coded by e1, e2, e3, and e4) gives positive feedback for each item indicator. the items that have a higher average score agreement of 90% are image clear, characters, and cartooning humour. the item that has the medium score agreement of 80% is visualization item, and that have a lower score agreement of 70% are setting place or background item. the result of the artwork shows more than 50% caused the characteristic of artwork that can be read by the readers. besides that, the cartoon things that visualized in the comic can stimulate the active involvement of the students in the learning of science (dalacosta, kamariotakipaparrigopoulou, palyvos, & spyrellis. 2009) the second one is the science content aspect. experts review through science content items shows a positive value and feedback. the higher score agreement is 90% placed on concept appropriateness and visualizes learning items. the medium score agreement is 80% set on the specific example, learning experience, and thinking figure 1 the example of two pages of the comic book developed in this study journal of science learning article doi: 10.17509/jsl.v3i1.18120 16 j.sci.learn.2019.3(1).12-18 triggering item. and the lower score agreement is 70% placed on the up-to-date item aspect. the third is about the language aspect. both english and indonesian language from the expert feedback shows that the sense of humour aspect gets a higher score of 100% agreement indonesian language and 93% agreement in english. the lower score gained by a grammatical structure that gets 73% in english and 60% in the indonesian language. the last aspect is about uniqueness aspect. from five items, one item of culture aspect gets the lower score agreement, which is 70%. it indicates that the cultural aspect in this comic is not quite enough to be added, so this aspect should be changed or revised to gain more score. besides the score, experts also give some comments and suggestions through this aspect. those aspects are commented on and suggested to gain a better result of science comics, especially for the humour aspect. the humour aspect has been proven to gain a good score of 100%. this humour aspect helps students to reduce tension in the classroom, relieving students’ embarrassment, and increase their boredom (ozdemir, 2017). 3.2 teachers’ feedback through science comic aspect the result of teachers’ feedback also divided into four aspects. the first aspect is the artwork of the science comic. the higher score gained on cartooning humour, visualization, balloon dialogue, and setting place. the score agreement is 93%. this score indicates that those item has filled the expectation of the teacher through the comic artwork. the rest items gain lower rating, which is framing, image clear, story flow, perspective, characters, body movement, and text. the average score agreement increased was 87% from 100% ideal score. the second aspect is about science content. the higher score is 100% achieved by learning experience items. the medium score earned by up-to-date and enjoyable learning items, which is an 87% score, and the lower score obtained by thinking triggering things, which is an 80% score from 100% ideal score. the third aspect is about language aspect. the higher score gained by understanding words, which is a 93% score. the medium score gained by word to expression, word to image, and scientific content dialogue, which is an 80% score. and the lower score gained by the grammatical structure, which is a 73% score from the ideal score agreement of 100%. the fourth aspect is about uniqueness. the data shows all item has the same score, which is 93% in all item of relatable, unique and new, enjoyable learning, sense of humour and culture. this score indicates the positive feedback from all science teachers 3.3 students responses through science comic students’ responses through the science comic divided into two different data. the first is about the students’ implementation test. this test was allowing the student to give the score on each aspect in table 3. the implementation test divided into four aspects, which are artwork, science content, language, and uniqueness. the first aspect of the artwork shows that 59.7% of students reputed that the image and text of the comic are perfect. besides that, there are 52.1% of students agree that the expression of and text of the character in the comic is related to the story. the second aspect of the science content aspect shows that 52.1% of students agree that the science content on the comic is useful to understand. besides that, 43.4% of the students agree that the image is excellent to explain the science concept more easily. the third aspect is the language aspect. there are 52.1% of students agree that the legibility of the story and character and image is excellent. besides that, there are 42.3% choose that the scientific term used in the comic is proper. the last aspect of uniqueness shows that 60.8% of the students choose that the science comic is excellent, which is very interesting and unique to read. the second is about the students’ questionnaire results. students’ questionnaire results conducted to 92 students of year eight from different schools. the analysis answer has been scored based on the rubric of each question. the lower score gained by students bj-f-54 and bj-f-55 (students code, bj means classroom two, f means female gender, and 54 means the number of the student), which is 50 score. the highest score gained by student pn-m-03 (pn means classroom one, male gender, number three), and student bd-m-85 (bd means class three, male gender, number 85), which is 92 score. the average score gained from 92 students is 75.9 score. after the average score already obtained, the comparison between how many students get the lower score and higher score from the average score can obtain. figure 2 shows from the total number of students, which are 92 students, there are 40 students get the lower score from the average score, and there are 52 students get the top score from the average score. the result means the science comic can help the students to gain the average figure 2 the comparison of students’ average score journal of science learning article doi: 10.17509/jsl.v3i1.18120 17 j.sci.learn.2019.3(1).12-18 score and ease them to learn about the human digestive system. those results prove by direct interview and observation, and the students enjoyed learning science through the comic. most of them are have fun with the relationship between visualization and science content. moreover, more of them are like the humour aspect too. it indicates that humour can be the basis of attraction that leads to life-long participation in science. 3.4 finding from students’ observation and interviews the observation class divided into three classrooms, which are pn, bj, and bd classroom. each classroom treated the same, but the language the media used is different. pn classroom gives an english science comic book, and the rest classroom is an indonesian language comic. all classrooms provide positive feedback and attitude, especially in classroom bj. in this classroom, all student shows a conducive class environment different from the other. besides that, the interview also conducted in this study. the researcher focuses on the students who got a lower score (student bj-f-55) in a questionnaire test, which is 50 score. the answer that she filled to the question was out of topic and different from the science concept given to the comic story. however, what makes the researcher pay attention to her result is, she gives an excellent score to the readability test, which means these students reputed that she can comprehend the comic. when the researcher asks this student, she is randomly giving the score and try to appreciate the researcher's hard work through the science comic made, when filling the questionnaire, she also cheated to her friend. this bj-f-55 student said that she could not focus on reading the science comic because she dislikes reading books, magazines, comics, and other reading materials. so that she cannot comprehend the science comic better. 4. conclusion based on the research result, there is some conclusion that can reach. firstly, the step of making the science comic should following several steps such as curriculum content analysis, material source analysis, aspects measure analysis, and creating the storyboard and first outline to the creative drawing process. the drawing process can be categorized as an essential process because, in this step, the application of all plans will realize in the form of sequential art and narrative text. secondly, the experts’ review toward science comic shows that all aspects consisted of the comic, which is of artwork, science content, language, and uniqueness aspect, mostly gain a good score, but it needs to be revised. the feedback that experts give also shows a positive value because most of them tried to give supportive feedback for science comic development. thirdly, the review from science teacher to all aspect measurement of science comic categorized as a good score. science teachers agree that this science comic could give good learning outcomes and the new experience to the students for learning science. science teachers mostly criticized the grammatical and content aspects of the comic. science teachers suggest to revise some grammatical structure on the comic text and reduce some science concepts that are not suitable for the 8th-grade student. forth, most of the students agree that science comics could help them to comprehend the science content more quickly because they can enjoy the story and the knowledge with the addition of humour in the comic. this data can prove because there are more 50% of students get a higher score compared to the average score (75.9 scores) gained in the questionnaire session. therefore, it concluded that developing science comics, which consist of some humourous figures and explicit narrative content, could bring some interest to the students in learning science. the additional suggestion to improve the next research can make by focusing on what aspect that matters to the science comic. those aspects will be contained and be a unique character for the science comic itself. acknowledgment the authors acknowledge mrs. nuryani y. rustaman and mr. ikmanda nugraha, who have given the best suggestion, support, and improving the quality of this research. references ahi, b. (2017). thinking about digestive system in early childhood: a comparative study about biological knowledge. cogent education, 4(1), 1278650. dalacosta, k., kamariotaki-paparrigopoulou, m., palyvos, j. a., & spyrellis, n. (2009). multimedia application with animated cartoons for teaching science in elementary education. computers & education, 52(4), 741-748. detjen, j. (1995). the media's role in science education. bioscience, 45, s58-s63. falk, j. h., storksdieck, m., & dierking, l. d. (2007). investigating public science interest and understanding: evidence for the importance of free-choice learning. public understanding of science, 16(4), 455-469. geerdts, m. s., van de walle, g. a., & lobue, v. (2015). daily animal exposure and children’s biological concepts. journal of experimental child psychology, 130, 132-146. gilbert, j. k., & lin, h. s. (2013). how might adults learn about new science and technology? the case of nanoscience and nanotechnology. international journal of science education, part b, 3(3), 267-292. hosler, j., & boomer, k. b. (2011). are comic books an effective way to engage nonmajors in learning and appreciating science?. cbe— life sciences education, 10(3), 309-317. istikomayanti, y., & mitasari, z. (2017). student’s misconception of digestive system materials in mts eight grade of malang city and the role of teacher’s pedagogic competency in mts. jurnal pendidikan biologi indonesia, 3(2), 103-113. lin, s. f., lin, h. s., lee, l., & yore, l. d. (2015). are science comics a good medium for science communication? the case for public journal of science learning article doi: 10.17509/jsl.v3i1.18120 18 j.sci.learn.2019.3(1).12-18 learning of nanotechnology. international journal of science education, part b, 5(3), 276-294. ozdemir, e. (2017). humour in elementary science: development and evaluation of comic strips about sound. international electronic journal of elementary education, 9(4), 837-850. tatalovic, m. (2009). science comics as tools for science education and communication: a brief, exploratory study. journal of science communication, 8(4), a02. a © 2019 indonesian society for science educator 85 j.sci.learn.2019.2(3).85-91 received: 31 may 2019 revised: 4 july 2019 published: 22 july 2019 analyzing student’s problem solving abilities of direct current electricity in stem-based learning rany apriyani1*, taufik ramlan ramalis1, irma rahma suwarma1 11department of physics education, sekolah pascasarjana, universitas pendidikan indonesia, bandung, indonesia *corresponding author. ranyapriyani912@gmail.com abstract this research has been conducted to analyze students' problem-solving abilities in direct current electricity in stem-based learning. the implementation of stem in this research is to train the domain of scientific practices and engineering practices that are associated with model problems and project-based learning. the research method used was pre-experiment with the design of one group pretest-posttest. the subjects of the research consist of 27 students at the 10th grade of one of the vocational schools in kabupaten bandung barat. the instrument of problem-solving ability in this study are four structured description questions, each of which consists of 5 questions, indicators of problem-solving ability, namely visualize the problem, describe the problem in physics description, plan the solution, execute the plan, and check and evaluate. as a result of the research, it was found that there was an increase in students' problem-solving abilities with the application of the integration model problem and project-based learning in stem-based learning. keywords stem, problem based learning, project-based learning, problem solving ability 1. introduction in the life of the 21st century requires a variety of skills that must be mastered by someone in facing every life problem, so that education can prepare students to learn the various skills needed to live life in the 21st century (pacific policy research center, 2010). the attachment was issued by the minister of education and culture regulation number 21 of 2017 concerning the contents of primary and secondary education standards. the 21st century skill is used to meet the future needs and to meet indonesian gold generation in 2045. it has established competency standards for graduates based on 21st century competence consisting of four competencies. consequently, students must have communication, critical thinking, and problem-solving, collaboration, and creative and innovative. this ability must be owned by graduates in indonesia (kemendikbud, 2016). one of the 21st-century skills is problem-solving ability. problem-solving ability as a mental and intellectual process used by students to relate previous knowledge and problems they face, and also recall the experience of solving problems in the past so that they get a solution to the problem. problem-solving ability is one of the competencies that students must have. in another opinion, adolphus, alamina, & aderonmu (2013) problem solving is identifying the gap between the issues and solutions using information (knowledge) and reasoning. with the adequate problem-solving ability, it will facilitate students in facing work situations that are filled with various problems that must be solved by them (yulindar, 2018). problem-solving ability is needed by students to face global competition. thus, students will be ready to jump in and participate in the real word (patnani, 2015). therefore, various efforts need to be made to improve problemsolving ability in students. these efforts include improving students' skills related to solving their problems and improving the quality of teaching by improving teacher methods and characteristics. thus, it is expected that students will be better prepared to face some problems, especially if they have been directly involved in the community. this is because when solving problems, an individual not only needs to think, but they need to think critically to be able to see the issues and think creatively to be able to solve problems. the ability to solve problems, in essence, learning to think or learning to reason, namely thinking or reasoning, applies the knowledge that has been obtained previously to mailto:ranyapriyani912@gmail.com journal of science learning article doi: 10.17509/jsl.v2i3.17559 86 j.sci.learn.2019.2(3).85-91 address new problems that have never been encountered (heller & heller, 2010). whereas according to ahiakwo (in adolphus, alamina, & aderonmu, 2013) states that, problem-solving is identifying gaps between problems and solutions using information (knowledge) and reasoning. to solve the problem at hand, an individual will take steps related to the problem-solving process. steps or steps that must be passed by students in solving problems there are five stages, namely (1) visualize the problem, (2) describe the problem in physics description, (3) plan the solution, (4) execute the plan, and (5) check and evaluate by heller and heller (2010). based on the facts in the research that has been done before, it was found that students' problem-solving abilities were still low. as in the study conducted by yulindar (2018); sutiadi & nurwijayaningsih (2016); jua, sarwanto, & sukarmin (2018); and from the results of a preliminary study in one of the high schools in west bandung district, it was found that students' problem-solving abilities for each indicator were still low. from the research that has been done before, it is known that, students' physics problem-solving abilities are improved by using appropriate learning models and approaches such as problem based learning models as in the research conducted by sutiadi & nurwijayantidingsih (2016), wahyu, sahyar, & ginting (2017), sahyar, sani & malau (2017), sahyar & fitri (2017), and argau, haile, & ayale (2017), ferreira & trudel (2012); collaborative learning (adolphus, 2013); direct instruction models; inquiry-based on just in time teaching-learning training models (turnip, wahyuni, & tanjung 2016); engineering design-based modeling approach (li, huang, jiang, & chang, 2016); project-based learning (tamba, motlan, & turnip, 2017); real engagement in active problem solving (yulindar, 2018); and so forth. so it can be concluded that the problem-solving abilities of students in indonesia are deficient. in practicing problem-solving skills, the right learning process is needed where students are motivated to solve problems faced. the difficulties faced cannot be separated from the use of technology, so the use of technology in learning is essential, so stem-based knowledge is suitable learning in fostering community interest (yasin, prima, & sholihin, 2018; wandari, wijaya, & agustin, 2018). stem approach (science, technology, engineering, and mathematics) is learning by integrating science, technology, engineering, and mathematics. the application of stem in learning can encourage students to design, develop, and utilize technology, sharpen cognitive, manipulative, and affective, and apply knowledge (kapila & iskander, 2014). therefore, the application of stem is suitable for use in physics learning. stem-based learning can train students to apply their knowledge to create designs as a form of solving problems related to the environment by utilizing technology. in applying stem can be supported by various learning methods. integrative stem allows multiple learning methods to be used to support its application (wirkala & kuhn, 2011). there are three essential domains in stem, namely: (1) practice: scientific practice and engineering practice, (2) discipline core ideas, and (3) crosscutting concept (national research council, 2011). in this study, the researcher took one domain, namely, practice. for scientific pbl models are used while for engineering practices, pjbl is used. in the national research council explained that scientific practice is a domain that discusses student involvement in science practice. the experience will help students understand how scientific knowledge develops; such direct relationship gives them an appreciation for the various approaches used to investigate, model, and explain the world. moreover, engineering practice or engineering practices assume that student involvement in engineering practice will help students understand the work of engineers, as well as the relationship between engineering and science. participation in these practices also helps students form an understanding of the concepts and ideas of overlapping scientific and engineering disciplines; besides that, it makes students' knowledge more meaningful and puts it deeper into their worldview. the pbl model is used for scientific practice because this learning model is a model that presents a problem related to physics concepts to students and provides opportunities for students to solve problems by conducting investigations or experiments. pbl can provide opportunities for students to apply knowledge to issues/problems as a form of problem solving. indirectly, the use of pbl also encourages students to master the knowledge needed to solve these problems (permanasari, 2016). this knowledge can be in the form of information or data which is then used as material for consideration to choose the right way of solving the problem through logical, critical, and systematic thinking. meanwhile, pjbl that is applied to engineering practices is able to guide students to solve the problem given and emphasize more on the product produced (chanlin, 2008). the products produced can be ideas or ideas that can be seen. products produced from the use of pjbl in science learning can be a student's contribution to improving the quality of life. the pjbl model is a student-centered learning model to develop and apply concepts from projects produced by exploring and solving real-world problems independently. according to bybee (awad & barak, 2014) stem realizes the importance of science and mathematics, and places particular emphasis on technology and engineering as a field that influences our lives, and is very important for people who are interested in continuing to renew. integration of stem in learning models in this case pbl and pjbl can improve problem-solving ability and help build relationships in real life. therefore it is possible to use journal of science learning article doi: 10.17509/jsl.v2i3.17559 87 j.sci.learn.2019.2(3).85-91 the stem approach to further enhance students' problemsolving abilities. 2. method in this study used the pre-experimental method with one group pretest-posttest design. the dependent variable in this study is the ability to solve students' physical problems in direct current electricity (o). while the learning model used is the pbl and pjbl model that is integrated into the stem approach is an independent variable (x). therefore, the study subjects used only one group without a comparison group. the design of one group pretest-posttest can be seen in table 1. the population in this study were all students of class 10th grade in one of the vocational schools in kabupaten bandung barat, jawa barat who were enrolled in the even semester of the 2018/2019 academic year. the sampling technique of this study is by taking a class randomly (random class). this sampling technique is because it does not allow changing the formation of students in an existing class if randomly chosen individual samples. so that one class is taken to be used as the research subject group. the instrument for problem-solving abilities used is tests and non-tests. the problem-solving ability test is given in two trials. the pretest and posttest in the description of the problem questionnaire consist of 4 structured items. each item consists of five problemsolving indicators. it refers to the indicator of problem solving according to heller where there are five steps to solving the problem, namely (1) visualize the problem, (2) describe the issue in physics description, (3) plan the solution, (4) execute the plan, and (5) check and evaluate each item. the non-test instruments used are student worksheets, which refer to the heller problem-solving indicators which consist of three meetings, namely ohm's law, electrical circuit and kirchoff's law, and household electrical installations. the instrument test was carried out by three experts. after the data is collected, the problem-solving ability score is from the pretest, posttest, and worksheets by calculating the average n-gain. 3. result and discussion the integration of the problem-based learning (pbl) and project-based learning (pjbl) models with the stem approach in this study aims to analyze the problem-solving abilities of smk students. learning is carried out in five meetings with direct current electricity. the first and fifth meetings were used to do the test and posttest while the meeting for learning used three meetings with the first and second learning details to train the scientific practices using the pbl model with the stem approach and the third meeting to prepare engineering practices using the pjbl model with the stem approach. direct current electricity is taught divided into three submissions, namely ohm's law, electrical circuits, and electrical installations. data on students' problem-solving abilities obtained in this study were from the data from the pretest and posttest results as well as student answers to the worksheet. the results of the pretest and posttest were measured using a problem-solving ability test in the form of a description problem with a score range of 0 4. the distribution of students' problem-solving abilities can be shown by comparing the average scores of the pretest and posttest and the gain and n-gain scores of all students direct current electricity. the improvement of students' problemsolving abilities as a whole from the results of the pretest, posttest, and gain data is presented in figure 1. students' problem-solving abilities are abilities that students must possess along with the development of the 21st century (oecd, 2014). the problem-solving ability referred to in this research is the ability of students to use their knowledge based on their experience in electricity or dynamic electricity laws that they learn to solve various problems that are often encountered in everyday life, especially those related to equipment or technological work. figure 1 is the average value of students ' problemsolving abilities before learning, which is measured using the problem-solving ability test shows 17.55 from a maximum amount of 100. whereas, the average value of students' problem-solving abilities after learning shows 73.75 from the maximum value 100. it is increasing students' problem-solving ability based on average gain values of 56.2. the average gain is then normalized into ngain; the result is 0.68. the n-gain results were confirmed by the category from hake (1999), so the improvement of students' problem-solving abilities as an impact of the integration of pbl and pjbl models in stem-based tabel 1 one group pretest-posttest design pretest treatment posttest o x o figure 1 percentage diagram of the average problem-solving ability of students journal of science learning article doi: 10.17509/jsl.v2i3.17559 88 j.sci.learn.2019.2(3).85-91 learning was included in the medium category. as explained earlier, this assessment was carried out by integrating two models, namely the pbl and pjbl models as stages of scientific practices and engineering practices. the same was found by other studies that conducted research to analyze the problem-solving abilities of students using problem-based learning models such as in research by wahyu, sahyar, & ginting (2017); sahyar, sani, & malau (2017); sahyar & fitri (2017); ferreira & trudel (2012) which could improve problem-solving abilities. the results of the study were the improvement of students' problem solving and critical thinking skills. a similar study was conducted by tamba, motlan, and turnip (2017) which aimed to analyze the effect of a project-based learning model (pjbl) on creative thinking skills and problem-solving. this finding also supports berry's (2012) statement that stem integration in learning models can improve problem-solving ability and can help with relationships in real life. based on table 2 it is known that there are 15 students experiencing an increase in problem-solving ability in the medium category with a percentage of 55% and 12 students in the high grade with a percentage of 45%. none of the students experienced an increase in the low category. this shows that the integration of pbl and pjbl models in stem-based earning can improve students' problemsolving ability in direct current electricity. the improvement of problem-solving ability in learning on direct current electricity is the impact of the integration of pbl and pjbl models in stem-based learning that is built through five stages in pbl (equipped with eight steps of scientific practices) and six stages in pjbl (equipped with eight engineering practices). each stage in the pbl and pjbl models is done using experiential learning. learning through direct experience is a process to get meaningful learning from direct encounter by mughal & zafar (2011). direct experience, including active experiments, makes it possible to implement stem in dealing with real situations (gilmore, 2013). this can strengthen the results of the study that the integration of pbl and pjbl models with stem-based learning can improve problem-solving abilities. in this study, the problem-solving abilities used in this study consisted of the ability to focus problems, describe problems in physical descriptions, plan problem-solving solutions, use problem-solving solutions, and examine and evaluate problem-solving solutions (heller, ronald & scott, 1992). the percentage of the average score of the pretest and posttest in each indicator of the problemsolving abilities measured is shown in figure 2. figure 2 shows the percentage of the average score of the results of the pretest and posttest of students' problemsolving abilities in each indicator. the average rate of the pretest score on the indicator (a) visualize the problem of 35.69%, (b) describe the issue in physics description of 26.16%, (c) plan the solution of 14.81%, (d) execute the idea of 7, 87%, and (e) check and evaluate of 3.24%. while the posttest results on the indicators focusing the problem were 94.91%, described the problem as 87.73%, planned a solution of 80.09%, used a solution of 61.11%, and evaluated 51.85%. based on figure 2, it can be seen that when compared between the average percentage of pretest scores and posttest on all indicators, the problem-solving ability has increased. the improvement of students' problem-solving skills based on the calculation of n-gain for each indicator is shown in figure 3. based on figure 3, it can be seen that the highest ngain is in the indicator visualize the problem. it was namely 0.92 as the high category. the lowest is to describe the issue in physics description with the n-gain value of 0.83 in the high category. planning the solution with the n-gain value table 2 percentage of the number of students in each category of improvement in problem-solving ability category number of students percentage (%) low 0 0 medium 15 55,6 high 12 44,4 figure 2 percentage of the average pretest-posttest score of students' problem-solving abilities for each indicator figure 3 average n-gain score of students' problem solving abilities for each indicator journal of science learning article doi: 10.17509/jsl.v2i3.17559 89 j.sci.learn.2019.2(3).85-91 of 0.77 is in the high category. executing the plan with an n-gain value of 0.58 in the medium category, and the lowest increase in the indicator and check and evaluate with an n-gain value of 0.5 at the medium category. so it can be concluded that indicators with a high category are visualized the problem, describe the problem in physics description, plan the solution, execute the plan, and check and evaluate. the indicator checks and evaluate is the indicator with the lowest increase. this happened because at the first and second meeting, there were still many students who ignored the indicators check and assess. some students do not review whether the problem-solving solution is in accordance with the problem or not. in general, the ability to solve problems in dynamic electrical material on each indicator has increased as a result of the integration of pbl and pjbl models in stem-based learning. based on the description of the data described illustrates that the integration of pbl and pjbl models in stem-based learning can improve students' problemsolving abilities on each indicator. increasing the indicator of the problem-solving skill which is calculated using the highest n-gain is the indicator visualize the problem (0.92), while the lowest is the indicator check and evaluate (0.50) this is similar to the findings of the research conducted by yulindar (2018) and sutiadi & nurwijayaningsih (2016). improved problem-solving skills at pretest and posttest both overall average and each indicator of problem-solving is not directly increased but first students are trained in solving problems in learning with the integration of pbl and pjbl models in stem-based learning. the value of problem-solving ability of each meeting which is reviewed from the student's worksheet answers can be seen in figure 4. based on figure 4 can be seen for each student's problem-solving abilities at each meeting. the meeting in this study was divided into three meetings, namely the first meeting to discuss ohm's law, the second meeting discussed the electric circuit and kirchoff's 1 law, and the third meeting discussed household electrical installations. for meeting 1, the average score of students is 38.31%, meeting 2 is obtained by the average value of students is 53.20%, and meeting 3 is obtained the average value of students is 75.32%. with the value of n-gain for meeting 1 to meeting 2 is 0.32 which is in the medium category and an increase in problem-solving ability from meeting 2 to meeting 3 increases to 0.47 which is still in the medium category. this proves that from each meeting the students' problem-solving abilities are increased seen from the value of n-gain students. the percentage of problem-solving abilities at each meeting for each indicator of problem-solving ability which is reviewed from the student's worksheet answers can be seen in figure 5. from figure 5 it can be seen that the score for each indicator is getting to the high indicator level, the smaller the score the student receives. based on figure 5, it can be seen that when compared to the average percentage of the first, second, and third meeting scores on all indicators, the problem-solving ability has increased. for further information, we will look at the capacity of each student's problem-solving in each of the indicators. from the findings, it can be concluded that the ability to visualize the problem has the highest increase taken from the scores obtained by students on the pretest and posttest problem-solving abilities in each question in part a with indicators focusing the problem. this is assumed because this learning model trains students to focus on problems. students must first understand the problem before solving a problem that must be faced. the question that is to be solved in the learning process is based on issues that have been encountered by students in everyday life. the ability to focus problems is trained with the stem approach in the asking questions and defining problems stage and in the pbl and pjbl models respectively at the first stage/phase, namely student orientation to the problem and determining the necessary questions. at this figure 4 average percentage diagram of students' problemsolving abilities at each meeting figure 5 percentage of the average score for students' problemsolving abilities for each meeting for each indicator journal of science learning article doi: 10.17509/jsl.v2i3.17559 90 j.sci.learn.2019.2(3).85-91 stage, the teacher acts as a facilitator who guides students to visualize the problem based on issues/problems that have been known or experienced by students. issues related to the topic regarding energy cycles that occur in the world, especially electricity. data on the implementation of teacher and student activities at this stage are in the overall category of activities carried out or are in an excellent grade (100%), meaning that both teachers and students do this stage very well. unlike indicators visualize the problem that has the highest increase, indicators check and evaluate the lowest growth, which is equal to 0.50 but still in the medium category. indicators check and assess at this meeting were trained with the stem approach at the engaging in argument from evidence and obtaining, evaluating, and communicating information stages in the pbl model and obtaining, evaluating, and communicating information on successive pjbl models at the last stage/phase, namely analyzing and evaluating problem solving process (pbl) and experience evaluation (pjbl). at this stage, students are trained to test the solutions they have made. the teacher guides students to evaluate problem-solving solutions that students make. then students evaluate whether the solutions they have made are by the problems they face or not. but this was done in one of the groups who made the presentation. this can be one of the causes of low ability to evaluate problems. from the data on the implementation of teacher activities the overall category of events was carried out or was in an excellent category (100%) but global student activities only 94% were in an outstanding category but not all were implemented, but in categories meant that both the teacher and students did the stages this very well. other factors that might cause a low average increase in evaluating data-based solutions are the form of a problemsolving ability test instrument in the form of a structured description. the disadvantage of structured description questions is that students must be able to answer the initial questions in order to work on the next question (prima & kaniawati, 2011). the ability to check and evaluate in test instruments is measured on the last item, after other abilities. therefore, the results obtained depend on the ability of students to work on the questions in the previous items. this is what causes the indicator to evaluate the solution is the indicator that has the lowest average increase among other problem-solving ability indicators. 4. conclusion problem-solving ability with the integration of problem and project-based learning models in stem-based learning as a whole have improved both overall in the value of the pretest, posttest, and worksheets as well as the indicator of problem-solving. if seen from each indicator, it is found that the highest increase is found in the indicator visualize the problem decreases according to the heller indicator stage where at the check and evaluate stage it has the lowest increase compared to other indicators and if it is seen from the tendency of the scoring results, the score decreases sequentially from visualizing the problem to check and evaluate. this is because the problem-solving ability is a series of mutually sustainable processes, where when you want to reach a good final stage you must go through the initial stages well. acknowledgment the authors acknowledge ms. noviana putri for helping in the course of the research and discussion about the learning. references adolphus, t., alamina, j., & aderonmu, t. (2013). the effects of collaborative learning on problem solving abilities among senior secondary school physics students in simple harmonic motion. journal of education dan practice, 5(25). 95-11. argau, a. s., haile, b. b., ayale, b. t., & kuma, s. g. (2017). the effect of problem based learning (pbl) instruction on students’ motivation and problem solving skills of physics. eurasia journal of mathematics science and technology education, 13(3). 857-871. arifuddin, m., mastuang, & mahardika, a. i. (2017). improving problem solving skill in physics through argumentation strategy in direct instruction model. international journal of sciences: basic and applied research (ijsbar), 35(3), 348-353. awad, n., & barak, m. (2014). sound, waves and communication: students’ achievements and motivation in learning a stemoriented program. creative education, 5, 1959-1968. berry, m., chalmers, c., & chandra, v. (2012). stem futures and practice, can we teach stem in a more meaningful and integrated way? 2nd international stem in education conference. brisben australia: queensland university of technology. chanlin, l.-j. (2008). technology integration applied to project-based learning in science. innovations in education and teaching international, 45(1), 55-65. ferreira, m.m. & trudel, a.r. (2012). the impact of problem based learning on student attitude toward science, problem solving skill and sense of community in the classroom. journal of classroom interaction, 47(1). 23-30. gilmore, m.w. (2013). improvement of stem educatiom: experiential learning is the key. modern chemistry and applications, 1(3), 1-3. hake, r. r. (1999). analyzing change/gain scores. unpublished.[online] url: http://www. physics. indiana. edu/~ sdi/analyzingchange-gain. pdf. heller, k., & heller, p. (2010). cooperative problem solving in physics a user’s manual. unpublished.[online] url: http://www. aapt. org/conferences/newfaculty/upload/coop-problem-solving-guide. pdf. jua, s. k. (2018). the profile of students’ problem-solving skill in physics across interest program in the secondary school. journal of physics: conference series, 1022(1), 012027. kapila, v. & iskander, m. (2014). lessons learned from conducting a k12 project to revitalize achievement by using instrumentation in science education. journal of stem education, 15(1), 46-51. kemendikbud. (2016). minister of education and culture regulation number 21 of 2017 concerning the contents of primary and secondary education standards. jakarta: kementrian pendidikan dan kebudayaan. li, y., huang, z., jiang, m., & chang, t. w. (2016). the effect on pupils' science performance and problem-solving ability through lego: an engineering design-based modeling approach. journal of educational technology & society, 19(3). mughal, f., & zafar, a. (2011). experiential learning from a constructivist perspective: reconceptualizing the kolbian cycle. international journal of learning and development, 1(2), 27-37. journal of science learning article doi: 10.17509/jsl.v2i3.17559 91 j.sci.learn.2019.2(3).85-91 national research council. (2011). a framework for k-12 science education: practices, crosscutting concepts, and core ideas. washington dc: the national academies press. organisation for economic co-operation and development (oecd). (2014). pisa 2012 result: creative problem solving: students skills in tackling real life problems (volume v). pisa: oecd publishing. patnani, m. (2015). upaya meningkatkan kemampuan problem solving pada mahasiswa. journal psikogenesis, 1(2), 130-142. prima, e. c., & kaniawati, i. (2011). penerapan model pembelajaran problem based learning dengan pendekatan inkuiri untuk meningkatkan keterampilan proses sains dan penguasaan konsep elastisitas pada siswa sma. jurnal pengajaran mipa, 16(1), 179-184. pacific policy research center. (2010). 21st century skills for students and teachers. honolulu: kamehameha schools, research & evaluation division. permanasari, a. (2016). stem education: inovasi dalam pembelajaran sains. prosiding seminar nasional pendidikan sains, 3, 23-34. sahyar, & fitri, r. y. (2017). the effect of problem-based learning model (pbl) and adversity quotient (aq) on problem-solving ability. american journal of educational research, 5(2). 179-183. sani, r. a., & malau, t. (2017). the effect of problem based learning (pbl) model and self-regulated learning (srl) toward physics problem solving ability (psa) of students at senior high school. american journal of educational research, 5(3), 279-283. sutiadi, a., & nurwijayaningsih, h. (2016). konstruksi dan profil problem solving skill siswa smp dalam materi pesawat sederhana. jurnal penelitian & pengembangan pendidikan fisika, 2(1). 37-42. tamba, p., motlan, & turnip, b. m. (2017). the effect of project based learning model for students’ creative thinking skills and problem solving. iosr journal of research & method in education, 7(5). 67-70. turnip, b., wahyuni, i., & tanjung, y. i. (2016). the effect of inquiry training learning model basedon just in time teaching for problem solving skill. journal of education and practice, 7(15). 177181. wahyu, e. s., sahyar., & ginting, e. m. (2017). the effect of problem based learning (pbl) model toward student's critical thinking and problem solving ability in senior high school. american journal of educational research, 5(6). 633-638. wandari, g. a., wijaya, a. f. c., & agustin, r. r. (2018). the effect of steam-based learning on students’ concept mastery and creativity in learning light and optics. journal of science learning, 2(1), 26-32. wirkala, c., & kuhn, d. (2011). problem-based learning in k-12 education: is it effective and how does it achieve its effects?. american educational research journal, 48(5), 1157–1186. yasin, a. i., prima, e. c., & sholihin, h. (2018). learning electricity using arduino-android based game to improve stem literacy. journal of science learning, 1(3), 77-94. yulindar, a. (2018). enhancement of problem solving ability of high school students through learning with real engagement in active problem solving (reaps) model on the concept of heat transfer. journal of physics: conference series, 1013(1), 012052. a © 2020 indonesian society for science educator 117 j.sci.learn.2020.3(2).117-123 received: 12 december 2019 revised: 5 march 2020 published: 19 march 2020 the relation between gender, reasons to participate in stem-related subjects, programs and the university supports on first-year university student’s satisfaction: a structural equation model ikmanda nugraha1*, tatang suranto2, asep kadarohman3, ari widodo4, i gusti darmawan5 1department of science education, universitas pendidikan indonesia, bandung, indonesia 2elementary school teacher study program, universitas pendidikan indonesia, serang, indonesia 3department of chemistry education, universitas pendidikan indonesia, bandung, indonesia 4department of biology education, universitas pendidikan indonesia, bandung, indonesia 5school of education, the university of adelaide, adelaide, australia *corresponding author. ikmandanugraha@upi.edu abstract first-year student's satisfaction has considered an important factor in the quality education offered by the university program. the objective of this study is to investigate the relationship among gender, a reason to participate in stem -related subjects, program, and the university support on first-year student's satisfaction of stem learning. the methods used in this study surveyed student's satisfaction using adapted interest and recruitment in the science questionnaire (iris q) instrument. the questionnaire comprised question items covering school science experiences, sources of inspiration for the choice of education, expectations for future job, first-year experiences as a stem student, and attitudes to gender equity in stem. a total of 448 students, first-year students from stem-related programs, have participated voluntarily in this study. the structural equating model assisted by computer program ibm spss amos 20 was employed to analyze the hypothesized model. the results from the model showed that reason and university support have a positive direct effect on first-year students' satisfaction with stem learning. from this study, it is suggested for the university to improve first-year student's satisfaction by helping stem students to develop appropriate expectations of the program, facilitating teaching quality to meet stem students' learning, and assisting students in developing positive attitudes toward their future carriers. keywords structural equation model, stem learning, first-year university students, satisfaction student’s satisfaction. 1. introduction in higher education, the issues of student satisfaction have been a great concern among researchers, policymakers, and practitioners. the point of student's satisfaction has become the major concern of many research, and the majority of the studies agreed that student's satisfaction has a strong correlation with university supports (e.g., crisp et al., 2009; duong, 2015; green, hood, & neumann, 2015; gruber, fuß, voss, & gläser‐zikuda, 2010; hakim, 2014; josephat, ismail, & martin, 2014; li & carroll, 2017; lo, 2010; mehdipour & zerehkafi, 2013; möller, 2006; naaj, nachouki, & ankit, 2012). most of the finding discuss that student's satisfaction can be used as the indicators to identify the areas where the university are performing well. moreover, it can use for performing strategies to retain and fit the students, providing insights or reputation of the university from people, and providing feedback for future program planning (billups, 2008). the first-year student becomes the concern because, this year, the student will decide whether they will stay or leave in stem-related subjects. in the middle of various issue related to the reducing interest of students to participate in science, technology, engineering and mathematics (stem) related subjects, the issue of lack of teaching quality come to the surface as one of the determinants of student's satisfaction. in the united kingdom, students from stem programs satisfied with funding and supporting facilities, but they were unsatisfied with the stem teaching. in contrast, students from nonstem programs experienced different directions (pawson, mailto:ikmandanugraha@upi.edu journal of science learning article doi: 10.17509/jsl.v3i2.21593 118 j.sci.learn.2020.3(2).117-123 2012). because teaching practices are context-specific (shulman, 1987), it is evident that certain teaching practices are common in a university stem teaching but different from another social discipline. moreover, students in a university stem classroom probably have different learning needs compared to students in different subject areas, such as laboratory activities or site visits (chang & park, 2014). this finding has encouraged stem teaching to satisfy the students expectation because teaching quality and expertise showed the most reliable relationship with student's satisfaction and their learning outcomes (green, hood, & neumann, 2015; hakim, 2014; lo, 2010; suarman, aziz, & yasin, 2013; burgess, senior, moores, 2018; khalil-ur-rehman, farooq, & younas, 2018; son, ha, thi, & khuyen, 2018). in terms of lecturer-student interaction, the study from a university in australia found that students expect the university to provide learning resources that can be accessed by the students anytime and anywhere. lectures need to engage students in interactive activities, student works must be read and give feedback, and the assessment marked and returned to them (crisp et al., 2009). on the other hand, the university's core service, i.e., the lecture, including the attainment of knowledge, class notes and materials, and classroom delivery, are more important than the physical aspects of the university services (douglas, douglas, & barnes, 2006). another study showed different findings such as (1) students' satisfaction with their university is based on a relatively stable person-environment relationship (gruber, fuß, voss, & gläser‐zikuda, 2010), (2) college curriculum needs to meet the student's expectation (zhou, 2016), (3) physical facilities and staff responsiveness also play an important role for student's satisfaction (ahmed & masud, 2014), (4) gender play different perspective, female students put higher expectation and demanding on the importance of university services than male students (grebennikov & skaines, 2009), (5) gender and the program do not have a significant influence on student's satisfaction (elhadary, 2016a), (6) a sense of belonging, and citizenship knowledge and skills were the best determinants of the level of first-year student satisfaction (al-sheeb, b., hamouda, a. m., & abdella, g. m. (2018). to sum up, it should also be noted that student satisfaction results not only from quality in teaching and learning, but it also comes from total student's experiences in their study (wilkins, balakrishnan, & huisman, 2012). therefore, to study student satisfaction, many other factors must come into consideration, such as ethnographic information, motives, assessment and feedback, learning resources, facilities, social life, etc. based on the literature above, the author formulated the following hypothesized model (figure 1) that describes the determinants of student satisfaction. accordingly, the author hypothesizes that the following factors will have a positive direct effect on student satisfaction: 1) gender, 2) reason to participate in stem-related subjects, 3) programs, and 4) university support. all the factors are included in the hypothesized model because they have the possibility to become the determinants of student satisfaction. using structural equation model (sem) figure 1 hypothesized model of first year student satisfaction journal of science learning article doi: 10.17509/jsl.v3i2.21593 119 j.sci.learn.2020.3(2).117-123 analysis, this study intended to test the good-fit of the hypothesized model. in the hypothesized model, gender as an observed variable. the reason to participate in stem-related subjects as a latent variable was formed from school science experience, influential others, and daily inspiration. the factor has confirmed through confirmatory factor analysis (cfa) as a single factor with rmsea value (0.101). the program, as observed variable, consists of seven major programs. they are mathematics, chemistry, physics, biology, agriculture, health, and engineering. in the hypothesized model, mathematic did not appear because it treated like a dummy. therefore only six programs appear. university support as a latent variable constructed from 9 items from the iris q questionnaire. 2. method the sample (n=448) were first-year university students who registered in stem-related subjects in bandung, west java indonesia. the sample consists of 237 male students and 211 female students from seven major stem-related subjects (mathematics, chemistry, physics, biology, agriculture, health, and engineering). the sample is randomly selected by distributing the online questionnaire. the sample gathered in this study was the students who voluntarily filled the online questionnaire completely. under the permission of iris international, the iris q questionnaire was translated into bahasa indonesia by one translator. from bahasa indonesia, the survey was backtranslated into english by another translator to ensure consistency and minimize the loss of meaning. the final form of the questionnaire was in bahasa indonesia after all table 1 iris q questionnaire no components code how important was each of the following school experiences in choosing your course? school experiences 1 your interset in related subjects scexp1 2 your previous attainment in related subjects scexp2 3 experiment/lboratory work scexp3 4 field work/excursions scexp4 5 lessong showing the relevance of the subject to the society scexp5 6 lesson showing practical applications of the subjects scexp6 7 using mathematics in lessons scexp7 8 clear feedback on wheter you got the right answer scexp8 how important were the following persons in choosing your course? influential others/people 9 mother or step mother pp1 10 father or step father pp2 11 good teachers pp3 12 friends (including boyfriend/girlfriend) pp4 13 sibling or other relatives pp5 14 carriers advisor in school pp6 how important were each of the following in choosing your course? daily inspiration 15 popular science books and magazines dlyexp1 16 science fiction or fantasy book/films dlyexp2 17 computer games dlyexp3 18 museum/science center dlyexp4 19 popular science television program (discovery channel, national geographic, etc.) dlyexp5 20 film or drama on television (csi, numbers, etc.) dlyexp6 21 science, technology, or mathematics competitions dlyexp7 22 other kinds of outreach activities (e.g. science festivals, science summer schools) dlyexp8 to what extend do you agree with the following statements about university supports and your experiences as student so far? university support/ experiences as student 23 i enjoy the company of the other students on my course uni1 24 i feel i fit in socially uni2 25 i feel i can keep up with the pace of the teaching uni3 26 i get personal feedback from lectures and teachers when i need it uni4 27 i feel my teachers care about whether students learn or not uni5 28 the university/college offers good working conditions (equipment, library, common areas, cafes, technical support uni6 29 i can see the relevance of what i learn uni7 30 i feel that my course suits the kind of person that i am uni8 31 i have become more interested in the subject since i started uni9 journal of science learning article doi: 10.17509/jsl.v3i2.21593 120 j.sci.learn.2020.3(2).117-123 of the translators meet their agreement. the original iris q comprises 65 items, but only 31 elements administered for this study (table 1). questions on the iris q responded using a 5-point likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). the adapted iris q consists of three components, school science experience (8 items), influential others (6 items), and daily source of inspirations (8 items) and university support/experiences as students (9 items). the adapted iris q then uploaded as an online questionnaire and randomly distributed to first-year students in stemrelated subjects by using the student's association network. first, confirmatory factor analysis (cfa) was performed to identify underlying factors of the reason to participate in stem-related subjects. the result showed that the constructs of the instrument are consistent with a researcher's understanding of the nature of that construct (or factor). second, the structural equation model (sem) analysis performed using ibm spss amos 20 (arbuckle, 2011) software to assess the causal relationships of the factors. sem that seek the relationship between multiple variables as a member of statistical models lies in two multivariate techniques: factor analysis and multiple regression analysis (hair, black, babin, & anderson, 2014). sem has been widely to test the model many studies such as sociology, psychology, business and marketing, education, and many other related studies (e.g., ahmed & masud, 2014; archambault, 2008; sembiring, 2016; camgoz-akdag & zaim, 2012; dyana & adeline, 2014; elhadary, 2016a, 2016b; gruber, fuß, voss, & gläser‐ zikuda, 2010; ijaz, irfan, shahbaz, awan, & sabir, 2011; li & carroll, 2017). hair, black, babin, & anderson (2014) explained there are six steps to decide on the fit model. they are: (1) define individual constructs, (2) develop overall measurement model, (3) formulate the study to produce the empirical results, (4) asses the measurement model validity, (5) specify the model, and (6) assessing structural model validity. in this study, the process of analysis was using those steps. in this study, the primary purpose of sem is to simultaneously test the hypothesized model relationship among a set of latent constructs. in terms of good-fit indices, this study used the standard fit indices for an sem, which are comparative fit index (cfi), tucker-lewis index (tli, or non-normed fit index), and rmsea. the tli and cfi values should be greater than or equal to 0.80 and the rmsea; the value should be between 0.08 to 0.10 to get mediocre fit (hooper, coughlan, & mullen, 2008). it should also be considered that there should be justification for the group of fit indices provided, it should converge to a conclusion that the data fit the sample model and also it is very crucial to determine the overall fit from multiple indices and not just a single fit index (schreiber, 2008). furthermore, the model chi-square value and degrees of freedom should also be reported because some fit indices are based on those values but are not typically used to justify the fit of the data to the model because of the chi-square value is sensitive to sample size (schreiber, 2008). at last, to be considered is because of the volume of fit indices, it becomes a temptation to choose the best-fit indices that indicate the best fit. it is should also be avoided at all costs removing important just because only based on the cut-off of fit indices. as a way to improve model fit, it is good practice to check the fit of each construct and its items individually to specify whether there are any poor items (hooper, coughlan, & mullen, 2008). at last, sem is an a priori method; therefore, the removal or addition of a relationship during model modification must consider theoretical, not just statistical sense. because modifying using statistical results may lead to producing a nonrepresentative model from the actual data (schreiber, 2008) 3. result and discussion the hypothesized model was tested by following the procedure to test the good-fit of the model (hair, black, babin, & anderson, 2014). there were seven tested models to find the good-fit of the model (table 2). the hypothesized model has not fit the actual data with tli value (0.499), cfi value (0.610), and rmsea value (0.136). then, to improve the good-fit of the model, the insignificant relationships indicated by p-value higher than 0.05 were excluded in the hypothesized model to form tested model 1. in this model (gender to satisfaction=0.952), (dummy2 to university support=0.717), (gender to university support=0.999), (dummy7 to reason=0.837), (gender to reason=0.777), and (dummy5 to reason=0.806). after modification, tested model 1 showed better fit to the actual data indicated by table 2 good-fit index for the tested models (n=448) no model cmin df cmin/df nfi rfi tli cfi rmsea 1 hypothesized model 1380.115 148 9.325 0.588 0.471 0.499 0.610 0.136 2 tested model 1 1380.291 154 8.963 0.588 0.492 0.521 0.612 0.133 3 tested model 2 1383.545 161 8.593 0.587 0.513 0.543 0.613 0.130 4 tested model 3 1038.124 147 7.062 0.653 0.597 0.633 0.684 0.116 5 tested model 4 1045.640 151 6.925 0.651 0.605 0.641 0.683 0.115 6 tested model 5 815.730 150 5.438 0.728 0.690 0.731 0.764 0.100 7 tested model 6 294.278 63 4.671 0.876 0.847 0.875 0.899 0.091 journal of science learning article doi: 10.17509/jsl.v3i2.21593 121 j.sci.learn.2020.3(2).117-123 tli value (0.521), cfi value (0.612), and rmsea value (0.133) but still has not met the required fit indices. tested model 1 then further analyzed to identify the insignificant relationship. to improve the model fit, insignificant in this model were excluded to form tested model 2 (dummy2 to reason=0.300), (dummy7 to university support=0.498), (dummy3 to satisfaction=0.699), (dummy2 to satisfaction=0.486), (dummy5 to satisfaction=0.641), (dummy6 to satisfaction=0.621) and (dummy7 to satisfaction=0.806). after modification, tested model 2 showed better fit to the actual data indicated by tli value (0.543), cfi value (0.613), and rmsea value (0.130) but still has not met the required fit indices. tested model 2 further modified to improve the model fit by exclude the insignificant relationship to form tested model 3, they were (dummy4 to satisfaction=0.177), (reason to satisfaction=0.115), (dummy6 to university support=0.153), (gender to dummy2=0.108), (dummy2 to reason=0.276), and (dummy6 to university support=0.153). after modification, tested model 3 showed better fit to the actual data indicated by tli value (0.633), cfi value (0.684), and rmsea value (0.116) but still has not met the required fit indices. tested model 3 further modified to improve the model fit by excluding the insignificant relationship to form tested model 4, they were (dummy4 to satisfaction=0.177), (reason to satisfaction=0.115), (dummy6 to reason=0.193), (dummy3 to university support=0.150) and (dummy4 to university support=0.144). after modification, tested model 4 showed better fit to the actual data indicated by tli value (0.641), cfi value (0.683), and rmsea value (0.115) but still has not met the required fit indices. they tested model 4 further modified to improve the model fit by excluding the insignificant relationship (dummy3 to university support=0.352) to form tested model 5. after modification, tested model 5 showed better fit to the actual data indicated by tli value (0.731), cfi value (0.764), and rmsea value (0.100) but still has not met the required fit indices. tested model 5 further modified to improve the model fit by excluding the insignificant relationship, but in this model, the relationships were all significant. to improve table 3 factor loading of factor in the good-fit models factor loading estimate unispprt <--reason .556 scexp <--reason .679 people <--reason .554 dailiy <--reason .617 uni1 <--unispprt .543 uni2 <--unispprt .572 uni3 <--unispprt .728 uni4 <--unispprt .587 uni5 <--unispprt .530 uni6 <--unispprt .469 uni7 <--unispprt .677 uni8 <--unispprt .832 uni9 <--unispprt .787 fryrstf <--unispprt .615 figure 2 final model of first year student satisfaction (standardized) journal of science learning article doi: 10.17509/jsl.v3i2.21593 122 j.sci.learn.2020.3(2).117-123 the model by considering closely related errors, the author correlated error4 and error5 because of both of the highly correlated. the author also excluded gender and program because they were not significantly having a direct effect on student satisfaction confirmed by an independent sample t-test and one-way anova. an independent sample t-test showed that there were no significant differences of satisfaction between male (m=10.01; sd=1.90) and female (m=9.99; sd= 1.82; t(446)=1.62, p=0.90, two-tailed). while program also showed that there was no significant difference at the p <.05 in the groups: f(6,441)= 0.74, p= 0.62. mathematics (m=10.03; sd=1.56), chemistry (m=10.40; sd=1.98), physics (m=9.85; sd=1.50), biology (m=10.50; sd=1.89), agriculture (m=9.86; sd=2.05), health (m=9.91; sd=1.92) and engineering (m=10.00; sd=1.86). after modification, tested model 6 showed better fit to the actual data indicated by tli value (0.875), cfi value (0.899), and rmsea value (0.091) and also met the requirement for model fit to the actual data (figure 2). all of the factor loadings in the tested model 6 met the minimum requirement (0.32), as can be seen in table 3. (hair, black, babin, & anderson, 2014). that signified that construct validity is adequate. the model also showed that reason, and university support indirectly explained 38% of the total variances of student's satisfaction and reason explained 31% of total variations of university supports. in this case, the model can consider contributing to the planning of the university program. this study explored the relationship gender, reasons to participate in stem-related subjects, programs, and the university supports on first-year university student's satisfaction. the sem results showed that students' satisfaction with stem learning is highly affected by university supports (experiences as a student) which is consistent with previous studies that university supports highly correlated with student's overall satisfaction (crisp et al., 2009; green, hood, & neumann, 2015; gruber, fuß, voss, & gläser‐zikuda, 2010; hakim, 2014; lo, 2010; suarman, aziz, & yasin, 2013; wilkins, balakrishnan, & huisman, 2012). this result confirms and strengthens the fact that student satisfaction can be used as indicators to identify the areas where the university is performing well. moreover, it can also apply for performing strategies to retain and fit the students, providing insights or reputation of the university from people, and providing feedback for future program planning (billups, 2008). gender and program are excluded in the model because both showed no difference in terms of satisfaction confirmed by independent sample t-test and one-way anova, so it is unnecessary to control the effect from the model. this finding also consistent with the previous study that gender and program do not have a significant influence on student's satisfaction (elhadary, 2016a). female students put higher expectations and demands on the importance of university services than male students (grebennikov & skaines, 2009). in the model, uni1 and uni2 are highly correlated; this is because after checking the items, the statements from both items are asking quite a similar question. uni1 (i enjoy the company of the other students on my course) and un2 (i feel i fit in socially). both of them asking the social life and interaction of the student. the model also showed that reason and university support indirectly explained 38% of the total variances of student's satisfaction and reason itself explained 31% of total variations of university supports. in this case, the higher the reason, the more the students need university support, and the more the university supports the students, the more they feel satisfied. this finding is in line with the previous research that highlight the more the university helps the students the more they feel satisfied (al-sheeb, hamouda, & abdella, 2018; son, ha, thi, & khuyen, 2018; burgess, senior, moores, 2018; khalil-ur-rehman, farooq, & younas, 2018: tandilashvili, 2019). knowing this finding, the university needs to manage the students as customers by providing the supports which able to prepare the students for successful carrier and worthy employment after graduation. seem like the business industry, which depends on customer satisfaction. when the students satisfied with the university, indirectly, they will share and encourage people to choose the programs that make them satisfied. if the university is able to achieve it, it is also indicating that the university is performing well to provide education that fit the students and the society in general. 4. conclusion this study is an exploratory way using sem that was conducted based on data obtained from students in stemrelated subjects in west java province. as a result, it should be considered to confirm if the final model in this study can be generalized to other university students in general. for the next future studies, it is essential to verify the existence of the model constructs and the structural relationships between constructs using the data obtained from other university students to generate a better generalization. acknowledgment the author would like to thanks ellen k. henriksen for permission to adapt the iris q instrument and che yee lye and aai awardees 2017 for their helpful lessons, advice, and supports during the data analysis. references ahmed, s., & mehedi masud, m. (2014). measure service quality of a higher educational institute towards student satisfaction. american journal of educational research, 2(7), 447–455. https://doi.org/10.12691/education-2-7-3 al-sheeb, b., hamouda, a. m., & abdella, g. m. (2018). investigating determinants of student satisfaction in the first year of college https://doi.org/10.12691/education-2-7-3 journal of science learning article doi: 10.17509/jsl.v3i2.21593 123 j.sci.learn.2020.3(2).117-123 in a public university in the state of qatar. education research international, 2018(1-14). https://doi.org/10.1155/2018/7194106 arbuckle, j. l. (2011). ibm spss amos. chicago: spss. archambault, l. z. (2008). measuring service performance, student satisfaction and its impact on student retention in private, postsecondary institutions. edu-com international conference. sustainability in higher education: directions for change., (november), 19–21. retrieved from http://ro.ecu.edu.au/ceducom/2 billups, f. d. (2008). measuring college student satisfaction: a multi year study of the factors leading to persistence measuring college student satisfaction: a multi-year study of the factors leading to persistence. nera conference proceedings, paper 14. retrieved from http://digitalcommons.uconn.edu/nera_2008%5cnhttp://digital commons.uconn.edu/nera_2008/14 burgess, a., senior, c., & moores, e. (2018). a 10-year case study on the changing determinants of university student satisfaction in the uk. plos one, 13(2), 1–15. https://doi.org/10.1371/journal.pone.0192976 camgoz-akdag, h., & zaim, s. (2012). education : a comparative structural equation modeling study. procedia social and behavioral sciences, 47, 874–880. https://doi.org/10.1016/j.sbspro.2012.06.750 chang, y., & park, s. w. (2014). exploring students ’ perspectives of college stem : an analysis of course rating websites. international journal of teaching and learning in higher education, 26(1), 90–101. retrieved from http://www.eric.ed.gov.proxy.library.msstate.edu/contentdelivery /servlet/ericservlet?accno=ej1043017 crisp, g., palmer, e., turnbull, d., nettelbeck, t., ward, l., lecouteur, a., … schneider, l. (2009). first year student expectations: results from a university-wide student survey. journal of university teaching and learning practice, 6(1), 13–26. retrieved from http://ro.uow.edu.au/jutlp/vol6/iss1/3 douglas, j., douglas, a., & barnes, b. (2006). measuring student satisfaction at a uk university. quality assurance in education, 14(3), 251–167. duong, m.-q. (2015). the factors influencing student satisfaction in vietnamese higher education. international research in education, 4(1), 27. https://doi.org/10.5296/ire.v4i1.8191 dyana, m. l., & adeline, y. l. (2014). student satisfaction with the service quality of cafeteria: a structural approach, 4(1), 105–111. elhadary, o. (2016a). gender and other determinants of undergraduate student satisfaction in stem, 15(3), 256–264. elhadary, o. (2016b). student satisfaction in stem : an exploratory study, 4(2), 195–199. https://doi.org/10.12691/education-4-2-7 grebennikov, l., & skaines, i. (2009). gender and higher education experience: a case study. higher education research & development, 28(1), 71–84. https://doi.org/10.1080/07294360802444370 green, h. j., hood, m., & neumann, d. l. (2015). predictors of student satisfaction with university psychology courses: a review. psychology learning and teaching, 14(2), 131–146. https://doi.org/10.1177/1475725715590959 gruber, t., fuß, s., voss, r., & gläser‐zikuda, m. (2010). examining student satisfaction with higher education services. international journal of public sector management, 23(2), 105–123. https://doi.org/10.1108/09513551011022474 hair, j. f., black, w. c., babin, b. j., & anderson, r. e. (2014). multivariate data analysis (7th ed.). harlow: pearson education limited. hakim, a. (2014). nursing students’ satisfaction about their field of study. journal of advances in medical education & professionalism, 2(2), 82–7. retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=4235 554&tool=pmcentrez&rendertype=abstract hooper, d., coughlan, j., & mullen, m. (2008). structural equation modelling : guidelines for determining model fit structural equation modelling : guidelines for determining model fit. electronic journal of business research methods, 6(1), 53–60. https://doi.org/10.1037/1082-989x.12.1.58 ijaz, a., irfan, s. m., shahbaz, s., awan, m., & sabir, m. (2011). an empirical model of student satisfaction : case of pakistani public sector business schools, vii(ii), 91–116. josephat, p., ismail, a., & martin, p. (2014). undergraduate students ’ satisfaction survey at the university of dodoma, 3(2), 45–53. khalil-ur-rehman, f., farooq, m., & younas, w. (2018). investigating the factors impacting the student satisfaction with the universities in the era of artificial intelligence: a comparative study of malaysia and pakistan. the journal of social sciences research, 2(3), 5–9. https://doi.org/10.31580/apss.v2i3.221 li, i. w., & carroll, d. r. (2017). factors influencing university student satisfaction, dropout and academic performance, 56. lo, c. (2010). how student satisfaction factors affect perceived learning. journal of the scholarship of teaching and learning, 10(1), 47– 54. retrieved from http://www.eric.ed.gov/ericwebportal/recorddetail?accno=e j882125 mehdipour, y., & zerehkafi, h. (2013). student satisfaction at osmania university. international journal of advancements in research & technology, 2(6), 233–240. möller, o. (2006). student satisfaction survey: the utrecht university approach. tertiary education and management, 12(4), 323–328. https://doi.org/10.1007/s11233-006-9009-0 naaj, m. a., nachouki, m., & ankit, a. (2012). evaluating student satisfaction with blended learning in a gender-segregated environment. journal of information technology education: research, 11, 185–200. retrieved from http://www.jite.org/documents/vol11/jitev11p185200abounaaj0979.pdf pawson, c. (2012). hea stem conference stem vs . non-stem programmes. psychology teaching review, 18(2), 16–21. schreiber, j. b. (2008). core reporting practices in structural equation modeling. research in social and administrative pharmacy, 4(2), 83–97. https://doi.org/10.1016/j.sapharm.2007.04.003 sembiring, m. g. (2016). student satisfaction and persistence : imperative features for retention in open and distance learning. asian association of open universities journal, 10(1), 1–11. shulman, l. s. (1987). knowledge and teaching: foundations of the new reform. harvard educational review, 57(1), 1–22. suarman, aziz, z., & yasin, r. m. (2013). the quality of teaching and learning towards the satisfaction among the university students. asian social science, 9(12 spl issue), 252–260. https://doi.org/10.5539/ass.v9n12p252 son, h. t., ha, n. t., thi, p., & khuyen, m. (2018). measuring students’ satisfaction with higher education service-an experimental study at thainguyen university. www.ijbmm.com international journal of business marketing and management, 3(4), 2456–4559. retrieved from www.ijbmm.com tandilashvili, n. (2019). education. the case of a georgian state university. in rais conference proceedings (pp. 39–54). https://doi.org/10.5281/zenodo.3267486 wilkins, s., balakrishnan, m., & huisman, j. (2012). student satisfaction and student perceptions of quality at international branch campuses in the united arab emirates. journal of higher education policy and management, 34(5), 543–556. https://doi.org/10.1080/1360080x.2012.716003 zhou, h. (2016). empirical study on university curriculum satisfaction of university graduates. open journal of social sciences, 4(january), 132–137. https://doi.org/10.1155/2018/7194106 http://digitalcommons.uconn.edu/nera_2008%5cnhttp:/digitalcommons.uconn.edu/nera_2008/14 http://digitalcommons.uconn.edu/nera_2008%5cnhttp:/digitalcommons.uconn.edu/nera_2008/14 https://doi.org/10.5539/ass.v9n12p252 http://www.ijbmm.com/ a © 2022 indonesian society for science educator 242 j.sci.learn.2022.5(2).242-249 received: 8 december 2021 revised: 16 march 2022 published: 27 july 2022 animated concept cartoons as a starter for cognitive conflict in online science learning: a case of circular motion ertugrul ozdemir1* 1department of basic education, faculty of education, artvin coruh universtity, artvin, türkiye *corresponding author: eozdemir@artvin.edu.tr abstract before taking formal science education, learners usually construct preconceptions based on their daily life experiences, many of which are scientifically unacceptable misconceptions. in formal science learning, new concepts often contradict these misconceptions. to correct a misconception, it is first needed to create dissatisfaction about it by initiating cognitive conflict. this study aims to create an animated concept cartoon about circular motion and evaluate its usability to start cognitive conflict about centrifugal force misconception in online learning. this is a one-group quasi-experimental study whose participants are 110 undergraduate students with non-science backgrounds. an animated concept cartoon about circular motion and an online data collection instrument were developed and implemented for the participants. results showed that half of the participants might be considered to be in a status of cognitive conflict after reading the animated concept cartoon. in addition, participants' ove rall satisfaction with the animated concept cartoon was found to be relatively high. participants' satisfaction with the animated concept cartoon was equal for students with and without cognitive conflict. in the context of these findings, the animated concept cartoon may be considered an enjoyable and useful learning activity to initiate cognitive conflict in online science learning. keywords animated concept cartoon, cognitive conflict, circular motion, online learning 1. introduction concerning cognitivist learning theory, learning a new concept, contradictory to learners' existing conceptual schema, needs a conceptual change in its structure (barrouillet, 2015). this learning type, called accommodation, starts with dissatisfaction with existing concepts (posner, strike, hewson, & gertzog, 1982). therefore, an instructional process aiming at accommodating a new concept should be designed to make learners dissatisfied with existing concepts to increase their readiness for learning new concepts. in science learning, new concepts often contradict the old ones because learners usually construct a unique schema of science concepts based on their personal daily life experiences before taking formal science education (pine, messer, & st. john, 2001). some of these introductory concepts involved in learners' existing schema are regarded as misconceptions. misconceptions are false or partially false concepts about nature that are highly resistant to change (clement, 1982). all the false concepts about nature declared by an individual are not regarded as misconceptions. individuals generally have their explanations about the reasons for the false concepts (franklin, 1992). besides, because of their robust nature, the false concepts, which the individual is sure about, may be regarded as misconceptions (kaltakci gurel, eryilmaz, & mcdermott, 2015). as mentioned above, correcting a misconception is first needed to create dissatisfaction about it. therefore, to take learners to the phase of dissatisfaction with a misconception, using a starter for cognitive conflict is suggested at the beginning of the instruction (kang, scharmann, & noh, 2004). in literature, concept cartoons are regarded as effective starters for cognitive conflict because they present correct concepts and misconceptions on the same page, all of which sound correct but contradict each other (keogh, naylor, & wilson, 1998). concept cartoons, which are one-frame illustrations demonstrating conflicting ideas about a daily life situation, have been used as classroom activities in science courses for several purposes (naylor & keogh, 2013). in literature, concept cartoons were mostly used to diagnose misconceptions (ekici, ekici, & aydin, 2007; kandil ingec, 2008; chin & teou, 2009; kusumaningrum & indriyanti, mailto:eozdemir@artvin.edu.tr journal of science learning article doi: 10.17509/jsl.v5i2.41191 243 j.sci.learn.2022.5(2).242-249 2018; atasoy, 2020; serttas & turkoglu, 2020) and to correct them (stephenson & warwick, 2002; ekici et al., 2007; taslidere, 2014; kumi-manu, 2021). as described above, concept cartoons are used to overcome misconceptions by creating cognitive conflict. a concept cartoon alone does not reveal which is the right idea; instead, it presents misconceptions and a scientifically acceptable concept together. to correct a misconception, the concept cartoon is usually integrated into a classroom discussion moderated and directed by the teacher (keogh et al., 1998). during this guided discussion, students are expected to realize the correct concept. in addition, to detect and correct misconceptions, concept cartoons are also used to teach a specific science concept. in this context, kabapinar (2005) used several concept cartoons as teaching/learning material for several science concepts such as mass, volume, phase changes, atoms, and molecules in primary science courses. similarly, sanliturk and zeybek (2022) created and used a series of digital concept cartoons to teach the concepts of reproduction, growth, and development in plants and animals. they found that digital concept cartoons are effective teaching material. concept cartoons are commonly recommended learning material in science education. however, because of their static nature, traditional concept cartoons printed on paper are not generally accepted as a suitable medium for science concepts related to motion. in other words, as a static illustration, a cartoon is expectedly insufficient to describe a situation involving motion (höffler & leutner, 2007). traditionally, motion is partially described by several supporting lines in cartoons that help imagine it. however, motion obviously can be better described by demonstrating it with an animation that is a dynamic medium rather than a still image. therefore, an animated concept cartoon may be helpful to start a cognitive conflict in learning science concepts related to motion by presenting the correct concept and misconceptions together. in this regard, circular motion, which may be appropriate to be presented in an animated concept cartoon, is one of the science concepts that has a prevalent misconception, called centrifugal force. according to this misconception, rotation is caused by the balance between centrifugal force and an external force opposite to it (volfson, eshach, & ben-abu, 2020). regarding newton's laws of motion, such a balance of forces yields an unaccelerated motion. it cannot create a circular motion, a kind of accelerated motion needing an unbalanced force. in this context, an animated concept cartoon presenting the centrifugal force misconception and the concept of centripetal force, which is the actual cause of circular motion, may be an effective instructional material to create cognitive conflict. traditionally, concept cartoons have been used in school science instruction to initiate cognitive conflict (naylor & keogh, 2013). on the other hand, concept cartoons, animated or not, may also be used in online science teaching for the same purpose (basarmak & mahiroglu, 2016). in the covid-19 pandemic, science instruction is primarily performed online in many countries without sufficient preparation. this urgent transition from face-to-face to online teaching caused quality issues in online science instruction. animated concept cartoons may help these issues by enriching the variety of e-learning activities in online science education. in addition, because of their animated structure, animated concept cartoons may be more suitable for science concepts, most of which have dynamic nature. in this context, the purpose of this study is to develop an animated concept cartoon about circular motion and evaluate whether it can create cognitive conflict or not in an online learning process. 2. method this study is a one-group post-test only quasiexperimental research study aiming to develop an animated concept cartoon about circular motion and to evaluate its capability to initiate cognitive conflict about centrifugal force misconception. in brief, to achieve this aim, an animated concept cartoon, which is integrated into an online questionnaire to measure cognitive conflict, was developed and implemented for a group of undergraduate students. data collected in this implementation were analyzed to conclude about the capability of the concept cartoon to initiate cognitive conflict about centrifugal force misconception in online learning. 2.1 participants in this study, participants are 110 undergraduate students who study primary education and social science education at a state university in turkey. in this sample, 78 (71%) female and 32 (29%) male students were selected through convenience sampling, representing the population of 408 students in these departments. participants do not have extensive science backgrounds and are attendees of the introductory science course or science, technology, and society course, which are online conceptual science courses for students with non-science backgrounds. in this study, students who do not have extensive science backgrounds were chosen as participants because they possibly have more preconceptions about circular motion than those studying science majors. this may make animated concept cartoons more beneficial for non-science majors students. 2.2 material and instruments in this study, an animated concept cartoon about circular motion was used as digital material to initiate cognitive conflict. an online questionnaire was used as a data collection instrument to measure cognitive conflict. the animated concept cartoon is a gif animation created by the researcher using computer drawing and animation software. the gif format is chosen for this animation because it is easy to create and integrate into online learning journal of science learning article doi: 10.17509/jsl.v5i2.41191 244 j.sci.learn.2022.5(2).242-249 environments. however, it is possible to convert it into other computer animation or video formats. the animated concept cartoon created in this study demonstrates a daily life situation in which a waiter rotates a tray with his hand, and three guests talk about why a glass of water on the rotating tray does not fall. tray rotation is a well-known traditional behavior of waiters in turkey. the first guest asks why the glass of water on the rotating tray does not fall. the second guest correctly answers this question regarding centripetal force, while the third guest answers it concerning the misconception of centrifugal force. figure 1 shows a screenshot from the animated concept cartoon created in this study. this animated gif image is integrated into an online questionnaire to measure cognitive conflict. in addition to the preliminary likert-type items about participants' opinions about science, this questionnaire mainly includes questions about who is correct in the concept cartoon and whether the participant is sure about who is correct. the participants, who are not sure, are assumed in a status of cognitive conflict. on the other hand, some of the sure participants may also feel a cognitive conflict before getting sure. therefore, the questionnaire includes an additional question for the sure participants asking whether they feel a conflict before getting sure or not. these participants are assumed in a status of temporary cognitive conflict. in addition to the preliminary items and the main questions, the questionnaire includes several supplementary likerttype items asking about satisfaction (enjoyment, interest, and usefulness) with the animated concept cartoon. table 1 items in the data collection instrument # question p1 i like science. p2 i am interested in science. p3 i think science is important. p4 i think i am successful in science. m1 you read the concept cartoon above. in your opinion, who is correct? m2 you picked one. are you sure that the person you picked is correct? m3 if you are sure about your answer, did you feel a conflict before your decision? s1 i do not like the animated concept cartoon. s2 i read the animated concept cartoon with enjoyment. s3 i found the animated concept cartoon fun. s4 the animated concept cartoon caught my attention. s5 reading the animated concept cartoon is boring. s6 the animated concept cartoon is intriguing. s7 the animated concept cartoon is remarkable. s8 the animated concept cartoon may be used as an introductory activity in distance education. s9 the animated concept cartoon is a suitable material for distance education. s10 i found the animated concept cartoon useful. s11 the animated concept cartoon is thought-provoking. s12 the animated concept cartoon is clear. s13 the animated concept cartoon is not suitable for my age. figure 1 a screenshot from the animated concept cartoon journal of science learning article doi: 10.17509/jsl.v5i2.41191 245 j.sci.learn.2022.5(2).242-249 supplementary satisfaction items were set to be ordered randomly for each participant. table 1 demonstrates the preliminary opinion items (p), the main cognitive conflict questions (m), and the supplementary satisfaction items (s) in the data collection instrument. both the animated concept cartoon about circular motion and the questionnaire for cognitive conflict were designed and constructed with the supervision of experts' opinions. in other words, initial versions of both instruments were given to two science education experts and revised concerning their opinions before implementing by the participants. 2.3 implementation as mentioned above, participants of this study are a group of undergraduate students attending the introductory science course or science technology and society course, which are offered as online courses because of the covid-19 pandemic. after revising the initial versions of instruments, an online form including the final versions of the animated concept cartoon and the questionnaire was constructed. after getting official permission from the research ethics committee, the link to the online form was shared with the participants before learning the concepts of circular motion in the introductory science course. the participants were also informed about the importance of their answers' independence. they responded to the online questionnaire in about two weeks using their computers or mobile devices. after completing the questionnaire, their responses data were downloaded and converted for statistical analyses. 2.4 statistical analysis before analyzing data from the main cognitive conflict items, participants' opinions about science measured by preliminary opinion items (p) were analyzed to describe the characteristics of the sample according to their opinions about science. after this preliminary analysis, data from the main cognitive conflict items (m) were analyzed. in this context, frequencies and percentages of participants who responded correctly and incorrectly about the cause of circular motion were calculated. then, the frequencies and percentages of participants who were sure and unsure about their responses to the first question were calculated. as mentioned in the previous sections, the students who are unsure about their responses to the first question are regarded as in the cognitive conflict status. in addition, frequencies and percentages of participants who were sure about their responses but felt a conflict before getting sure were also calculated. these participants are accepted as in the status of temporary cognitive conflict. in addition to these descriptive statistics, a statistical comparison of the frequencies of the participants in the status of the cognitive conflict who responded correctly and incorrectly about the cause of circular motion was performed by using the chisquare independence test to check whether the concept cartoon starts cognitive conflict equally for correct and incorrect responses or not. in addition to statistical analyses on cognitive conflict described above, several supplementary descriptive statistics were also calculated based on data from supplementary items in the questionnaire. in addition to descriptives, the pearson-r reliability of these supplementary items was also calculated. these supplementary statistics demonstrate participants' opinions about using the animated concept cartoon in an online learning environment in the dimensions of enjoyment, interest, and usefulness. furthermore, the findings of this analysis were used to evaluate participants' overall satisfaction with the animated concept cartoon. finally, participants' satisfaction with the animated concept cartoon was compared with regard to being in the status of cognitive conflict by using the mann-whitney u-test because of the non-normal distribution of students' satisfaction scores. the results of these analyses will be used as feedback to improve the animated concept cartoon. 3. results and discussion this study's results of statistical analyses were classified into three main categories. at first, the descriptive results of participants' opinions about science were reported. then, the results of the statistical analysis of the items about the cognitive conflict were presented. finally, the analysis results of participants' opinions about the animated concept cartoon were reported. 3.1 participants’ opinions about science participants' opinions about science were measured by four items representing four dimensions of attitude: the enjoyment of science, interest in science, importance (value) of science, and perception of self-success in science. these items are not an actual attitude scale to calculate the individual attitude scores of the participants. instead, they are just descriptive items to describe the characteristics of the sample in the affective domain. table 2 demonstrates participants' opinions about science. as shown in table 2, most participants have positive opinions about the enjoyment (p1) and importance (p3) of science. however, their responses to interest in science (p2) and self-perception of success in science (p4) seem to be more dispersed. in other words, for most participants, interest in science and self-perception of success in science does not seem to be high. these dispersed results are somehow expected for a sample with non-science majors. 3.2 participants’ conceptions and cognitive conflict as mentioned in the previous sections, participants were asked three questions about centrifugal force misconception and cognitive conflict. before performing deeper analyses, descriptive statistics give some idea about participants' conceptions and their status of cognitive conflict. table 3 shows the frequencies and percentages of participants' conceptions of circular motion. journal of science learning article doi: 10.17509/jsl.v5i2.41191 246 j.sci.learn.2022.5(2).242-249 as shown in table 3, more than half of the participants seem to choose centrifugal force as the cause of circular motion in the related question. however, it does not mean that these participants have the centrifugal force misconception. in this study, the criterion of having a misconception is to be sure about it. in this context, it may be safely stated that 29.1 percent of the participants, who are sure about their answers, have the centrifugal force misconception. on the other hand, 21.8 percent of the participants are sure about their correct answers to centripetal force. all of the other participants are not sure about their conceptions. although the frequencies of the four groups shown in table 2 seem to differ, these four frequencies are not statistically different (chi-squared = .564; p = .453). after demonstrating the participants' conceptions of circular motion, descriptive statistics of their cognitive conflict status may be useful for evaluating the capability of the animated concept cartoon. table 4 shows participants' status of cognitive conflict. as demonstrated in table 4, slightly less than half of the participants, who are unsure about their answers, may be regarded as being in the status of cognitive conflict after reading the animated concept cartoon. therefore, the frequencies of the participants, who are in cognitive conflict and not in cognitive conflict, can be accepted as equal (chi-squared = .036; p = .849). on the other hand, 24.5 percent of the participants were sure about their answers and declared that they felt a conflict before getting sure. therefore, these participants may be regarded as being in the status of temporary cognitive conflict after reading the animated concept cartoon. therefore, the frequencies of the participants, who are in temporary cognitive conflict and not in temporary cognitive conflict, can also be accepted as equal (chi-squared = .019; p = .891). 3.3 satisfaction with the animated concept cartoon at first, the frequencies of participants' responses to the satisfaction items were calculated. next, the items with negative meaning were reversed for further analyses, then participants' individual and average scores, named the concept cartoon satisfaction score, were calculated. finally, the frequencies of participants in each satisfaction item were demonstrated in table 5. as shown in table 5, most participants have positive opinions about the animated concept cartoon. as mentioned above, participants' concept cartoon satisfaction scores were calculated based on their responses to 13 satisfaction items. their average concept cartoon satisfaction score is 54.3 out of 65 with 0.945 reliability, which may be regarded that the participants have a high level of satisfaction with the animated concept cartoon. in addition to average satisfaction scores, the distribution of the individual scores is quite skewed that most participants are massed at high satisfaction scores. figure 2 shows the histogram of participants' satisfaction scores. in addition to the central tendency and distribution of satisfaction scores, a statistical comparison of satisfaction scores of the participants who are in the status of cognitive conflict and the others is useful to make deeper inferences about the capability of the animated concept cartoon. in table 2 participants’ opinions about science frequencies* items sd d n a sa i like science (p1) 8 7 10 61 24 i am interested in science (p2) 7 21 29 34 19 i think science is important (p3) 8 2 3 37 60 i think i am successful in science (p4) 12 29 34 26 9 * sd: strongly disagree, d: disagree, n: neutral, a: agree, sa: strongly agree table 3 descriptive statistics of participants’ conceptions on circular motion conception (m1) centripetal force freq. (%) centrifugal force freq. (%) 51 (46.4) 59 (53.6) sure (%) not sure (%) sure (%) not sure (%) 24 (21.8) 27 (24.5) 32 (29.1) 27 (24.5) table 4 participants’ status of cognitive conflict cognitive conflict (m2) not sure (%) sure (%) 54 (49.1) 56 (50.9) temporary cognitive conflict (m3) feel conflict (%) do not feel conflict (%) missing (%) 27 (24.5) 26 (23.6) 3 (2.7) journal of science learning article doi: 10.17509/jsl.v5i2.41191 247 j.sci.learn.2022.5(2).242-249 this regard, mann-whitney u-test was performed to make this comparison because of the non-normal distribution of satisfaction scores. it was found that there is no significant difference in satisfaction scores of the participants who are in the status of cognitive conflict and not in the status of cognitive conflict (u = 1308, p = .221). the descriptive and inferential findings of the animated concept cartoon are discussed in the next section. 3.4 discussion concept cartoons are usually used to detect the existence of misconceptions and/or to start cognitive conflict at the beginning of the teaching/learning process. instead of the traditional static version printed on paper, animated concept cartoons may be useful for the same purpose in an online learning environment, especially for the concepts related to motion. in this study, an animated concept cartoon about circular motion was developed and implemented for a group of undergraduate students in an online science course to evaluate the extent to which reading this cartoon creates a cognitive conflict. results showed that about half of the participants chose the cartoon character justifying the centrifugal force misconception. however, it does not mean that all these participants have the centrifugal force misconception. in this study, the criterion of having a misconception is to be sure about it, as mentioned in the previous sections. in this regard, about 29 percent of all participants seem to be sure about centrifugal force misconception. this finding is similar to the ones in the literature related to the misconceptions about circular motion (bani-salameh, 2016; volfson et al., 2020). as described above, this study does not focus on detecting this particular misconception. instead, it investigates whether or not the animated concept cartoon starts the cognitive conflict. students' cognitive conflict status is measured based on being unsure about their responses. results showed that students who chose centripetal and centrifugal forces tend equally to be in cognitive conflict, and this finding may be interpreted that narrations of the cartoon characters are sound correct, which is one of the must-have features of a good concept cartoon (keogh et al., 1998). in this study, the students who are sure about their responses are simply accepted not to be in the status of cognitive conflict. however, some may feel a temporary conflict before getting sure about their responses. therefore, an additional item measured the status of temporary cognitive conflict in the data collection instrument. results indicated that half of the participants who are eventually sure about their responses are in the status of temporary cognitive conflict before getting sure. table 5 participants' responses to the satisfaction items # item sd d n a sa s1 i do not like the animated concept cartoon.* 57 42 5 1 5 s2 i read the animated concept cartoon with enjoyment. 3 5 9 45 48 s3 i found the animated concept cartoon fun. 2 8 8 46 46 s4 the animated concept cartoon caught my attention. 4 5 5 45 51 s5 reading the animated concept cartoon is boring.* 62 39 8 0 1 s6 the animated concept cartoon is intriguing. 4 6 10 41 49 s7 the animated concept cartoon is remarkable. 3 7 6 44 50 s8 the animated concept cartoon may be used as an introductory activity in distance education. 4 7 9 39 51 s9 the animated concept cartoon is a suitable material for distance education. 4 6 11 41 48 s10 i found the animated concept cartoon useful. 3 6 7 44 50 s11 the animated concept cartoon is thought-provoking. 4 9 6 49 42 s12 the animated concept cartoon is clear. 4 10 8 50 38 s13 the animated concept cartoon is not suitable for my age.* 50 38 14 6 2 * items with an asterisk have a negative meaning and were reversed for analysis. figure 2 histogram of participants’ satisfaction scores journal of science learning article doi: 10.17509/jsl.v5i2.41191 248 j.sci.learn.2022.5(2).242-249 this result may indicate that the animated concept cartoon starts a temporary cognitive conflict as well as the longterm cognitive conflict. regardless of choosing correct or incorrect concepts in the concept cartoon, approximately half of the students were observed to have long-term cognitive, and onequarter of the students were observed to have temporary cognitive conflict after reading the animated concept cartoon. these findings indicate that the animated concept cartoon sufficiently achieved the aim of initiating cognitive conflict. as mentioned above, one of the reasons behind this result may be that the narrations in the speech balloons sound equally correct. as expected, the narration stating centrifugal force misconception sounds correct because it is a widespread, persistent misconception (volfson et al., 2020). another reason the concept cartoon successfully starts cognitive conflict may be its daily life context. misconceptions often originate and are consolidated by individuals' daily life experiences (kinik topalsan & bayram, 2019). therefore, the concept cartoon demonstrating a daily life situation about circular motion may evoke or excite a dormant centrifugal force misconception and initiate a cognitive conflict. as described above, this study also focuses on participants' overall satisfaction with the animated concept cartoon to get feedback for future revisions. participants' overall satisfaction with the animated concept cartoon, including enjoyment, interest, and usefulness, is relatively high in these dimensions. in addition, no difference in the satisfaction scores of participants in the status of cognitive conflict and not in the status of cognitive conflict was found. this result indicates that animated concept cartoons may be regarded as enjoyable, remarkable, and useful learning activities in the online science teaching process regardless of being in cognitive conflict. similarly, literature reports that concept cartoons are funny, exciting, and useful learning aids (keogh & naylor, 2000; cinar, 2017; norfarah, mohd ali, & chong, 2019). conclusion this study aims to evaluate the usability of an animated concept cartoon about circular motion; however, it mainly relies on participants' opinions. therefore, a definite causeand-effect relationship cannot be established as a result of this study, which is the study's main limitation. an experimental research setting can investigate a cause-andeffect relationship between two variables. therefore, the other researchers are suggested to perform experimental research studies about the causality between the use of concept cartoons and cognitive conflict. this study may also suggest educational policymakers, instructional material developers, and science teachers. due to their dynamic nature, animated concept cartoons are very suitable for teaching/learning science concepts, most of which are somehow related to motion. in addition, as digital material, animated concept cartoons are appropriate for computer and internet-based teaching/learning. in addition to online learning, animated concept cartoons may also be used in real classrooms equipped with computer technologies. therefore, science curricula may be revised to offer some digital activities initiating cognitive conflict, such as animated concept cartoons. in addition, digital instructional material and activity developers may produce a series of animated concept cartoons covering all science concepts. after initiating cognitive conflict, to present scientifically correct concepts, developers may also design and develop supplementary digital learning activities that may be used after concept cartoons, such as simulations and virtual experiments. besides, through in-service training, science teachers may be trained to use animated concept cartoons effectively in online and face-to-face classroom teaching. the animated concept cartoon developed in this study was used as a standalone online activity to create cognitive conflict about students' preconceptions about circular motion in online science instruction. results indicated that this animated concept cartoon might be a suitable digital activity for this specific purpose. in conclusion, concerning statistical findings of this study reported and discussed previously, animated concept cartoons have a serious potential to be remarkable, enjoyable, and useful digital activities that can be used to initiate cognitive conflict about students' preconceptions at the beginning of teaching/learning process in online science education. acknowledgment the author would like to thank h. pesman and k. e. bagriyanik for contributing to developing the animated concept cartoon by giving invaluable feedback about its content, appearance, and implementation. references atasoy, s. (2020). using concept cartoons to identify the epistemological beliefs of middle school students. journal of science learning, 3(3), 165-173. https://files.eric.ed.gov/fulltext/ej1267613.pdf bani-salameh, h. n. (2016). how persistent are the misconceptions about force and motion held by college students? physics education, 52(1), 1-7. https://doi.org/10.1088/1361-6552/52/1/014003 barrouillet, p. (2015). theories of cognitive development: from piaget to today. developmental review, 38(1), 1-12. https://doi.org/10.1016/j.dr.2015.07.004 basarmak, u., & mahiroglu, a. (2016). the effect of online learning environment based on caricature animation used in science and technology course on the success and attitude of the student for humor. turkish online journal of educational technologytojet, 15(4), 107-118. https://files.eric.ed.gov/ fulltext/ej1117641.pdf chin, c., & teou, l. y. (2009). using concept cartoons in formative assessment: scaffolding students’ argumentation. international journal of science education, 31(10), 1307-1332. cinar, d. (2017). the opinions of teacher candidates about the use of concept cartoon in science and technology teaching courses. contemporary educational researches journal, 7(2), 50-56. https://files.eric.ed.gov/ journal of science learning article doi: 10.17509/jsl.v5i2.41191 249 j.sci.learn.2022.5(2).242-249 https://pdfs.semanticscholar.org/ 5174/ed4d8661c300b65cec8129226da17591f68a.pdf clement, j. (1982). students’ preconceptions in introductory mechanics. american journal of physics, 50(1), 66-71. http://zyzx.haust.edu.cn/moocresource/data/20091115/u/mit 20091115083/ocwweb/urban-studies-and-planning/11125spring2009/calendarandassignments/students_preconceptions_in_intr oductory_mechanics.pdf ekici, f., ekici, e., & aydin, f. (2007). utility of concept cartoons in diagnosing and overcoming misconceptions related to photosynthesis. international journal of environmental and science education, 2(4), 111–124. franklin, b. j. (1992). the development, validation, and application of a twotier diagnostic instrument to detect misconceptions in the areas of force, heat, light and electricity [doctoral dissertation, lousiana state university]. https://digitalcommons.lsu.edu/gradschool_disstheses/5302 höffler, t. n., & leutner, d. (2007). instructional animation versus static pictures: a meta-analysis. learning and instruction, 17(6), 722738. https://www.sciencedirect.com/science/article/pii/s0959475207 001077 kabapinar, f. (2005). effectiveness of teaching via concept cartoons from the point of view of constructivist approach. educational science: theory & practice, 5(1), 135–146. https://www.researchgate.net/profile/filiz-kabapinar2/publication/ 265411400_effectiveness_of_teaching_via_concept_cartoons_f rom_the_point_of_view_of_constructivist_approach/links/57 d03fc708ae5f03b4890a3c/effectiveness-of-teaching-viaconcept-cartoons-from-the-point-of-view-of-constructivistapproach.pdf kaltakci gurel, d., eryilmaz, a., & mcdermott, l. c. (2015). a review and comparison of diagnostic instruments to identify students’ misconceptions in science. eurasia journal of mathematics, science & technology education, 11(5), 989–1008. https://open.metu.edu.tr/bitstream/handle/11511/38957/index. pdf kandil ingec, s. (2008). use of concept cartoons as an assessment tool in physics education. us-china education review, 5(11), 47– 54. https://files.eric.ed.gov/fulltext/ed503880.pdf kang, s., scharmann, l. c., & noh, t. (2004). reexamining the role of cognitive conflict in science concept learning. research in science education, 34(1), 71-96. keogh, b., naylor, s., & wilson, c. (1998). concept cartoons: a new perspective on physics education. physics education, 33(4), 219– 224. https://link.springer.com/content/pdf/10.1023/b:rise.000002 1001.77568.b3.pdf keogh, b., & naylor, s. (2000). teaching and learning in science using concept cartoons: why dennis wants to stay in at playtime. investigating, 16(3), 10-14. kinik topalsan, a., & bayram, h. (2019). identifying prospective primary school teachers’ ontologically categorized misconceptions on the topic of "force and motion". journal of turkish science education, 16(1), 85-109. https://www.tused.org/index.php/tused/article/download/221/ 174/ kumi-manu, r. n. (2021). concept cartoon as a teaching technique for conceptual change: a ghanaian junior high school experience. american journal of educational research, 9(9), 587-599. http://article.scieducationalresearch.com/pdf/education-9-95.pdf kusumaningrum, i. a., & indriyanti, n. y. (2018). concept cartoons for diagnosing student’s misconceptions in the topic of buffers. journal of physics: conference series, 1022(1), 012036 naylor, s., & keogh, b. (2013). concept cartoons: what have we learnt? journal of turkish science education, 10(1), 3–11. https://www.tused.org/index.php/tused/article/download/273/ 223/ norfarah, n., mohd ali, s., & chong, l. y. (2019). concept cartoon: a constructivist strategy to enhance positive attitude towards science in malaysia: an ancova repeated measures approach. international journal of education, psychology and counseling, 4(27), 2840. pine, k., messer, d., & st. john, k. (2001). children's misconceptions in primary science: a survey of teachers' views. research in science & technological education, 19(1), 79-96. https://www.tandfonline.com/doi/pdf/10.1080/0263514012004 6240 posner, g. j., strike, k. a., hewson, p. w., & gertzog, w. a. (1982). accommodation of a scientific conception: toward a theory of conceptual change. science education, 66(2), 211-227. http://www.ud.infn.it/urdf/laurea/idifo1/materiali/g5/posne retal.pdf sanliturk, a. d., & zeybek, g. (2022). the effect of using digital concept cartoons in science lesson on students’ achievement. journal of science learning, 5(1), 1-13. https://ejournal.upi.edu/index.php/jslearning/article/view/121/ pdf serttas, s., & turkoglu, a. y. (2020). diagnosing students’ misconceptions of astronomy through concept cartoons. participatory educational research, 7(2), 164-182. https://dergipark.org.tr/en/download/article-file/1086989 stephenson, p., & warwick, p. (2002). using concept cartoons to support progression in students' understanding of light. physics education, 37(2), 135. taslidere, e. (2014). effect of conceptual change instruction on remedying misconceptions concerning direct current circuits. bartin üniversitesi egitim fakültesi dergisi, 3(1), 200. volfson, a., eshach, h., & ben-abu, y. (2020). identifying physics misconceptions at the circus: the case of circular motion. physical review physics education research, 16(1), 1-11. https://doi.org/10.1103/physrevphyseducres.16.010134 https://www.researchgate.net/profile/filiz-kabapinar-2/publication/ https://www.researchgate.net/profile/filiz-kabapinar-2/publication/ a © 2022 indonesian society for science educator 334 j.sci.learn.2022.5(2).334-341 received: 04 january 2022 revised: 24 april 2022 published: 27 july 2022 chemistry learning using multiple representations: a systematic literature review margaretha bhrizda permatasari1, sri rahayu1*, i wayan dasna1 1departement of chemistry, faculty of mathematics and natural science, universitas negeri malang, malang, indonesia *corresponding author: sri.rahayu.fmipa@um.ac.id abstract the abstractness of the chemistry concept can be understood easily through chemistry learning using multiple representations. this article used the systematic literature review (slr) method to review eleven articles published from 2012 to 2021 and focused on chemistry learning using various representations. the articles are systematically obtained from the online article database eric, scopus, and sinta. the purpose of a review is to give information to teachers and researchers in chemistry education about the definition of multiple representations, the influence of multiple representations on chemistry learning outcomes, and how to implement various representations in chemistry learning models or strategies. the review results showed that the definition of numerous representations referred to both three levels of chemical representation and the tetrahedral representation of chemistry. also, it referred to the use of various media. the influence of multiple representations on chemistry learning outcomes included improving concept understanding, improving performance, reducing mental effort, improving self-efficacy, making better cognitive structures, improving mental models, and reducing misconceptions. multiple representations have also been implemented in several learning models or strategies such as inquiry, inquiry 5e, guided inquiry, problem solving, thinking, aloud pair problem solving (tapps), problem posing (pp), cognitive dissonance, and multiple representation based learning (mrl). keywords multiple representations, chemistry learning outcomes, learning models/strategies 1. introduction chemistry is one of the branches of natural science which studies the properties of substances, the structure of imports, changes of substances, laws, and principles that describe changes in senses, as well as concepts and theories about it (effendy, 2016). it can also be interpreted as a science that seeks answers about what, why, and how natural phenomena happen that relate to substances, including structure, composition, nature, dynamics, kinetics, and energetics involving skills and reasoning (huddle & pillay, 1996; chang & overby, 2011). chemistry has many abstract and complex concepts, but unfortunately, many learners have limited ability to think abstractly, leading to struggle to understand chemical concepts and sometimes experience misconceptions (milenković, segedinac & hrin, 2014; nakhleh, 1992). this causes chemistry to be often seen as a difficult subject (sirhan, 2007). the chemistry concepts' abstractness can be easily understood by involving multiple representations in the chemistry learning process. wiyarsi, sutrisno & rohaeti, (2018) explained that chemistry learning with multiple representations could be a bridge for learners in understanding chemistry concepts. widarti, marfu’ah & parlan (2019) stated that multiple representations of intermolecular force learning in organic chemistry i class could help learners better understand concepts. domin & border (2004) also assert that multiple representations can help learners to solve problems related to chemistry concepts. based on these explanations, it is important to do an in-depth literature review of chemistry learning using multiple representations to understand better the implementation and the influence of involving multiple representations in chemistry learning. this literature review article presents the results of studies from several articles that focused on chemistry learning using multiple representations. the reports studied in this literature review were obtained through a systematic synthesis of articles from 2012 to 2021. through this literature review article, it is hoped that teachers and researchers in chemistry education get insights and information about multiple representations, the influence mailto:sri.rahayu.fmipa@um.ac.id journal of science learning article doi: 10.17509/jsl.v5i2.42656 335 j.sci.learn.2022.5(2).334-341 of involving multiple representations in chemistry learning, and how to implement them in chemistry learning models or strategies. in addition, it also provides information for researchers in chemistry education about trends and patterns of research over chemistry learning using multiple representations and gives direction for further investigation. the research questions that guided the writing of this article are: 1. what is the definition of multiple representations? 2. how does the effect of multiple representations on the chemistry learning outcomes? 3. how does the implementation of multiple representations on chemical learning models or strategies? 2. method the method used in this article was systematic literature review (slr). xiao & watson (2019) explain that systematic literature review (slr) is a literature review method that follows standard rules for identifying and synthesizing relevant research articles and assessing what is known from the studied topic. the articles analyzed in this literature review were obtained by searching the online database eric, scopus, and sinta (indonesian research database). literature and analysis were collected from september 30 to december 17, 2021. in this study, the keywords used are "impact multiple representations in chemistry learning", "chemistry learning with multiple representations", "the effect of multiple representations", and "multiple representation". after searching for a keyword, the researcher reads the title of the article to select articles that meet the following inclusion criteria: (1) relate to learning using multiple representations and the impact of learning using multiple representations; (2) the year of publication articles from 2012 to 2021; (3) articles from reputable journals that indexed internationally by scopus or indexed nationally by sinta. based on the article search results, there were 49 titles suitable with inclusion criteria. by then, the abstracts of those 49 articles were analyzed. the result of the abstract analysis was 26 articles following chemistry learning using multiple representations. meanwhile, the other 23 articles were inappropriate because three articles were numerous representations based on learning for other subjects, and 20 were studies to prove the effectiveness of teaching materials or teaching media that use multiple representations. furthermore, all content of 26 articles was read. based on the results of content readings, 11 articles contained the research design, types of research inclined to quantitative or mixed methods, and incorporate syntax/ learning process. finally, these 11 articles provided suitable information on what will be discussed. the searching and selection process can be seen in figure 1. 3. result and discussion 3.1 definition of multiple representations chemistry is one of the sciences that is often considered difficult for many people because it consists of abstract figure 1 the process of search and selection articles eric n = 35 scopus n = 9 sinta n = 5 n = 49 articles n = 26 articles n = 11 articles title selection abstract selection content selection relate with learning using multiple representations and the impact of learning using multiple representations. include design of research, the type of research inclined to quantitative or mixed method, and include syntax or learning process. accordance with chemistry learning using multiple representations. eric = 6 articles; scopus = 3 articles; sinta = 2 articles journal of science learning article doi: 10.17509/jsl.v5i2.42656 336 j.sci.learn.2022.5(2).334-341 concepts and topics (santos & arroio, 2016). the abstractness of chemistry concepts can be easily understood through the three levels of representation described by johnstone (1993): macroscopic, submicroscopic, and symbolic. according to chandrasegaran, treagust & mocerino. (2007) and rahayu & kita (2010), macroscopic representation relates to observable phenomena or everything that can be seen, touched, and felt by the five senses. submicroscopic representation relates to the level of particulates such as atoms, ions, and molecules in chemical reactions. symbolic representation relates to elemental symbols, chemical reaction equations, and chemical formulas. these three levels of representation in chemistry are commonly known as multiple representations (wiyarsi, sutrisno & rohaeti, 2018). some articles that were obtained systematically as "multiple representations, multiple-level representations, or multiple knowledge representations" referred to the involvement of macroscopic, submicroscopic, and symbolic representations in chemistry learning (milenković, segedinac & hrin, 2014; supasorn, 2015; kimberlin & yezierski, 2016; derman & ebenezer, 2020; tima & sutrisno, 2018; widarti, permanasari, mulyani, rokhim & habiddin, 2021). indriyanti, saputro & sungkar (2020) use the "multiple representations" word to refer to the tetrahedral representation of chemistry. mahaffy (2006) explained that tetrahedral representation consists of macroscopic, submicroscopic, symbolic, and the human element as supplementary. the human element emphasizes case studies, active learning, and investigative projects to link school chemistry with daily activities. the word “multiple representations” also refers to the usage of various media to represent a concept or a process. the press mentioned can be in the form of diagrams, equations, tables, text, graphics, animations, sounds, videos, and dynamic simulations (ainsworth, 2006). the multiple representations which refer to the usage of various media are commonly known as multiple external representations (mers). in chemistry learning, mers are regularly integrated with all three levels of chemical representation (macroscopic, submicroscopic, and symbolic) as performed by baptista, martins, conceição and reis (2019), sunyono & meristin (2018), and priyasmika (2021). based on the explanation above, the definition of multiple representations in chemistry learning refers to the involvement of three levels of johnstone representation (1993) or tetrahedral representation of mahaffy (2006). in addition, the definition of multiple representations also refers to the involvement of various media in the learning process (ainsworth, 2006). although there is a definition of multiple representations that refers to the use of various media, in chemistry learning, these different media are often integrated with macroscopic, submicroscopic, and symbolic representations to make it easier for learners to understand chemistry concepts. 3.2 the influence of multiple representations on chemistry learning outcomes the involvement of multiple representations in chemistry learning positively impacts learners. the research articles obtained systematically in this literature review showed some positive influences of multiple representations on the learning outcomes of several chemistry topics, as table 1 shows the influences of multiple representations on the learning outcomes of several chemistry topics the influences of multiple representations on the learning outcomes are summarized in figure 2. based on figure 2, the greatest influence of multiple representations on chemistry learning outcomes is improving learners' concepts and understanding. a statistical result from several articles showed that the impact of the posttest was higher than the results of the pretest. it explained that understanding concepts increased because it involved multiple representations in chemical learning (bridle & yezierski, 2012; supasorn, 2015; kimberlin & yezierski, 2016; sunyono & meristin, 2018; widarti, marfu’ah & parlan, 2019; indriyanti, saputro & sungkar 2020). learners’ understanding of concepts can increase as multiple representations become the bridge for learners to understand the abstract concepts of chemistry that require depiction (wiyarsi, sutrisno & rohaeti, 2018; widarti, marfu’ah & parlan, 2019). chemistry learning, which highlights three levels of chemistry representation, helps learners see the relationships between the three levels of chemistry representation to understand the concept better afterward. when learners understand the concept well, their mental effort (cognitive burden) during working on problems is figure 2 summary of the influence of multiple representations on chemistry learning outcomes 43% 15% 7% 7% 14% 7% 7% the influence of multiple representations on chemistry learning outcomes increasing students concept understanding incresing students performance reducing cognitive load increasing self efficacy develop students cognitive structure develop students mental model reducing misconception journal of science learning article doi: 10.17509/jsl.v5i2.42656 337 j.sci.learn.2022.5(2).334-341 reduced, so their performance (achievement or learning outcomes) can be improved (milenković, segedinac & hrin, 2014; priyasmika, 2021). a good understanding of concepts also increases learners' self-efficacy (tima & sutrisno, 2018). self-efficacy can be interpreted as the confidence of learners in working on problems. involving multiple representations in chemistry learning can also reduce misconceptions that occur in learners when using cognitive dissonance strategies. misconceptions learners can reduce because integrating multiple representations in cognitive dissonance strategies allows learners to discover concepts, connect new concepts with existing knowledge, and solve problems related to the concepts learned (widarti, permanasari, mulyani, rokhim & habiddin, 2021). multiple representations also affect the development of a better cognitive structure for learners. it is evidenced by the pretest-posttest results of several articles using wat (word association test). when the posttest was conducted, learners could write more response words and connect response words compared to the pretest. differences in pretest and posttest results indicated that the cognitive structures in learners were developed. multiple representations allow learners to build a deeper and more structured understanding of concepts, so the cognitive structure of learners creates for the better (derman & ebenezer, 2020; baptista, martins, conceição tabel 1 the influences of multiple representations toward the learning outcomes of several chemistry topics no research and year topic the influences of multiple representations toward the learning outcomes 1 bridle and yezierski (2012) particulate nature of matter improve the understanding concepts of learner about particulate nature of matter 2 milenkovic et al. (2014) inorganic reaction improving performance of learner reducing mental effort of learner 3 supasorn (2015) galvanic cells improve the understanding concepts of learner about galvanic cells improving mental models of learner about galvanic cells 4 kimberlin and yezierski (2016) stoichiometry improve the understanding concepts of learner about stoichiometry 5 sunyono and meristin (2018) chemical bonding improve the understanding concepts of learner about chemical bonding 6 baptista et al. (2019) saponifikasi reaction cognitive structures of learner better than before 7 derman and ebenezer (2020) physical and chemical change cognitive structures of learner better than before 8 indriyanti et al. (2020) mole concept improve the understanding concepts of learner about mole concept 9 tima and sutrisno (2018) chemical equilibrium improve the understanding concepts and self efficacy of learner about chemical equilibrium 10 widarti et al. (2021) volumetric analysis reducing misconceptions about volumetric analysis on learmer 11 priyasmika (2021) chemical equilibrium improving learning outcomes about chemical equilibrium of learner figure 3 students’ pre and post mental model journal of science learning article doi: 10.17509/jsl.v5i2.42656 338 j.sci.learn.2022.5(2).334-341 and reis, 2019). besides influencing the development of cognitive systems, multiple representations can also improve learners' mental models (supasorn, 2015). it is evidenced by the higher post-model mental score of learners than the pre-model mental score. the improvement of the mental model of learners can also be observed in figure 3. based on figure 3, the pre-mental model of learners was correct in drawing the oxidation half-cells where there are zn atoms in the anode and zn2+ ions in the solution. however, learners were still incorrect in drawing the reduction half-cells with ni2+ ions in the cathode and ni atoms in the solution. in the post-mental model, learners correctly drew the reduction half-cells with ni atoms in the cathode and ni2+ ions in the solution. the increasing mental model of learners resulted from learning with multiple representations, which provided a learning experience for learners to go through three levels of chemistry representation. thus, initially, learners only have macroscopic and symbolic understanding to achieve a submicroscopic understanding (supasorn, 2015). 3.3 the implementation of multiple representations on chemistry learning models / strategies nowadays, learners must be more involved in the learning process. therefore, the learners are expected to play an active role in the learning process. thus, the activeness of learners is reviewed from their role in learning, such as asking, answering questions, and responding (nugrahaeni, redhana & kartawan, 2017). in addition, the activeness of learners is also reviewed from their efforts to learn everything on their will and ability, so the teacher or educator only acts as motivators, guides, and facilitators. currently, c hemistry learning uses various learning models or strategies to improve learners' activeness, learning interests, and learning outcomes. besides using various learning models or strategies, implementing multiple representations is essential in chemistry learning, considering that multiple representations can positively influence learners, as previously described. the systematic research articles in this literature review showed that multiple representations had been implemented on several models or strategies in chemistry learning. the implementation of multiple representations on several models or strategies can be seen in table 2. based on table 2, it can be seen that multiple representations are often implemented on chemistry learning-based inquiry. for example, the inquiry learningbased multiple representations conducted by bridle & yezierski (2012) and kimberlin & yezierski (2016) begin by asking learners who have been divided into groups to explore the particulate level (submicroscopic) of the topic studied. furthermore, students engaged in practicum activities were then asked to connect the level of particulates that have been explored with macroscopic and symbolic representations and mathematical calculations (specifically stoichiometric topics). meanwhile, inquiry learning-based multiple representations conducted by milenković, segedinac & hrin (2014), baptista, martins, conceição and reis (2019), and derman & ebenezer (2020) begins by asking learners who have also been divided into groups to observe a phenomenon or experiment. then, they were asked to link their observations to submicroscopic and symbolic levels. afterward, they were asked to explain phenomena or experiments observed macroscopically, submicroscopically, and symbolically in class discussions. the results showed that implementing multiple representations in inquiry learning can improve learners' conceptual understanding. in addition, some articles explain that implementation of multiple representations in inquiry learning can develop better cognitive structures of learners. multiple representations also are implemented in the 5e inquiry learning activity (suparson, 2015). in this case, the learning activity of the 5e inquiry is assisted by a simple practicum combined with a kit model on electrochemical topics. the learning process consists of five stages, namely: tabel 2 the implementation of multiple representations on several models or strategies no research learning models/strategies 1 bridle and yezierski (2012) inquiry 2 milenkovic et al. (2014) inquiry 3 kimberlin and yezierski (2016) inquiry 4 baptista et al. (2019) inquiry 5 derman and ebenezer (2020) inquiry 6 supasorn (2015) inquiry 5e 7 priyasmika (2021) guided inquiry 8 tima and sutrisno (2018) problem solving 9 indriyanti et al. (2020) thinking aloud pair problem solving (tapps) problem posing (pp) 10 widarti et al. (2021) cognitive dissonance 11 sunyono and meristin (2018) multiple representation based learning (mrl) / simayang journal of science learning article doi: 10.17509/jsl.v5i2.42656 339 j.sci.learn.2022.5(2).334-341 (1) engagement, where learners are involved in scientific questions related to the topics; (2) exploration, in which learners are asked to explore and collect data to answer questions through the planning and implementation of experiments; (3) explanation, in which learners are asked to formulate explanations based on data and scientific knowledge to answer questions; (4) elaboration, in which learners are required to explain, expand, connect, and apply their macroscopic and symbolic findings in experiments to the submicroscopic level through interaction with the kit model; (5) evaluation, in which learners are involved in the evaluation of understanding through group and class discussions along with demonstrating the kit model on experimental concepts. the results showed that implementing 5e inquiry learning assisted by a simple practicum combined with the kit model can improve the conceptual understanding and mental model of learners. besides 5e inquiry, multiple representations are also implemented in guided inquiry learning. for example, priyasmika (2021) implements multiple representations (macroscopic, submicroscopic, and symbolic) in guided inquiries at the exploration and concept formation stage. at that stage, learners can process information through investigations related to the presented representations, so their understanding of concepts becomes better. the results showed that implementing multiple representations in chemistry learning with guided inquiry models can improve learners' understanding of concepts and learning outcomes. this is in line with research conducted by pikoli (2020), where chemistry learning using guided inquiry models with multiple representations can improve the understanding of learners' concepts so that misconceptions that occur in learners can be reduced. articles that discuss the implementation of multiple representations in inquiry learning that have been described above, when viewed from the methodology, most of these articles use pre-experimental and quasi-experiment quantitative research methods. when studying the effectiveness, the most effective implementation of multiple representations in inquiry learning is carried out by milenković, segedinac & hrin (2014). this is because milenković, segedinac & hrin (2014) use pretest-posttest control group research designs, while others use one group pretest-posttest research designs. the method of the pretest-postest control group showed the existence of a control class for comparison. so, the effectiveness of treatment is more tested, and the results can be generalized. meanwhile, the one-group pretest-posttest research design has not shown the existence of a control class for comparison. so, the effectiveness of the treatment is still doubtful and untested. in addition, the research results due to treatment are still assumptions that need to be followed up again and cannot be generalized. besides the inquiry learning model, multiple representations are also implemented in the problemsolving model, as done by tima & sutrisno (2018). the problem-solving learning process integrated with multiple representations consists of four stages: (1) understanding the problem. at this stage students are asked to read and understand issues about chemical equilibrium that contain macroscopic, submicroscopic, symbolic, and mathematic aspects; (2) devising a plan, at this stage students identify chemical equilibrium based on multiple representations and find a way to solve problems from the source of books; (3) carrying out the plan, at this stage students, answer all issues of chemical equilibrium in the worksheet which contains macroscopic, submicroscopic, symbolic, and mathematical aspects; (4) looking back, at this stage students were asked to review the answers and match them with the sourcebooks. the results showed that chemistry learning using problem-solving models with multiple representations made students more understand concepts. so, their cognitive learning outcomes and self-efficacy became high. this is in line with the results of domin & bodner's (2012) study which showed that the implementation of multiple representations in problemsolving could improve the understanding of learners so that they can solve problems in organic chemistry. indriyanti, saputro & sungkar (2020) implemented multiple representations in two learning models, namely the thinking aloud pair problem solving (tapps) model and the problem posing (pp) model. the multiple representations implemented in both models are multiple tetrahedral representations. learning activities in tapps and pp classes begin with teachers reminding learners of the topics learned last week. then the teacher gives illustrations about phenomena in the environment and provides a stimulus to calculate the number of particles that are very small and cannot be seen with the naked eye or microscope. furthermore, tapps and pp classes’ activities are different. in tapps classes, teachers give two problems to each group to solve. while in the pp class, each group is given the same two grids to build two issues (questions), then the completed questions are exchanged with other groups to be solved by that group. the results showed that tapps and pp classes had higher posttest scores than pretests, meaning multiple representations, especially tetrahedral representations implemented in learning, can improve the understanding concepts of learners in both classes. however, the understanding of the concept of learners in pp classes was higher than in the tapps model. it was because the atmosphere of tapps class discussion was passive and skewed one way compared to pp classes. the implementation of multiple representations in learning can be combined with cognitive dissonance strategies to reduce learners' misconceptions, as done by widarti, permanasari, mulyani, rokhim & habiddin (2021). cognitive dissonance is one of the strategies for creating cognitive conflict (lee et al., 2003). the learning syntax of cognitive dissonance strategies are: (1) invites the journal of science learning article doi: 10.17509/jsl.v5i2.42656 340 j.sci.learn.2022.5(2).334-341 initial knowledge of learners; (2) create cognitive dissonance; (3) implements new knowledge and feedback; (4) reflects; and (5) closing. cognitive dissonance strategies combined with multiple representations (macroscopic, submicroscopic, and symbolic) in images, curves, and videos can raise questions that motivate students to try to understand and find answers. so, there is understanding concepts change of learners towards clear understanding. the results showed that cognitive dissonance strategies combined with multiple representation-based learning could increase the mastery of learners' concept concepts. in addition, it can reduce misconceptions in learners. aside from being implemented in learning models or strategies, multiple representations are also developed into learning models. for example, multiple representation based learning (mrl), called simayang, is a learning model developed by sunyono in 2013. this model consists of four learning phases: orientation, explorationimagination, internalization, and evaluation (sunyono, yuanita & ibrahim, 2015). the results showed that the mrl model could improve the mastery concepts of learners and problem-solving abilities, especially for learners with medium and low initial abilities (sunyono & meristin, 2018). in addition, the mrl model also effectively optimizes learners' imagination skills so their ability to think and argue to solve problems can increase. based on the above, it can be observed that multiple representations have been implemented in several chemistry learning models or strategies such as inquiry, inquiry 5e, guided inquiry, problem solving, thinking aloud pair problem solving (tapps), problem posing (pp) and cognitive dissonance. multiple representations are also developed into learning models, namely multiple representation based learning (mrl) or the simayang learning model. the results of implementing multiple representations on such learning models or strategies mostly increased mastery or understanding of concepts of learners, and reviewed from the results of the pretestposttest, models or methods integrated with multiple representations mostly increased posttest value. it is indicated that each model or system combined with multiple representations effectively improves learners' mastery or understanding of concepts. conclusion based on studies from the articles obtained systematically, the definition of multiple representations refers to the involvement of both three levels of chemical representation and the tetrahedral representation of chemistry. it can also refer to various media, but these media are used to integrate chemical representations (macroscopic, submicroscopic, and symbolic). the involvement of multiple representations in chemical learning has a positive influence, including improving understanding of concepts, improving performance, reducing mental effort (cognitive load), improving selfefficacy, making cognitive structures develop for the better, and improving mental models, and can reduce misconceptions. based on the study, it was also obtained information that multiple representations have been implemented in several chemical learning models or strategies such as inquiry, inquiry 5e, guided inquiry, problem solving, thinking aloud pair problem solving (tapps), problem posing (pp), cognitive dissonance, and multiple representation based learning (mrl). the results showed that implementing multiple representations in some learning models or strategies effectively improved learners' mastery or understanding of concepts. based on the results of the article review that has been done, it is essential to involve multiple representations in chemical learning so that students learning outcomes can increase. teachers should involve multiple representations in chemistry learning because, through multiple representations, the abstractness of chemistry concepts will become more real and easier to understand for learners. as a result, their understanding of chemistry concepts can increase, and the impact of their learning outcomes can also increase. in addition, involving multiple representations in learning can reduce learners' assumptions about chemistry as a difficult subject. the suggestion for further research is to implement multiple representations on other models or strategies that have not been mentioned or make modifications for integrating multiple representations in the models or strategies that have been presented previously. future research may also implement mahaffy tetrahedral representations on models or strategies that still use johnstone multiple representations. references ainsworth, s. (2006). deft: a conceptual framework for considering learning with multiple representations. learning and instruction, 16(3), 183–198. https://doi.org/10.1016/j.learninstruc.2006.03.001 baptista, m., martins, i., conceição, t., & reis, p. (2019). multiple representations in the development of students’ cognitive structures about the saponification reaction. chemistry education research and practice, 20(4), 760–771. https://doi.org/10.1039/c9rp00018f. bridle, c. a., & yezierski, e. j. (2012). evidence for the effectiveness of inquiry-based, particulate-level instruction on conceptions of the particulate nature of matter. journal of chemical education, 89(2), 192– 198. https://doi.org/10.1021/ed100735u. chandrasegaran, a. l., treagust, d. f., & mocerino, m. (2007). the development of a two-tier multiple-choice diagnostic instrument for evaluating secondary school students’ ability to describe and explain chemical reactions using multiple levels of representation. chemistry education research and practice, 8(3), 293–307. https://doi.org/10.1039/b7rp90006f chang, r., & overby, j. (2011). general chemistry, 6th ed. new york: the mcgraw-hill companies, inc. effendy.(2016). ilmu kimia untuk siswa sma dan ma jilid 1a [chemistry for high school and ma students volume 1a]. malang : indonesia academic publishing. derman, a., & ebenezer, j. (2020). the effect of multiple representations of physical and chemical changes on the https://doi.org/10.1021/ed100735u journal of science learning article doi: 10.17509/jsl.v5i2.42656 341 j.sci.learn.2022.5(2).334-341 development of primary pre-service teachers cognitive structures. research in science education, 50(4), 1575–1601. https://doi.org/10.1007/s11165-018-9744-5. domin, d. & bodner, g. (2004). mental models: the role of representations in problem solving in chemistry. proceedings, university chemistry education, 4(1) : 24-30. domin, d., and bodner, g. (2012). using students’ representations constructed during problem solving to infer conceptual understanding. journal of chemical education, 89(7), 837–843. https://doi.org/10.1021/ed1006037. huddle, p. a., & pillay, a. e. (1996). an in-depth study of misconceptions in stoichiometry and chemical equilibrium at a south african university. journal of research in science teaching: the official journal of the national association for research in science teaching, 33(1), 65-77. indriyanti, n. y., saputro, s., & sungkar, r. l. (2020). problem-solving and problem-posing learning model enriched with the multiple representation in tetrahedral chemistry to enhance students’ conceptual understanding. edusains, 12(1), 123–134. https://doi.org/10.15408/es.v12i1.13282. johnstone, a. h. (1993). the development of chemistry teaching. journal of chemical education, 70(9), 701–705. kimberlin, s., & yezierski, e. (2016). effectiveness of inquiry-based lessons using particulate level models to develop high school students’ understanding of conceptual stoichiometry. journal of chemical education, 93(6), 1002–1009. https://doi.org/10.1021/acs.jchemed.5b01010. lee, g., kwon, j., park, s. s., kim, j. w., kwon, h. g., & park, h. k. (2003). development of an instrument for measuring cognitive conflict in secondary-level science classes. journal of research in science teaching, 40(6), 585–603. https://doi.org/10.1002/tea.10099. mahaffy, p. (2006). moving chemistry education into 3d: a tetrahedral metaphor for understanding chemistry: union carbide award for chemical education. journal of chemical education, 83(1), 49–55. https://doi.org/10.1021/ed083p49. milenković, d. d., segedinac, m. d., & hrin, t. n. (2014). increasing high school students’ chemistry performance and reducing cognitive load through an instructional strategy based on the interaction of multiple levels of knowledge representation. journal of chemical education, 91(9), 1409-1416. nakhleh, m. (1992). why some students don’t learn chemistry. journal of chemical education, 69 (3), 191−196. nugrahaeni, a., redhana, i. w., & kartawan, i. m. a. (2017). penerapan model pembelajaran discovery learning untuk meningkatkan kemampuan berpikir kritis dan hasil belajar kimia [application of discovery learning learning model to improve critical thinking skills and chemistry learning outcomes]. jurnal pendidikan kimia indonesia, 1(1), 23-29. pikoli, m. (2020). using guided inquiry learning with multiple representations to reduce misconceptions of chemistry teacher candidates on acid-base concept. international journal of active learning, 5(1), 1-10. priyasmika, r. (2021). the effect of multiple representation-based guided inquiry on learning outcomes reviewed from scientific thinking skills. educhemia (jurnal kimia dan pendidikan), 6(1), 5566. https://doi.org/10.30870/educhemia.v6i1.8985. rahayu, s., & kita, m. (2010). an analysis of indonesian and japanese students’ understandings of macroscopic and submicroscopic levels of representing matter and its changes. international journal of science and mathematics education, 8(4), 667–688. https://doi.org/10.1007/s10763-009-9180-0. santos, v. c., & arroio, a. (2016). the representational levels: influences and contributions to research in chemical education. journal of turkish science education (tused), 13(1). sirhan, g. (2007). learning difficulties in chemistry: an overview. journal of turkish science education, 4(2), 2−20. sunyono, s., & meristin, a. (2018). the effect of multiple representation-based learning (mrl) to increase students’ understanding of chemical bonding concepts. jurnal pendidikan ipa indonesia, 7(4), 399–406. https://doi.org/10.15294/jpii.v7i4.16219. sunyono, s., leny, y., & muslimin, i. (2015). supporting students in learning with multiple representation to improve student mental models on atomic structure concepts. science education international, 26(2), 104-125. supasorn, s. (2015). grade 12 students’ conceptual understanding and mental models of galvanic cells before and after learning by using small-scale experiments in conjunction with a model kit. chemistry education research and practice, 16(2), 393–407. https://doi.org/10.1039/c4rp00247d. tima, m. t., & sutrisno, h. (2018). influence of problem solving based on multiple representations model on teaching and learning of chemistry on student academic self-efficacy and students’s cognitive achievement. american journal of educational research, 6(7), 887-892. widarti, h. r., marfu’ah, s., & parlan. (2019). the effects of using multiple representations on prospective teachers’ conceptual understanding of intermolecular forces. journal of physics: conference series, 1227(1). https://doi.org/10.1088/17426596/1227/1/012006. widarti, h. r., permanasari, a., mulyani, s., rokhim, d. a., & habiddin, h. (2021). multiple representation-based learning through cognitive dissonance strategy to reduce student’s misconceptions in volumetric analysis. tem journal, 10(3), 1263– 1273. https://doi.org/10.18421/tem103-33. wiyarsi, a., sutrisno, h., & rohaeti, e. (2018). the effect of multiple representation approach on students’ creative thinking skills: a case of “rate of reaction” topic. journal of physics: conference series, 1097(1). https://doi.org/10.1088/1742-6596/1097/1/012054. xiao, y., & watson, m. (2019). guidance on conducting a systematic literature review. journal of planning education and research, 39(1), 93–112. https://doi.org/10.1177/0739456x17723971 https://doi.org/10.1007/s11165-018-9744-5 https://doi.org/10.1021/ed1006037 https://doi.org/10.30870/educhemia.v6i1.8985 https://doi.org/10.1088/1742-6596/1227/1/012006 https://doi.org/10.1088/1742-6596/1227/1/012006 https://doi.org/10.1088/1742-6596/1097/1/012054 a © 2022 indonesian society for science educator 469 j.sci.learn.2022.5(3).469-477 received: 13 january 2022 revised: 23 april 2022 published: 27 november 2022 online flipped classroom: developing postgraduate science education students’ critical thinking skills wahono widodo1* 1department of science education, faculty of mathematics and natural sciences, universitas negeri surabaya, surabaya, indonesia *corresponding author: wahonowidodo@unesa.ac.id abstract during the covid-19 pandemic, online learning should be carried out through innovative methods. this study aims to determine whether implementing an online flipped classroom, with converting all face-to-face sessions into virtual face-to-face sessions, can facilitate the development of critical thinking skills of postgraduate science education students in the school physics course. this quantitative study used one group pretest-postest research design. the subjects were postgraduate science education students who took school physics courses in the first semester at one of the public universities in indonesia. online flipped classroom was supported by google classroom learning management system (lms). critical thinking skills data in school physics were obtained through tests. data were analyzed using a description of normalized changes (c) and nonparametric-inferential analysis with the related-samples wilcoxon signed rank test. the results showed that the online flipped classroom could facilitate the development of the students’ critical thinking skills with an average normalized change of .71 in a high category. even though the students felt the benefits of an online flipped classroom, they still wanted a face-to-face learning mode to be carried out soon. keywords online flipped classroom, critical thinking skills, school physics, postgraduate science education students. 1. introduction higher education is responsible for developing students’ critical thinking skills (bassham, irwin, nardone, & wallace, 2013). necessary thinking skills are essential in coping with academic or professional interests (costa & kallick, 2014; kraisuth & panjakajornsak, 2017) and become an indicator of lifelong learning (oecd, 2018). postgraduate students majoring in science education are prepared to become academics who can later act as researchers, lecturers, educational consultants, or teacher advisors. as shown in the overview of the university of british columbia's master of education in science education (med), the postgraduate interest is a wide range of research and professional interests in the stem education field (ubc, 2020). in addition, moeti, mgawi & waitshega (2017) and ashour & hawamdeh (2012) emphasize that postgraduates need to perceive critical thinking skills to be professional researchers or lecturers. the learning process conducted in science education major at the universitas negeri surabaya postgraduate program focuses on developing critical thinking, creative thinking and problem solving, communication, and collaboration skills (s2pendsains, 2020). physics is a way to understand our world and recognize the underlying principles and laws that connect to the disparate phenomena of our physical world, which is full of content and meaning (rau, 2017). it is an authentic, essential, logical, and rational science that underlies the development of science and technology. in connection with its materials, problem-solving and decision-making skills can be carried out through some stages of a thinking process, which directly promote critical thinking skills and conceptual understanding of physics (magno, 2010). critical thinking requires students to be thorough, purposeful, and deliberate, focus on the main issue, and evaluate all parts of its complex and challenging claims and arguments (sasson, yehuda, & malkinson, 2018). in a scientific context, the evaluation aspect involves interpreting data, drawing accurate conclusions from the data, comparing and evaluating models and data, estimating methods, and deciding how to proceed in an investigation (walsh, quinn, wieman, & holmes, 2019). in a science classroom, students' critical thinking can be identified by their ability to respond to daily life problems (suhirman, prayogi, & asy‘ari, 2021). journal of science learning article doi: 10.17509/jsl.v5i3.43107 470 j.sci.learn.2022.5(3).469-477 regardless of the education level, the learning process is currently required to adapt to the conditions of the covid-19 pandemic. the covid-19 outbreak was first detected in wuhan city, china, in december 2019 and was declared a pandemic by who on march 11, 2020. in education, various countries have responded to adjusting the learning process according to the conditions of their respective countries. in indonesia, the learning process is conducted from home, based on the letter of the minister of education and culture of the republic of indonesia number 4 of 2020 concerning the implementation of education policies in an emergency for the spread of coronavirus disease (covid-19). in higher education, the home learning process is carried out online. online lectures have been practiced for a long time, but during the covid19 pandemic, this method has been compulsory in many countries. kuntarto (2017) states that online learning is more effective and able to increase students’ mastery of subject material compared to the use of a face-to-face learning mode. the online lecture system positively contributes to encouraging disparities in the quality of higher education in indonesia (mustofa, chodzirin, & sayekti, 2019). rosdiana, widodo, nurita & fauziah (2018) state that learning vibrations and waves through online learning improves science teacher prospectives’ problem-solving and graphing skills. however, various obstacles arise when a learning process is carried out entirely online without a face-to-face presence for a long time. the most prominent obstacle is that learning activities are too boring for students. livana, mubin, & basthomi (2020) state that 55.8% of students complained that online learning was boring. cao et al. (2020) also convey that academic activity delays were positively related to student anxiety during the covid-19 pandemic. the cause of students’ stress might involve academic problems, interpersonal problems, learning activities, social relationships, encouragement and desire, and group activity (yusoff & rahim, 2010). on the other hand, it was also found that, in general, assignments did not have a significant effect on student stress levels (mardiati, hidayatullah, & aminoto, 2018). therefore, the idea of learning to bridge the boredom of online lectures, appropriate assignments, and adequate results is necessary when the situation demands a complete online learning process. therefore, one of the ideas applied in this study is an online flipped classroom. the flipped classroom is a learning approach that uses an online platform with flexible time (karabulut-ilgu, cherrez, & jahren, 2018). in its application, students complete several online activities to prepare the face-toface learning on campus. it is dissimilar to the traditional lecture, where a professor must present new materials (reidsema, kavanagh, hadgraft, & smith (eds), 2017). instead, in a flipped classroom, instructors provide online materials for students to read, watch, outline, summarize, understand, and present on their own so that the classical method can be allocated to student-centered learning activities (baytiyeh, 2017). although studies on flipped classrooms are still required, most publications (e.g., peerreviewed) describe it as a type of blended learning (abeysekera & dawson 2015). the application of flipped classrooms in lectures has several advantages, for example, offering flexibility, increasing interaction between lecturers and students as well as students and students, and developing professional skills (karabulut-ilgu, cherrez, & jahren, 2018). baepler, walker, & driessen (2014) found that students who learned using flipped classroom model had better performance than those who knew with the traditional model. the application of flipped classrooms is considered appropriate because students can study literature or lecture assignments efficiently and flexibly. they also can discuss the materials or tasks face-to-face. marlowe (2012) shows that flipped classrooms could reduce students’ stress levels and increase their semester scores. however, the nature of flipped classrooms cannot be applied in many countries due to the existing covid-19 pandemic. in the flipped classroom, there is a face-to-face session. however, during the covid-19 pandemic, many countries still do not allow face-to-face meetings. therefore, the face-to-face session in flipped classrooms is replaced by face-to-face online sessions called online flipped classroom. for example, reidsema, kavanagh, hadgraft, & smith (eds), (2017) have not included face-to-face online ideas in the flipped classroom section in books on flipped classrooms for higher education. to the researchers’ knowledge, there is little research on an online flipped classroom. weinhandl, lavicza, & houghton (2020) conducted a study about an online flipped classroom for mathematics class, which is different from the present study's focus. smith (2020) also promoted online, face-to-face sessions that showed effective learning times and better student performance. therefore, implementing a fully online flipped classroom with a duration of one semester in physics lectures to facilitate the development of critical thinking skills is worth researching. thus, this study aims to determine whether the implementation of the online flipped classroom in school physics lectures can develop science education postgraduate students' critical thinking skills. 2. method the study was pre-experimental, using one group pretest-posttest design. this design in table 1 let the participants be measured on their outcomes before and after the treatment (mertens, 2010). the research subjects comprised 13 students who took the school physics course for the academic year of 2020/2021 at the science education postgraduate program at one public university in east java province, indonesia. journal of science learning article doi: 10.17509/jsl.v5i3.43107 471 j.sci.learn.2022.5(3).469-477 regarding gender, the ratio of male and female students was 5:8. the treatment in this study was using an online flipped classroom with school physics materials. the treatment encompassed several learning strategies: learning the material, conducting investigations, creating presentation files, carrying out an online face-to-face discussion, and accomplishing assignments. the response variable of the present study was critical thinking skills in school physics, defined as the student's skills in explaining, making arguments, and assessing the truth shown by the critical thinking skills test score in school physics. the researcher hypothesized that the treatment was effective, so the outcome scores improved. on the contrary, the treatment had no effect when the scores did not change. however, the researchers did not claim that the online flipped classroom was the best strategy among other strategies for online learning. the online flipped classroom strategy was implemented in the lecture process by changing lectures from face-toface to online, according to the flow chart in figure 1. in the online flipped classroom research, learning began with introductory sessions carried out synchronously. based on the introduction, students conducted inquiries and searched for learning resources to make presentation files on certain topics according to the results of the distributed tasks. the lecturer gave feedback on the uploaded presentation files, and then the student corrected them according to the lecturer's suggestion. afterward, face-toface online learning was conducted where students presented their work while the other students responded. lecturers provided reinforcement, clarification, and further suggestions if necessary. this online face-to-face was continued with the task of analyzing, making explanations, and initiating arguments related to potential misconceptions according to physics topics discussed asynchronously. these activities were carried out using google classroom learning management system (lms). this lms was chosen because it was easy, quickly accessible, and integrated with various hardware owned by students. students conducted studies encompassing concepts, principles/laws along with their applications, and potential misconceptions on the topics, including 1) kinematics, 2) dynamics, 3) heat and thermodynamics, 4) waves, sound, and optics, 5) electricity, 6) magnetism and electromagnetic induction, and 7) earth and space. the inquiry activity in this study was an inquiry designed independently by students according to the topic using existing equipment and materials according to the context and virtual laboratories. students were required to formulate problems, hypotheses, and methods to test hypotheses, present data, analyze data and draw conclusions. thus, students were expected to develop their abilities in building arguments based on deductive and inductive reasoning. a rubric assessed this activity that two experts on science education had validated. the activities of lecturers and students were observed based on the activities at lms by detailing the personalization o f lecturers and students in the pre-class and in-class phases based on koh's (2019) idea, however, focusing on various activities. critical thinking skills were measured through a critical thinking skills test in school physics. the test instrument used consisted of 11 questions on critical thinking skills, of which the indicators of essential skills of thinking referred to ennis (2011), namely: 1) making arguments equipped with mathematical models based on deductive analysis of physical symptoms; 2) making arguments equipped with a mathematical model based on an inductive analysis of physical symptoms; 3) explaining the statement of principles/laws of physics, and 4) assessing the truth of the statement accompanied by arguments based on the concepts/principles/laws of physics. the test questions consisted of a series of question formulation according to the indicators, internal reviews, revisions, and expert validation and revision. pre-test questions differed from post-test ones; however, both are equivalent. in addition, science education experts validated the content, construction, and language use questions. table 1 experiment design o1 x o2 initial measurement (pretest) of critical thinking skills related to school physics the treatment given using an online flipped classroom in school physics course the final measurement (posttest) of critical thinking skills related to school physics figure 1 the implementation of the online flipped classroom used in the present study ntroduction to ectures synchronous) inquiry, information retrieval, and file presentation (asynchronous) revision of the presented file based on lecturer’s feedback asynchronous) presentation and discussion (synchronous) the further assignments: conducting analysis, making explanations, and initiating arguments related to potential misconceptions according to the topics discussed (asynchronous) journal of science learning article doi: 10.17509/jsl.v5i3.43107 472 j.sci.learn.2022.5(3).469-477 this study also used a questionnaire to reveal students’ responses to the school physics learning process with an online flipped classroom strategy. this response was mandatory to obtain an overview of what science education graduate students experienced and felt regarding the learning process using the online flipped classroom strategy. in addition, the response was essential to get feedback that could be improved in the future. the questionnaires were administered online through the lms. this study used descriptive and inferential data analysis techniques. descriptive analysis was a description of increasing critical thinking skills using the normalized change (c) equation proposed by marx & cummings (2007) (see table 2). sriyansyah & azhari (2017) also suggested normalized change analysis compared to normalized gains. the inferential analysis was a median difference between the post-test and pre-test scores using nonparametric-inferential analysis of the related-samples wilcoxon signed rank test (siebert & siebert, 2018). this table 2 equations and criteria for normalized change score conditions equation criteria c value category postest < pretest pre prepost c − = negative decline postest = pretest 0 no improvement postest > pretest pre prepost c − − = 100 0 < c  0.3 low 0.7 < c  0.7 moderate 0.7 < c  1 high table 3 implementation of online filpped classroom meeting activities lecturer activities student activities 1 explanation of the lesson plan in one semester ahead and pretest, how to learn, assignments, and assessments providing information, relevant learning resources, and assignments listening, reading, answering, conducting collaborative groups 2 independent tasks: inquiry with a virtual laboratory, collaboratively online, making reports monitoring, providing assistance working on assignments collaboratively, consulting 3 synchronous discussion of the results of inquiry, assignment to conduct studies according to the selected topic listening, providing feedback, giving follow-up assignments conducting presentations and discussions, carrying out follow-up assignments 3-4 independent assignments: students collaboratively conducted studies including concept maps, conceptual descriptions, principles/laws, applications, potential misconceptions, misconceptions remediation ideas, and inquiry ideas on an agreed topic. the results of the study were in the form of presentation file. monitoring, providing assistance providing feedback on the results of the study working on assignments collaboratively, consulting and making revisions 5 students made video presentations monitoring, providing assistance. working on assignments collaboratively, consulting 6-14 presentation and discussion synchronously and asynchronously about the study results performed by each group providing feedback, reinforcement, and clarification, if necessary, as well as giving the task of making explanations and arguments related to the conceptions of the topic carrying out further tasks (applications in problem solving and building arguments), consulting 15 wrap-up summing up a review, carrying out reinforcement, and giving insight for the betterment clarifying 16 postest and questionnaire completion sending posttest and questionnaire answering journal of science learning article doi: 10.17509/jsl.v5i3.43107 473 j.sci.learn.2022.5(3).469-477 analysis was performed to determine whether the difference in pre-test and post-test scores was significant. because of the small data, the researcher could not assume that the subjects were normally distributed and homogeneous, so a nonparametric analysis was undertaken. the significance level was .05. the inferential analysis was carried out using spss version 26.0. 3. result and discussion table 3 shows the online flipped classroom implemented in the first semester of 2020/2021. table 3 depicts that the online flipped classroom took place with students who conducted inquiries and studies first, followed by virtual face-to-face presentations and discussions. then, the activity was continued with a followup assignment. this pattern followed the flipped classroom strategy according to bergmann & sams (2012), reidsema, kavanagh, hadgraft, & smith (eds), (2017), and baytiyeh (2017), but every learning stage was conducted online, named an online flipped classroom. figure 2 shows an example of the online flipped classroom along with its example of the results of students’ collective assignments in lms. critical thinking aspects related to providing explanations, making deductive arguments, and making arguments inductively were developed through online inquiry activities in the form of investigating gas simulations in closed spaces. table 4 portrays the results of the student inquiry report data analysis. table 4 conveys that, on average, students obtained online inquiry results of 97 with a standard deviation of 4.9, which were in a very good category. the point that was emphasized during the online discussion was analyzing data. some students took logical leaps in data analysis by deductively mentioning specific laws. supposedly, in the data analysis phase, students formulated a mathematical model based on data trends (e.g., graphs). that was also to develop the ability to make arguments based on students' inductive reasoning. table 5 shows the results of the descriptive analysis of the pre-test, post-test, and c values for students' critical thinking skills. figure 2 examples of online flipped classroom in lms journal of science learning article doi: 10.17509/jsl.v5i3.43107 474 j.sci.learn.2022.5(3).469-477 based on table 5, it was found that there was an increase in students’ critical thinking skills, which was indicated by the elevating average pre-test score of 42 to 83 in the posttest. such improvement was categorized in a high category, with the average normalized change (c) of 0.71. a difference test in pre-test and post-test scores was conducted to determine whether online flipped classroom learning significantly affected students' critical thinking skills (see table 6). in addition, hypothesis testing of the median difference between pre-test and postest scores was carried out using related-samples wilcoxon signed rank test. the hypotheses were drawn as follows: ho: there is no median difference between pre-test and post-test scores. ha: there is a median difference between pre-test and post-test scores. table 6 shows that ho was rejected, and ha was accepted. thus, the post-test and pre-test scores for critical thinking were significantly different. these results reinforced the findings of he et al. (2017), who claim that flipped classrooms in architectural physics produced better students’ learning performances and abilities than the traditional classes. this study showed deeper learning in conducting investigations, browsing the literature, formulating presentation files, and discussing synchronously in the virtual face-to-face phase. the strength of the online flipped classroom to facilitate students to search various literature and conduct collaborative online investigations was also depicted by sun, wu & lee (2017). sun, wu & lee. (2017) argued that the flipped classroom model promoted a learning environment that prompted its learners to seek external table 4 inquiry results no inquiry aspect mean score standard deviation interpretation errors that occur 1 formulating a problem 100 0 very good 2 formulating a hypothesis 98 6.9 very good incomplete hypothesis 3 designing a hypothesis testing steps 100 0 very good 4 presenting data 100 0 very good 5 analyzing data 88 13.0 very good inductive analysis mixed with deductive reasoning (not based on data) 6 drawing conclusion 96 9.4 very good not all conclusions were supported by data overall average 97 4.9 very good table 5 descriptive analysis of students’ critical thinking skills no pretest posttest c values improvement criteria 1 45 82 0.67 medium 2 44 85 0.73 high 3 38 86 0.78 high 4 41 79 0.64 medium 5 56 87 0.70 high 6 24 82 0.76 high 7 31 76 0.65 medium 8 48 82 0.65 medium 9 38 84 0.74 high 10 46 86 0.75 high 11 51 83 0.65 medium 12 38 81 0.70 medium 13 47 88 0.76 high average 42 83 0.71 high standard deviation 8.43 3.31 0.05 table 6 hypothesis test summary null hypothesis test sig. decision the median of differences between pretest and postest equals 0. related-samples wilcoxon signed rank test .001 reject the null hypothesis. asymptotic significances are displayed. p < .05 journal of science learning article doi: 10.17509/jsl.v5i3.43107 475 j.sci.learn.2022.5(3).469-477 helps proactively. that was consistent with what one student said, "... i think it is exciting. honestly, this is my first time with a system like this implemented. the lecture was unique and challenging, making me have to search various sources, question ideas in my mind, and discuss with friends… ”. from this statement, the online flipped classroom allowed students to self-regulate, foster selfefficacy, and increase their learning outcomes, of which chen & hwang's (2019) study also revealed similar findings. in addition, the lecturers believed that the constructive feedbacks were helpful for the students. "in my opinion, synchronous face-to-face lectures let me understand the material discussed easily because, when the discussion is over, the lecturer still provides the explanation and restrengthens any misconceptions. that is attractive to me". that means that this online flipped classroom provided opportunities for lecturers to provide feedback to reinforce students’ motivation (tricomi & depasque, 2016). zainuddin, haruna, li, zhang, & chu (2019) emphasized that the flipped classroom yielded positive learning outcomes in students' learning activities, such as learning motivation and engagement, social interaction, and self-directed learning skills. figure 3 shows radar score diagram based on critical thinking indicators indicator 1: making arguments supported by a mathematical model based on deductive analysis of the physical context; indicator 2: making arguments supported by a mathematical model based on an inductive analysis of the physical context; indicator 3: explaining the statement based on the principles/laws of science-physics; indicator 4: assessing the validity of statements that were strengthened by arguments following figure 3, each average score on indicators of critical thinking portrayed an increase. normalized change (c) to arguments supported with a mathematical model based on a deductive and inductive analysis of physical phenomena was in a high category. that indicated that providing opportunities for students to carry out activities to formulate problems, generate hypotheses, conduct investigations to obtain data, analyze data, and conclude inquiry activities had an impact on the ability to build arguments accompanied by inductive mathematical models. this fact shows that implementing online learning with flipped classrooms could give students scientific investigation skills. these results were in line with a study by tan, yangco, & que (2020), which revealed that online science learning with flipped classrooms could improve students' science process skills and conceptual understanding. marshall & kotska (2020) also said that online learning with flipped classrooms strengthened teachers' presence virtually in a class. in other words, when students worked on assignments monitored by lecturers online at lms, students felt the presence of lecturers. many relevant studies stated that deductive thinking skills required continuous practice (eglington & kang, 2018; carriera, amando, & jacinto, 2020). this study portrayed that tracing learning resources, making presentation files, and discussing them facilitated the students a lot in training their critical thinking. moreover, mclean, attardi, faden, & goldszmidt (2016) showed that flipped classrooms produced better learning processes and academic performance (romero-garcía, buzón-garcía, & touron, 2018). although the online flipped classroom facilitated students’ learning activities, lecturers’ feedback, and increased critical thinking skills, students still preferred offline face-to-face meetings. some students said, “... it is better face-to-face; eye fatigue; unstable network constraints; extravagant; although flexible, the combination with face to face is better; can't feel interpersonal relationship... ”. therefore, in the context of online learning, it was still in line with susanti, hidayati, anggreiny & maputra, (2020), who claimed that 83.6% of students stated online learning in the last semester was not effective because of difficulty in understanding learning materials, neglected task, boring, and no teacher’s explanation. troublesome signals, not optimal face-to-face learning, overload task, low economy, challenging to ask questions, and less attention to grades. irawan, dwisona & lestari (2020) showed that students started to get bored during online learning after the first two weeks of the learning process. although the online flipped classroom implementation showed that students wanted some offline, face-to-face activities (to feel good, not to get tired, and not to have the feeling of interpersonal relationships), they showed good learning performance. they produced better critical thinking skills in school physics. figure 3 radar score diagram based on critical thinking indicators 0 20 40 60 80 100 1 2 3 4 pretest postest journal of science learning article doi: 10.17509/jsl.v5i3.43107 476 j.sci.learn.2022.5(3).469-477 4. conclusion the online flipped classroom can facilitate the development of science education postgraduate students’ critical thinking skills in the school physics course. based on the pre-test and post-test analysis, it is found that there is a significant median difference between the pre-test and post-test scores, of which the average normalized change of 0.71 is in a high category. developing these skills through collaborative online inquiry activities could work on analyzing concepts, principles, potential misconceptions, and ideas to overcome the misconceptions, face-to-face online discussions, and follow-up assignments. at each stage, there are reinforcement and feedback. if online learning is still a must, for example, because of a pandemic, then an online flipped classroom is worth choosing. this study, finally, suggests that different materials and subjects in online flipped classrooms might need to be undertaken references abeysekera, l., & dawson, p. (2015). motivation and cognitive load in the flipped classroom: definition, rationale and a call for research. higher education research & development, 34(1), 1–14. https://doi.org/10.1080/07294360.2014.934336. ashour, r. q. & hawamdeh, m. f. (2012). critical thinking skills among postgraduate university students in jordan. journal of institutional research south east asia, 10(2), 16-28. baepler, p., walker, j.d., & driessen, m. (2014). it's not about seat time: blending, flipping, and efficiency in active learning classrooms. computers & education, 78, 227–236. https://psycnet.apa.org/doi/10.1016/j.compedu.2014.06.006. bassham, g., irwin, w., nardone, h., & wallace, j. m. (2013). critical thinking: a student's introduction (5th ed.). new york: mcgraw-hill. baytiyeh, h. (2017). the flipped classroom model: when technology enhances professional skills. the international journal of information and learning technology, 34(1), 51-62. https://doi.org/10.1108/ijilt-07-2016-0025. bergmann, j. & sams, a. (2012). flip your classroom: reach every student in every class every day, washington dc: international society for technology in education. cao, w., fang, z., hou, g., han, m., xu, x., dong, j., & zheng, j. (2020). the psychological impact of the covid-19 epidemic on college student in china. psychiatri research. 287, 112934. https://doi.org/10.1016/j.psychres.2020.112934. carriera, s., amando, n., & jacinto, h. (2020). venues for analytical reasoning problems: how children produce deductive reasoning. educ. sci., 10. 169, 1-23. https://doi.org/10.3390/educsci10060169. chen, p.y. & hwang, g.j. (2019). an irs-facilitated collective issuequest approach to enhancing students’ learning achievement, selfregulation and collective efficacy in flipped classrooms. british journal of educational technology, 50(4), 1996–2013. https://doi.org/10.1111/bjet.12690. costa, a., & kallick, b. (2014). dispositions: reframing teaching and learning. thousand oaks, ca: corwin press. eglington, l. g. & kang, s. h. k. (2018). retrieval practice benefits deductive inference educational psychology review, 30(1), 215-228. https://doi.org/10.1007/s10648-016-9386-y. ennis, r. h. (2011). the nature of critical thinking: an outline of critical thinking dispositions and abilities. university of illinois. https://education.illinois.edu/docs/default-source/facultydocuments/robertennis/thenatureofcriticalthinking_51711_000.pdf. irawan, a.w., dwisona, & lestari, m. (2020). psychological impacts of students on online learning during the pandemic covid-19. konseli, 07(1), 53-60. https://doi.org/10.24042/kons.v7i1.6389. karabulut-ilgu, aliye, cherrez, & n.j., jahren, c.t. (2018). a systematic review of research on the flipped learning method in engineering education. british journal of techology, 49(3), 398-411. https://doi.org/10.1111/bjet.12548. koh, j. h. l. (2019). four pedagogical dimensions for understanding flipped classroom practices in higher education: a systematic review. educational sciences: theory and practice, 19(4), 14 33. https://doi.org/10.12738/estp.2019.4.002 . kraisuth, d., & panjakajornsak, v. (2017). thai engineer asean readiness: a structural equation model analysis. asia-pacific social science review, 16(3), 96-117. https://doi.org/10.1177/2158244017745346. kuntarto, e. (2017). keefektifan model pembelajaran daring dalam perkuliahan bahasa indonesia di perguruan tinggi [the effectiveness of the online learning model in indonesian language lectures at universities]. jile, 3(1), 99-110. http://dx.doi.org/10.24235/ileal.v3i1.1820. livana, p.h., mubin, m.f. & basthomi. y. (2020). tugas pembelajaran penyebab stress mahasiswa selama pandemi covid-19 [learning tasks cause student stress during the covid-19 pandemic]. jurnal ilmu keperawatan jiwa, 3(2), 203-208. magno, c. (2010). the role of metacognitive skills in developing critical thinking. metacognition learning 5, 137–156. https://doi.org/10.1007/s11409-010-9054-4. mardiati, i., hidayatullah, f. & aminoto, c. (2018). faktor eksternal tingkat stres mahasiswa keperawatan dalam adaptasi proses pembelajaran [external factors of nursing students' stress levels in adapting the learning process], 7th univ. res. colloqium 2018 stikes pku muhammadiyah surakarta, pp. 172–181. marlowe, c. (2012). the effect of the flipped classroom on student achievement and stress. unpublished master’s thesis, montana state university. marshall, h.w. & kostka, i. (2020). fostering teaching presence through the synchronous online flipped learning approach. the electronic journal for english as a second language, 24(2), 1-14. marx, j. d. & cummings, k. (2007). normalized change. american journal of physics 75(1), 88-91. https://doi.org/10.1119/1.2372468. mclean, s., attardi, s.m., faden, l., & goldszmidt, m. (2016). flipped classrooms and student learning: not just surface gains. adv physiol educ, 40(1), 47-55. https://doi.org/10.1152/advan.00098.2015. mertens, d. m. (2010). research and evaluation in education and psychology : integrating diversity with quantitative, qualitative, and mixed methods 3rd ed. california: sage publications. moeti, b., mgawi, r. k., & waitshega. (2017). critical thinking among post-graduate diploma in education students in higher education: reality or fuss? journal of education and learning, 6(2), 13-24. https://doi.org/10.5539/jel.v6n2p13. mustofa, m. i., chodzirin, m., dan sayekti, l. (2019). formulasi model perkuliahan daring sebagai upaya menekan disparitas kualitas perguruan tinggi [the formulation of the online lecture model as an effort to reduce the disparity in the quality of higher education]. walisongo journal of information technology, 1(2), 151-160. https://doi.org/10.21580/wjit.2019.1.2.4067. oecd (2018). the future of education and skills education 2030. http://www.oecd.org/education/2030/e2030%20position%20p aper%20(05.04.2018).pdf. rau, a. r. p. (2017). what is physics? the individual and the universal, and seeing past the noise. physics essays, 30(1), 60-69. https://doi.org/10.4006/0836-1398-30.1.60. reidsema, c., kavanagh, l., hadgraft, r., & smith, n (eds). (2017). the flipped classroom. singapore: springer. romero-garcía, c., buzón-garcía, o., & touron, j. (2018). the flipped learning model in online education for secondary teachers, journal of technology and science education, 9(2), 109-121. https://doi.org/10.3926/jotse.435. https://doi.org/10.1080/07294360.2014.934336 https://psycnet.apa.org/doi/10.1016/j.compedu.2014.06.006 https://doi.org/10.1108/ijilt-07-2016-0025 https://doi.org/10.1016/j.psychres.2020.112934 https://doi.org/10.3390/educsci10060169 https://doi.org/10.1111/bjet.12690 https://doi.org/10.1007/s10648-016-9386-y https://education.illinois.edu/docs/default-source/faculty-documents/robert-ennis/thenatureofcriticalthinking_51711_000.pdf https://education.illinois.edu/docs/default-source/faculty-documents/robert-ennis/thenatureofcriticalthinking_51711_000.pdf https://education.illinois.edu/docs/default-source/faculty-documents/robert-ennis/thenatureofcriticalthinking_51711_000.pdf https://doi.org/10.24042/kons.v7i1.6389 https://doi.org/10.1111/bjet.12548 https://doi.org/10.12738/estp.2019.4.002 https://doi.org/10.1177%2f2158244017745346 http://dx.doi.org/10.24235/ileal.v3i1.1820 https://doi.org/10.1007/s11409-010-9054-4 https://doi.org/10.1119/1.2372468 https://doi.org/10.1152/advan.00098.2015 https://doi.org/10.5539/jel.v6n2p13 https://doi.org/10.21580/wjit.2019.1.2.4067 http://www.oecd.org/education/2030/e2030%20position%20paper%20(05.04.2018).pdf http://www.oecd.org/education/2030/e2030%20position%20paper%20(05.04.2018).pdf https://doi.org/10.4006/0836-1398-30.1.60 https://doi.org/10.3926/jotse.435 journal of science learning article doi: 10.17509/jsl.v5i3.43107 477 j.sci.learn.2022.5(3).469-477 rosdiana, l., widodo, w., nurita, t., & fauziah, a. (2018). e-learning task analysis making temporal evolution graphics on symptoms of waves and the ability to solve problems. international conference on science education (icosed). https://doi.org/10.1088/17426596/1006/1/012025. s2pendsains. (2020, 12 20). profil singkat magister pendidikan sains [brief profile of master of science education]. http://s2pendsains.pasca.unesa.ac.id/post/profil-singkat. sasson, i., yehuda, i. & malkinson n. (2018). fostering the skills of critical thinking and question-posing in a project-based learning environment. thinking skills and creativity, 29, 203-212. https://doi.org/10.1016/j.tsc.2018.08.001. siebert, c. f. & siebert, d. c. (2018). data analysis with small samples and non-normal data: nonparametrics and other strategies. new york: oxford university press. smith, k.d. (2020), "is it face time or structure and accountability that matter? moving from a flipped to a flipped/hybrid classroom", journal of applied research in higher education, vol. ahead-of-print no. ahead-of-print. https://doi.org/10.1108/jarhe-08-20190229. sriyansyah, s.p. & azhari, d. (2017). addressing an undergraduate research issue about normalized change for critical thinking test. jpii, 6(1), 138-144. doi: https://doi.org/10.15294/jpii.v6i1.9602. suhirman, prayogi, s., & asy‘ari, m. 2021). problem-based learning with character-emphasis and naturalist intelligence: examining students critical thinking and curiosity. international journal of instruction, 14(2), 217-232. https://doi.org/10.29333/iji.2021.14213a. sun, j.c.y., wu, y.t., & lee, w. (2017). the effect of the flipped classroom approach to open courseware instruction on students’ self-regulation. british journal of educational, 48(3), 713-729. https://doi.org/10.1111/bjet.12444. susanti, m., hidayati, i., anggreiny, n., & maputra, y. (2020). school from home during covid-19 pandemic, a descriptive study: effectivity of learning towards high school students in west sumatra. international conference of psychology, kne social sciences, 430– 445. https://doi.org/10.18502/kss.v4i15.8231. tan, r. m., yangco, r. t., & que, e. n. 2020). students’ conceptual understanding and science process skills in an inquiry-based flipped classroom environment. malaysian journal of learning & instruction, 17(1), 159-184. https://doi.org/10.32890/mjli2020.17.1.7 tricomi, e. & depasque, s. (2016), "the role of feedback in learning and motivation", recent developments in neuroscience research on human motivation (advances in motivation and achievement, vol. 19), emerald group publishing limited, bingley, pp. 175-202. https://doi.org/10.1108/s0749-742320160000019015. ubc (2020, 12th 20). master of education in science education (med). retrieved january 3 2021 from graduate and postdoctoral studiesgraduate school: https://www.grad.ubc.ca/prospectivestudents/graduate-degree-programs/master-of-education-scienceeducation. walsh, c., quinn, k.n., wieman, c., & holmes, n.g. (2019). quantifying critical thinking: development and validation of the physics lab inventory of critical thinking. phys. rev. phys. educ. res., 15, 010135, 1-17. https://doi.org/10.1103/physrevphyseducres.15.010135. weinhandl, r., lavicza, z., & houghton, t. (2020). designing online learning environments for flipped approaches in professional mathematics teacher development. journal of information technology education: research. 19. 315-337. https://doi.org/10.28945/4573. yusoff, m.s.b, & rahim, a.f.a. (2010). the medical student stressor quistionnaire (mssq) manual. kota bharu: kkmed publication. zainuddin, z., haruna, h., li, x., zhang, y. & chu, s.k.w. (2019). a systematic review of flipped classroom empirical evidence from different fields: what are the gaps and future trends? on the horizon, 27(2), 72-86. https://doi.org/10.1108/oth-09-2018-0027. https://doi.org/10.1088/1742-6596/1006/1/012025 https://doi.org/10.1088/1742-6596/1006/1/012025 http://s2pendsains.pasca.unesa.ac.id/post/profil-singkat https://doi.org/10.1016/j.tsc.2018.08.001 https://www.emerald.com/insight/search?q=kara%20diane%20smith https://www.emerald.com/insight/publication/issn/2050-7003 https://doi.org/10.1108/jarhe-08-2019-0229 https://doi.org/10.1108/jarhe-08-2019-0229 https://doi.org/10.15294/jpii.v6i1.9602 https://doi.org/10.1111/bjet.12444 https://doi.org/10.18502/kss.v4i15.8231 https://doi.org/10.32890/mjli2020.17.1.7 https://doi.org/10.1108/s0749-742320160000019015 https://doi.org/10.1103/physrevphyseducres.15.010135 https://doi.org/10.28945/4573 https://doi.org/10.1108/oth-09-2018-0027 a © 2019 indonesian society for science educator 19 j.sci.learn.2019.3(1).19-28 received: 17 june 2019 revised: 30 october 2019 published: 28 november 2019 exploring view of nature of science and technology pre-service chemistry teachers devita marlina venessa1*, hernani2, heli siti halimatul2 1masters program on chemistry education, postgraduate school, universitas pendidikan indonesia, bandung, indonesia 2department of chemistry education, faculty of mathematics and science education, universitas pendidikan indonesia, bandung, indonesia *corresponding author. dy2psihite@gmail.com abstract this study aims to explore the views of pre-service chemistry teachers on the nature of science and technology. pre-service chemistry teachers’ view of nature of science and technology (vnost) is very important to help their students later in understanding the concepts of science. this research is the initial stage in didactical design research involving 48 pre-service chemistry teachers from sriwijaya university. the descriptive method is used to explain the results of the analysis of pre-service chemistry teachers’ vnost. student views were assessed using the vnost questionnaire, consisting of 8 questions and guided by the discourse of ionic liquid technology. data collection uses a survey containing the views of pre-service chemistry teachers on vnost and where each statement grouped according to three categories, namely "realistic" (r), "has merit" (hm), and naïve (n). the results showed that in general, students have the view of has merit (hm), or the belief that they chose is not entirely correct even though there are parts of statements that are still by the general view of science. the results of this student vnost exploration are the basis for developing learning designs that are oriented to the realist answers of each question so that learning designs are produced based on aspects of the nature of science and technology. keywords view of nature of science and technology, pre-service chemistry teachers, nature of science, kind of technology 1. introduction science and technology are an influential part of developing scientific literacy abilities. science literacy consists of several general dimensions, namely the nature of science, the kind of scientific knowledge, scientific concepts, scientific principles, and theories related to science (shwartz, ben-zvi, & hofstein, 2005). from some of the general dimensions of scientific literacy, it is widely believed that if a science teacher does not understand the nature of science and technology (nost), it will be challenging for them to assist their students in gaining a good understanding of scientific concepts (murcia & schibeci, 1999; tairab, 2001; ayvaci, & ozbek, 2019). science teachers' knowledge of the nature of science and technology is fundamental to prepare students to be able to participate in society with the development of science that is continuously changing scientifically and technologyoriented (tairab, 2001). besides, science teachers are also expected to be able to influence students in making the right decisions about the problems faced, especially in explaining phenomena related to science and technology (rotherham, & willingham, 2010; tairab, 2001). the importance of science teachers' understanding of nost is related to the low results of the scientific literacy of students in indonesia. based on pisa data, measurements of the level of scientific literacy of indonesian students in 2000-2015 are still far below the international average. the results of the 2015 program for international student assessment (pisa) study show that indonesia is 64th out of 72 countries. the results of students' scientific literacy mastery are arranged into seven levels from level 1b to level 6, where the higher the level, the better scientific literacy mastery. the position of indonesian students is shown to be below level 1 by 1.2%; 14.4% at level 1b; 40.4% at level 1a; 31.7% at level 2; 10.6% at level 3; 1.6% at level 4; 0.1% at level 5 and none at level 6 (oecd, 2016). the data shows that the majority of indonesian students still trapped below the second level of 87.7% and 41.6% of students are below level 1b. these results indicate that students in indonesia are still difficult mailto:dy2psihite@gmail.com journal of science learning article doi: 10.17509/jsl.v3i1.17757 20 j.sci.learn.2019.3(1).19-28 to identify and apply scientific concepts to the phenomena they have (bybee & mccrae, 2011; mudzakir, widhiyanti, arifin, lestari, & jauhariansyah, 2017). based on these facts, how a teacher views the nature of science and technology will influence what they choose to teach and how they will teach (lederman, 1992; mansour, 2010). as expressed by tairab (2001), science teachers must work out better ways to improve students' understanding of the nature of science and technology. a science teacher needs to have adequate knowledge of science and technology because the views they hold on nost will influence directly or indirectly in the way they present learning experiences in class (palmquist & finley, 1997; tairab, 2001). based on this, research to explore how the views of science teachers to nost needs to done and in this study, selected pre-service chemistry teachers as research subjects. pre-service chemistry teachers chose because later, they would play an essential role in influencing students to be literate in science (tairab, 2001; lederman, lederman, & antink, 2013). exploration the pre-service chemistry teachers' vnost is essential to know whether the understanding of pre-service chemistry teachers about the nature of science and technology is adequate. the results of the exploration of the initial views of pre-service chemistry teachers will be a follow-up to the next research in determining the right solution to develop the opinions of pre-service chemistry teachers towards realistic or get an adequate understanding of nost. previous research to explore pre-service chemistry teachers 'views on the relationship between science and technology has been carried out by mansour (2010), where educators' views on the relationship between science and technology scattered in the naïve, has merit and realistic categories. initially educators have a naïve view or do not have an adequate understanding of nature of science and technology (nost), but with a change of view of nature of science and technology (nost) in the realist category provides a practical difference in the development of pedagogy and teaching of educators (mansour, 2010). figure 1 ionic liquid technology discourse figure 2 discourse on ionic formation process in nacl journal of science learning article doi: 10.17509/jsl.v3i1.17757 21 j.sci.learn.2019.3(1).19-28 likewise research conducted by kusuma, mudzakir, & widhiyanti (2019) where the views of pre-service chemistry teachers are generally in the has merit category or statements that selected by pre-service chemistry teachers on the vnost questionnaire are groups of reports that indicate conditions that are not entirely true even though there are parts of the statement that are still in accordance with the general view of science, scientific concepts, and scientific theories (rubba & harkness, 1993). based on the background stated, this study aims to explore the pre-service chemistry teachers’ vnost in looking at the nature of science and technology. exploration of the views of pre-service chemistry teachers to nost focuses on four aspects offered by tairab (2001), namely the characteristics of science and technology, the purpose of science and scientific inquiry, the features of scientific knowledge, and scientific theory, and the relationship between science and technology. exploration of the views of pre-service chemistry teachers uses the vnost questionnaire adapted from tairab (2001) but has modified by the addition of an ionic fluid technology-based discourse. through the acquisition of the discussion of ionic liquid technology, pre-service chemistry teachers are expected to understand the relationship of the nost context from the viewpoint of the latest philosophical and historical analysis of the scientific activities and practices of scientists in developing ionic liquids as triggers for technological development in the world. 2. method the descriptive method is used in this study to explain the results of an analysis of the views of pre-service chemistry teachers to nost. data collection techniques using the vnost questionnaire which adapted from the journal "views on science-technology society ©" (aikenhead, ryan, & fleming. 1989) and modified again by tairab (2001) and then changed back by researchers by adding discourse on ionic liquid technology. participants in this study were 48 pre-service chemistry teachers from the chemistry education study program at sriwijaya university. vnost questionnaire consists of 8 questions / statements in the form of 7 multiple choice and one essay. seven multiple-choice items require students to choose a statement that fits their views while one essay item requires students to give their opinion in writing about the difference between science and technology. data collected for seven multiple-choice questions were analyzed using a frequency distribution to characterize students' views of the nature of science and technology, while description questions regarding the general opinions of pre-service chemistry teachers' differences in science and technology described descriptively. the frequency distribution provides a characterization of the views held by pre-service chemistry teachers based on the categories suggested by rubba & harkness (1996). the results of pre-service chemistry teachers’ vnost for each question item categorized by the r / realistic category (choice expresses figure 3 discourse on the determination of nacl structure by x-ray diffraction journal of science learning article doi: 10.17509/jsl.v3i1.17757 22 j.sci.learn.2019.3(1).19-28 an appropriate view), hm / has merit (option is not realistic, but shows a legitimate thing), n / naïve (decision shows ideas that are not correct/invalid) and uncategorized (choices that reveal that none of the options in items 1-7 are in accordance with the views of pre-service chemistry teachers (rubba & harkness, 1996). 3. result and discussion the view held by pre-service chemistry teachers about the nature of science and technolgy (nost) is presented in the i – iv table. results pre-service chemistry teachers’ vnost grouped by 4 categories, namely (1) on the table i offered the view of pre-service chemistry teachers to the scientific definition of science, the purpose of science and the fact of the scholarly research, (2) on the table ii presented views pre-service chemistry teachers about table 1 frequency and percentage of students’ views on what is science, its aim, and natural scientific research statement category frequency % definition of science the development of the invention and application of liquid salt and ionic liquid technology is a scientific activity. in your opinion, basic science is... the fields of science, such as biology, chemistry, and physics. hm 3 6.25 principles, laws, and theories, which explain the world around us such as matter, energy, and life hm 35 72.9 investigate the unknown and discover new things about the world, the universe, and how it works. r 4 8.33 involve experiments to solve problems around us. hm 3 6.25 create and designing things (for example, artificial hearts, computers, and space vehicles). n find and using knowledge to make a better world (for example, cure diseases, overcome pollution, and improve agriculture). hm 2 4.17 a group of people called scientists who have ideas and techniques to discover new knowledge. n i do not know n 1 2.08 i do not have enough knowledge to make a choice n the purpose of science the objectives of the sainspembenionic nacl from its elements have a purpose, as well as science. in your opinion, the purpose of science is... believing that what has found about the world is an essential truth n 1 2.08 understand, explain, and interpret ongoing changes in nature and its characteristics r 32 66.7 find, collect and classify facts about nature hm 10 20.8 discovering new ways to make life a better age hm 3 6.25 i do not understand. n i do not have enough knowledge to determine the choice. n none of the above choices fit my view 2 4.17 scientific research the activities undertaken by bragg and friends are a form of scientific research. why do you think scientists do scientific research? to create a new invention. n to test their explanation about why things can happen. r 24 50 to make something that can help human life. hm 3 6.25 to collect as much data as possible, and conclude a scientific law based on that data. hm 21 43.8 i do not understand. n i do not have enough knowledge to determine the choice n none of the above choices fit my view notice : hm : has merit, r = realistic, n = naive journal of science learning article doi: 10.17509/jsl.v3i1.17757 23 j.sci.learn.2019.3(1).19-28 scientific knowledge and scientific theory, (3) on table iii presented the picture of pre-service chemistry teachers about technological characteristics and the relationship between science and technology, (4) on table iv gave a general view pre-service chemistry teachers about the differences in science and technology. the results of the student views of chemical teachers on the definition of science, science objectives, and the nature of scientific research presented in table 1. based on table 1, only 8.33% of pre-service chemistry teachers choose the science definition statement is investigating the unknown and discovering new things about the world, the universe as well as how it works. this view is considered realistic by rubba and harkness (1993). science as an investigation process also stated by scientists compiled by tairab (2001), stating that science is a tool to explain the world. similarly, the opinions expressed by mc ginn (1991) where science is an organized body of knowledge and is a systematic field of investigation into nature. however, based on the results of grouping of categories corresponding to the rubba & harkness (1993) of the statements selected by pre-service chemistry teachers, generally or about 72.9% of pre-service chemistry teachers choose the statement that science is a knowledge, like principle, law, and theory, which explains the world around us (matter, energy, and life). the statement categorized into the has merit category or comment with the condition that is not entirely correct, but there is part of the report still by the general view of science, the concept of science, and the theory of science (rubba & harkness, 1993). as discussed earlier, the vnost questionnaire given to pre-service chemistry teachers guided by the technological discourse of ionic liquid (figure 1). speech given by presenting the melting point data of liquid salt and research on the ionic liquid should be able to affect the pre-service chemistry teachers to choose the statement that science is the process of investigating the unknown and discover new things about the world and the universe. however, the sheer view of pre-service chemistry teachers on the statement suggests that so far, they think that the principles, laws, and theories expressed by previous scientists are at the heart of the science's characteristics. epistemological science, where science is a way to know or constitute the values and beliefs inherent in the development of scientific knowledge (lederman, 1992; lederman, laderman, & antink, 2013) poorly understood by pre-service chemistry teachers as a whole. also, the learning experienced by pre-service chemistry teachers who are more likely to memorize and receive confidently all scientific explanations obtained from teaching materials or classes of instruction can potentially affect the view them to science. pre-service chemistry teachers rarely make direct connections or investigate things around them where principles, laws, and theories should be used to explain the unknown so that they can tell the phenomena they observe or can build new knowledge that can complement the experience developed by previous scientists. as revealed by fernandes, rodrigues, & ferreira, (2018) that an understanding of the nature of science evolved the time they became students and remained constant for many years, making it very difficult to reduce previous perceptions held by pre-service chemistry teachers about science. table 2 frequency and percentage of students’ views on the nature of scientific knowledge and scientific theory statement category frequency % scientific knowledge investigation of the melting point of ionic liquids yields scientific knowledge. in your opinion, the following statement is following your understanding of scientific knowledge? scientific knowledge is a collection of well-organized facts. r 21 43.8 current scientific knowledge based on scientific perspectives, ideas, and interpretations of scientists from the past. r 26 54.2 scientific knowledge was at one time produced by scientists at that time. hm scientific knowledge only contains statements that are 100% true. n i do not understand. n 1 2.08 i do not have enough knowledge to make a choice n none of the above options are in line with my view scientific theory based on the discourse above, in your opinion, a scientific theory is... an idea of what will happen n 3 6.25 the most appropriate interpretation and explanation which has been agreed by scientists hm 13 27.08 a fact that has proven through various experiments r 24 50.00 i do not have enough knowledge to make a choice n 1 2.08 i do not understand n 3 6.25 there is no one choice above that fits my point of view 4 8.33 notice : hm : has merit, r = realistic, n = naive journal of science learning article doi: 10.17509/jsl.v3i1.17757 24 j.sci.learn.2019.3(1).19-28 unlike the case with the goals of science, pre-service chemistry teachers generally have a realistic view. 66.67% of the total number of students chose the statement that the purpose of science is to understand, explain, and interpret sustainable changes in nature and their characteristics. the results of this study also show the same thing as the research conducted by lokollo, hernani, & mudzakir (2019) wherein science goals, 44.1% of preservice chemistry teachers have a realistic view. likewise, a study conducted by kusuma, mudzakir, & widhiyanti (2019) in which 70.73% of pre-service chemistry teachers had a practical perspective. in this section, there is a change that is better than the first view of students regarding the definition of science, which was previously in the has merit category. however, as many as 20.8% of the number of pre-service chemistry teachers consider that the purpose of science is to discover, gather, and classify facts about nature and find new ways to make human life right. this view categorized into has merit, or the statement they choose contains part of the report, which is process-oriented and still by the purpose of science. the discourse given regarding the phenomenon of the process of forming ionic salts, as shown in figure 2, can guide pre-service chemistry teachers to understand that science aims to understand, explain, and interpret sustainable changes in nature and their characteristics. if this view is related to the opinions of pre-service chemistry teachers regarding the definition of science, it can conclude that pre-service chemistry teachers believe that knowledge such as principles, laws, and theories are the main characteristics of science. pre-service chemistry teachers do not understand that experience in the form of policies, rules, and methods constructed by previous scientists is a way for scientists to explain the results of their investigation of the world and the universe. likewise, the views of pre-service chemistry teachers for the nature of scientific research evenly distributed in the realistic and has merit categories. half of the students think that scientists conduct scientific research to test their explanations of why things can happen. this view categorized in realistic (r). the student choice of the statement is by the general opinion of science, where the primary purpose of scientific research is to gather the knowledge needed to compile an explanation of phenomena that exist in the world through specific rules (tairab, 2001). however, half of the pre-service chemistry teachers chose the statement that scientists conduct scientific research to make something that can help human life and collect as much data as possible and conclude a scientific law based on that data. the students' views are categorized in has merit and show that they believe scientific research is related to social aspects. students table 3 frequency percentage of students’ views on the definition of technology and the relationship between science-technology statement k f % definition of technology ionic liquids are an opportunity for environmentally friendly technology. in your opinion, what is meant by technology application of science that is useful to improve the quality of life of hm 19 39.6 various objects made by humans such as devices, tools, and instruments (e.g. computers) r 10 20.8 objects, techniques, processes, and people related to hm devices, tools and instruments hm 3 6.25 creating, designing, developing and testing devices, tools, and instruments r 13 27.1 very similar to science n the process of producing and knowing how to make a product n 1 2.08 i do not understand n i don't have enough knowledge to make n choices n there is no one choice above that fits my point of view 2 4.17 the relationship between science and technology from the explanation above, science and technology interrelated with daily life. in your opinion, the statement below that fits your understanding is technological innovation and / or science development can cause environmental problems hm 4 8.33 science and technology often make our lives healthier, easier, and more comfortable hm 4 8.33 the prosperity of a nation depends on the extent of the development of science and technology. hm 10 20.83 science and technology rarely endanger human life. n we cannot solve all the problems we face using only science and technology. r 3 6.25 because science, technology, and society are not related to one another, they do not influence each other. n 1 2.08 on the one hand, science and technology affect society, but on the other hand, society also influences the development of science and technology. r 19 39.6 i do not know. n 4 8.33 i do not have enough knowledge to make a choice. n 1 2.08 there is no one choice above that fits my point of view 2 4.17 notice : hm : has merit, r = realistic, n = naive journal of science learning article doi: 10.17509/jsl.v3i1.17757 25 j.sci.learn.2019.3(1).19-28 think that scientific research is needed to produce a product that can use for the needs of many people. the view of pre-service chemistry teachers in the has merit category shows that they have an inadequate look at scientific research. although the discourse gives (figure 3) presents one example of a study conducted by scientists to test their explanation of the structure of nacl, but the learning process, teaching methods, and their learning methods generally through memorization may be the cause of inadequate views of pre-service chemistry teachers towards scientific research. of course, it would be difficult for pre-service chemistry teachers to influence later students' ability to view science. in table 2, students' views on the nature of scientific knowledge generally distributed in the realistic category. 98% of students have an understanding that scientific knowledge is a well-organized collection of facts and based on scientific perspectives, ideas, and interpretations of scientists from the past. the majority of students have a realistic view and show that they have an understanding by scientists where scientific knowledge is a collection of knowledge obtained through scientific research compiled through certain scientific principles. however, 2.08% of students are naïve and think that scientific knowledge only contains statements that are 100% true. unlike the case with scientific theory, in general, students' views of the scientific method are spread in the realistic category, where 50% of students consider that scientific method is a fact that has proven through various experiments. 27.08% of students choose the statement that scientific theory is the most appropriate interpretation and explanation, which has been agreed by scientists and this view categorized into has merit. both comments still show an opinion by the light of scientists where scientific theory is the simplest explanation of a phenomenon that can prove through a series of experiments (tairab, 2001). however, 14.58% of students have a naïve view. students assume that scientific theory is an idea of what will happen. also, students do not have enough knowledge to determine their choices and do not understand. the fact "no knowledge or no understanding" proves that some preservice chemistry teachers are less sure about their choice of scientific theory. 8.33% of students chose "there is no choice that fits my point of view." shows that students have table 4 pre-service chemistry teachers prospects views of differences in science and technology no science technology 1 a rational, systematic, logical, and consistent process of thought and analysis part of the set of information included in scientific knowledge, containing perspective information regarding the creation of systems and their operation 2 a knowledge that uses in everyday life the application of science is made to improve the quality of life 3 the study of life on earth and the research and resolution of problems by experimentation and analysis devices that are run with or without human assistance and are useful to help work and observe things that humans have not been able to achieve. example: microscope, etc. 4 the study of nature, conscious efforts to investigate, discover and enhance human understanding of the reality of life various objects made by humans such as devices that can help humans to find information and knowledge 5 science-based on collecting data, developing a theory and establishing a theory that can explain natural phenomena that occur scientifically. discovery, development, testing, and determination of tools or instruments that can be used in scientific activities scientifically. 6 something that is already in and investigated for its truth through specific knowledge and a process of discovery human skills using natural resources through rational methods to be applied 7 a process of understanding or opinion about something that developed and logical, natural knowledge prescriptive information about system creation and operation 8 natural knowledge developed the result of developed natural knowledge 9 knowledge or knowledge learning regarding aspects related to tools knowledge or knowledge learning regarding aspects related to tools. 10 a science or understanding that studies about life in nature that can be associated with everyday life. a breakthrough in developing a knowledge that will bring new understanding to develop further 11 the basis of the embodiment of technology is absolute and scientific. following what data is there and is a science-based on facts the results of science or the application of scientific innovations that can support/facilitate life 12 science is more focused on the concept of science and scientific facts that occur. science of knowledge that continues to develop technology is more a product of science/science and also the development of previous concepts. technology is more towards developing better 13 the process of thinking and analysis is rational, systematic, logical and consistent technology is part of the information included in scientific knowledge, which follows the path of the times. journal of science learning article doi: 10.17509/jsl.v3i1.17757 26 j.sci.learn.2019.3(1).19-28 other views about scientific methods but not revealed when filling out the vnost questionnaire. the results of pre-service chemistry teachers’ vnost regarding technology and the relationship between science and technology shown in table 3. in table 3, the views of pre-service chemistry teachers on the definition of technology generally scattered in the realistic category that is equal to 47.9%. pre-service chemistry teachers assume that technology is a variety of objects made by humans, such as devices, tools, and instruments (e.g., computers). this view shows that students believe that technology is all forms of objects that are used to facilitate human life. the majority of pre-service chemistry teachers choose technology as a tool and instrument by the estimation of gardner (1999), where most people tend to believe that technology is the application of science in the form of useful products to serve humanity. also, students' realistic views focused on the definition of technology is creating, designing, developing, and testing devices, tools, and instruments. this view is consistent with the opinion of the general public, where technology is an application of science. it is in line with research conducted by tairab (2001), where students believe that technology is an artifact such as equipment, tools, and materials used to make certain types of techniques. although technology is identical to tools and instruments, the assumption that technology influences the quality of human life also chosen by some students (45.9%) who categorized in has merit. the student's view shows that technology developed as a social goal for the welfare of social life. it is a concern expressed by aikenhead & ryan (1992), where tools or so-called instrumentalists often confuse science in terms of technology, especially those relating to their social goals. the statement chosen by preservice chemistry teachers that technology is the application of science that is useful for improving the quality of life shows that they agree with things like devices, tools, and instruments that engineered are examples of technology. they assume that technology is the application of science. it is also consistent with the view of gardner (1999), where technology generally is seen as a science application. however, it needs to understand that technology is not only an application of science, but technology creates objects to be investigated and sometimes technological innovation is a direct goal of research (lacey, 2012). besides, 2.08% of students consider that technology is a process for producing and knowing how to make products. this view is naïve. students do not understand that the outcome demonstrates that the technique is not a step or process that is done to produce something. likewise, when students asked to choose their understanding of the relationship between science and technology, as many as 45.38% of the number of students want that on the one hand, science and technology affect society, on the other hand, culture also influences the development of science and technology. this view categorized as realistic. students display the belief that science and technology have an impact on society and that the effect depends on how to use science and technology itself. these results prove that pre-service chemistry teachers agree that science and technology are two parts that are not mutually exclusive and mutually reinforcing. science and technology are two different, but interconnected and inseparable subjects. science and technology are involved in complex and interactive twoway interactions, as revealed by gardner (1999). on the other hand some of the views of pre-service chemistry teachers about the relationship between science and technology are categorized into has merit which is detailed as follows, (1) 8.33% of students choose technological innovation and /or science development can cause environmental problems, (2) 8.33% of students select science and technology often makes our lives healthier, easier and more comfortable, (3) 20.83% of students choose the prosperity of a nation depends on the rapid development of science and technology. this choice, although not realistic, reveals a valid statement about the nature of science, technology and its interactions in society. these results indicate that students agree with the view that technology does indeed influence society. it is consistent with the belief that reveals that technology is intricately woven in human activities and is influenced or influences human capabilities, cultural values, public policies, and environmental constraints (herman, 2013). on the other hand, 12.49% of the number of students have a naïve view. students consider science, technology, and society not related to one another so that they do not influence each other. they also choose "do not know" and do not have enough knowledge to make a choice. these results indicate that students who consider science and technology to be independent and unrelated. this view is not by the opinions of gardner (1999), where technology and science are involved in two-way and interrelated interactions. pre-service chemistry teachers seem less aware that technological developments that have an impact on society are closely related to science. meanwhile, scientists agree on the science, science resolution, goal science, scientific research, exact science, scientific theory, technological fission, and the relationship between science and technology discussed in figure 4. the results of the vnost questionnaire for the 8th item about the views of pre-service chemistry teachers about the differences in science and technology described descriptively shown in table 4. based on the opinions expressed by pre-service chemistry teachers about the differences in science and technology, science generally regarded as knowledge, thought, analysis, science, and a process of understanding while technology considered as the application of science, devices, information sections, journal of science learning article doi: 10.17509/jsl.v3i1.17757 27 j.sci.learn.2019.3(1).19-28 scientific products and a breakthrough from science. the views of pre-service chemistry teachers are not entirely by the general look of science where science is an organized body of knowledge and is a field of systematic inquiry into nature while technology is everything that is used to facilitate human life. therefore, students' understanding of science and technology needs to be developed to be adequate in a more realistic direction. 4. conclusion this study explores the views of pre-service chemistry teachers on the nature of science and technology (nost). the research findings show that students' views generally scattered in the realistic and has merit categories. regarding the specific aspects of science and technology, students' pictures show the opposite results. the look of the definition of science, generally spread in the has merit category while for the meaning of technology is usually covered in the realist category. the results with these two different categories show that students are less confident in the definition of science as a tool to explain phenomena. however, it is different from the description of technology where students believe that technology is everything that is used to facilitate human life. these findings indicate that pre-service chemistry teachers used to deal with a variety of technological innovations, but they are difficult to connect the interrelationships of science and technology in them. therefore, the nature of science and technology needs to be explicitly discussed and discussed so that preservice chemistry teachers can later develop an appropriate view of the characteristics of science and technology. other findings on the specific aspects of scientific knowledge and scientific theory show that students' views generally scattered in the realistic category. almost all students believe that scientific knowledge is a collection of knowledge obtained through scientific research compiled through specific rules. on the other hand, half of the students believe that scientific theory is the simplest explanation of a phenomenon that can prove through a series of experiments in the aspect of the objectives of science and scientific research, students' views generally spread in the realistic category. students believe that the purpose of science is process-oriented to explain a phenomenon that proven through scientific research. so it is with aspects of the relationship between science and technology. in general, students' views spread in the realistic category. students agree that science and technology are two different subjects but related in two directions, where, on one side, science requires technology, and on the other hand, the technique involves science. also, their views prove that science and technology are closely related and significantly affect people's lives. in general, it can conclude that the results of the view of nature of science and technology (vnost) pre-service chemistry teachers are in the has merit category where the statements they choose about the nature of science and technology are only partially consistent with the general view of science. these results prove that although preservice chemistry teachers have been given discourse in the form of an ionic liquid phenomenon, their lights have not yet led to realistic. therefore, the results of the opinions of pre-service chemistry teachers who participated in this study had a significant influence on the teaching and learning of students' science. an adequate understanding of nost is an urgent basis at all levels in science education (tairab, 2001). by understanding the nature of science and technology, pre-service chemistry teachers can understand the phenomena that occur in their environment and can relate them to the concept of science as a whole. if a science teacher's view of the nature of science reflected in his pursuit, then that view will have a significant impact on the teaching and learning that they do. acknowledgment the authors acknowledge members of the chemical curriculum content innovation field who participated in giving ideas and suggestions for improving this article. references aikenhead, g. s., ryan, a. g., & fleming, r. w. (1989). views on sciencetechnology-society. social science and humanities research council. aikenhead, g. s., & ryan, a. g. (1992). the development of a new instrument:‘views on science—technology—society’(vosts). science education, 76(5), 477-491. ayvaci, h. s., & ozbek, d. (2019). the effect of documentary films on preservice science teachers' views of nature of science. journal of science learning, 2(3), 97-107. bybee, r., & mccrae, b. (2011). scientific literacy and student attitudes: perspectives from pisa 2006 science. international journal of science education, 33(1), 7-26. fernandes, g. w. r., rodrigues, a. m., & ferreira, c. a. (2018). conceptions of the nature of science and technology: a study with children and youths in a non-formal science and technology education setting. research in science education, 48(5), 1071-1106. figure 4 percentage of views of pre-service chemistry teachers on nost aspect journal of science learning article doi: 10.17509/jsl.v3i1.17757 28 j.sci.learn.2019.3(1).19-28 gardner, p. l. (1999). the representation of science-technology relationships in canadian physics textbooks. international journal of science education, 21(3), 329-347. herman, b. c. (2013). a convergence of postman and vygotsky perspectives regarding contemporary media’s impact on learning and teaching. the nature of technology (pp. 291-328). brill sense. kusuma, d. c., mudzakir, a., & widhiyanti, t. (2019). pre-service chemistry teachers’ vnost and their conceptions about the context of oled and related chemistry contents. in journal of physics: conference series (vol. 1157, no. 4, p. 042037). iop publishing. lacey, h. (2012). reflections on science and technoscience. scientiae studia, 10(spe), 103-128. lederman, n. g. (1992). students' and teachers' conceptions of the nature of science: a review of the research. journal of research in science teaching, 29(4), 331-359. lederman, n. g., lederman, j. s., & antink, a. (2013). nature of science and scientific inquiry as contexts for the learning of science and achievement of scientific literacy. international journal of education in mathematics science and technology, 1(3), 138-147. lokollo, l., hernani, h., & mudzakir, a. (2019). pre-service chemistry teacher’s view about the nature of science and technology. in journal of physics: conference series (vol. 1157, no. 4, p. 042036). iop publishing. mansour, n. (2010). science teachers’ perspectives on sciencetechnology-society (sts) in science education. eurasian journal of physics and chemistry education, 2(2), 123-157. mc ginn, r. e. (1991). science and technology, and society (no. 303.483 m4599s ej. 1). prentice hall. mudzakir, a., widhiyanti, t., hernani, arifin, m., lestari, a. n., & jauhariansyah, s. (2017, august). the nature of science and technology for pre-service chemistry teacher: a case of technochemistry experiment “from stannum metalicum to conductive glass.” in aip conference proceedings (vol. 1868, no. 1, p. 030015). aip publishing. murcia, k., & schibeci, r. (1999). primary student teachers' conceptions of the nature of science. international journal of science education, 21(11), 1123-1140. oecd, p. (2016). results (volume i): excellence and equity in education. palmquist, b. c., & finley, f. n. (1997). preservice teachers' views of the nature of science during a postbaccalaureate science teaching program. journal of research in science teaching: the official journal of the national association for research in science teaching, 34(6), 595-615. rotherham, a., & willingham, d. (2010). new, but a worthy challenge. american educator, 17-20. rubba, p. a., & harkness, w. l. (1993). examination of preservice and in-service secondary science teachers' beliefs about sciencetechnology-society interactions. science education, 77(4), 407-31. rubba, p. a., & harkness, w. j. (1996). a new scoring procedure for the views on science‐technology‐society instrument. international journal of science education, 18(4), 387-400. shwartz, y., ben‐zvi, r., & hofstein, a. (2005). the importance of involving high‐school chemistry teachers in the process of defining the operational meaning of ‘chemical literacy.’ international journal of science education, 27(3), 323-344. tairab, h. h. (2001). how do pre-service and in-service science teachers view the nature of science and technology?. research in science & technological education, 19(2), 235-250. a © 2020 indonesian society for science educator 124 j.sci.learn.2020.3(3).124-131 received: 5 december 2019 revised: 26 march 2020 published: 15 july 2020 prospective physics and science teachers' mental models about the concept of work ayşegül sağlam-arslan1, işık saliha karal2, hava i̇pek akbulut1* 1mathematics and science education, trabzon university, turkey 2mathematics and science education, giresun university, turkey *corresponding author havaipek@gmail.com abstract work, as a concept, is often encountered in daily life, but the meaning of work in physics is closely related to power, force, and energy. studies made about various concepts and subjects have proved that using words in ways different from their scientific meanings has a negative effect on teaching, and causes conceptual complexities and/or generation of alternative mental models. this study aimed to determine the mental models of prospective physics and science teachers about the concept of work. this study was conducted with the participation of 107 prospective teachers in the physics and science teacher training program. the trainee teachers' understandings were determined by using an achievement test, consisting of three open-ended questions, developed by the researchers. data obtained were first analyzed according to the level of understanding demonstrated, and then the mental models were determined by using these levels. four types of mental models about work were identified, specifically the scientific,, the scientific synthesis, the initial synthesis, and the initial models. the synthesis model is the dominant mental model, and it has been developed by prospective teachers from two disciplines. keywords mental model, work concept, prospective teachers 1. introduction conceptual perceptions, learning difficulties, epistemological and pedagogical obstacles all shape teaching and learning and have attracted much attention by researchers for many years. as is known, these factors have a negative impact on teaching activities and student learning, leading to misconceptions, or alternative understandings, about scientific subjects. the fact that a concept is the subject of more than one discipline and/or may have different meanings in the curriculum and daily life is an underlying issue of the above dynamics in teaching (yıldırım, 1996; sağlam-arslan, 2016). when an individual meets new information, he/she perceives it through the communication channel, and this perception is frequently somewhat different from their existing knowledge on the topic (sağlam-arslan, 2016). theoretical approaches in this field, i.e., the anthropological theory of didactics (chevallard, 1998), emphasized the importance of the 'first identification/encounter' in mental structures of learners and discussed the effects of this encounter on scientific learning. by the time learners come to school, they have already faced, in their daily lives, much knowledge that will be taught with scientific meaning in the classroom. for this reason, the meanings assigned to terms in daily life have great importance in the teaching of concepts. as is known, the use of concepts in daily life that are different from their scientific meaning makes them difficult to teach and learn in the right way (lubben, netshisuaulu, & campell, 1999; bennett, hogarth, & lubben, 2003). this situation requires the association of daily life meanings with scientific meanings in concept teaching. giving a meaning to one concept in different forms and in more than one discipline independent from each other has negative effects on learning (aydın & balım, 2005). accordingly, the curricula in turkey have been redesigned by placing interdisciplinary and context-based teaching at the center. the concept to be taught, which is difficult because of its epistemological nature, becomes even more difficult when the meaning in daily life is different from its scientific meaning. the concept of work is known in daily life as an activity which is done by spending energy to produce something. it is not defined independently of its scientific mailto:havaipek@gmail.com journal of science learning article doi: 10.17509/jsl.v3i3.21660 125 j.sci.learn.2020.3(3).124-131 meaning but is explained within its mathematical relations (aguiar, sevian, & el-hani, 2018; bächtold, 2018). in scientific statements, work has a meaning in connection with concepts of power, force, and energy. the explanations regarding work within the scope of science and physics courses emphasize the conditions necessary for doing physical work but do not give a specific definition of the concept of work. a literature review reveals that the concept of work is discussed in conjunction with the concepts of power, force, and energy. these studies are grouped under six headings according to their objectives: (a) explaining the correlation between work and energy (adamczyk & willson, 1996; doménech et al., 2015; gutlerrez, zuza, & gulsasola, 2015; hartmann & priemer, 2018); (b) determing incorrect comprehension levels of students about work, power, and energy (bahar, öztürk, & ateş, 2002; erduran-avcı, ünlü, & yağbasan, 2009); (c) identifying alternative concepts in work, power and energy subjects; (küçük, çepni, & gökdere, 2005; pastırmacı, 2011), (d) evaluating students' perception levels about energy and related concepts (duit, 1984; goldring & osborne 1994; hırça, 2008; küçük, çepni, & gökdere, 2005; trumper, 1998; ünal-çoban, aktamış, & ergin, 2007; watts, 1983), (e) eliminating alternative conceptions about work and energy (i̇pek akbulut, şahin, & çepni, 2013); and (f) applying new approaches in teaching work, power and energy subjects; (büyükdede & tanel, 2018; cerit berber, 2008; cerit berber & sarı, 2009; desianna, nugroho, & ellianawati, 2019; ergin, 2011; mustofa & asmichatin, 2019). there are a limited number of studies focusing specifically on work, for example, misconceptions about the concept (erduran avcı, kara, & karaca, 2012) or prospective teachers' skills of distinguishing the difference between negative-positive work (uzunkavak, 2009). the literature also shows that there are some difficulties related to learning and teaching the concept of work. it is important to determine the perceptions of prospective teachers who will be responsible for teaching this concept. as it is known, the cognitive structures of teachers affect student learning. therefore, it is important to determine the characteristics of the cognitive structures of prospective teachers about work. a multidimensional analysis of cognitive structures is called a mental model in the literature. mental models are defined as personal and internal presentations created to understand and perceive real phenomena (franco & colinvaux, 2000). as cognitive representations, they are used for reasoning, identifying, explaining, estimating, and controlling phenomena (buckley & boulter, 2000; örnek, 2008). mental models have characteristics such as being individual, changeable, developable, reconfigurable and incomplete (norman, 1983; franco & colinvaux, 2000; buckley & boulter, 2000; harrison & treagust, 2000; barquoero, 1995, as cited in greca & moreira, 2000; coll & treagust, 2003; ünal & ergin, 2006, jalmo & suwandi, 2018). accordingly, mental models are structured and arranged by the individual in their learning process (ünal-çoban, 2009; nongkhunsarn, yuenyong, tupsai, & sranamkam, 2019; chanserm, tupsai, & yuenyong, 2018). due to these characteristics, determining mental models about any subject will reveal important indicators about the learning of individuals (putri, samsudin, nugraha, & fratiwi, 2019). for these reasons, this study set out to determine the mental models and alternative conceptions about the concept of work held by prospective physics and science teachers. 2. method this descriptive research was carried out with trainee science (n=56) and physics (n=51) teachers who had completed their subject matter knowledge courses and were attending their last term on methods in education in the 2016–2017 academic year. trainee science teachers arrive at teacher's college, having followed physics courses throughout at least four years of high school and would have been exposed to the concept of work several times. at the college level, prospective physics teachers attend different physics courses in the teacher training program, including general physics i in the first semester, and general physics ii in the second semester (both weekly 4 hours theoretical and 2 hours practical). data on the mental model held by the college students in this study was compiled in several ways following kurnaz (2011): determining the mental model by classifying the characteristics of student responses (e.g. borges & gilbert, 1999; lin & chiu, 2007); referring to models known in the literature (gökdere & çalık, 2010; şengören, 2010); determining the mental model according to predetermined classifications based on the nature of the subject (e.g. sağlam-arslan, 2004); and considering the level of understanding of students (e.g., i̇yibil, 2010). in the current study, the latter method was followed in order to identify the mental models about work by trainee teachers. an achievement test focussed on three elements of concept knowledge describing, connecting, and interpreting. the first question on the test required a definition of work, its units, and type designation. the second question necessitated explanations as to whether or not physical work is done in three different systems. the third question demanded a prediction of the relationship between the amount of work done along three different possible paths (see appendix for details). 2.1. validity and reliability for the validity and reliability of the data collection tool, expert opinion on the research question, and then a pilot application was performed. in this context, the data collection instrument was examined by a group of two faculty members and two physics teachers with ten years of experience. after the necessary arrangement of the data journal of science learning article doi: 10.17509/jsl.v3i3.21660 126 j.sci.learn.2020.3(3).124-131 collection instrument, the pilot application was carried out with 70 freshmen who had taken the general physics course. thus the conformity of the data collection instrument with the aim of the study and its comprehensibility was established. 2.2. data analysis the data obtained from the study were analyzed in two stages. firstly according to the level of understanding and secondly according to the mental model. the following scale, developed by abraham, williamson & wetsbrook (1994), was used in the comparative analysis to analyze the responses according to the level of understanding of the concept of work by prospective teachers. [0]: no understanding (nu): blank, rewriting the question, irrelevant or unclear response. [1]: alternative conception (ac): scientifically incorrect responses containing illogical or incorrect information. [2]: partial understanding with a specific alternative conception (pu/ac): responses showing that the concept is understood but also containing an alternative conception. [3] partial understanding (pu): responses containing some components of the scientifically accepted response. [4]: sound understanding (su): responses containing all components of the scientifically accepted response. the mental models related to work were identified by conducting an individual holistic analysis of the level of understanding. understanding levels are associated with mental models (initial, synthesis, and scientific) in the literature (vosniadou, 1994). the synthesis model is divided into scientific synthesis and initial synthesis. accordingly, in this study, four types of mental models were identified: the scientific, the scientific synthesis, the initial synthesis, and the initial models. the relationship between the general characteristics of these models, and their levels of understanding, are presented in table 1. 3. result and discussion findings are discussed using two themes: prospective teachers' understanding level and their mental model related to the concept of work. table 1 mental models and their characteristics mental model properties of model relation between model and understanding level scientific a model consisting of conceptions exactly compatible with scientific knowledge. all responses to different types of questions are in level [3] or [4]. scientific synthesis a model consisting of conceptions based on daily life and experience, resembling scientific knowledge. two of the responses to different types of questions are in level [3] or [4] and the other is in level [0], [1] or [2]. initial synthesis a model consisting of alternative conceptions based on daily life and experiences, and/or where false knowledge is dominant. two of the responses to different types of questions are in level [0], [1] or [2] and the other is in level [3] or [4]. initial a model type consisting of alternative conceptions irrelevant to scientific knowledge, and/or involving false information. all responses to different types of questions are in level [0], [1] or [2]. [0]: no understanding (nu), [1]: alternative conception (ac), [2]: partial understanding with specific alternative conceptions (pu/ac), [3]: partial understanding (pu), [4]: sound understanding (su). table 2 distribution of understanding levels according to the responses question type level physics science total f % f % f describing [0] 1 2 1 [1] 2 4 14 25 16 [2] 19 37 22 39 41 [3] 19 37 18 32 37 [4] 11 22 1 2 12 connecting [0] 1 2 1 [1] 5 10 22 39 27 [2] 30 59 11 20 41 [3] 10 20 12 21 22 [4] 6 12 4 7 10 interpreting [0] 1 2 2 4 3 [1] 1 2 9 16 10 [2] 5 10 26 46 31 [3] 43 84 13 23 56 [4] 1 2 1 journal of science learning article doi: 10.17509/jsl.v3i3.21660 127 j.sci.learn.2020.3(3).124-131 3.1 prospective teachers' understanding level the distribution, according to understanding levels of the responses given to the questions, has been summarized in table 2. the results on connecting questions in table 2 show that more than half (59%) of physics prospective teachers' responses in this category were at level [2], while other responses were at levels [1], [3] and [4] with respective ratios of 10%, 20%, and 12%. no responses were recorded at level [0]. the responses of prospective science teachers in the connecting category demonstrated 39% at level [1], followed by similar ratios 20% and 21% at levels [2] and [3]. the remaining responses of trainee science teachers were located at levels [0] and [4] (2% and 7%, respectively). examination of the responses of prospective teachers in interpreting questions showed that 56% of responses were at level [3], and 31% were at level [2]. the majority (84%) of trainee physics teachers' responses in this category were at level [3], followed by 10% at level [2]. while nearly half (46%) of prospective science teachers' responses in this category were at level [2], there were no level [4] responses. the other responses were distributed at levels [3], [1] and [0] with ratios 23%, 16% and 4% respectively. table 2 shows that a majority of the prospective teachers have some alternative conceptions about the concept of work. a detailed analysis of responses in level [2] revealed various alternative conceptions about work, and are summarized in table 3. based on table 3, the most common alternative conceptions displayed by prospective teachers were as follows. ac1: work is a vector quantity as it depends on the vector quantities force and displacement. ac2: in order to do work, there must be a force other than the force of gravity and displacement. ac3: work done changes according to the path taken. two additional alternative table 3 prospective teachers’ alternative conceptions related to ‘work’ alternative concepts sample answers frequency p s t ac1. work is a vector quantity as it depends on the vector quantities force and displacement -the displacement in the direction of the force is called work. it is a vector quantity (f19). -it is the displacement of an object in the direction of the applied force. they are vector quantities (fb22). 14 6 20 ac2. in order to have work there must be a force other than the force of gravity and displacement. a child walking on an inclined path moves in the direction of the movement, but not work is done physically because there is not force acting on the child (f40). as a free falling object does not use any force, it doesn't do any work (f6). no work. because there is no force applied to the object (f7). 8 1 9 ac3. work done changes according to the path taken -wc> wa> wb, because the work done is proportional to the distance. w= f.x, the path taken at c is the maximum (f39). c is in the opposite direction to gravity, but more energy is consumed, because of the curved path (fb50). 1 7 8 ac4. work done only changes according to displacement w = f.x, a and c are the same but bigger than b, stairs are not important. the path having hypotenuse is the same as a, the path of b is small (f41). -work is done more in objects a and c due to the distance of the path, it is less in b than them (f16). 4 4 8 ac5. work done is directly proportional to displacement, inversely proportional to the slope of the path -b>a=c, work is done in b, directly, because the taken path is the shortest (fb24). -wb>wa=wc, because as the b path itself is in the direction of force (inverse), maximum work is done there. since the displacements of the a and b paths are equal and the vertical components are in the direction of the force, work is less (f50). 2 6 8 ac6. the work done depends on the difficulty of the path -a is the most difficult to take out, b is the easiest; c is the medium difficulty (fb15). one who gets out of the a path uses less energy. one who gets out of the c path uses less energy, the path is long, b path is short, spends more energy (fb34). 7 7 ac7. if force acting on an object and the displacement of the object are in a horizontal direction, the work is done. since the displacement of a free falling object in the horizontal plane is 0, there is no work done (f11, f24). 3 3 6 ac8. work is done if energy is spent. -the child walking on the inclined path did work. he had difficulty to go upwards. he loses energy (fb30). 2 2 journal of science learning article doi: 10.17509/jsl.v3i3.21660 128 j.sci.learn.2020.3(3).124-131 conceptions closely followed these three. ac4: work done only changes according to displacement. ac5: work done is directly proportional to the displacement and inversely proportional to the slope of the path. it has been seen that the alternative conceptions of trainee physics and science teachers differ. according to table 3, the physics group mostly demonstrate ac1 and ac2, while the prospective science teachers mostly show ac3 and ac6 coded alternative conceptions. while the prospective teachers in both branches are equivalent in terms of the ac4 coded alternative conception, ac6 and ac8 coded alternative concepts were only seen in trainee science teachers. 3.2. prospective teachers' mental models about the concept of work the mental models on the concept of work held by the trainee teachers in this study and analyzed by integral evaluation of levels of responses to the questions in the achievement test are summarized in table 4. when table 4 is examined, it can be seen that most of the prospective teachers hold an initial synthetic model (42%), followed by 35% with a synthetic scientific model, 15% with a scientific model, and 9% holding an initial model. the trainee teachers holding an initial synthetic model (physics 31%; science 56%) were not able to scientifically explain the conceptual structure of work and displayed various alternative conceptions. a typical response given by a trainee science teacher holding an initial synthetic model is illustrated below: describing: it is the type of energy that occurs in unit time. its' unit is j, which is a scalar quantity. w=f.x. connecting: he doesn't do work. with the effect of gravity, it already falls by itself. he does work, and it is in a horizontal plane and weight on his shoulder. interpreting: c>a>b. in the path c, it is in the opposite direction to gravity, but a curved path, more energy is spent. (fb50) the synthetic scientific model was the second most frequently displayed mental model among the participants (physics 51%, science 16%). it can be seen that the prospective teachers who have this model have explained the conceptual structure of work scientifically, and have answered at least two of the questions about work in a scientific way, but they also displayed various alternative conceptions. the characteristics of synthetic scientific model responses given by a trainee physics teacher are illustrated below: describing: the displacement on the direction of the force is called work. w=f.x, it is a scalar quantity. connecting: the object doesn't do work. gravity does work on the object. it has done work as the multiplication of the force is applied to the load, and the distance the load was carried. the child has done work against gravity. interpreting: the important thing in the work done against gravity is how high it arises (in equal mass). since the heights are equivalent, the work is equivalent, too (f9). the scientific model was demonstrated by a limited number of prospective teachers (physics: 18%, science: 10%). these trainee teachers were able to explain work scientifically (definition, units, and magnitude) and transfer their knowledge to real cases and interpret different physical events. a typical response by a prospective physics teacher with a scientific model is illustrated below: describing: when a force is applied to an object if the object displaces in the direction of the applied force, work is done. w = f. x, the unit is the joule (n. m), it is a scalar quantity connecting: when released, a falling object has done work physically. because of the weight of the object, and its displacement is in the same direction. a man who carries a load on his shoulders on a horizontal plane is not regarded as doing work physically. because of the weight of the load and the man's direction of movement is not in the same direction. a child walking on a sloping path is regarded to have done work because its weight's horizontal element and direction of movement are in the same direction. interpreting: the work done by all of them is equivalent to each other because the displacement of all three objects is equal to each other (f44). table 4 shows that the initial mental model is held only by trainee science teachers. prospective teachers holding this model responded without demonstrating scientific knowledge about work, were not able to scientifically explain the concept, and displayed various alternative conceptions. the responses given by a trainee teacher identified as holding the initial model are illustrated below: describing: it is said to change the way and direction of an object by the result of an applied force. connecting: work has been done because there is a displacement. work is not done unless the load carried contacts the ground. it did work because there is a displacement. table 4 mental models of prospective teachers subject area mental model type scientific scientific synthetic initial synthetic initial f % f % f % f % physics 9 18 28 51 14 31 science 6 10 7 16 28 56 9 18 total 15 35 42 9 journal of science learning article doi: 10.17509/jsl.v3i3.21660 129 j.sci.learn.2020.3(3).124-131 interpreting: c>a>b. in c, the work done against gravity is highest because we consume a lot of energy while climbing a ladder. since there is a slope in a, it is a lot too. however, it will be less in b if we directly put it upstairs (fb26). the results indicate that prospective physics teachers generally hold the synthetic scientific model, and none of the prospective physics teachers hold the initial model. while prospective science teachers mostly adopted the initial synthetic or initial models, over a third of this group were held to the synthetic scientific or scientific models as well. the findings show that the level of knowledge of trainee teachers about work varies according to the type of question. in general, they demonstrated a partial understanding of specific alternative conceptions. in the describing and interpreting categories, they showed a partial understanding. also, it was determined that they had a partial understanding of specific alternative conceptions in the connecting category. it may be that this is caused by the education system, which includes teaching practices that only require cognitive knowledge and skills towards lower levels of the cognitive learning area. for this reason, it seems that participants' responses to the type of associative questions, requiring high-level cognitive skills, are below current scientific knowledge levels. the results of some studies (sağlam-arslan, 2004; kurnaz, 2007; kurnaz & sağlam-arslan, 2009), showing that student learning is shaped according to teaching practices, also supports this idea. the prospective teachers' responses reflecting their level of understanding revealed alternative conceptions about the concept of work. this analysis showed that prospective teachers developed common alternative conceptions related to work, and the incidence of these concepts differs according to the participant group. some of the alternative conceptions in this study have also been revealed in previous studies. for example, the alternative conception, 'work depends on displacement', is found by erduran avcı (2019) and erduran avcı, kara, & karaca (2012) in studies with trainee science teachers and students. aydoğmuş (2008), with 6th-grade students, palmer (2001), with 6th and 10th-grade students, and pastırmacı (2011), with the 7th-grade students, identified students holding the alternative conception 'if work should be done, a force other than gravitation and displacement should be present.' on the other hand, madanoğlu found an alternative conception held by 9th-grade students of 'more or less work is done if the inclination is more or less,' was similar to the alternative conception of 'work done is inversely proportional to the slope of the path' exhibited by prospective science teachers (madanoğlu, 2015). it can be argued that, in general, the insufficient use of interdisciplinary approaches in concept teaching may affect the development of alternative conceptions. the difficulty of holding in one's mind a holistic structure of common concepts that have been attributed to different meanings in separate courses necessitates the use of a better approach. the analysis conducted to determine the mental models of the participants about the work concept identified four different models: the scientific, the scientific synthesis, the initial synthesis, and the initial. it was found that prospective physics teachers mainly displayed scientific synthesis and initial synthesis models, while prospective science teachers displayed initial synthesis and initial models about work. the characteristics of these types of models indicate that prospective teachers have mental structures that are not fully aligned with the scientific concept of work. synthesis models (initial synthesis and scientific synthesis) based on the knowledge structures in which learners combine scientific knowledge with their experiences are the most common types of mental models (e.g., kikas, 2005; hannust & kikas, 2007). this situation suggests that the mental schemas of learners are structured according to knowledge obtained as a result of ordinary, or daily life processes, as well as scientific information (greca & moreira, 2001). it is seen that the mental models about work by trainee science and physics teachers differ. accordingly, prospective physics teachers developed mental models more compatible with scientific knowledge compared to prospective science teachers. this situation can be accounted for by the fact that work is a common subject in different courses of the science teacher training program (mechanics, electricity and magnetism, modern physics, thermodynamics, etc.). when the results of this study on comprehension levels, alternative concepts and mental models are evaluated within the framework of the postulates of the anthropological theory of chevallard (1989), deficiencies related to learning by the individual can be defined as a reflection of educational reality. for this reason, learning environments to promote scientifically accurate mental models should be designed by taking into consideration these results, and the effects of, for example, written course resources used during teaching, teachers' knowledge, as well as students' daily life experiences. we suggest that learning environments and activities should include multiple and frequent representations of concepts and be based on interdisciplinary approaches. 4. conclusion this study revealed that prospective science teachers have various alternative conceptions and different mental models about work that are more or less compatible with scientific knowledge. because students first meet with formal meanings of basic science concepts in science courses at school, the science teachers' mental models about these concepts are very important. for this reason, it is recommended that some courses, including special teaching techniques and laboratory journal of science learning article doi: 10.17509/jsl.v3i3.21660 130 j.sci.learn.2020.3(3).124-131 approaches/applications, be designed to address this issue in science teacher training programs. references abraham m. r., williamson v. m., & wetsbrook s. l. (1994) a crossage study of the understanding of five chemistry concepts. j res sci teach, 31(2), 147–165. adamczyk, p., & willson, m. (1996). using concept maps with trainee physics teachers. physics education, 31(6), 374. aguiar, o., sevian, h., & el-hani, c. n. (2018). teaching about energy. science & education, 27(9-10), 863-893. aydın, g., & balım, a. g. (2005). yapılandırmacı yaklaşıma göre modellendirilmiş disiplinler arası uygulama: enerji konularının öğretimi. ankara üniversitesi eğitim bilimleri fakültesi dergisi, 38(2), 145-166. aydoğmuş, e. (2008). lise 2 fizik dersi iş-enerji konusunun öğretiminde 5e modelinin öğrenci başarısına etkisi [doctoral dissertation, selçuk üniversitesi fen bilimleri enstitüsü]. bächtold, m. (2018). how should energy be defined throughout schooling?. research in science education, 48(2), 345-367. bahar, m., öztürk, e., & ateş, s. yapilandirilmiş grid metodu ile lise öğrencilerinin newton’un hareket yasasi, iş, güç ve enerji konusundaki anlama düzeyleri ve hatali kavramlarinin tespiti. bennett, j., hogarth, s., & lubben, f. (2003). a systematic review of the effects of context-based and science-technology-society (sts) approaches in the teaching of secondary science. eppi-centre and university of york. borges, a., t., & gilbert, j., k. (1999). mental models of electricity. international journal of science education, 21(1), 95-97. buckley, b. c., & boulter, c. j. (2000). investigating the role of representations and expressed models in building mental models. in developing models in science education (pp. 119-135). springer, dordrecht. büyükdede, m., & tanel, r. (2018). iş-enerji ve itme-momentum konularina yönelik fetemm etkinliklerinin kavramsal anlama üzerine etkisi. diyalektolog, (19), 379-395. berber, n. c. (2008). i̇ş-güç-enerji konusunun öğretiminde pedagojik-analojik modellerin kavramsal değişimin gerçekleşmesine etkisi: konya ili örneği [doctoral dissertation, selçuk üniversitesi fen bilimleri enstitüsü]. cerit berber, n., & sarı, m. (2009). iş-güç-enerji konusunun öğretiminde kavramsal değişimin gerçekleşmesine pedagojikanalojik modellerin etkisi. gazi eğitim fakültesi dergisi, 29(1), 257-277. chanserm, t., tupsai, j., & yuenyong, c. (2018). grade 11 student's mental model of the nature of light. journal of physics: conference series, 1340(1). chevallard, y. (1989). rapport au savoir. séminaire de didactique des mathématiques et de l'informatique. chevallard, y. (1998). analyse des pratiques enseignantes et didactiques de mathématiques: l’approche anthropologique. actes de l’université d’eté, la rochelle, irem de clermont-ferrand, 91-120. coll, r. k., & treagust, d.f. (2003). learners' mental models of metallic bonding: a cross-age study. science education, 87, 685-707. desianna, i., nugroho, s. e., & ellianawati, e. (2019). phenomenon of buying and selling as bridging analogy of learning work and energy. physics communication, 3(1), 10-20. doménech, j. l., gil-pérez, d., gras-martí, a., guisasola, j., martíneztorregrosa, j., salinas, j., & vilches, a. (2007). teaching of energy issues: a debate proposal for a global reorientation. science & education, 16(1), 43–64. duit, r. (1984) learning to the energy concept in school-empirical results from the philippines and west germany. physics education 19(2),59–66. erduran avcı, d. (2019). fen öğretiminde kavram yanılgıları tespiti ve giderilmesi içinde (s.191-217). pegem a. erduran avcı, d., kara, i̇., & karaca, d. (2012). fen bilgisi öğretmen adaylarının iş konusundaki kavram yanılgıları. pamukkale üniversitesi eğitim fakültesi dergisi, 31(1), 27-39. ergin, s. (2011). fizik eğitiminde 4mat öğretim yönteminin farklı öğrenme stillerine sahip lise öğrencilerinin iş, güç ve enerji konusundaki başarısına etkisi [unpublished doctoral dissertation, gazi üniversitesi eğitim bilimleri enstitüsü]. franco, c., & colinvaux, d. (2000). grasping mental models. in j. k. gilbert & c. j. boulter (eds.), developing models in science education. kluwer academic publishers. goldring, h., & osborne, j. (1994). students' difficulties with energy and related concepts. physics education, 29(1), 26. gökdere, m., & çalık, m. (2010). a cross-age study of turkish students' mental models: an atom concept. didactica slovenica – pedagoska obzorja, 2, 185-199. greca, i. m., & moreira, m. a. (2001). mental, physical, and mathematical models in the teaching and learning of physics. science education, 86(1), 106-121. greca, m. i., & moreira m. a. (2000). mental models, conceptual models and modeling. international journal of science education, 22(1) 1-11. gutierrez, j., zuza, k., & guisasola, j. (2015). what engineering students understand on the first principle of energy in mechanics at introductory physics courses. in edulearn15 proceedings of the 7th international conference on education and new learning technologies (barcelona, spain,) (pp. 6811-6817). hannust, t., & kikas, e. (2007). children's knowledge of astronomy and its change in the course of learning. early childhood research quarterly, 22(1), 89-104. harrison, a. g., & treagust, d. f. (2000). a typology of school science models. international journal of science education, 22(9), 10111026. hartmann, b., & priemer, b. (2018). introducing energy through observations and measurements. physics education, 53(6), 1-7. hırça, n. (2008). 5e modeline göre “iş, güç ve enerji” ünitesiyle i̇lgili geliştirilen materyallerin kavramsal değişime etkisinin incelenmesi [unpublished doctoral dissertation, atatürk university]. i̇pek akbulut, h., şahin, ç., & çepni, s. (2013). i̇ş ve enerji konusu ile ilgili kavramsal değişimin incelenmesi: i̇kili yerleşik öğrenme modeli örneği. mehmet akif ersoy üniversitesi eğitim fakültesi dergisi, 13(25), 241-268. i̇yibil, ü. (2010). farklı programlarda öğrenim gören öğretmen adaylarının temel astronomi kavramlarını anlama düzeylerinin ve ilgili kavramlara ait zihinsel modellerinin analizi [unpublished doctoral dissertation, karadeniz teknik üniversitesi]. jalmo, t., & suwandi, t. (2018). biology education students' mental models on genetic concepts. journal of baltic science education, 17(3), 474-485. kikas, e. (2005). development of children's knowledge: the sky, the earth and the sun in children's explanations. electronic journal of folklore, 31, 3156. küçük, m., çepni, s., & gökdere, m. (2005). turkish primary school students alternative conception about work, power and energy. journal of physics teacher education, 3(2), 22-28. kurnaz, m. a., & sağlam arslan, a. (2009). using the anthropological theory of didactics in physics: characterization of the teaching conditions of energy concept and the personal relations of freshmen to this concept. journal of turkish science education, 6(1), 72-88. kurnaz, m. a. (2007). enerji kavramının üniversite 1. sınıf seviyesinde öğrenim durumlarının analizi [unpublished doctoral dissertation, karadeniz teknik üniversitesi]. kurnaz, m. a. (2011). enerji konusunda model tabanlı öğrenme yaklaşımına göre tasarlanan öğrenme ortamlarının zihinsel model gelişimine etkisi [unpublished doctoral dissertation, karadeniz teknik üniversitesi]. lin, j. w., & chiu, m. h. (2007). exploring the characteristics and diverse sources of students' mental models of acids and bases. international journal of science education, 29(6), 771-803. lubben, f., netshisualu, t., & campell, b. (1999). students' use of cultural metaphors and their scientific understandings related to heating. science education, 83, 761-774. http://void(0)/ journal of science learning article doi: 10.17509/jsl.v3i3.21660 131 j.sci.learn.2020.3(3).124-131 madanoğlu, n. (2015). 9. sınıf öğrencilerinin iş ve enerji konusundaki kavramsal anlamalarının incelenmesi [unpublished master’s thesis, balıkesir university]. mustofa, z., & asmichatin, a. (2019). modeling instruction to promote student's understanding of system and model of system of mechanical energy. abjadia, 3(1), 17-29. nongkhunsarn, a., yuenyong, c., tupsai, j., & sranamkam, t. (2019). grade 11 student's mental model of fluid and analytical thinking in science teaching through science technology and society (sts) approach. journal of physics: conference series, 1340(1). norman, d. (1983). some observations on mental models. in d. gentner & a. l. stevens (eds.), mental models. lawrence erlbaum associates. örnek, f. (2008). models in science education: applications of models in learning and teaching science. international journal of environmental and science education, 3(2), 3545. palmer, d. (2001). students' alternative conceptions and scientifically acceptable conceptions about gravity. international journal of science education, 23(7), 691706. pastırmacı, e. (2011). 7. sınıf öğrencilerinin iş ve enerji konusundaki alternatif fikirlerinin belirlenmesi ve kavramsal gelişimlerinin incelenmesi [unpublished master's thesis, balıkesir university]. putri, n., samsudin, a., nugraha, m. g., & fratiwi, n. j. (2019). the analysis of tenth grade lmm on work and energy. journal of physics: conference series, 1280(5). sağlam-arslan, a. (2016). didaktiğin antropolojik teorisi. in e. bingölbali, s. arslan, i̇. ö. zembat (eds.), matematik eğitiminde teoriler (377-392). pegem a akademi. sağlam-arslan, a. (2004). les équations différentielles en mathématiques et en physique: étude des conditions de leur enseignement et caractérisation des rapports personnels des etudiants de première année d’université a cet objet de savoir [unpublished ph.d. thesis, universite joseph fourier]. şengören, k. s. (2010). turkish students' mental models of light to explain the single slit diffraction and double slit interference of light: a cross-sectional study. journal of baltic science education, 9(1), 61-71. trumper, r. (1998). a longitudinal study of physics students' conceptions on energy in pre-service training for high school teachers. journal of science education and technology, 7(4), 311–318. ünal-çoban, g. (2009). modellemeye dayalı fen öğretiminin öğrencilerin kavramsal anlama düzeylerine, bilimsel süreç becerilerine, bilimsel bilgi ve varlık anlayışlarına etkisi: 7. sınıf ışık ünitesi örneği. [unpublished doctoral dissertation, dokuz eylül university]. ünal-çoban, g., aktamış, h., & ergin, ö. (2007). i̇lköğretim 8. sınıf öğrencilerinin enerjiyle ilgili görüşleri. kastamonu eğitim dergisi, 15(1), 175-184. ünal, g., & ergin, ö. (2006). fen eğitimi ve modeller. milli eğitim dergisi, 171, 188-196. uzunkavak, m. (2009). revealing discrimination skills of students between positive and negative work by writing and drawing method. international journal of technologic sciences, 1(2), 10-20. vosniadou, s. (1994). capturing and modeling the process of conceptual change. learning and instruction, 4, 45-69. watts, d. m. (1983). some alternative views of energy. physics education, 18, 213-217. yıldırım, a. (1996). disiplinlerarası öğretim kavramı ve programlar açısından doğurduğu sonuçlar. hacettepe üniversitesi eğitim fakültesi dergisi, 12(12). appendix 1. define the work concept? explain if it is scalar or vector quantity and its unit. 2. explain with reasons whether or not work has been done physically in the given situations below? a) a free falling object b) man carrying a load on his shoulder in the horizontal plane c) a child walking on an inclined path 3. what are the relations among the works done against gravity when three bodies with identical mass are removed from the a, b and c paths to the point m? explain your answer. http://void(0)/ a © 2020 indonesian society for science educator 79 j.sci.learn.2020.3(2).79-88 received: 23 november 2019 revised: 25 february 2020 published: 17 march 2020 an investigation on the effect of stem practices on sixth grade students’ academic achievement, problem solving skills, and attitudes towards stem müzdelife kurt1, semra benzer2* 1education institute, gazi university, turkey 2science education, gazi university, turkey *corresponding author. sbenzer@gazi.edu.tr abstract this study aims to identify the effect of stem practices, integrated into science courses in 6th grades of middle schools, on students' academic achievement, problem-solving skills, their attitudes towards stem, and their interest level of stem fields. the population of the study consists of 6th-grade students enrolled in a middles school in the 2018-2019 academic year. during the research, academic achievement test, stem attitude scale, stem career interest survey, and problem solving inventory were applied to experimental and control groups as pre and post-tests by quasi-experimental research design. in data analysis, a statistics package program was used. results of the study showed that the academic achievement test scores of experimental groups that received stem practices were seen to be higher than that of the control group to which the constructivist approach was applied. the difference was observed to be meaningful. also, when the post-test scores of the stem attitude scale, stem career interest survey, and problem solving inventory were compared, a meaningful difference was found between the experimental and control group. keywords academic achievement, attitude, interest, problem solving, stem 1. introduction in this era of fast-developing technology, countries compete with intellectual power rather than physical power and adapt to developing information and technology. the only way to attain this success is education. however, raising individuals who can produce creative solutions and address problems from various viewpoints is only possible with integrating multidisciplinary practices into educational systems (çevik & özgünay, 2018). in the dynamic process of developing communities, many fields, including economy, science, medicine, social sciences, engineering, mathematics, and finance, comprise of complicated systems (sabelli, 2006). different educational approaches have been developed for such operations as data analysis, data conversion, and new data generation (english & sriraman, 2010). in order to utilize new approaches in education, individuals who are keen on science and enjoy lifelong learning, open to innovation, able to solve a problem, productive and enterprising are needed. known as 21stcentury skills with the created educational settings, critical thinking and problem-solving, collaboration, and across networks and leading by influence, agility and adaptation, capable oral and written communication, accessing and analyzing information, curiosity, and imagination should be taught to students (wagner, 2008). the idea of raising a responsible, problem-solver, and an active individual in the process of interpreting information has produced the opinion of organizing the curriculum according to progressivism philosophy (yıldırım & altun, 2015). as the world's new educational trend, stem, consisting of the initials of science, technology, engineering, and mathematics, is an educational approach that aims to help individuals gain information and skills with an interdisciplinary approach. another aim of stem is to raise individuals who can handle the economic power that can offer an advantage in the industrialization of countries and to transform mailto:sbenzer@gazi.edu.tr journal of science learning article doi: 10.17509/jsl.v3i2.21419 80 j.sci.learn.2020.3(2).79-88 countries into economically and industrially developed countries (çevik & özgünay 2018). besides, increasing the number of students who study stem fields and wish to pursue their professional life in these fields and raising stem literate individuals are among the targets of this approach (nrc, 2011). stem education provided in educational environments promotes interdisciplinary information and skills for life. it prepares individuals for an informationbased economy (nrc 2011). the general purpose of stem education is to advance future generations with an innovative approach (çorlu, capraro, & capraro, 2014). in the stem approach, the practice setting is arranged as it is in real life. students gain experience in these settings with problem-solving and project-based learning methods by pretending as real engineers, scientists, and technologists (breiner, harkness, johnson, & koehler, 2012). in today's world, where information and technology are rapidly developing, one of the skills known as "21stcentury skills", defined by p21, is the problem-solving skill (p21, 2018). apart from being an integrated approach, stem education is essential for individuals to develop 21st-century skills (yıldırım & altun, 2015). students who gain problem-solving skills can produce new information and improve reasoning skills and creativity (abdullah, halim & zakaria, 2014). in order for individuals to develop interest and attitude towards science and mathematics, they need to know the concepts related to these fields and interpret science and mathematics. the reason why students do not have any positive or negative attitude towards engineering is that their engineering achievements are not included in the primary and secondary school curriculum (yılmaz, yiğit-koyunkaya, güler, & güzey, 2017). if individuals are asked to pursue a career in stem, the interest and attitude towards stem should be changed positively (moore & richards, 2012). technology and engineering, which enables the application of science and mathematics in our lives, spread to all areas of daily life and offers solutions to the problems of all times of humanity (nrc, 2012). there is also a relationship between variables of interest and attitude and academic achievement. for this reason, the academic success of students will be examined in this research. national and international studies conducted in primary education level are mostly conducted with 5th, 7th, and 8th grades (elliott, oty, mcarthur, & clark, 2001; robinson, 2005; riskowski, todd, wee, dark, & harbor, 2009; schnittka & bell, 2011; olivarez, 2012; wyss, heulskamp, & siebert, 2012; king & english, 2016; christensen & knezek, 2017). when studies on stem practices are analyzed, no study jointly investigating variables of problem-solving, attitude towards stem, and interest for stem fields are not found. thus, this study is believed to contribute to the literature. in this study, the aim is to investigate the effect of stem practices on 6th-grade students' academic achievement, problem-solving skills, their attitudes towards stem, and to identify their interest level for stem fields. it will be determined whether there is a statistically significant difference between the academic achievements of the control and experimental group students, their problemsolving skills, their attitudes towards stem, and their interest in stem areas before and after the study. 2. method 2.1 research model in this study, the pretest posttest-controlled group design was employed as a quasi-experimental research design. (büyüköztürk, çakmak, akgün, karadeniz, & demirel, 2016). the independent variable of pretest posttest-controlled group design is stem practices; dependent variables are students' problem-solving skills, their interest in stem fields, their attitudes towards stem practices, and their academic achievement. in this regard, academic achievement test (aat), stem attitude scale (sac), stem career interest survey (stem-cis), and problem solving inventory (psi) were applied as pre and posttests. the research design of the study is presented in table 1. 2.2 study consists the population of the study consists of 6th-grade students enrolled in a middle school in the central district of çankırı province in turkey. table 2 shows the distribution of students according to gender. table 1 research design group learning model pre-tests post-tests control group constructivist approach aat, sac, psi, stem-cis aat, sac, psi, stemcis experimental group stem approach aat, sac, psi, stem-cis aat, sac, psi, stemcis, interview form table 2 descriptive statistics results for students in experimental and control groups in terms of gender group gender female male total n % n % control group 7 54.00 6 46.00 13 experimental group 10 76.92 3 23.08 13 total 17 65.38 9 34.61 26 journal of science learning article doi: 10.17509/jsl.v3i2.21419 81 j.sci.learn.2020.3(2).79-88 2.3 data collection tool academic achievement test (aat) aat has been developed in this study with the aim of assessing students' academic achievement before and after the application. a pilot study of aat was conducted with 137 students in the 2018-2019 academic year. the analysis of aat was performed on a statistics program. three items with a discrimination index below 0.20 were excluded from the test. analyses proved that aat is valid and reliable, and the test consisting of 26 items was employed in the study. item analyses of aat are presented in table 3. problem solving inventory (psi) problem solving inventory, developed by heppner & peterson (1982) and adapted to turkish by şahin, şahin, & heppner (1993), was used in this study. stem attitude scale (sac) stem attitude scale, developed by faber, unfried, wiebe, corn, townsend, & collins (2013) and adapted to turkish by yıldırım & selvi (2015), was employed in the study. there are eight questions on "mathematics", nine questions on "science", nine questions on "engineering and technology", and 11 questions on "21st century skills". stem career interest survey (stem-cis) stem career interest survey developed by kier, blanchard, osborne, & albert (2013) and adapted to turkish by pekbay (2017) was used in the study. there are nine questions on "mathematics", nine questions on "science", nine questions on "engineering", and nine questions on "technology". table 3 item analyses of aat pre-test form for the unit of electrical conduction item no difficulty index (pj) discrimination index (rjx) item no difficulty index (pj) discrimi nation index (rjx) 1 0.51 0.66 16 0.93 0.18** 2 0.64 0.56 17 0.93 0.13** 3 0.65 0.51 18 0.82 0.51 4 0.68 0.61 19 0.40 0.00** 5 0.63 0.63 20 0.28 0.63 6 0.64 0.48 21 0.60 0.48 7 0.07 0.68 22 0.28 0.71 8 0.53 0.58 23 0.23 0.30 9 0.74 0.43 24 0.82 0.38 10 0.64 0.48 25 0.76 0.48 11 0.60 0.53 26 0.42 0.73 12 0.50 0.46 27 0.61 0.76 13 0.74 0.56 28 0.45 0.76 14 0.39 0.63 29 0.18 0.38 15 0.80 0.33 **note: excluded items are shown as bold and with **. table 4 stages of practices done with the control group subject area topic and concepts let’s experiement! interactive activies assessment and evaluation conductors and non-conductors conductors non-conductors fields of usage for conductors and non-conductors which materials conduct electricity? are water and air conductors? eba morpa kampus okulistik subject assessment test 1 electrical resistance and related factors electrical resistance factors affecting electrical resistance which makes light brighter? thick or thin? short or long? eba morpa kampüs okulistik subject assessment test 2 assessment of the unit table 5 stages of practices done with the experimental group 1st stage 2nd stage 3rd stage 4th stage 1. conductors and nonconductors angles building electrical circuit presenting the informationbased life problem model house design by using engineering design processes the relationship between model house lightning and electrical conduction 2. real life usage of electrical conductance and resistivity angles presenting the informationbased life problem water transfer pump design by using engineering design processes water transfer pump 3. building electrical circuit energy transformation conductors and nonconductors presenting the informationbased life problem liquid level control circuit design by using engineering design processes liquid level control circuit 4. angles algebra real life usage of electrical conductance and resistivity presenting the informationbased life problem vacuum cleaner design by using engineering design processes vacuum cleaner journal of science learning article doi: 10.17509/jsl.v3i2.21419 82 j.sci.learn.2020.3(2).79-88 stages of application the teaching process for the control group, conducted in line with the constructivist approach, lasted eight weeks. the control group consisting of 13 students, were divided into four groups. concepts provided in the 6th grade science coursebook were taught to students, and they were asked to take notes. experiments in the coursebook were conducted together with the groups. practices done with the control group are presented in table 4. examples from students' activities that have been produced during implementation using the stem approach in class are shown in figure 1. the teaching process for the experimental group, conducted in line with the stem approach, lasted eight weeks. stem integration process steps offered by yıldırım & selvi (2017) were applied in the study. practices done with the experimental group are presented in table 5. these steps are explained as follows: • identifying the field: the field where stem is to be integrated is identified. in this study, the field was chosen as the "science course". • identifying the learning domain: learning domains within the identified field are chosen. in this study, the learning domain was chosen as "conductors and nonconductors" and "electrical resistance and related factors" presented in the unit called "electrical conduction". • identifying the relationship with disciplines: the relationship between the identified learning domain and mathematics, technology, and engineering disciplines is identified. it is the most important and difficult stage of the stem integration process. in this study, a connection was established between learning domains of "conductors and non-conductors" and "electrical resistance and related factors" in science course and "angles and algebra" learning domains of mathematics course. as there is no course related to technology and engineering in the primary education curriculum, the integration of these domains was provided with combining learning outcomes of technology and design course with technology and engineering. • designing stem practices: in this study, four practices appropriate for stem integration were designed. • application: application and assessment of designed stem practices (yıldırım & selvi, 2017). 3. result and discussion results obtained from the independent samples t-test carried out for determining whether there is a difference among academic achievement, problem-solving skills, attitudes towards stem, and interest for stem fields for students in both control and experimental groups before the application are given in table 6. there is no meaningful difference among aat, sac, psi, stem-cis pretest mean scores of students in the control and experimental group. thus, it is possible to say that academic achievement, problem-solving skills, attitudes towards stem, and interest for stem levels of students in both groups are equal (table 6). results obtained from the t-test carried out for determining how academic achievement, problem-solving skills, attitudes towards stem, and interest for stem fields for students figure 1 examples from students' activities of developed in this study table 6 t-test results regarding pre-test scores of aat, sac, psi, stem-cis for control and experimental group students group scale n �̅� ss t p control group aat 13 0,.42 0.03 -2.941 0.060 experimental group 13 0.46 0.11 control group pci 13 2.81 0.46 -2.678 0.052 experimental group 13 3.79 0.22 control group sac 13 3.01 0.32 -2.954 0.064 experimental group 13 3.33 0.21 control group stemcis 13 3.10 0.35 0.328 0.746 experimental group 13 3.16 0.52 *p<0.05 journal of science learning article doi: 10.17509/jsl.v3i2.21419 83 j.sci.learn.2020.3(2).79-88 in both control and experimental groups differ before and after the application are given in table 7. there is a meaningful difference between aat pretest and posttest mean scores of students in the control group while there is no difference among pretest and posttest mean scores of sac, psi, stem-cis. a meaningful difference is observed among experimental group students' pretest, and posttest mean scores of aat, sac, psi, stem-cis (table 7). results obtained from the independent samples t-test carried out for determining whether there is a difference among academic achievement, problem-solving skills, attitudes towards stem, and interest for stem fields for students in both control and experimental groups differ after the application is given in table 8. there is a meaningful difference among aat, sac, psi, stem-cis posttest mean scores of students in the control and experimental group (table 8). results obtained from the independent samples t-test carried out for determining how data obtained from the sub-dimensions of sac and stem-cis for students in both control and experimental groups differ before and after the application are given in table 9. there is no meaningful difference between control group students' pretest and posttest mean scores of sac sub-dimensions, which are "science", "mathematics", "engineering and technology", and "21st-century skills". while there is a meaningful difference between pretest and posttest mean scores of "science" and "mathematics" sub-dimensions of stem-cis, there is no meaningful relationship between pretest and posttest mean scores of "technology" and "engineering" sub-dimensions (table 9). results obtained from the independent samples t-test carried out for determining how data obtained from the sub-dimensions of sac and stem-cis for students in the experimental group differ before and after the application are given in table 10. there is a meaningful difference between pretest and posttest mean scores of "science", "mathematics", "engineering and technology", and "21st-century skills" of "science" sub-dimensions of sac. there is no meaningful difference in "science", "mathematics", and "engineering" sub-dimensions of stem-cis while there is no meaningful difference in "technology" sub-dimension (table 10). table 7 comparative t-test results regarding pre-test and post-test scores of aat, sac, psi, stem-cis for control and experimental group students groups scale n pre-test post-test t p �̅� ss �̅� ss control group aat 13 0.42 0.03 0.55 0.12 -4.124 0.001* experimental group 13 0.46 0.11 0.71 0.15 -4.830 0.000* control group pci 13 2.81 0.46 3.42 0.48 4.042 0.65 experimental group 13 3.79 0.22 3.84 0.52 5.961 0.000* control group sac 13 3.01 0.32 3.21 0.84 -1.178 0.261 experimental group 13 3.33 0.06 4.07 0.08 -8.053 0.000* control group stem-cis 13 3.10 0.35 3.78 0.69 -3.767 0.090 experimental group 13 3.16 0.52 3.82 0.72 -3.532 0.004* *p<0.05 table 8 comparative t-test results regarding post-test scores of aat, sac, psi, stem-cis for control and experimental group students group scale n �̅� ss t p control group aat 13 0.55 0.12 -2.891 0.008* experimental group 13 0.71 0.15 control group pci 13 3.42 0.48 0.483 0.033* experimental group 13 3.84 0.56 control group sac 13 3.21 0.84 -3.446 0.002* experimental group 13 4.07 0.31 control group stem-cis 13 3.78 0.69 -2.453 0.009* experimental group 13 3.82 0.72 *p<0.05 journal of science learning article doi: 10.17509/jsl.v3i2.21419 84 j.sci.learn.2020.3(2).79-88 results obtained from the independent samples t-test carried out for determining whether there is a difference between the sub-dimensions of sac and stem-cis for students in both control and experimental groups after the application are given in table 11. there is a meaningful difference among posttest mean scores of "science", "mathematics", and "21st-century skills" subdimensions of sac while there is no meaningful difference in "engineering and technology" subdimension. also, there is a meaningful difference among posttest mean scores of "science" sub-dimension of stem-cis while there is no meaningful relationship in "mathematics", "engineering", and "technology" subdimensions (table 11). a meaningful difference was found between aat pretest and posttest scores of control group students who pursued their science course in its normal procedure. this result indicates that the constructivist approach increases the learning level of students in science courses. an increase was also found in students' learning levels after the application of stem practices in the experimental group. this shows that stem practices integrated into science course is effective in increasing students' learning level. in the comparison of control and experimental table 9 t-test results regarding pre-test and post-test scores of sac and stem-cis sub-dimensions for control group students scale sub-dimension n �̅� ss t p sac mathematics pre-test 13 2.84 0.30 0.537 0.601 post-test 13 2.76 0.55 science pre-test 13 2.99 0.46 -1.624 0.130 post-test 13 3.23 0.76 engineering and technology pre-test 13 3.00 0.42 -1.205 0.251 post-test 13 3.48 1.71 21st century skills pre-test 13 3.18 0.55 -0.450 0.661 post-test 13 3.27 1.27 stem-cis mathematics pre-test 13 3.23 0.54 -4.754 0.000* post-test 13 4.11 0.76 science pre-test 13 3.23 0.39 -5.103 0.000* post-test 13 3.72 0.66 technology pre-test 13 3.29 0.78 -2.630 0.103 post-test 13 3.91 0.87 engineering pre-test 13 2.88 0.71 -3.210 0.260 post-test 13 3.38 0.95 *p<0.05 table 10 t-test results regarding pre-test and post-test scores of the sub-dimensions of aat, sac, psi, stem-cis for control and experimental group students scale sub-dimension n �̅� ss t p sac mathematics pre-test 13 3.17 0.41 -3.635 0.003* post-test 13 4.74 0.56 science pre-test 13 3.36 0.31 -4.641 0.000* post-test 13 4.11 0.48 engineering and technology pre-test 13 3.21 0.66 -3.692 0.003* post-test 13 4.00 0.47 21st century skills pre-test 13 3.48 0.19 -2.276 0.000* post-test 13 4.37 0.34 stem-cis mathematics pre-test 13 2.99 0.73 -2.763 0.017* post-test 13 3.83 0.77 science pre-test 13 3.65 0.49 -2.665 0.021* post-test 13 4.04 0.54 technology pre-test 13 3.24 0.43 -1.337 0.206 post-test 13 3.46 0.89 engineering pre-test 13 2.52 0.48 -3.136 0.009* post-test 13 3.35 1.0 *p<0.05 journal of science learning article doi: 10.17509/jsl.v3i2.21419 85 j.sci.learn.2020.3(2).79-88 groups' aat posttest results, it is seen that there is a meaningful difference in favor of the experimental group. this proves that the stem integrated science course has a more positive effect on increasing students' learning levels comparing to the constructivist approach. this result is consistent with previous studies showing that stem practices are effective in increasing students' academic achievement. marulcu & mercan höbek (2014) investigated the effect of activities they developed with the engineering design method on the academic achievement of 8th-grade students. their results indicate that the engineering design method has a meaningful effect on students' learning levels. yılmaz, gülgün, & çağlar (2017) stated that stem activities they developed for "power and energy" unit had a meaningful effect on students' academic achievement and their attitudes towards stem. in a study by çevik & abdioğlu (2018), it is shown that stem activities positively increase the academic achievement of 8th-grade students. in another study conducted by i̇nce, mısır, küpeli, & fırat (2018), it was concluded that stem practices positively affected 5th-grade students' academic achievement and problemsolving skills. ergün & balçın (2019) also found in their single-group pretest and posttest study that problembased stem practices increased students' academic achievement. no meaningful difference was found between psi pretest and posttest scores of control group students who pursued their science course in its normal procedure. this result indicates that the constructivist learning approach has no effect on improving students' problem-solving skills. however, in the experimental group, a meaningful increase was found after stem-based activities. this may suggest that stem practices integrated into science courses are effective in improving students' problemsolving skills. when the posttest results of the control and experimental group were compared, a meaningful difference was found in favor of the experimental group. this shows that stem integrated science course has a more positive effect in improving students' problemsolving skills comparing to the constructivist approach. results obtained in this study confirm previous studies in the literature. in a doctoral dissertation by ceylan (2014), the effect of stem activities on 8th-grade students was investigated. the results of the study showed that stem activities positively affected students' problem-solving skills. in a study by şahin, ayar, & adıgüzel (2014), the effect of extracurricular stem activities based on collaboration was investigated, and they were found to improve students' problem-solving skills. dewaters & powers (2006) also found that holistic stem education improved students' problem-solving skills in real life. no meaningful difference was observed in attitudes towards stem in control group students, who were applied constructivist approach, before and after the application. however, in the experimental group provided with stem practices, a meaningful difference was found in students' attitudes towards stem before and after the application. this finding suggests that stem practices integrated into a science course are effective in improving students'attitudes towards stem. before and after the application, no meaningful difference was observed in sub-dimensions of attitudes towards stem, which are "science", "mathematics", "engineering and technology", "21st-century skills" in the control group. however, in the control group where stem integrated science course was table 11 comparative t-test results regarding post-test scores of sac and stem-cis sub-dimensions for control and experimental group students scale sub-dimensions group n �̅� ss t p sac mathematics pre-test 13 2.76 0.55 -5.029 0.000* post-test 13 3.74 0.56 science pre-test 13 3.23 0.48 -2.964 0.012* post-test 13 4.11 0.76 engineering and technology pre-test 13 3.48 0.47 -0.976 0.349 post-test 13 4.00 1.71 21st century skills pre-test 13 3.27 1.27 -2.718 0.019* post-test 13 4.37 0.34 stem-cis mathematics pre-test 13 3.83 0.76 0.868 0.403 post-test 13 4.11 0.77 science pre-test 13 3.72 0.54 -3.137 0.042* post-test 13 4.04 0.66 technology pre-test 13 3.46 0.87 1.291 0.221 post-test 13 3.91 0.89 engineering pre-test 13 3.25 0.95 0.088 0.931 post-test 13 3.38 1.00 *p<0.05 journal of science learning article doi: 10.17509/jsl.v3i2.21419 86 j.sci.learn.2020.3(2).79-88 conducted, a meaningful difference was observed in subdimensions of attitudes towards stem which are "science", "mathematics", "engineering and technology", "21st-century skills". a meaningful difference was found in attitudes towards stem in both control and experimental group students after the application. these results indicate that stem integrated science course has a more positive effect in improving students' attitudes towards stem comparing to the constructivist approach. when the literature is reviewed, it is seen that there are studies that have reached similar results. aydın, saka, & guzey (2017) adapted the stem attitude scale, developed by guzey, harwell, & moore (2014), to turkish and aimed to identify middle school students' attitudes towards stem. their results showed that students who have not received stem education had a moderate level attitude. in a study by damar, durmaz, & önder (2017), the effect of stem practices on middle school students' attitudes towards stem was investigated. according to obtained data, stem practices were found to increase the attitude towards stem. karışan & yurdakul (2017) investigated the effect of stem practices on 6th-grade students' attitudes towards stem and found a positive change after the study. in a study by özcan & koca (2019), the effect of stem practices on 7th-grade students' attitudes towards stem was investigated. results of the study showed that when academic achievement of control and experimental group were compared, a meaningful difference was observed in favor of the experimental group. also, practices were seen to improve experimental group students' attitudes towards stem. no meaningful difference was observed in interest for stem in control group students, who were applied constructivist approach, before and after the application. this indicates that the constructivist approach has no meaningful effect on students' interest in stem. however, in the experimental group, a meaningful difference was found in students' interest in stem before and after the application. this result shows the meaningful effect of stem practices on students' interest in stem. before and after the application, a meaningful difference was detected in sub-dimensions of interest for stem, which is "science" and "mathematics" while no meaningful effect was observed in sub-dimensions of "engineering" and "technology" in the control group. this result can be explained by the fact that they receive science and mathematics courses in the primary school curriculum. before and after the application, a meaningful difference was observed in sub-dimensions of interest for stem, which are "science", "mathematics" and "engineering"; however, no meaningful difference was seen in sub-dimensions of "technology" in the experimental group. a meaningful difference was found in both the control and experimental group students' interest in stem after the application. this may suggest that stem integrated science course has a more positive effect in improving students' interest in stem comparing to the constructivist approach. although no previous studies which are in complete agreement with this result have been found in the literature, studies investigating students' interest in stem fields will be mentioned. mohr-schroeder, jackson, miller, walcott, little, speler, schooler, & schroeder (2014) aimed to improve 5-8th grade students' interest in stem in the blue stem camp. after the 5-day camp organized with moderate level students who have a discipline problem in the class, it was seen that their interest and motivation increased, they found the activities fun, and their interest in a career in stem fields showed a 3% increase. in a study by irkıçatal (2016), students' interest in professions in stem fields was investigated. the results of the study indicated that there was a meaningful difference in "engineering" and "technology" sub-dimensions while no difference was observed in "mathematics" and "science" sub-dimensions. benek & akçay (2018) identified middle school students' opinions on stem designs. results of the study conducted with stem drawing form showed that students made use of technology the most and mathematics the least. 4. conclusion this study explores stem applications on the 6thgrade students' academic success, problem-solving skills, and attitudes towards stem. the data obtained from aat, pci, sac, and stem-cis, which are applied to the control and experimental group students as pre-test and post-test, are evaluated; it can be said that the control group students of pci "mathematics", "engineering and technology" sub-dimensions have a significant effect on the pre-test mean scores and the "math" sub-dimension general pre-test mean scores. in light of the results obtained in the current study, the following suggestions on stem practices can be made: first, turkey's educational system can be organized in accordance with the stem approach and enhanced with collaboration among university, school, and community. second, by opening science centers in each city of turkey, the collaboration of schools with science centers and inservice training for teachers in these centers can be provided. third, prospective teachers' skills in stem can be increased by conducting studies on stem education in the faculties of education. also, by promoting the collaboration among faculties of education, engineering, and science, prospective teachers can be provided with interdisciplinary stem education. fourth, stem practices can be done starting from pre-school education to higher education without limiting it to one class level only. fifth, though it can be difficult to integrate stem into each topic in the curriculum, its effect can be investigated on appropriate topics by conducting stem journal of science learning article doi: 10.17509/jsl.v3i2.21419 87 j.sci.learn.2020.3(2).79-88 practices. sixth, by providing collaborative in-service training for science and mathematics teachers working in the schools of the ministry of national education, engineering education can also be included in this training. seventh, stem education can be provided for gifted students at a more advanced level, and their opinions on having a career in stem fields can be motivated. eighth, more research on stem can be made, and awareness of stem can be increased. acknowledgment the authors would like to acknowledge the teacher and the students who participated in and made this article possible. the authors would also like to thank the reviewers for their valuable comments. this article is based on the part of the first author's master's thesis. references abdullah, n., halim, l., & zakaria, e. (2014). stops: a thinking strategy and visual representation approach in mathematical word problem solving toward enhancing stem literacy. eurasia journal of mathematics, science and technology education, 10(3), 165-174. aydın, g., saka, m., & guzey, s. (2017). science, technology, engineering, mathematic (stem) attitude levels in grades 4 8. mersin university journal of faculty of education, 13(2), 787-802. benek, i̇. & akçay, b. (2018). stem in the my imaginary world! investigation of student’s drawings in stem field. journal of steam education, 1(2), 79-107. breiner, j., harkness, s., johnson, c. c., & koehler, c. m. (2012). what is stem? a discussion about conceptions of stem in education and partnerships. school science and mathematics, 112(1), 3-11. büyüköztürk, ş., çakmak, e. b., akgün, ö. e., karadeniz, ş., & demirel, f. (2016). bilimsel araştırma yöntemleri. ankara: pegem. çevik, m. & abdioğlu, c. (2018). an investigation of the effects of a science camp on the stem achievements, science motivations and metacognitive awareness of 8th grade students. journal of human ad social science research, 7(5), 304-327. çevik, m. & özgünay, e. (2018). stem education through the perspectives of secondary schools’ teachers and school administrators in turkey. asian journal of education and training, 4(2), 91-101. ceylan, s. (2014). a study for preparing an instructional design based on science, technology, engineering and mathematics (stem) approach on the topic of acids and bases at secondary school science course. msc thesis, uludağ university institute of education, bursa, turkey. p 279. christensen, r. & knezek, g. (2017). relationship of middle school student stem interest to career intent. journal of education in science, environment and health (jeseh), 3(1), 1-13. çorlu, m. s., capraro, r. m., & capraro, m. m. (2014). introducing stem education: implications for educating our teachers for the age of innovation. education and science, 171(39), 74-85. damar, a., durmaz, c., & önder, i̇. (2017). middle school students’ attitudes towards stem applications and their opinions about these applications. journal of multidisciplinary studies in education, 1(1), 47-65. dewaters, j. & powers, s. e. (2006). improving science and energy literacy through project-based k-12 outreach efforts that use energy and environmental themes. proceedings of the 113th annual asee conference and exposition, chicago, il. elliott, b., oty, k., mcarthur, j., & clark, b. (2001). the effect of an interdisciplinary algebra/science course on students‟ problem solving skills, critical thinking skills and attitudes towards mathematics. international journal of mathematical education in science and technology, 32(6), 811–816. english, l. & sriraman, b. (2010). problem solving for the 21 st century. in theories of mathematics education. springer, berlin, heidelberg. ergün, a. & balçın, m. d. (2019). the effects of problem-based stem applications on academic success. journal of limitless education and research, 4(1), 40-63. faber, m., unfried, a., wiebe, e. n., corn, j., townsend, l.w., & collins, t. l. (2013). student attitudes toward stem: the development of upper elementary school and middle/high school student surveys. 120th asse annual conference & exposition. atlanta. guzey, s. s., harwell, m., & moore, t. (2014). development of an instrument to assess attitudes toward science, technology, engineering, and mathematics (stem). school science and mathematics, 114(6), 271-279. heppner, p. p. & peterson, c. h. (1982). the development and implications of a personal problem-solving inventory. journal of counseling psychology, 29, 66-75. i̇nce, k., mısır, m. e., küpeli, m. a., & fırat, a. (2018). examining the effect of stem-based approach on the problem-solving ability and academic success of students in teaching the enigma of the earth’s crust unit of the 5th grade life sciences course. journal of stem education, 1(1), 64-78. irkıçatal, z. (2016). stem related afterschool program activities and associated outcomes on student’s success and on their stem perception and interest. msc thesis. akdeniz university institute of education, antalya, turkey. karışan, d. & yurdakul, y. (2017). the effects of microprocessors based science technology engineering and mathematics (stem) investigations on 6th grade students’ attitudes towards these subject areas. adnan menderes university education faculty journal of education science, 8(1), 37-52. kier, m. w., blanchard, m. r., osborne, j. w., & albert, j. l. (2013). the development of the stem career interest survey (stemcis). research in science education, 44(3), 461-481. king, d. & english, l. d. (2016). engineering design in the primary school: applying stem concepts to build an optical instrument. international journal of science education, 38(18), 27622794. marulcu, i̇. & mercan höbek, k. (2014). teaching alternate energy sources to 8th grades students by engineering design method. middle eastern & african journal of educational research, 9, 42-58. mohr-schroeder, m. j., jackson, c., miller, m., walcott, b., little, d. l., speler, l., schooler, w., & schroeder, d. c. (2014). developing middle school students’ interests in stem via summer learning experiences: see blue stem camp. school science and mathematics, 114(6), 291–301. moore t. & richards l. g. (2012). p-12 engineering education research and practice. advances in engineering education, 3(2), 1-9. nrc (national research council) (2011). succesful k-12 stem education, washington, dc: national academy. olivarez, n. (2012). the impact of a stem program on academic achievement of eighth grade students in a south texas middle school, doctoral thesis, texas a & m university, corpus christi, texas. özcan, h. & koca, e. (2019). the impact of teaching the subject “pressure” with stem approach on the academic achievements of the secondary school 7th grade students and their attitudes towards stem. education and science, 44(198), 201-227. p21 (2018). partnership for 21st century learning 2015. http:// www.p21.org/storage/documents/p21_framework_0515.pdf pekbay, c. (2017). effects of science technology engineering and mathematics activities on middle school students. phd thesis, hacettepe university, education science institute, ankara.turkey. riskowski, j. l., todd, c. d., wee, b., dark, m., & harbor, j. (2009). exploring the effectiveness of an interdisciplinary water resources engineering module in an eighth-grade science course. international journal of engineering education, 25(1), 181-195. http://www.p21.org/storage/documents/p21_framework_0515.pdf journal of science learning article doi: 10.17509/jsl.v3i2.21419 88 j.sci.learn.2020.3(2).79-88 robinson, m. (2005). robotics-dricen activities: can they improve middle school science learning? bulletin of science, technology and society, 25(1), 73-84. sabelli, n. h. (2006). complexity, technology, science, and education. the journal of the learning sciences, 15(1), 5-9. şahin, a., ayar, m. c., & adıgüzel, t. (2014). stem related afterschool program activities and associated outcomes on student learning. educational sciences: theory and practice, 14(1), 309–322. şahin, n., şahin, n. h., & heppner, p. p. (1993). the psychometric properties of the problem-solving inventory. cognitive therapy and research, 17(4), 379-396. schnittka, c. & bell, r. (2011). engineering design and conceptual change in science: addressing thermal energy and heat transfer in eighth grade. international journal of science education, 33(13), 18611887. wagner, t. (2008). rigor redefined. educational leadership, 66(2), 20-24. wyss, v. l., heulskamp, d., & siebert, c. j. (2012). increasing middle school student interest in stem careers with videos of scientists. international journal of environmental and science education, 7(4), 501522. yıldırım, b. & altun, y. (2015). investigating the effect of stem education and engineering applications on science laboratory lectures. el-cezeri journal of science and engineering, 2(2), 28-40. yıldırım, b. & selvi, m. (2015). adaptation of stem attitude scale to turkish. turkish studies-international periodical for the languages, literature and history of turkish or turkic, 10(3), 11071120. yıldırım, b. & selvi, m. (2017). an experimental research on effects of stem applıcatıons and mastery learning. journal of theory and practice in education, 13(2), 183-210. yılmaz, a., gülgün, c., & çağlar, a. (2017). teaching with stem applications for 7th class student’s unit of" force and energy": let's make a parachute, water jet, catapult, intelligent curtain and hydraulic work machine (bucket machine) activities. journal of current researches on educational studies (jocures), 7(1), 97-116. yılmaz, h., yiğit-koyunkaya, m., güler, f., & güzey, s. (2017). turkish adaptation of the attitudes toward science, technology, engineering, and mathematics (stem) education scale. kastamonu education journal, 25(5), 1787-1800. a © 2020 indonesian society for science educator 36 j.sci.learn.2020.3(2).36-45 received: 24 october 2019 revised: 26 november 2019 published: 13 december 2019 examination of prospective science teachers’ levels of designing project directed to national support programs: a profile of turkey gülşah sezen vekli1* 1department of elementary science education, yozgat bozok university, yozgat, turkey *corresponding author. gulsahsezen28@gmail.com abstract this study aims to determine prospective science teachers’ levels of designing projects about the scientific and technological research council of turkey, written in the abbreviation tubitak’s 2209-a “support for university students’ research projects” and their views on the process. the study was conducted with the participation of 67 prospective science teachers. project proposal forms and reflective reports were used in the study, which was conducted in the pedagogical activity research methodin collecting the data. the data coming from the reflective reports were then put to content analysis. it was remarkable in consequence that the majority of the prospective teachers ’levels of designing projects were inadequate. the prospective science teachers stated that they had learned the stages of scientific projects in the process of project designing and that they also became skillful at collaborating, researching, problem-solving, writing in academic language, working in a planned manner and at communicating. keywords national support programs, prospective science teacher, designing project 1. introduction one of the fundamental goals of science teaching is to raise individuals who are knowledgeable about the ways of accessing knowledge and who are capable of using scientific knowledge effectively. in other words, the goal is to raise scientifically literate individuals. students should be provided with authentic learning environments that are rich in content, in which activities are done, which are based on real-life problems and which are research-inquiry oriented (timur & i̇mer çetin, 2017). activities such as national or international scientific projects in which students can participate are, undoubtedly, important at this point. it was demonstrated by studies that developing scientific projects were influential in making them gain scientific methods (özer & özkan, 2012), that it contributed to developing interest in and positive attitudes towards science (camcı, 2008; şahin, 2012; çeliker & erduran avcı, 2015; yıldırım & şensoy, 2016) and to students’ choice of a profession related to science (fisanick, 2010; dionne, reis, trudel, guillet, kleine & hancianu, 2012). on the other hand, teachers’ ways of counseling and guidance to students in preparing projects are fundamental in obtaining the gains (capraro et al., 2016). teachers are expected to monitor their students in the process of planning and implementing scientific projects and to guide them as far as possible in the process (erdem, uzal & ersoy, 2006). however, it is apparent that teachers do not comprehend the concept of guidance adequately, that they cannot lead students adequately and that they need help in this respect (tortop, 2013; kaplan & coşkun, 2012; ünver, arabacıoğlu & okulu, 2015). in addition to that, it was also found that teachers had problems in such matters as inventing an idea of the project and writing a project in the process of preparing projects (kaplan & coşkun, 2012; özel & akyol, 2016; timur & imer çetin, 2017). as oğuz ünver, arabacıoğlu and okulu (2015) also point out, teachers’ inadequacy in their experience with and their skills at projects set considerable obstacles in front of students’ process of preparing projects. sözer (2017) states that teachers and students should be exposed to training in scientific research methods and project development so that they can develop projects of better quality. in a similar vein, timur and imer çetin (2017) recommend that science teacher and prospective science teachers should be offered mailto:gulsahsezen28@gmail.com journal of science learning article doi: 10.17509/jsl.v3i2.20740 37 j.sci.learn.2020.3(2).36-45 sufficient applied rather than theoretical training in project development. it is evident that teachers who are not knowledgeable about the project development process during undergraduate education do not include project work in their professional life or that they include it in very restricted ways (güven, 2013). it was also demonstrated through research that teachers found themselves inadequate in guiding their students in conducting a project since they had not taken a practical course on project development during their undergraduate education (baki & bütüner, 2009; bencze, 2010; tatlı, 2016). prospective teachers in our country do not have training through which they can gain skills in writing such projects during their education at university nor do they have environments of experience in which they can gain skills in writing and conducting projects although several institutionsmainly scientific and technological research council of turkey (tubitak) offer financial support and although there are several programs for which teachers can apply (peten metin, yaman, vekli sezen & çavuş, 2019). tubitak offers scientists to support programs to increase the number of applications for projects. however, because such activities are limited, all the prospective teachers across the country cannot benefit from such training on projects. on the other hand, there are hardly any studies in the literature on training prospective teachers for tubitak support programs, and on analyzing the effectiveness of them. peten metin, yaman, vekli sezen & çavuş (2019) demonstrated that, of the tubitak support programs, project preparation training for 4006 science fairs and for 4007 science festival projects improved prospective science teachers’ project preparation/writing skills and that they supported the prospective teachers especially in terms of choosing the types of projects and in terms of writing acceptable projects. however, no studies were found about tubitak 2209-a support for university students' research projects in relation to prospective science teachers. this current study, aiming to determine prospective science teachers’ levels of designing projects in relation to tubitak’s 2209-a “support for university students research projects” and their views on the project designing process, is believed to fill the gap in the literature due to the fact that it is the first study in the literature to determine prospective science teachers’ levels of designing projects within the scope of projects coded as tubitak 2209-a. considering the fact that prospective science teachers might apply for such projects in their schools upon graduation from university or that they might guide their students, it is essential to determine their levels of designing projects in exhibiting the current situation and in making the necessary modifications for their training by taking the necessary precautions. accordingly, this study asked prospective science teachers to design a project for the tubitak 2209/a support program for university students projects, and thus their project designing levels in addition to their views on the process were determined. in accordance with the purpose of the study, answers are sought to the following research questions: 1. what are the project designing levels of prospective science teachers in terms of tubitak 2209-a support for university students’ research projects? 2. what are the views of prospective science teachers on the process of designing projects for tubitak 2209a support for university students’ research projects? 2. method this study aims to determine prospective science teachers’ levels of designing projects in relation to tubitak’s 2209-a “support for university students’ research projects” and their views on the process. to attain its purpose, the research was conducted in the method of pedagogical action research. pedagogical action table 1 the stages followed pedagogical action research, and what is done in the process stages in a pedagogical action research what is done in the process step1 identifying a problem/paradox/difficulty informal observation focal group interviews step 2 searching for ways to solve the problem planning the content for technology and project design i course as applied. step 3 performing the action designing project proposals for tubitak 2209-a support for university students’ research projects within the scope of the technology and project design i course writing individual reflective reports related to the experience at the end of the project designing process step 4 evaluating the results of the action plan analysing the project proposals designed by prospective science teachers for tubitak 2209-a support for university students’ research projects through an evaluation form analysing the reflective reports written by the prospective science teachers step 5 making modifications for future research making recommendations on the basis of the results obtained in the study journal of science learning article doi: 10.17509/jsl.v3i2.20740 38 j.sci.learn.2020.3(2).36-45 research involves identifying a problem or a restriction usually in a university environment or in the environment researchers work, developing a solution to it, applying the solution and monitoring and sharing the results (norton, 2009). the study was performed in turkey. the content of the study, which was conducted in the steps of pedagogical action research, is described below (see table 1). step 1. identifying a problem/paradox/ difficulty the researcher observed a year ago while informing prospective science teachers of a project supported by tubitak (tubitak 2209-a support for university students’ research projects) to encourage participation in the project, that the prospective teachers were not informed of the purpose, scope, and content of the project in question. the focal interviews with two different groups of prospective teachers revealed that they had not even heard of such a project before. step 2. searching for ways to solve the problem the researcher planned the content for the course technology and project design ian elective course which was taught two hours a weekin the form of project design for tubitak 2209-a support for university students’ research projects to offer them training for preparing projects. the process was completed in seven weeks. step 3. performing the action firstly, the prospective teachers were informed of the technology and project design i course (the goal, scope, evaluation process, etc.). secondly, they were offered general information on tubitak 2209-a support for university students’ research projects and they were told that they would be trained in project writing/preparation for the support programme for seven weeks and that they would design a project. information on the process of training in project preparation is shown in table 2. in week one of project preparation training, the project coded as tubitak 2209-a was introduced, the project proposal form used for the application was examined in detail and the groups to design projects were formed. in week two, the proposal forms for the accepted projects were examined and information on the evaluation process was offered. during those two weeks, the groups tried to identify original project subjects outside the classes. in week three, the groups were informed of how to review the literature and how to write the research questions and purpose of the research. outside the classes, the groups reviewed the literature for the subjects they had identified and they wrote the research questions and the purpose of their research. in the following weeks, the prospective teachers were offered information on such parts as eigenvalue, methodology, study plan, widespread impact and abstract; and they were expected to write those parts outside the class time. they received feedback from the lecturer teaching the course throughout the process, both face to face and via e-mails. four of the projects designed table 2 programme for training in project writing/preparation programme for training in project writing/preparation weeks project training programme/content groups’ duties outside classes 1 general introduction to tubitak 2209-a university students’ research projects detailed examination of project proposal forms for tubitak 2209-a university students’ research projects forming the groups identifying an original project subject 2 examining the sample proposal forms for tubitak 2209-a university students’ research projects informing the prospective teachers of the evaluation process for the projects coded as tubitak 2209-a presenting the project subjects in the classroom and voting identifying an original project subject 3 how is literature review done? how are problem statement, research questions and purpose written in the project proposal form? presenting the project subjects in the classroom and voting writing the problem statement, research questions and purpose in the project proposal form 4 how is eigen value written in the project proposal form? writing the problem statement, research questions, purpose and eigen value in the project proposal form 5 how is the methodology written in the project proposal form? how is the study plan written in the project proposal form? writing the methodology and study plan parts in the project proposal form 6 how is widespread impact written in the project proposal form? writing the conclusion part in the project proposal form 7 how is the abstract written in the project proposal form? discussing the factors influencing the acceptance of a project (eigen value, methodology, widespread impact) writing the abstract part in the project proposal form and giving the final shape to the proposal form choosing the projects for applying journal of science learning article doi: 10.17509/jsl.v3i2.20740 39 j.sci.learn.2020.3(2).36-45 by the groups were chosen by the lecturer to send tubitak. step 4. evaluating the results of the action plan the project proposals designed by the prospective teachers within the scope of this study were used as the tool of data collection. additionally, the reports reflecting the prospective teachers’ experiences in relation to the project design processes were also used for evaluation at the end of the process. step 5. making modifications for future research the necessary regulations will be made in accordance with data coming from the prospective teachers’ project proposals and from their reflective reports. the regulations are shown in the recommendations part of this study. 2.1 participants 67 prospective teachers (51 female and 16 male) who were the second and the third-year students and who had taken the course technology and project design i took part in the research (see table 3). of the participants, those who were the third-year students had taken the course scientific research methods whereas those who were the second-year students had not taken the course. the participants were coded as pst1, pst2, pst3, pst67. 2.2 data collection tools two different tools of data collection were used in this study. one of them was the project proposal form (26) designed and prepared by prospective science teachers to determine their levels of project designing in relation to tubitak 2209-a “support for university students’ research projects”. the form was designed by tubitak and was shared on its official website for completing the process of application for the project. the other data collection tool was the “reflective reports” that the prospective teachers were asked to write at the end of the designing process to determine their views on the process of project designing. the reflective report contained the instruction stated as “share the experiences you had (the parts you thought you were good at or the parts you had difficulty with/the knowledge and skills that the process caused you to gain if there are any) while designing the project proposal for 2209 support for university students’ research projects within the scope of the technology and project design course”. in accordance with the instruction given, the participants wrote the reflective reports. table 4 shows the distribution of the data collection tools according to the research questions. 2.3 support programme for tubitak 2209/a university students’ projects support program for tubitak 2209/a university students’ projects aims to encourage undergraduate students to do research through projects. the programme, which is included in the support programs of the department of science and society, provides university students with a donation for the machinery/equipment, consumables, travel and service purchasing expenses required by the research projects they prepare. undergraduate students can apply for the program individually or in groups two times a year (in october and in march). if they apply in groups, one of the students has the responsibility as the project coordinator. a project can have 3 partners apart from the project coordinator at the maximum. 2.4 data analysis an evaluation form was developed by the researcher to evaluate the prospective science teachers’ project proposal forms for tubitak 2209-a support for university students’ research projects quantitatively. the project evaluation form was prepared on the basis of the three basic criteria set by science and society projects, in which the tubitak 2209-a support program for university students’ research projects was also included. the three criteria were labeled as “content and transfer”, “widespread impact” and “feasibility”. in addition to that, the parts included in the project proposal form for tubitak 2209-a support for university students’ research projects were also examined and thus a 22-item evaluation form was created for the parts. two experts in science education and chemistry education were consulted for expert opinion to attain the content validity of the evaluation form. the final shape was given to the evaluation form on the basis of the feedback coming from expert opinion. the evaluation form developed was graded and arranged (see table 5). accordingly, the prospective teachers received 3 points if they answered all the questions fully, they received 2 points if they answered them partially and they received 1 point if they did not have any table 3 demographic properties of the prospective science teachers second year third year total female male female male female male n % n % n % n % (n)% (n)% 41 85.4 7 14.6 10 52.6 9 47.4 (51) 76.1 (16) 23.9 table 4 data collection tools according to the research questions research questions data collection tools 1. what are the project designing levels of prospective science teachers in terms of tubitak 2209-a support for university students’ research projects? project proposal forms 2. what are the views of prospective science teachers on the process of designing projects for tubitak 2209-a support for university students’ research projects? reflective reports journal of science learning article doi: 10.17509/jsl.v3i2.20740 40 j.sci.learn.2020.3(2).36-45 applications with the questions. thus, while the maximum score receivable from the 22-item form with 3 points for each question was 66, the minimum score receivable was 22. based on the scores received from the projects, the projects were classified as “acceptable” (>50), “partially acceptable” (30≤… ≤ 50) and as “unacceptable” (≤30). the reflective reports in which the participants individually had written their views on the process of designing projects in relation to tubitak 2209-a support for university students’ research projects were, on the other hand, put to content analysis. primarily, codes were created on the basis of the prospective teachers’ statements written in the reflective reports. then, the reflective reports were revised again, the frequencies and the percentages of the codes were found and then the data were tabulated. at this stage, the codes created were ranked from the highest frequency to the lowest frequency, and only the frequencies with one repetition were found. besides, samples were quoted from the prospective teachers having the codes with the highest frequencies in the tables. eight of the prospective teachers’ reflective reports were re-coded and analyzed by an independent researcher to test the reliability of the data analysis. accordingly, an agreement between the researchers was found to be 83%. miles and huberman (1994) state that it is considered reliable if the reliability value is above 70%. thus, reliability was attained in this context. 3. result and discussion 3.1 findings concerning research question one the prospective teachers developed 26 projects in total. the projects were classified as “acceptable” (>50), “partially acceptable” (30≤… ≤ 50) and as “unacceptable” (≤30) (see table 6). according to the classification, 34.6% of the projects developed by the participants were “unacceptable” whereas only 11.53% were “acceptable”. on the other hand, approximately half of them (53.84%) were considered to be partially acceptable. descriptive findings concerning the criteria in the evaluation form are presented in detail below. analyzing the prospective science teachers’ tubitak 2209/a project designs in terms of content and transfer on analyzing the students’ projects in terms of content and transfer, such parts as “abstract, problem statement, research questions and the purpose” were examined (see table 7). when the projects were analyzed from the aspect of content and transfer, it was found that the prospective science teachers were not at the adequate level in writing the abstract, in doing sufficient literature review and in stating the research questions and the purpose of the project accurately. in this context, it became evident that table 5 evaluation criterion for the “feasibility” of projects evaluation criteria stages of tübi̇tak 2209/a project design f e as ib il it y m e th o d no questions no (1) % partia lly (2) % yes (3)% 9 were the steps to be taken in the project planned well? 10 were appropriate/right measurement-evaluation instruments used to evaluate the project outcome? 11 was information on how to analyse the data coming from the project offered? 12 were the possibilities for infrastructure which is available/to be supplied to actualise the project described? 13 was the scientific competence of the project team to actualise the project described? 14 was the budgeting for infrastructure/equipment/environment to be supplied done correctly? 15 were the precautions to be taken for hitches probable to be encountered during the project described? s tu d y p la n 16 was the work/package of work to be done in the study plan tabulated? 17 is the total length of time specified in the study plan for the project enough in terms of feasibility? 18 is the length of time for work to be done in the study plan enough? table 6 the quality of the projects developed within the scope of the course technology and project design i the projects developed within the scope of the course technology and project design i acceptable partially acceptable unacceptable total 3 14 9 percentages 3/26 11.53% 14/26 53.84% 9/26 34.61% journal of science learning article doi: 10.17509/jsl.v3i2.20740 41 j.sci.learn.2020.3(2).36-45 the participants did not state the purpose of the project (42%) and that they did not explain the way the project would contribute to science and to the society (42%). it was remarkable that they did not review sufficient literature (38%). in a similar vein, more than half of the prospective teachers (58%) did not state the purpose of their project accurately. in addition to that, it was also remarkable that they had problems in formulating research questions of researchable quality (38%). analyzing the prospective science teachers’ tubitak 2209/a project designs in terms of feasibility the methodology and study plan of the projects were analyzed from the aspect of feasibility. accordingly, it became apparent that the majority of the prospective teachers had problems in writing the methodology and study plan parts of their projects (see table 8). it was found that the prospective science teachers were not adequate in planning the stages to be followed in the process of project making (54%), in using the appropriate evaluation table 7 descriptive findings concerning the prospective science teachers’ tubitak 2209/a project designs in terms of content and transfer evaluation criteria the stages of tubitak 2209/a project design content and transfer abstract no questions no (1)% partially (2) % yes (3)% 1 is the purpose of the project stated? 42% 15% 42% 2 is the method of the project (measurement and evaluation instruments, data analysis, etc.) stated? 31% 58% 12% 3 are the ways the project will contribute to science and to the society stated? 42% 46% 12% problem statement 4 was problem statement appropriate to the project summon formulated? 27% 23% 50% 5 was sufficient literature reviewed in presenting the problem statement? 38% 46% 15% research questions 6 are the research questions of researchable quality? 35% 23% 42% 7 are the research questions accurate in scientific/academic language? 38% 27% 35% purpose 8 is the purpose of the project scientifically/academically accurate? 58% 19% 23% table 8 descriptive findings concerning the prospective science teachers’ tubitak 2209/a project designs in terms of feasibility evaluation criteria stages of project design no questions n o (1 ) % p a rt ia ll y (2 ) % y e s (3 )% feasibility methodo logy 9 are the stages to be followed in the project planned well? 54% 31% 15% 10 are the appropriate/accurate measurement-evaluation instruments used to evaluate the project outcomes? 54% 23% 23% 11 is the information given on how to analyse the data collected in the project? 81% 15% 4% 12 are the infrastructure/equipment possibilities available/needing to be supplied to conduct the project stated? 73% 23% 4% 13 is the scientific competence and occupational experience of the project team to conduct to project stated? 69% 27% 4% 14 is the budgeting for the infrastructure/equipment/environment to be supplied for the project done accurately? 73% 23% 4% 15 are the precautions to be taken for hitches probable to be experienced in the project stated? 100% 0% 0% study plan 16 is the work/package of work to be done in the study plan tabulated? 35% 46% 19% 17 is the total length of time specified in the study plan for the project sufficient in terms of feasibility? 58% 31% 12% 18 is the length of time specified sufficient for the work to be done in the study plan? 58% 35% 8% journal of science learning article doi: 10.17509/jsl.v3i2.20740 42 j.sci.learn.2020.3(2).36-45 instruments to evaluate the project outcomes (54%), in informing on how to analyze the collected data (81%) and in informing on the infrastructure/equipment possibilities which are available or which need to be supplied (73%). besides, it was also found that the project team was not adequate in stating their competence (69%), in budgeting the project expenses accurately (73%) and in specifying the appropriate length of time for each activity in the study table 9 descriptive findings concerning the prospective science teachers’ tubitak 2209/a project designs in terms of widespread impacts evaluation criteria stages of project design no questions no (1) % partially (2) % yes (3)% widespread impacts results/widespr ead impacts 19 are the ways the project outcomes will contribute to science and to the society stated? 38% 42% 19% 20 are the ways to share the project outcomes with large masses stated? 73% 8% 19% resources 21 are all the resources used in preparing the project proposal form stated? 69% 15% 15% 22 are the resources used in preparing the project proposal form written academically correctly? 62% 31% 8% table 10 the stages with which the prospective science teachers had difficulty in designing the projects, and sample quotations from their statements stages with this they had difficulty f (n:67) examples from their statements specifying an original project subject 48% pst14: “i had great difficulty inspecifying the problem statement because the problem statements i found had been found by others before and studies had been performed about them.” pst22: “firstly, we had a little difficulty in choosing our project.” pst26: “we primarily had a little difficulty in finding an idea for the project because it needed to be both original and researchable.” ö61: “we had difficulty in specifying the project subject because the project should be original, designable and useful.” writing about methodology 25% pst34: “the part we had difficulty in preparing this project proposal was the methodology part…” pst50: “… but we had certain difficulties in writing about the methodology because methodology needed to include the whole project. we needed to plan and list everything from the description of the problem to the conclusion step by step.” academic writing 9% pst48: “i felt inadequate in using academic language in writing the project and i had difficulty in it.” pst59: “i personally had difficulty in putting the project to writing because i had difficulty in making sentences in academic language since it was my first experience.” preparing a study plan 7.4% pst14: “but in preparing the work package, especially, i had difficulty in expressing them in tables or graphs, in dividing them into months and in deciding on devoting time to each item.” pst42: “we also had difficulty in the final partin the work packagebecause we were naturally delayed in finishing the project when we had more stages to follow in each month.” most of the stages of a scientific project 7.4% pst20: “the part i had difficulty with was the preparation of the stages of the project.” pst63: “most of the time i had difficulty in what stages to follow in writing the project and in how to write the project.” reviewing the literature 6% pst21: “i had difficulty in reviewing the literature and the resources because there were lots of resources and i had difficulty in reviewing the ones appropriate for our project.” pst49: “another part i had difficulty was the literature review part. i had difficulty in including in the project what we found while investigating what had been done about the subject we worked on and who else had studied the subject”. writing about widespread impacts 6% pst48: “we had difficulty in finding where we could use a game assuming that games had effects after using the game.” ö66: “we all had difficulty in writing the widespread impacts.” group work 4.5% pst28: “we had difficulty in agreement with my friends in the group while completing the project proposal.” pst41: “we mostly had difficulty in acting as a group”. writing the purpose 3% pst44: “… we also had difficulty in writing the purpose of the project.” formulating the research questions 3% pst48: “i had difficulty in formulating the research questions in the project because i thought the questions would be more general.” journal of science learning article doi: 10.17509/jsl.v3i2.20740 43 j.sci.learn.2020.3(2).36-45 plan (58%) and for all the activities (58%). it was also found that the prospective science teachers did not state the precautions which might be taken for hitches to arise in the process of conducting the project in any of the projects. analyzing the prospective science teachers’ tubitak 2209/a project designs in terms of widespread impact the results/widespread impacts of the projects were analyzed in the criterion of widespread impacts. on analyzing the projects from the aspect of widespread impacts, it was found that the majority of the prospective science teachers (73%) did not offer information on how to share the project outcomes with large masses (see table 9). in most of the projects, on the other hand, it was found that the resources used in the process were not stated (69%) and that the resources were not written correctly academically (62%). 3.2 findings concerning research question two the stages with which the prospective science teachers had difficulty it was found on examining the reflective reports that the prospective teachers had difficulty in specifying an original project subject, in writing about the methodology, in academic writing, in preparing a study plan and in the literature review (see table 10). a detailed examination of these themes demonstrated that almost half of the prospective teachers (48%) had difficulty in specifying an original project subject in relation to tubitak 2209-a support for university students’ research projects. additionally, they also stated that they had difficulty in expressing such scientific project stages as methodology, work package, literature, widespread impacts and formulating research questions and writing the purpose of the project in academic language. the stages that the prospective science teachers thought they were good on examining the stages that the prospective teachers thought they were good in the process of project design, such themes as specifying original project subjects, writing the purpose of the project, working in coordination groups, preparing work packages, writing research questions, writing abstracts, reviewing the literature and writing the widespread impacts were found to be remarkable (see table 11). only a small number of prospective science teachers thought that they were good at writing the stages of a scientific project in the process of project design. for instance, 10.4% said that they were good at specifying an original project subject. on the other hand, they were found to think that they were good at writing about the methodology (6%), preparing package of work (6%), reviewing the literature (3%), writing the widespread effects (3%), formulating the research questions (6%) and writing the purpose (6%). table 12 gives a summary of the knowledge and skills the prospective science teachers gained in the process of designing projects. accordingly, the prospective teachers stated that they learned the stages of a scientific project (41.8%) and that they became skillful at collaborating (27%), doing research (17.9%), problem-solving (17.9%), writing in academic language (11.9%), working according to a plan (11.9%) and communicating (9%) in the process of designing projects (see table 12). a small number of them, however, said that they gained skills in multidimensional thinking and in creative thinking. some of them, on the other hand, said that they learned such things as writing the references and preparing the package of work. table 11 the stages at which the prospective science teachers were good in designing the projects, and sample quotations from their statements stages of project f (n:67) examples from their statements specifying an original project subject 10.4% pst6: “the part we were good at was…. we had good ideas pst46: “i think we were good at the parts where we specified the problem statement.” writing the purpose 9% pst47: “i think we were good at stating the purpose of the project.” working in coordination in groups 9% pst5: “…i think i worked in coordination with my friends in the group n the process.” preparing packages of work 6% pst57: “… the part we did best was to prepare the timetable for work.” formulating the research questions 6% pst4: “the part we were the best at was the research questions.” writing abstracts 6% pst14: “i had no difficulty in the abstract because it was written after all the project work had finished.” writing about methodology 6% pst67: “i liked preparing the package of work very much. i think was good at writing about the methodology.” literature review 3% pst60: …we did the literature review effectively with my partners.” writing the widespread impacts 3% pst50: “i didn’t have any difficulty in writing the widespread impacts because i think that we reflected well the information on how to use the results across the country and across the world” journal of science learning article doi: 10.17509/jsl.v3i2.20740 44 j.sci.learn.2020.3(2).36-45 3.3 discussion this study was conducted to determine prospective science teachers’ levels of project design and their views on the process of project design in relation to tubitak 2209-a support for university students’ research projects. according to the results, the fact that a small proportion of the projects (11.56%) were acceptable demonstrated that the majority of the prospective teachers were inadequate in terms of designing projects. this was a result in parallel to the ones obtained by baki and bütüner (2009), by bencze (2010) and by tatlı (2016). considering the fact that the majority of the prospective teachers had prepared project proposals for the first time and that they had not taken a course in research methods (70%), the result could be regarded as natural. another result obtained in this study demonstrated that the feasibility of the projects developed by the prospective science teachers was inadequate. for example, they were found to have problems in planning the process of project design, in choosing appropriate evaluation instruments to evaluate the project outcomes and in stating the length of time needed and the materials which are available/ which should be supplied to actualize the project. their views that they had difficulty in writing the methodology part in their reflective reports were also supportive of this result. in a similar way, it might have stemmed from the fact that most of the participants had not taken a course in scientific research methods. considering the length of the course technology and project design 1 (two hours a week), the fact that in-depth information was not offered about scientific research methods could be understood. therefore, it may be recommended that a similar study be repeated with students who have taken a course in scientific research methods and that the results obtained to be compared with the ones obtained in this current study. on the other hand, the fact that the course technology and project design 1 was included in the second-year curriculum and that the course scientific research methods was included in the third-year curriculum might have influenced the effectiveness of the project training in negative ways. thus, it can be recommended that the table 12 knowledge and skills the prospective science teachers gained in the process of project design, and sample quotations from their statements knowledge and skills gained f (n:67) examples from their statements stages of preparing a scientific project (research questions,widespread impacts, feasibility) 41.8 % pst19: “i was informed of how to write a project. i learnt about what should be cared in writing a project, how to prepare a project, research methods used in a project and how to form the content pst25:“ i think i learnt about the stages from research problem to justifications, purpose, hypotheses, methodology and conclusion.” ö58: “i learnt what i should pay attention to while preparing a project and what properties a project should have (widespread impact, originality, feasibility, meeting a need, economy, etc.). most importantly, i learnt how to write a project. collaborating 27% pst24: “… ıt taught me how to work in division of work, and how to apply it.” pst37: “ıt secured working in groups in coordination in a project and sharing the work voluntarily and achieving group success.” skills in doing research 17.9% pst19: “i learnt how to do comprehensive research.” pst30: “ i learnt to do research based on various resources.” problem-solving 17.9% pst18: “what we gained from it: we learnt how to find solutions to the problems we encounter.” pst65: “we designed a project about the problem we specified in the scope of this course. i can say we improved our problem solving skills.” writing in academic language 11.9% pst10: “i learnt to write a project draft in academic language.” working according to a plan 11.9% pst38: “i understood that preparing a project was a demanding job and that it required a lot of research and regular and disciplined work in order for it to be original.” communication skills 9% pst35: “ıt contributed to our communicating effectively with our partners in the group while designing a project idea and while talking to others.” preparing package of work/work schedule 7.4% pst52: “….ıt made us gain skills in what way to pursue while preparing a study plan.” multi-dimensional thinking 4.5% pst4: “designing projects taught me to think broadly and to analyse a problem from different perspectives.” creative thinking skills 3% pst13: “ i think i gained the skill to think creatively in this course.” writing the references 3% pst19: “i learnt how to prepare the references part.” setting up cause and effect relationships 3% pst37: “… i learnt to set up cause-effect relationships and to generate original ideas.” selfconfidence 3% pst56: “the fact that our project was chosen and sent to tubitak increased my self-confidence.” journal of science learning article doi: 10.17509/jsl.v3i2.20740 45 j.sci.learn.2020.3(2).36-45 undergraduate program for the science teaching department should be re-arranged by taking this factor into consideration. it was found in this study that the prospective science teachers could not specify the problem statement appropriately to the project summon or that they could not state it inappropriate academic language. the data coming from the reflective reports also demonstrated that they had difficulty in this respect. relevant literature also shows that the greatest problem prospective science teachers had in developing projects was “failure to generate a new idea/to find a project subject” (oğuz ünver, arabacıoğlu & okulu, 2015; özel & akyol, 2016; timur & imer çetin, 2017). project support programs for which teachers can apply individually and in which they need to guide their students which are supported by several institutions-mainly by tubitak are available in turkey. when seen from this perspective, it might be influential for prospective teachers to receive pre-service training about projects and to have experience in such project designing and coordinating programs so that they can gain skills in designing and coordinating projects (timur & imer çetin, 2017). 4. conclusion the results obtained from the prospective teachers’ reflective reports demonstrated that they gained such different skills as collaborating, researching, problemsolving, writing in academic language, working according to a plan and communicating in addition to learning about the stages of scientific projects. it was a finding in parallel to the one obtained by peten metin, yaman, vekli sezen & çavuş (2019), who analyzed the effects of tubitak’s support project preparation training (coded as 4006 and 4007) on prospective science teachers’ project writing/preparation skills. this study could not determine prospective science teachers’ levels of project design prior to project training for support coded as 2209 because it was found as mentioned earlier that the prospective teachers did not have any prior knowledge about the support program. a similar study could be performed with two groups who are knowledgeable about the support program coded as 2209-a by using the experimental method with a control group, and thus the effectiveness of project training in terms of different variables could be demonstrated more concretely. references baki, a., & bütüner, s. ö. (2009). kırsal kesimdeki bir ilköğretim okulunda proje yürütme sürecinden yansımalar. i̇lköğretim online, 8(1), 2-14. bencze, j. l. (2010). promoting student-led science and technology projects in elementary teacher education: entry into core pedagogical practices through technological design. international journal of technology and design education, 20(1), 43-62. camcı, s. (2008). bilim şenliğine katılan ve katılmayan öğrencilerin bilim ve bilim insanlarına yönelik ilgi ve imajlarının karşılaştırılması. hacettepe üniversitesi, ankara. capraro, r. m., capraro, m. m., scheurich, j. j., jones, m., morgan, j., huggins, k. s., ... & han, s. (2016). impact of sustained professional development in stem on outcome measures in a diverse urban district. the journal of educational research, 109(2), 181196. çeliker, d., & erduran avci, d. d. d. (2015). i̇lkokul öğrencilerinin bilim i̇nsanı algıları: öğrencilerin bilimsel faaliyetlere katılması bilim i̇nsanı algılarını nasıl etkiler?. mehmet akif ersoy üniversitesi eğitim fakültesi dergisi, 1(36), 90-104. dionne, l., reis, g., trudel, l., guillet, g., kleine, l., & hancianu, c. (2012). students’sources of motivation for participating in science fairs: an exploratory study within the canada-wide science fair 2008. international journal of science and mathematics education, 10(3), 669693. erdem, a., uzal, g., & ersoy, y. (2006). fen bilgisi/fizik öğretmenlerinin eğitim sorunları: gelişmeleri sürekli i̇zlemeleri ve gerekli yenilikleri edinmeleri. araştırma raporu, tfv yayını, tekirdağ. fisanick, l. m. (2010). a descriptive study of the middle school science teacher behavior for required student participation in science fair competitions. güven, i̇. (2013). fen ve teknoloji öğretmen adaylarının proje yönetimi deneyimlerinin değerlendirilmesi [evaluation of prospective science and technology teachers’ experiences of project management in turkish][special issue]. hacettepe university journal of education, 1, 204-218. huberman, a. m., & miles, m. b. (1994). data management and analysis methods. in n. k. denzin & y. s. lincoln (eds.), thousand oaks, ca, us: sage publications, inc. kaplan, a. ö., & coşkun, y. d. (2012). proje tabanlı öğretim uygulamalarında karşılaşılan güçlükler ve çözüm önerilerine yönelik bir eylem araştırması. mersin üniversitesi eğitim fakültesi dergisi, 8(1), 137-159. norton, l. (2018). action research in teaching and learning: a practical guide to conducting pedagogical research in universities. routledge. özel, m., & akyol, c. (2016). bu benim eserim projeleri hazırlamada karşılaşılan sorunlar, nedenleri ve çözüm önerileri. gazi üniversitesi gazi eğitim fakültesi dergisi, 36(1). özer, d. z., & özkan, m. (2012). proje tabanlı öğretimin fen bilgisi öğretmen adaylarının bilimsel süreç becerileri üzerine etkisi. journal of turkish science education, 9(3), 119-130. peten metin, d., yaman, f., sezenvekli, g. & çavuş, m. (2019). improving preservice science teachers’ ability to write/prepare projects to tubitak grant program. international journal of social sciences and education research, 5(1), 78-90. şahin, ş. (2012). bilim şenliklerinin 10. sınıf öğrencilerinin kimya dersine yönelik tutumlarına olan etkisi. uşak üniversitesi sosyal bilimler dergisi, 5(1), 89-102. sözer, y. (2017). investigation of the project development process of the pupils who attend to the tubitak research project contest: an action research. inonu university journal of the faculty of education, 18(2), 139-158. tatli, z. (2016). proje geliştirme sürecine dair öğretmen görüşleri. yüzüncü yıl üniversitesi eğitim fakültesi dergisi, 13(1), 631-656. timur, b., & i̇mer çetin, n. (2017). fen ve teknoloji öğretmenlerinin proje geliştirmeye yönelik yeterlikleri: hizmet içi eğitim programının etkisi. journal of kirsehir education faculty, 18(2). tortop, h. s. (2013). science teachers’ views about the science fair at primary education level. turkish online journal of qualitative inquiry, 4(2), 56-64. ünver, a. o., arabacıoğlu, s., & okulu, h. z. (2015). öğretmenlerin bu benim eserim proje yarışması rehberlik sürecine ilişkin görüşleri. muğla sıtkı koçman üniversitesi eğitim fakültesi dergisi, 1(3). yıldırım, h. i̇., & şensoy, ö. (2016). bilim şenliklerinin 6. sınıf öğrencilerinin fen bilimleri dersine yönelik tutumlarına etkisi. türk eğitim bilimleri dergisi, 14(1), 23-40. a © 2022 indonesian society for science educator 500 j.sci.learn.2022.5(3).500-508 received: 13 june 2022 revised: 16 october 2022 published: 27 november 2022 9th grade students' learning of designing an incubator through instruction based on engineering design tasks fethiye karsli baydere1*, aydın murat bodur1 1department of mathematics and science education, faculty of education, giresun university, giresun, turkey *corresponding author: fethiyekarsli28@gmail.com abstract in this study, a stem activity was designed in which 9th-grade students can complete the task of making incubators by overcoming the difficulties they face in the engineering design process. this activity has been handled in the context of energy conversion and prepared based on the engineering design process consisting of 9 stages. the activity was applied to 34 (19 females and 15 males) 9th-grade students studying at a public school in the eastern black sea region in turkey in the fall semester of the 2019-2020 academic year. this application took 7 lesson hours (7x40 minutes) in total. at each stage of the engineering design process, students worked like an engineer and scientists by collaboratively conducting scientific research and inquiry. throughout the process, students were confronted with several difficulties, given the time and opportunity to help them develop stem literacy. more importantly, the students had the opportunity to experience a stem activity by putting the steps of the engineering design process into practice. keywords stem education, design of an incubator, engineering design process 1. introduction science lessons aim to raise individuals who can develop rational and alternative solutions to our problems in real life (bybee, 2010; 2013; national research council [nrc], 2012). it is necessary to use well-planned, responding to today's needs and contemporary teaching methods and techniques in the field of science education in order to raise individuals with this characteristic (uluçınar, cansaran, & karaca, 2004). in other words, it is necessary to create learning environments where memorization is pushed to the background, developing 21st-century skills, where experience is at the forefront, and allowing creative solutions to real-life problems. one of the approaches that create learning environments with these characteristics is the stem education approach (akgündüz et al., 2015; eroğlu & bektaş, 2016). the educational approach that enables individuals to identify the problem situation, to be able to develop alternative and practical solutions to the problem, and offer creative and original solutions by integrating science, technology, engineering, and mathematics disciplines is called "stem" (altunel, 2018; bybee, 2010, 2011; yılmaz, yiğit koyunkaya, güler, & güzey, 2017). while science or mathematics in education is often mentioned in science and mathematics curricula, technology or engineering is rarely referred to, and this is a matter to be solved (bybee, 2010; katehi, pearson, & feder, 2009). these areas included in the concept of stem are interrelated and cannot be considered separately (thomas, 2014). the main reason for the stem education approach to become widespread is the lack of knowledge in stem fields and the insufficient workforce (kennedy & odell, 2014; rosenblum & kazis, 2014). since the stem education approach is based on a social constructivist framework, we can characterize science learning as putting social acculturation and personal construction of ideas into practice. stem disciplines offer a variety of opportunities to gain 21st century skills. students studying according to the stem education approach can develop 21st-century skills such as adaptation, effective communication, social skills, non-routine problem-solving, self-directedness/selfimprovement, and systems thinking (nrc, 2012). some studies on the stem education approach state that it gives all students a chance to apply their knowledge (ritz & fan, 2015; yıldırım & altun, 2015). in other words, stem education and engineering practices offer students the time and opportunity to understand where and how to use mailto:fethiyekarsli28@gmail.com journal of science learning article doi: 10.17509/jsl.v5i3.47226 501 j.sci.learn.2022.5(3).500-508 scientific knowledge. at the same time, it was concluded that the participation of students in activities related to stem applications increased their interest in stem fields (dabney et al., 2012; maltese & tai, 2011; tindall & hamil, 2004). the stem approach allows students to blend information. as a result of the blending of the information, it becomes easier for the students to create alternative solutions to the problems they encounter (morrison, 2006; niess, 2005; yıldırım, 2016, 2017; wang, 2012). this allows students to improve their skills and use them in solving different problems. the stem approach has made a noticeable difference in the students' interest and motivation toward the lessons (niess, 2005). in the stem education approach, students are first given a problem from daily life. then, the students think about and investigate the problem situation. students participate in the inquiry processes of the stem education approach individually or as a class. doing research and inquiry enables students to take responsibility for learning and increase their self-confidence (lewis, 2006). then they use alternative solution suggestions to solve the problem. next, they determine the best solution in the context of the problem and make a prototype for the solution. then, they test whether the prototype solves the problem and improve their design until the prototype is successful. finally, they share the information they learned and the designs they developed during this process, called the engineering design process, with their peers. all these processes provide students with important experiences. in this context, it is aimed to develop and implement an activity based on the engineering design process in this study. in the activity developed, a real-life problem was presented to the students in which they could use the primary fields of science, technology, mathematics, and engineering. this way, students will have the opportunity to experience an applied activity based on integrating four main disciplines by using their science, technology, mathematics, and engineering skills and by conducting research and inquiry. in this way, they will be productive students. 1.1 engineering design process (edp) turkey's education policies aim to generalize stem education, train students with engineering skills, and direct students with these characteristics to engineering fields (meb-yeği̇tek, 2016; mne, 2018). the need to introduce engineering design concepts and processes to achieve these goals is recognized. the purpose of introducing edp to students is not to "build things", which is a common misconception. instead, edp aims to teach students that engineering is about organizing thoughts to improve decision-making to develop high-quality solutions and/or products to problems (bybee, 2010; dym, agogino, eris, frey, & leifer, 2005; gencer, doğan, bilen, & can, 2019). edp is a process that involves students producing practical solutions to the problem, choosing the most appropriate solution, and designing a product using the solution they choose (nrc, 2010). in this process, students try to find solutions to real-life problems and design and test an original product by applying the edp steps sequentially. they consider some criteria and limitations to successfully solve the problem (brunsell, 2012; itea, 2007; nrc, 2012). in this process, students will share responsibility by doing collaborative work in groups and developing their creativity while finding solutions to real-life problems. experienced engineers work together to systematically explore and evaluate design ideas before investing substantial resources to fully implement them (daly, adams, & bodner, 2012; wendell, andrews, & paugh, 2019; wells, 2016). in addition, students critical thinking skills will be employed while finding a solution to the problem in this process. in addition, students will use their communication skills to influence others in the engineering design process as part of presenting and disseminating their designs. similarly, the task of the team of engineers who have designed and successfully prototyped a new product continues beyond there. after that, they need to explain the basic elements that determine how their designs will go into mass-produce to their production engineers (wendell et al., 2019). in other words, experienced engineers must also have high communication skills. design tasks assigned to students enable the development of critical thinking skills and are often associated with engineering and technology literacy. stem experiences significantly impact students' critical thinking development (duran & sendag, 2012; mater et al., 2020). with actual engineering practices, high school students will learn that design is not just about building things. instead, they will realize it is a process in which the need or problem is clearly defined. moreover, they will realize that designing is a process that includes research, planning, brainstorming, testing, evaluation, communication, and more. engineering design learning experiences are increasingly offered, but research on how to support learners' knowledge construction during the engineering design process in the classroom is still preliminary (wendell et al., 2019). in this context, the current research is based on the development of a stem activity according to edp, its application to students, how to support learners' knowledge construction, and the students gain experience in this process. in addition, this research aims to provide students with stem experience and present an example to practitioners and teachers on stem integration. 1.2 energy conversion and design of an incubator in science, energy transformation constitutes a privileged and crucial conceptual field due to its abstract and interdisciplinary characteristics. energy transformation has been significant in science, engineering, and mathematics in the past and current centuries. it will continue to be given. researchers generally investigated students' alternative concepts and level of understanding journal of science learning article doi: 10.17509/jsl.v5i3.47226 502 j.sci.learn.2022.5(3).500-508 within the scope of energy transformation subjects (e.g., liu & tang, 2004; park & liu, 2016; töman & çimer, 2012; watts, 1983). they also investigated how the effects of different pedagogical approaches on students' understanding of energy transformation differed (yavuztopaloğlu & balkan-kıyıcı, 2017; ozkan & umdutopsakal, 2020). although these studies have created much information about how students think about energy transformations and which learning approach facilitates students' conceptual understanding of this subject, they did not address whether or how students applied the concepts they learned by selecting appropriate objects and materials for practical tasks. in addition, stem activity development researches in turkey are very limited compared to other types (aydın günbatar & tabar, 2019; ormancı, 2020). in the present study, it was considered necessary that students both learn about energy conversion and gain experience in the application of the subject during the incubator-making process. 2. method since this research was planned to design and implement a stem activity, the engineering design process-based teaching model was used. the stem activity developed within the scope of the research was applied to a total of 34 (19 girls and 15 boys) 9th-grade students studying at a public school in the eastern black sea region in the fall semester of the 2019-2020 academic year. participants ages range from 15 to 16. the study's sample was selected from the students at the school where one of the study's authors was a teacher. this ensures that the research is conducted in a natural learning environment. a combination of both convenience and purposeful sampling procedures was used in the study. all study participants were informed about the research. in addition, all the students participating in the study declared that they participated voluntarily. the students in the study learned that energy is conserved by transforming kinetic and potential energy types in the 8th grade. the students participating in the study have not previously participated in any teaching activity designed according to the edp of the stem education approach. 2.1 the implementation of the stem activity in this study, a stem activity was developed and implemented using the edp steps to convert electrical energy. the objectives of the stem activity have been determined according to acquisitions of secondary school science, mathematics, and technology courses' curriculums in turkey (mne, 2018). while developing activities according to the edp steps of the stem education approach, the 9th-grade physics textbooks and the studies on the stem approach in the literature were examined (aydın & karslı-baydere, 2019; hacıoğlu & dönmez-usta, 2020; wendell & lee, 2010; karslı-baydere, hacıoğlu, & kocaman, 2019; karahan, bilici, & ünal, 2015). by presenting real-life problem/design challenges to the students by the researchers, they were made to work like scientists and engineers (hynes et al., 2011; wendell & lee, 2010). there are three basic concepts in the successful implementation of edp: (i) students are engineers; (ii) teachers need to listen to their students; (iii) classroom environments need to change to enable learning through edp properly. the goal of learning engineering design is to encourage students to interact with engineering in hands-on activities as a practical application of mathematics and science. design challenges or real-life problems presented to students must have various features. these include (1) it should facilitate students' learning; (2) the design challenge must be carefully structured and from real life; (3) the problem must be open-ended with many possible solutions; (4) it must have the criteria and limitations that will lead students to the target; and (5) the designed product should be able to be tested and evaluated (havice, 2009; moore et al., 2014; wendell, 2008; brunsell, 2012; silk, schunn, & cary, 2009). in this context, a design challenge with these features was presented to the students in the study. the activity was presented as a worksheet for students to be guided correctly in the problem-solving process, to write their thoughts easily, to discuss the topic among themselves, and to pass the application process efficiently. the activity prepared was reviewed by two experts: an academic with studies on stem education and a teacher with more than ten years of experience in the field. the second author made a teaching intervention in the research. the researcher who made the teaching intervention took a postgraduate course about the stem education approach and is knowledgeable about the application of this approach. before implementing the activity, during one lesson hour, students were given detailed information about edp, the different aspects of the process from other lessons, and what they will do. it is crucial to create a social learning environment because edp includes identifying problems encountered in daily life, developing alternative solutions to them, and making prototypes that create solutions to the problem (wendell & lee, 2010). thirty-four students were divided into six groups, two groups of 5 and 4 groups of 6 people. in the opening lesson, students learn that their engineering design challenge is to create an incubator model with the necessary conditions for developing a fertilized egg. over the following six lessons, the students, guided by their teacher and worksheet, conduct engineering tests to identify materials to meet these design requirements. to facilitate their efforts throughout the process, useful websites and science workbooks prompt students to reflect on experiments and observations. students were expected to produce multiple answers within the framework of the assigned task and do the necessary research. each group was then asked to plan their designs to solve the problem presented to them. the students identified their needs, journal of science learning article doi: 10.17509/jsl.v5i3.47226 503 j.sci.learn.2022.5(3).500-508 imagined their planned product, and drew their designs. after providing the necessary materials, they freely designed and tested their products. they used light sensors and temperature sensors for their tests. as the students tested materials and began prototyping, they were asked to make scientific arguments about the materials they would use and their reasons for an incubator. the products designed by all groups were evaluated in terms of criteria and limitations in the classroom. in this study, we focused on students' learning how electrical energy is transformed into other energies through describing, selecting, designing, and testing. also, we focus on students' conducting scientific argumentation about the solution to the problem through an engineering design component. in this study, stem activity was developed according to the 9-step edp recommended by hynes et al. (2011) for 9th-grade students on converting electrical energy (see figure 1). what has been done to all steps of edp is explained in detail below. step 1. identify and define problems: it is essential to determine the problem at the beginning of the process in edp. because when the teacher asks students to find a problem to solve, they will be forced to state the problem in their own words. this approach will increase the likelihood that students will embrace their class challenge and gain critical thinking skills (hynes et al., 2011; lemons, carberry, swan, rogers, & jarvin, 2010). care was taken that the classroom challenge presented to students is close to a real-world engineering challenge as possible and have open-ended with many possible solutions. in addition, attention was paid to criteria and limitations that guide students to gain knowledge and skills aligned with the targets. moreover, care was taken to ensure that the design challenge presented to students could be tested and evaluated in the context of limitations and criteria. therefore, the application process was first started by leaving the students alone with the design challenge. for this purpose, the design challenge and directions given to students are presented in figure 2. step 2. research the need or problem: after determining the design challenge and the problem to be solved, there are better ideas for students to try to solve the problem with the first idea that comes to mind. at this stage of edp, students should be made aware that there are many things they need to consider and know to solve the problem. as students investigate their needs and problem and discover new constraints or ideas, they are more likely to redefine and clarify the problem (hynes et al., 2011). when dealing with any number of problems embedded in a design challenge, students should decide what information sources o draw upon and what past figure 1 the steps of edp recommended by hynes et al. (2011) for 9th-grade students figure 2 the design challenge and directions given to students journal of science learning article doi: 10.17509/jsl.v5i3.47226 504 j.sci.learn.2022.5(3).500-508 experiences to apply most effectively (crismond & adams, 2012). after ensuring that the students fully understand the problem, the teacher asks them the questions such as "what do you know to design an incubator?" and "what do you need to know?" it was emphasized that students should collect information from different sources in order to complete their designs successfully. for this, the teacher provides the necessary internet, computer, and library facilities for all students to do their research. step 3. develop a possible solution(s): no single solution to edp problems is chosen from daily life (brunsell, 2012; silk et al., 2009). at this stage, the students were expected to generate many ideas (at least three) by considering the criteria and limitations of the problem given to them. students wrote their solution suggestions on their worksheets. it was emphasized that they should work in teams and plan for this. during this process, students are expected to brainstorm to use their creativity. this step aimed to help students produce various ideas by asking and guiding questions about the information that students will use while creating ideas. in addition, students exchange ideas with others using their communication skills within the problem criteria and constraints. students can develop their ideas using words, drawings, and prototypes. experienced designers participate in continuous learning by brainstorming, drawing, generating ideas, and communicating with people (lawson & dorst, 2013). step 4. select the best possible solution: at this stage, the students are expected to choose the best possible solution from the solution proposals they have developed within the given criteria and limitations in overcoming the design challenge given to them (hynes et al., 2011; nrc, 2012). the teacher emphasized that the students should pay attention to the criteria and limitations of the design challenge while deciding on the best possible solution. at this stage, students think like engineers and evaluate whether their solution proposals meet the criteria and limitations (brunsell, 2012; mentzer, 2011; nrc, 2012). for this, the teacher provides the students with instructions: "choose the best possible solution among your proposed solutions, taking into account the criteria and limitations of the problem, and present your solution to your other group friends by justifying it." finally, "draw the design of the best possible solution you have decided." these instructions are also provided to enable students to use their mathematics and science knowledge to make informed decisions and continuously evaluate each (hynes et al., 2011). the students were then asked to prepare for the next lesson by preparing a plan to implement their chosen solution. in this step, the students planned how to design their solutions and drew what they imagined (see figure 3). step 5. construct a prototype: a prototype is a model (physical, virtual or mathematical) that represents the final solution (hynes et al., 2011; karslı-baydere et al., 2019; nrc, 2012). the most important feature of this state is that iterative prototyping until an acceptable product is reached, and the students physically create a solution model (koehler, latif, faraclas, sanchez, & kazerounian, 2005). it is important to let students fail in making their prototypes and learn from those failures as they replicate their solutions. even if students fail in the end, they can learn the features of the final solution during the challenge and gain knowledge and skills in many subjects. at this stage, the teacher asked the students to design their products following the solution suggestions put forward by each group. for this, the teacher instructs the students to create their designs. at this stage, the teacher gives all the materials the students will need. during this period, students were encouraged to work just like scientists, engineers, artists, mathematicians, or technologists and began making their designs with their groupmates with the materials they needed. step 6. test and evaluate the solution: the usefulness of the presented prototype is vital for a successful solution (koehler et al., 2005; karslı-baydere et al., 2019). therefore, it is scientifically necessary to test and evaluate the prototype with tests that include criteria and limitations. as a result of this evaluation, the design should be developed until the prototype is successful (hynes et al., 2011; brunsell, 2012; karslı-baydere et al., 2019). at this stage, the teacher offers students the opportunity to test their products after they have finished their designs (figure 4). to do this, it instructs, "implement your design and test it experimentally. if it does not work, redesign and come to the retesting stage". students complete and test their designs. the designs created at this stage were tested in groups. first, the group discussed why the designs that failed the test did not work. then, the detected errors and deficiencies were corrected. finally, the design was rearranged and transformed into a solution to the problem. thus, all groups had the opportunity to see the success of their prototypes. figure 3 group work on selecting the best possible solution journal of science learning article doi: 10.17509/jsl.v5i3.47226 505 j.sci.learn.2022.5(3).500-508 steps 7-8. communicate the solution and redesign: implementing engineers share their ideas, solutions, and designs with others for feedback and marketing purposes (hynes et al., 2011). as engineers do, students can communicate their solutions through presentations, written documents, or other tools. in this process, students must indicate their design presentations' conditions, performances, problems, limitations, and criteria. in doing so, students will make an oral presentation using a language understandable to the target audience. at this stage, the teacher asks all groups to present their designs to other groupmates (see figure 5). at the same time, the teacher asked other groups to comment on the designs presented. students presented to their group friends how their designs transform electrical energy into heat and light energy. the students explained the reasons for using the materials they used to solve the problem and the difficulties they faced. they documented the extent to which they met the criteria and limitations of the design challenge presented to them. with these practices, students' communication and problem-solving skills were put to work. the students worked in continuous cooperation with their group mates. this situation contributed to the students developing group consciousness. an evaluation rubric was given to the students. the target audience evaluated the designs presented according to various criteria such as offering a solution to the problem, not harming the environment, or economy, suitability for human health, durability, and usefulness. it was decided which one of the designs made by the groups was the best. thus, the students realized that a good design depends not on a single criterion but on several variables. the teacher instructs the students "improve your design until a final product that meets all requirements and criteria is produced." the goal is to ensure that your prototype passes all tests and evaluations. step 9. completion: in this step, students decide that they adequately meet the design requirements and are ready to implement their prototype as a final product (see figure 6). in this step, after the students have decided that their prototype is ready to implement as a final product, the teacher asks them to introduce their designs by giving instructions, "prepare a product introduction or user manual to popularize your design". the aim here is to realize the importance of marketing a developed product and developing a product. 3. result and discussion in this study, a stem activity was developed and implemented using the edp steps to convert electrical energy. edp is one of the most suitable ways to give students a stem experience. this experience process also allows students to share responsibility through collaborative work in groups and develop their creativity while trying to find original solutions to real-life problems. students designing in this process use many learning styles: learning by doing; learning by brainstorming; learning by prototyping; learning from iteration, feedback, and failure; learning by noticing and troubleshooting; learning by drawing and ideas; learning by dialogue with materials; and learning from people and thought (hathcock, dickerson, eckhoff, & katsioloudis, 2015; lawson & dorst, 2013). students also benefit from different disciplines and improve their skills while solving the problem (kolodner, 2002a, 2002b; leonard, 2004). in this research, ninth-grade students were faced with many difficulties in edp and were guided to overcome them. as a result, they found solutions to the real-life figure 4 3rd group that created their prototypes figure 5 group 6 presents their prototype to other groups figure 6 the final design of the 5th group journal of science learning article doi: 10.17509/jsl.v5i3.47226 506 j.sci.learn.2022.5(3).500-508 problem presented by adding their original ideas and employing the disciplines of science, engineering, technology, and mathematics. in addition, they eventually overcame their difficulties and completed the task of making incubators. educators argue that if students are to improve their stem literacy, they have to face and solve several difficulties (bybee, 2010). in this process, the students not only learned how to make an incubator but also learned with fun. they also had the opportunity to develop many aspects, such as harmonious working, effective communication, social skills, new idea development, responsibility, problem-solving, and decision-making. after the students were challenged to make an incubator, the students began to research, think, and make how the incubator could be made. the researchers observed that not all groups were equally successful in this process. the designs that some groups initially dreamed of and the products they made did not match. in other words, the designs imagined by some groups offered more realistic solutions to the problem, while the products they designed needed help finding a solution to the problem. one of the reasons for this is that they need to correctly combine the tools and equipment necessary to realize their imagined design. in addition, it was observed that some students had problems using their manual skills. at the stage of testing their products, the students could adjust the temperature, light, and humidity values that should be in the environment during the incubation. they tested this by measuring them with appropriate tools. also, the students followed the process of hatching chicks from a real fertilized egg after the activity was completed. however, no group hatched chicks from fertilized eggs utilizing the products they designed. it is important to let students fail from time to time and learn from these failures as they repeat their solutions (bybee, 2010) because students can learn while questioning the reasons for failure. the activity was presented as a worksheet for students to be guided correctly in the problem-solving process, to write their thoughts easily, to discuss the topic among themselves, and to pass the application process efficiently. while this activity is being implemented, it has been observed that the worksheets act as an excellent organizer to guide the students and help the teacher. this shows that worksheets have an important role that students can play in facilitating science learning through authentic activities such as engineering design. 4. conclusion considering the whole process, the students needed help creating a solution to the problem and, at the same time, had much fun. in addition, this activity applied to students provided the time and opportunity to help them develop stem literacy. therefore, when the whole process is considered, the implementation of the developed activity meets the learning outcomes to a great extent. however, on the other hand, this activity is an example of how science, mathematics, and technology can be integrated into edp. based on these results, it is recommended to carry out and disseminate similar studies in different interdisciplinary subjects or concepts that will solve reallife problems. 5. implications and further research based on these results, it is recommended to carry out and disseminate similar studies in different interdisciplinary subjects or concepts that will solve real-life problems. in this research, a stem activity was designed and implemented. although this is an engineering design development study, the effects of the developed and implemented activity on students could not be examined. however, the effects of the activity developed in further research on various learning products of students can be investigated. after the activity was completed, the students followed the process of hatching chicks from a real fertilized egg. however, no group hatched chicks from fertilized eggs utilizing the products they designed. when they investigated the reason for this, they found that turning the eggs in the incubator was of great importance during the incubation period. therefore, the students failed to meet the criteria for rotating the eggs continuously and at an appropriate speed in their designs. this situation could be given as a separate criterion in the problem scenario, or more attention could be paid to this issue with teacher guidance. for this reason, it is recommended that the teachers who will do this activity give more guidance on making a mechanism in the incubator that will perform the proper rotation during the incubation process. references akgündüz, d., aydeniz, m., çakmakçı, g., çavaş, b., çorlu, m. s., öner, t., & özdemir, s. (2015). stem education turkey report. i̇stanbul: scala basım. altunel, m. (2018). stem education and turkey: opportunities and risks. siyaset, ekonomi ve toplum araştırmaları vakfı, 207, 1-7. aydın, e., & karslı-baydere, f. (2019). 7th grade students’ views about stem activities: example of separation of mixtures. ondokuz mayis university journal of education faculty, 38(1), 35-52. aydın-günbatar, s., & tabar, v. (2019). content analysis of science, technology, engineering and mathematics (stem) research conducted in turkey. yyu journal of education faculty, 16(1), 10541083. brunsell, e. (2012). the engineering design process. in brunsell, e. (ed.), integrating engineering + science in your classroom (pp. 3-5). arlington, virginia: national science teacher association [nsta]. bybee, r. w. (2010). advancing stem education: a 2020 vision. technology and engineering teacher, 70(1), 30-35. bybee, r. w. (2011). scientific and engineering practices in k-12 classrooms: understanding “a framework for k-12 science education. science and children, 49(4), 10-16. bybee, r. w. (2013). the case for stem education: challenges and opportunities. nsta press. crismond, d. p., & adams, r. s. (2012). the informed design teaching and learning matrix. journal of engineering education, 101(4), 738. https://dergipark.org.tr/en/pub/omuefd https://dergipark.org.tr/en/pub/omuefd journal of science learning article doi: 10.17509/jsl.v5i3.47226 507 j.sci.learn.2022.5(3).500-508 dabney, k. p., tai, r. h. almarode, j. t. miller-friedmann, j. l. sonnert, g. sadler, p. m., & hazari, z. (2012). out-of-school time science activities and their association with career interest in stem. international journal of science education, part b, 2(1), 63-79. daly, s., adams, r., & bodner, g. (2012). what does it mean to design? a qualitative investigation of design professionals’ experiences. journal of engineering education, 101(2), 187–219. duran, m., & sendag, s. (2012). a preliminary investigation into critical thinking skills of urban high school students: role of an it/stem program. creative education, 3(2), 241. doi: 10.4236/ ce.2012.32038. dym, c. l., agogino, a. m., eris, o., frey, d. d., & leifer, l. j. (2005). engineering design thinking, teaching, and learning. journal of engineering education, 94(1), 104-120. eroğlu, s., & bektaş o. (2016). ideas of science teachers took stem education about stem based activities. journal of qualitative research in education, 4(3), 43-67. gencer, a. s., doğan, h., bilen, k., & can, b. (2019). integrated stem education models. pau journal of education, 45(45), 38-55. hacıoğlu, y., & dönmez usta, n. (2020). digital game design-based stem activity: biodiversity example. science activities, 57(1), 1-15. hathcock, s. j., dickerson, eckhoff, d. l., & katsioloudis, p. (2015). scaffolding for creative product possibilities in a design-based stem activity. research in science education, 45(5), 727-748. havice, w. (2009). the power and promise of a stem education: thriving in a complex technological world. the overlooked stem imperatives: technology and engineering, 10-17. hynes, m., portsmore, m., dare, e., milto, e., rogers, c., hammer, d., & carberry, a.. (2011). infusing engineering design into high school stem courses. retrieved from https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=116 5&context=ncete_publications. international technology education association (itea). (2007). standards for technological literacy: content for the study of technology. reston, va: author. karahan, e., bilici, s. c., & ünal, a. (2015). integration of media design processes in science, technology, engineering, and mathematics (stem) education. eurasian journal of educational research, 15(60), 221-240. karslı-baydere, f., hacıoğlu, y., & kocaman, k. (2019). an example of the science, technology, engineering, and mathematics (stem) education activity: anticoagulant drugs. kastamonu education journal, 27(5), 1935. katehi, l., pearson, g., & feder, m. (2009). engineering in k-12 education: understanding the status and improving the prospectus. washington, dc: national academies press. kennedy, t. j., & odell, m. r. l. (2014). engaging students in stem education. science education international, 25(3), 246-258. koehler, c., latif, s. k., faraclas, e., sanchez, s., & kazerounian, k. (2005, june). engineering frameworks for a high school setting: guidelines for promoting technical literacy for high school students. in 2005 annual conference (pp. 10-550). kolodner, j. l. (2002a). facilitating the learning of design practices: lessons learned from an inquiry into science education. journal of industrial teacher education, 39(3), 9-40. kolodner, j. l. (2002b). learning by design: iterations of design challenges for better learning of science skills. cognitive studies: bulletin of the japanese cognitive science society, 9(3), 338-350. lawson, b., & dorst, k. (2013). design expertise. routledge. lemons, g., carberry, a., swan, c., rogers, c., & jarvin, l. (2010, june). the importance of problem interpretation for engineering students. in 2010 annual conference & exposition (pp. 15-1238). leonard, m. j. (2004). toward epistemologically authentic engineering design activities in the science classroom. online submission. lewis, t. (2006). design and inquiry: bases for an accommodation between science and technology education in the curriculum? journal of research in science teaching: the official journal of the national association for research in science teaching, 43(3), 255-281. liu, x., & tang, l. (2004). the progression of students’ conceptions of energy: a cross-grade, cross-cultural study. canadian journal of science, mathematics & technology education, 4(1), 43-57. maltese, a. v., & tai, r. h. (2011). pipeline persistence: examining the association of educational experiences with earned degrees in stem among us students. science education, 95(5), 877-907. mater, n. s., hussein, m, j, h., salha, s. h., draidi, f. r., shaqour, a. z., qatanani, n., & affouneh, s. (2020). the effect of the integration of stem on critical thinking and technology acceptance model. educational studies, 1-17. doi.org/10.1080/03055698.2020.1793736. meb-yeği̇tek. (2016). stem eğitim raporu [stem education report]. ankara: yenilik ve eğitim teknolojileri genel müdürlüğü. mentzer, n. (2011). high school engineering and technology education integration through design challenges. journal of stem teacher education, 48(2), 7. mne. (2018). secondary school physics course (9, 10, 11 and 12th grades) curriculum. ankara, turkey: talim terbiye kurulu başkanlığı. moore, t. j., stohlmann, m. s., wang, h. h., tank, k. m., glancy, a. w., & roehrig, g. h. (2014). implementation and integration of engineering in k-12 stem education. in engineering in pre-college settings: synthesizing research, policy, and practices. ed. s¸. purzer, j. strobel, m. c. cardella), pp. 35–60. west lafayette, india: purdue university press. morrison, j. (2006). attributes of stem education: the student, the school, the classroom. teaching institute for excellence in stem, 20, 2-7. national research council [nrc]. (2010). exploring the intersection of science education and 21st century skills: a workshop summary. washington, dc: national academies press. national research council [nrc]. (2012). a framework for k-12 science education: practices, crosscutting concepts, and core ideas. washington dc: the national academic press. niess, m. l. (2005). preparing teachers to teach science and mathematics with technology: developing a technology pedagogical content knowledge. teaching and teacher education, 21(5), 509-523. ormanci, ü. (2020). thematic content analysis of doctoral theses in stem education: turkey context. journal of turkish science education, 17(1), 126-146. ozkan, g., & umdu-topsakal, u. (2020). investigating the effectiveness of steam education on students’ conceptual understanding of force and energy topics. research in science & technological education, 1-20, doi: 10.1080/02635143.2020.1769586. park, m., & liu, x. (2016). assessing understanding of the energy concept in different science disciplines. science education, 100(3), 483-516. ritz, j. m., & fan, s. c. (2015). stem and technology education: international state-of-the-art. international journal of technology and design education, 25(4), 429-451. rosenblum, i., & kazis, r. (2014). middle-skill stem state policy framework. jobs for the future. retrieved from https://files.eric.ed.gov/fulltext/ed556763.pdf. silk, e. m., schunn, c. d., & cary, m. s. (2009). the impact of an engineering design curriculum on science reasoning in an urban setting. journal of science education and technology, 18(3), 209-223. thomas, t. a. (2014). elementary teachers' receptivity to integrated science, technology, engineering, and mathematics (stem) education in the elementary grades (doctoral dissertation). university of nevada, reno. tindall, t., & hamil, b. (2004). gender disparity in science education: the causes, consequences, and solutions. education, 125(2). töman, u., & çimer, s. o. (2012). an investigation into the conception energy conversion at different educational levels. erzincan university journal of education faculty, 14(2), 289312. uluçınar, ş., cansaran, a., & karaca, a. (2004). the evaluation of laboratory studies in science. türk eğitim bilimleri dergisi, 2(4), 465-475. https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=1165&context=ncete_publications https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=1165&context=ncete_publications https://doi.org/10.1080/02635143.2020.1769586 https://files.eric.ed.gov/fulltext/ed556763.pdf journal of science learning article doi: 10.17509/jsl.v5i3.47226 508 j.sci.learn.2022.5(3).500-508 wang, h. h. (2012). a new era of science education: science teachers ‘perceptions and classroom practices of science, technology, engineering and mathematics (stem) integration (unpublished doctoral thesis). the university of minnesota. watts, d. m. (1983). some alternative views of energy. physics education, 21, 154-156. wells, j. g. (2016). efficacy of the technological/engineering design approach: imposed cognitive demands within design-based biotechnology instruction. journal of technology education, 27(2), 420. wendell, k. b. (2008). the theoretical and empirical basis for design-based science instruction for children (unpublished qualifying paper). tufts university. wendell, k. b., & lee, h. s. (2010). elementary students learning of materials science practices through instruction based on engineering design tasks. journal of science education and technology, 19(6), 580-601. wendell, k. b., andrews, c. j., & paugh, p. (2019). supporting knowledge construction in elementary engineering design. science education, 103(4), 952-978. yavuz topaloğlu, m., & balkan kiyici, f. (2017). the effect of hydroelectric power plants trip on students' conceptual understandings. mersin university journal of the faculty of education, 13(3). yıldırım, b. (2016). 7. sınıf fen bilimleri dersine entegre edilmiş fen teknoloji mühendislik matematik (stem) uygulamaları ve tam öğrenmenin etkilerinin incelenmesi [examining the effects of science technology engineering mathematics (stem) applications and mastery learning integrated into 7th grade science course] (unpublished phd thesis). gazi üniversitesi, eğitim bilimleri enstitüsü, ankara. yıldırım, b., & altun, f. (2015). investigating the effect of stem education and engineering applications on science laboratory lectures. el-cezeri journal of science and engineering, 2(2), 28-40. yıldırım, p. (2017). a qualitative study on integration of science, technology, engineering, and mathematics (stem). atatürk university kazım karabekir education faculty journal, 35, 31-55. yilmaz, h., koyunkaya, m. y., güler, f., & güzey, s. (2017). turkish adaptation of the attitudes toward science, technology, engineering, and mathematics (stem) education scale. kastamonu education journal, 25(5), 1787-1800. a © 2022 indonesian society for science educator 531 j.sci.learn.2022.5(3).531-539 received: 07 june 2022 revised: 03 august 2022 published: 27 november 2022 science teachers' view on sustainable development in covid-19 pandemic process hakan türkmen1* 1department of mathematics and science education, faculty of education, ege university & izmir, turkey corresponding author: hakan.turkmen@ege.edu.tr abstract this study examines what science teachers think about sustainable development and its' goals, how they integrate sdgs into their science lessons, and how the covid-19 pandemic affects their thoughts about sustainable development. the study group was selected by convenience sampling from 51 middle schools in the central district of one of the big cities. the questionnaire, including five open-ended questions, was prepared and checked by two experts. after their feedback, it was re-corrected and then applied to the participants. it was conducted using the "google forms" application. then, it was delivered to 165 teachers’ e-mail accounts. answering the questionnaire takes approximately 10-15 minutes for each participant. one hundred five teachers answered the questionnaire. the content analysis method was used to analyze data. the results showed that teachers mainly described sustainable development with the goal of "responsible consumption and production", "decent work and economic growth", and "quality education". half of the teachers are still trying to integrate sd using traditional science subjects strategies. and generally focused on science subjects of matter cycles, environmental problems, biodiversity, fuels, and domestic waste and recycling. in this process, their problems are usually intense curriculum, insufficient time, and attitudes of school administration. keywords sustainable development goals, education for sustainable development, primary science teachers 1. introduction one of the main focal points of the 21st century understanding of education is the concept of sustainable development. it is hard to define because of its continuous evolution. briefly, we can determine that sustainable development (sd) means to develop the needs of the present without harming the next generations. this perspective emerged with the concept of environmental education in the 1970s. according to united nations (un), “sd has been recognized as an overarching goal for institutions at the national, regional and international levels since 1972” (yiu & saner, 2014). in the intergovernmental conference on environmental education in 1977, scientists declared that environmental education should enable students, who will become citizens of the future worlds, to acquire awareness, knowledge, skills, and participation in environmental issues that concern society (unesco, 1978). the framework of environmental education contained conservation and pollution issues in the ’70s, then slowly broadened with the public's concerns about land-use management, population rate, energy, and production. today, sd, climate change, and biodiversity concepts arose in environmental education (hungerford, 2009). in the united nations conference on environmental and development (1992), it was reported that sd has three mutually reinforcing dimensions, social, economic, and environmental, to handle issues for supplying natural resources for next generations all over the world (albareda-tiana, vidal-raméntol, & fernándezmorilla, 2018; sachs et al., 2019). societies’ living standards and living conditions determine their consumerism and production trends. if we increase the welfare of their societies, we can draw people's attention to environmental problems more easily. united nations conference on sustainable development rio+20 conference (2012) paid attention to 7 areas, which were jobs, energy, cities, food, water, oceans, and disasters. it declared the main aspect of sustainable development goals (sdgs), which is to supply the national needs of countries worldwide. sdgs were tried to build logically on the eight millennium development goals, (1) end poverty and hunger (2) universal education (3) gender and equality (4) child health (5) maternal health (6) combat hiv/aids (7) journal of science learning article doi: 10.17509/jsl.v5i3.46743 532 j.sci.learn.2022.5(3).531-539 environmental sustainability (8) global partnership, designed to improve the lives of the poor people, were adopted by 189 nations in 2000 (griggs, et al. 2014; blanc, 2015; leal filho et al., 2019). millennium development goals notable success until it ended in 2015. from this point of view, the sdgs conducted by united nations development program went into operation in 2015 and are targeted by 2030. these 17 goals are named (1) no poverty, (2) zero hanger, (3) good health & well-being, (4) quality education, (5) gender equality, (6) clean water & sanitation, (7) affordable & clean energy, (8) decent work & economic growth, (9) industry, innovation & infrastructure, (10) reduced inequalities, (11) sustainable cities & communities, (12) responsible consumption & production, (13) climate action, (14) life below water, (15) life on land, (16) peace, justice, & strong institutions, (17) partnership for the goals (esquivel & sweetman, 2016; fleming, wise, hansen & sams, 2017; griggs, et al. 2014). the united nations development program (undp) supports these goals by integrating 170 countries’ governmental political plans and policies (united nations development programme, 2016). briefly, sd aims to represent nice, equal, and sustainable lifestyles for all living. the main terminology is “sustainability”, which essentially established three cornerstones, economy, environment, and society, which is a way of national development. sustainability helps people achieve their ambitions and improves their living standards by protecting their economic, social, and environmental systems (brito, 2012; leal filho et al., 2019; moyer & hedden, 2020; omole & ozoji, 2014). society members should be responsible for not harming the environment so that our kid’s future is not endangered. one of the important ways to integrate sd into their governmental policies is to educate people. the significance of sd in education lies within its vision of gaining environmental and ethical awareness, attitudes, and behavior. moreover, it is helpful for decision-making in local/global problems and establishes connections with the dimensions under the sdgs (davidson, 2014; vladimirova & blanc, 2016). in 2005, the un general assembly took a big step and invited all educational institutions to join the decade of education for sustainable development (desd) (2005-2014) project to help new education perspectives for sustainability. according to desd, integrating sd in education is the institutional responsibility. this responsibility is to reshape the curriculum and provide better teaching for current and next generations. of course, this process will be longlasting, and adopting a flexible and comprehensive point of view to change is necessary (albareda-tiana, vidalraméntol, & fernández-morilla, 2018). one proof of why integrating sd is a long journey is moyer & hedden’s (2020) study. they analyzed 186 countries by 9 sdgs indicators in their research. the data showed that 43 percent of countries-values had already been reached by 2015, and 53 percent of country values will be achieved by 2030. thus, integration of sustainability in education is a complex, lengthy process and multidisciplinary approach to the environmental, development, socio-cultural issues, and demographic aspects. this approach improves the social dimension of sustainable development and its goals while providing innovative context in education (rauch, 2002). although turkey lacks practices in sustainable development, it is attentive to integrating economic, social, and environmental dimensions of sd at many levels of policy (öztürk, 2018). according to kaya, çobanoğlu & artvinli (2011), some of the key examples of turkish sustainability policies, which aim to ensure economic, cultural, and social sustainable development, are the "national environmental strategy", prepared for 2007 2013 and the "integration of sd into sectoral policies project" with a large corporate domain. these new policies affected education. environmental education-related subjects involving sustainability and sustainable development concepts were implemented in related courses in national educational program. with the 2013 science education curriculum (3-8 grades), science-technology-society concepts, containing socio-scientific and sustainable development issues, included learning areas of the primary science courses to educate scientifically literate children. in 2018, the ministry of national education added the four objectives to the science education program. the program is concerned with environmental issues such as recycling and waste in general, the relationship between humans and nature, health care, global warming, and social awareness such as smoking cessation and organ donation, and 33 of the total 305 objectives in the program are related to sd in all the levels of the 3-8 grades level. additionally, the “sustainable development” subject in 8-grade science program aims to integrate sd as one of the general objectives of science education programs and to develop an awareness of sd in society (ateş, 2019). in the literature, there are many studies about that. lawale & bory-adams (2010) suggested that integrating sd into lessons provides a convenient learning environment for the four pillars of learning, learning to know, learning to do, learning to live together, and learning to be, which are the cornerstone of 21st-century education suggested by the 1996 delors report (2010). tuncay-yüksel, yılmaz-tüzün & teksoz (2011) examined the relationship between moral reasoning patterns and environmental dilemmas. they found that pre-service teachers’ thoughts were directly related to nature, and their environmental dilemmas were related to global issues, not local ones. thus, they suggested examining environmental problems in lessons to help students develop moral reasoning patterns about environmental problems. borg, gericke, höglund & bergman (2014) states that one teacher cannot be expected journal of science learning article doi: 10.17509/jsl.v5i3.46743 533 j.sci.learn.2022.5(3).531-539 to handle all the dimensions of sd extensively. thus, it is essential to follow an interdisciplinary approach. besides, the interconnected nature of sdgs allows teachers to associate science subjects across the lessons. yücel & özkan (2014) state science teachers think that the environment-related subjects in the curriculum are sufficient to acquire the ability to comprehend the 'sciencetechnology-society-environment' relationship whereas insufficient to acquire an awareness of the environment. however, simsekli (2015) examines the environmental practices, including activities and experiment sets, investigates the effect on the awareness of primary students’ and found rises in their awareness level of local problems. therefore, science teachers must apply curriculum goals to their lessons with student-centered strategies. çobanoğlu & türer (2015) investigated the preservice teachers’ awareness of the social, economic, and environmental dimensions of sd. their study showed that preservice science teachers have a higher awareness level for the environmental dimension of sd but lower awareness of the social dimension of sd than preservice social science teachers. harman (2017) aimed to investigate the awareness of preservice science teachers about sustainability. analyzed their drawings about their dreams of school, most preservice science teachers had low awareness, because they were not enough to connect the knowledge learned in the lessons with real life. annan-diab & molinari (2017) showed how important to adopt an interdisciplinary approach in education for sd in their study and defined that sd-related subjects provide postgraduate mba students to activate their previous knowledge and use knowledge from other disciplines. andersen (2018) argues that to encourage teachers to use action-based and task-based learning for sustainable context, science textbooks should embody action-based and task-based activities, which would be efficient to overcome the problems related to material insufficiency and pedagogical deficiencies that limit teachers in lessons based on sdgs. aytar & özsevgenç (2019) evaluated how the interdisciplinary approach affects the 7th grade students’ sd development, based on some courses. they declared that if practices are applied in science lessons for sd, students' conceptual understanding will increase, especially in biodiversity, soil pollution, hunger, renewablenonrenewable energy sources, and recycling concepts. wang, li, malik & anwar (2021) study was related to educational technology and online education with the middle school learners. they gained positive attitudes and satisfaction with online education. this result can be easily implied that when we improved the learner satisfaction and increased the implementation of online education in our countries, we could achieve sdg4 (quality education). people faced the covid-19 pandemic at the beginning of 2020. this pandemic affected health, economy, society, and education significantly. the most important known effects were curfews and the closures of factories, institutions, and schools to avoid spreading the virus. at that time, closing the schools was a big step in fighting the covid pandemic for education. the closure of schools led governments to quickly change education perspectives from face-to-face teaching to the use of online teaching. the idea/practice of conducting the science lessons with direct face-to-face student participation to teach science content effectively changed within that process. all teachers had to make the rapid transition to distance learning and online teaching in a short time. unfortunately, the teaching of sd topics in science lessons was influenced by the pandemic process. because sd education is a transdisciplinary process that addresses the cognitive, affective, and psychomotor domains, it can be difficult for teachers in the distance education process during the pandemic process. although it is crucial to gain knowledge, awareness, interest, and attitude toward sustainable development goals for students, we do not have enough information about how our teachers handled this problem and how students are affected in that learning process. this study aims to analyze what science teachers think about sd, how they integrate sdgs into their science lessons, and how the covid-19 pandemic process affects their thought about sd. 2. method 2.1. the research design a qualitative research methodology was used in this study. the research aimed to discover what science teachers do for students to gain knowledge, awareness, interest, and attitude towards sustainable development goals in 5-8 grades' science lessons, which course subjects they do, and which techniques they use. moreover, what limitations do they face when integrating sustainable development goals in their science lessons? 2.2. study group the sample in this research comprised 105 science teachers lecturing in 5-8 grades through convenience sampling. the convenience samples consist of participants that are easy to access thus, non-random, with % a 41.6 average response rate (yu & cooper, 1983). in this study, the average response rate was 63.6%. thus, we assumed that the sample reflects the population and that the sample responds to the questions sincerely and correctly. 2.3. data collection tools data collection is provided by an online questionnaire of 5 open-ended questions applied to the participants. the researcher prepared questions the asked to analyze by three expert opinions. according to their feedback, questions were revised and finalized to be applied to the study. 2.4. data analysis the content analysis method was applied to evaluate the teachers’ responses by categorizing the study's theme and code system. the fundamental process in the content journal of science learning article doi: 10.17509/jsl.v5i3.46743 534 j.sci.learn.2022.5(3).531-539 analysis method is to categorize the collected data within the framework of certain themes, organize it into similar codes, and interpret these themes and codes to understand readers (selçuk, palanci, kandemir & dündar, 2014). the teacher responses to 5 questions were analyzed particularly, with simultaneous coding of 2 experts in science education. 3. result and discussion the data are analyzed for each question separately. the participants asked the first question, “what does sustainable development mean?”. responses to the first questions were examined in the light of sdgs. the ninetytwo (87.6%), eighty-eight (83.3%), eighty (76.2%), and seventy-nine (75.2%) of those 105 science teachers point out “good health & well-being,” “responsible consumption & production,” “decent work & economic growth” and “quality education” goals which are the highest frequencies. twenty-eight (26.7 %) and thirty-five (33.3%) of 105 science teachers’ responses, “peace, justice & strong institutions” and “life below water,” were the lowest frequencies. on the other hand, four science teachers (3.8%) responses could not be paired with sdgs (table 1). the second question was, “what do you do for students to gain knowledge, awareness, interest, or attitude towards sdgs in your lessons?” after the content analysis of answers based on teaching/learning strategy, their responses were analyzed into three themes, “expository teaching” (68.6%) and “inquiry-based learning” (31.4%). in the expository teaching theme, science teachers generally prefer to use “direct lecturing” (42.9%), “exemplifying” (33.3%), “informing” (23.8%), “demonstration” (19.1%), and “field trip after lecturing” (9.5%) techniques in their lessons. the inquiry-based learning / hands-on theme includes 18 codes. while the highest ratio of code is the “5e” teaching model (19.1%), “project” (19.1%), and “discussion” (17.1%), the lowest ratio of code is “philips 66” (1.9%). the exciting result was that there are many codes were detected. thus, almost one-third of participants were still trying to use modern teaching/learning approaches, but the rest still used a traditional perspective on science teaching. table 2 shows teaching strategies of teachers to integrate sdgs into lessons the third question, “in what science subjects do you connect with sdgs in your lessons?” was designed to discover science subjects teachers associate with sdgs in their lessons. the answers were categorized into three units of science area. “in the earth and universe” unit, there were found 13 science subjects. the highest ratio was “matter cycles and environmental problems” (13.5%), “sustainable development” (11.8%), and “biodiversity” (9.6%). the lowest rates were “reproduction, growth, & development in human” (1.3%) and “reproduction, growth, & development in plants & animals” (1.3%). “in the matter and its nature” unit, four science subjects were found. while the highest ratio was “fuels” (10.6%) and “domestic wastes and recycling” (9.6%), the lowest rate was "acids and bases” (0.9%). the only science subject, table 1. definition of sd sdgs f (%) 3. good health & well-being 92 87.6 12. responsible consumption & production 88 83.8 8. decent work & economic growth 80 76.2 4. quality education 79 75.2 10. reduced inequality 69 65.7 2. zero hunger 68 64.8 5. gender equality 60 57.1 13. climate action 56 53.3 7. affordable & clean energy 54 51.4 6. clean water & sanitation 53 50.5 9. industry, innovation, & infrastructure 53 50.5 1. no poverty 49 46.7 17. partnerships to achieve the goal 49 46.7 15. life on land 46 43.8 11. sustainable cities & communities 42 40.0 14. life below water 35 33.3 16. peace, justice, & strong institutions 28 26.7 none 4 3.8 table 2. teaching strategies of teachers to integrate sdgs into lessons theme code f % expository teaching (f: 72, 68.6%) lecturing 45 42.9 exemplifying 35 33.3 informing 25 23.8 demonstration 20 19.1 after lecturing, virtual field trips 10 9.5 inquiry-based learning / hands-on (f: 33, 31.4%) 5e 20 19.1 projects 20 19.1 discussions 18 17.1 open-ended questions 12 11.4 open-lab approaches 12 11.4 brainstorming 9 8.6 argumentation 5 4.8 narrative (problem-based scenario) 5 4.8 student tasks (presentation) 5 4.8 compare & contrast examples 4 3.8 6 thinking hat 4 3.8 before lecturing, the field trip 4 3.8 concept cartoon 3 2.9 flip-flop 3 2.9 research studies 3 2.9 using visualtechnological tools 3 2.9 philips 66 2 1.9 journal of science learning article doi: 10.17509/jsl.v5i3.46743 535 j.sci.learn.2022.5(3).531-539 “transformation of electrical energy” (5.8%) in the “physical events” unit, was said by participants (table 3). the fourth question, “what are the reasons that limit your studies on sdgs in your lessons?” was asked , and participants' answers were analyzed. the results showed that 14 codes appeared as limitations when science teachers connect with science subjects and sdgs in their lessons. according to 105 science teachers, the high limitations for teachers were “insufficient time” (57.1%), “intense curriculum” (55.2.1%), and "exam-based education system" (52.8%). additionally, teachers mentioned educational policy, school physical situations, technological deficiencies, families’ perspectives, students’ knowledge levels, and pedagogical deficiencies as their limits. the lowest limit declared by science teachers was “science has abstract knowledge” (9.5%), and only two science teachers declared they had not faced obstacles because they said, "i do not do anything related to sdgs." table 4 discusses reasons that limit teachers from making connections with sdgs in their lessons. the last question of the study was, “could you explain how the coronavirus (covid-19) pandemic has affected your thoughts about teaching sd? the 105 participants’ answers were separated into two categories, positive and negative thoughts. the teaching of sd in science lessons was generally affected negative way. teachers’ negative thoughts were analyzed in 3 themes, teachers, students, and administrators. according to teachers, they were caught unprepared during the covid process because they didn’t have enough knowledge about pedagogical teaching models for distance learning (83.8%), online teaching experience (80.9%), about time to make the transition (76.2%), and about the evaluation of student's performance (66.7%). moreover, they have technological device problems, internet access problems (49.5%), and computer problems (47.6%). they also do not know how to use new technological devices (47.6%) and do not have enough experience with virtual field trips (47.6%) and virtual lab experiences (43.8%). furthermore, about one-quarter of the participants said they lacked personal table 3. sdgs related science subjects areas science subjects  % the earth and universe unit 5 grade biodiversity 30 9.6 human-environment relationship 24 7.7 destructive natural events 10 3.2 6 grade systems and health in our body 7 2.2 7 grade reproduction, growth, & development in human 4 1.3 reproduction, growth, & development in plants and animals 4 1.3 8 grade matter cycles & environmental problems 42 13.5 sustainable development 37 11.8 biotechnology 22 7.1 food chain and energy flow 17 5.5 energy conversions 10 3.2 climate & air movements 7 2.2 dna & genetic code 4 1.3 matter and its nature unit 6 grade fuels 33 10.6 7 grade domestic wastes & recycling 30 9.6 8 grade chemical industry in turkey 10 3.2 acids and bases 3 0.9 physical events unit 8 grade transformation of electrical energy 18 5.8 total 312 100.0% table 4. reasons that limit teachers to make connections with sdgs in their lessons limitations  % insufficient time 60 57.1 intense curriculum 58 55.2 exam based educational system 55 52.8 informal learning environment constraints 40 38.1 economic reasons / financial resources 35 33.3 crowded population 35 33.3 pedagogical deficiencies 34 32.4 student disinterest 28 26.7 student's knowledge level 25 23.8 attitudes of parents/families 20 19 attitudes of school administration 18 17.1 physical deficiency in school 18 17.1 science is abstract 10 9.5 none 2 1.9 journal of science learning article doi: 10.17509/jsl.v5i3.46743 536 j.sci.learn.2022.5(3).531-539 interaction/dialogue with colleagues (26.7%) and students (22.9%). in the student’s theme, there were seen internet access problems (66.7%), computer problems (64.8%), and lack of interest/motivation (42.8%). forty teachers (38.1%) thought they did not have enough support from the administration. given positive thoughts by teachers are generally future perceptions of education. they thought that people let nature breathe (ex. reducing carbon emission, global warming) (71.4%) and understood how the importance of some sdgs, which are sdg3 (health & well-being) (92.9%); sdg4 (quality education) (66.7%); sdg8 (decent work & economic growth) (52.9%); and sdg12 (consumption & production) (51.4%). additionally, politics understood and made some efforts in education quality (especially technology integration) (33.3%) and equality in education (26.7%). table 5 tabulates the covid-19 pandemic effects of sd teaching according to quality assurance agency in the uk, the main aim of education of sd is to raise knowledgeable and skilled students who can contribute to the development of humanity and, for that purpose, guard the environmental, social, and economic well-being, both in the present and for next generations (kemp, bellingham & longhurst, 2014). science teachers know this purpose. their thoughts generally focused on good health & well-being (sdg3), responsible consumption & production (sdg12), decent work & economic growth (sdg8), and quality education (sdg4). this result has similarities with shulla et al., (2021) analyzing how the covid19 pandemic affected the sdgs. they defined a model of the state of being connected with sdg3 (health & well-being), sdg4 (quality education), sdg8 (decent work & economic growth), sdg12 (consumption & production), sdg13 (climate action), and covid-19 consequences. moreover, this study highlighted social and economic than environmental dimensions because of the covid-19 pandemic process. the restriction of people's social life, quarantines, and other occupational groups having to close businesses may be the reasons for their thoughts. while one-third of the teachers still used modern science teaching methods to teach sd in their lessons, the remaining participants were using conservative methods because they probably did not know contemporary teaching methods. although the transition began from a conservative teaching perspective to a student-centered teaching perspective in 2005, they still have a problem with to use of techniques and methods of the student-centered approach. according to the ministry of national education board, “the science curriculum in turkey, have general goals that needed to implement that all students be able to “use science process skills and scientific research designs to produce solutions for human-environment related issues” (goal 2), and “to develop reasoning patterns, scientific thinking habits, and decision-making skills by using socioscientific issues” (goal 9) (p.9). unfortunately, two-thirds table 5. the covid-19 pandemic effects of sd teaching thoughts source of problem codes f % negative teachers lack of pedagogical knowledge of distance learning 88 83.8 lack of online teaching experience 85 80.9 the limited time to make the transition, 80 76.2 evaluation of students’ performance 70 66.7 internet access problems 52 49.5 computer problems 50 47.6 lack of expertise regarding new technologies 50 47.6 lack of virtual field trips experience 50 47.6 lack of virtual labs experience 46 43.8 lack of materials/resources 45 42.8 lack of personal interactions/dialogues with colleagues 28 26.7 lack of personal interactions/dialogues with students 24 22.9 students computer problems 70 66.7 internet access problems 68 64.8 lack of interest/motivation 45 42.8 administrator lack of support from the administration 40 38.1 positive human beings let nature breathe (ex. reducing carbon emission, global warming … ) 75 71.4 understanding of some sdgs sdg3 (health & well-being), 97 92.9 sdg4 (quality education), 70 66.7 sdg8 (decent work & economic growth), 55 52.9 sdg12 (consumption & production) 54 51.4 government effort education quality (technology integration) 35 33.3 equal education 28 26.7 journal of science learning article doi: 10.17509/jsl.v5i3.46743 537 j.sci.learn.2022.5(3).531-539 of teachers mainly use conservative teaching methods (lecturing, informing), which scarcely gives opportunity to students to gain these goals. on the other hand, the 5e teaching model, projects, and discussion techniques were mostly preferred, rarely preferred philips 66, visual technological tools, research studies, and flip-flops to apply by science teachers. the findings of this study reveal those science teachers consider teaching environment-related sdgs more than social-related sdgs. because they focused generally on the unit of the earth and universe. the high subjects, biodiversity, matter cycle, environmental problems, fuels, domestic wastes, and recycling, are remembered chiefly in their science lessons. çobanoğlu & türer (2015) found the same results in their study of science teacher candidates. as science teachers have environment lessons in higher education and their teaching area includes environmental science, awareness of sd in the ecological dimension is assumed to be higher than in other dimensions. therefore, they attribute mostly environmental sdgs to define sd. the science teaching program in turkey is a spiral curriculum with repetitive objectives and explanations in different subjects and grade levels, besides holistic outcomes. this spiral approach enables teachers to create a multidisciplinary lesson context in which students use their prior knowledge from science and all other related courses. in agenda21, interconnections between sociocultural aspects of the environment and development issues are suggested through multidisciplinary curricula (unced, 1992). however, jimenez, lerch & bromley (2017) reviewed over 1000 books worldwide, and findings reveal that not all issues at the global level that are compatible with sdgs are evenly included in the school curriculum. for instance, violence against women and lgbt rights still seems too delicate for school discussions. likewise, although science contexts are convenient for the sdgs like gender equality and reducing inequalities, explicitly related subjects and objectives couldn't find in the 2018 science curriculum in turkey, but the linkages that teachers' emphasis or guidance on implicitly related subjects. the lack of integrated and multidisciplinary structure is also found in un reports as there are poor connections between education and energy, water, urbanization, and a lack of policy implications dimensions (vladimirova & blanc, 2016). thus, in general, reports and policies, education should have a better place for educational system developers and educators to devote themselves to integrating sdgs into learning environments. according to results of limitations of implementing sd in science lessons findings, insufficient time, intense curriculum, and the exam-based education system are the most significant limitations for science teachers in integrating sdgs into their course subjects. özsevgeç & artun (2005) also define some related difficulties, such as teachers' inadequate time for environmental subjects and deficiency of objectives. their study examines teachers' challenges in the "human and environment" unit. to overcome these problems, sdgs and environment-related subjects can place in the other subjects by taking advantage of their multidisciplinary and multidimensional characteristics. the teacher, who states that s/he didn't do anything related to sdgs because of the exam-based educational system and attitudes of school administration, parents, and students regard to environment-related subjects, defines crucial problems in environmental education. although one of the goals of the science curriculum in turkey is to enable students "to recognize the interaction between the individual, environment, and society; develop an awareness of sd in society, economy, and natural resources.” the aim of science education may correlate to people's knowledge of science content (smith & siegel, 2004). thus, it is vital to people understand that to resolve environmental crises, raise individuals who are aware of the environmental consequences of their actions, determine their behaviors accordingly, and act on the issues by examining the environmental aspects of state and private sector policies through environmental education (teixeira, 2013). almost half of the participants declared they did not know how to use virtual labs, virtual field trips, and new technological software. one-fourth of participants have struggled with interactions/dialogues with students and colleagues. moreover, all students must have a computer, tablet, or cell phone to join online lessons. this fact is the barrier to an equal education. because science teachers defined that students have computers, internet access, and motivational problems. the government should consider getting higher education quality and equal education standards for the pandemic effects. this result parallels dovletmurzaeva, magomadova & barzaeva (2021), which invited the governments to solve the problem by creating additional organizations and financial support to afford the pandemic process's effect on sdg 4. 4. conclusion the covid-19 pandemic influenced the whole education system, of course, the integration of sd in science teaching. unfortunately, there was not enough time to adapt the requirements of the solution to the pandemic problem for education. many countries found the solution via applying distance/online learning strategies. however, science teachers who participated in this study mostly do not have enough knowledge about pedagogical teaching models for distance learning, do not have enough experience in online teaching, and do not know how to evaluate their students on online lessons besides using new technological devices and internet access problems. as known, the best way of teaching sdgs is to use studentcentered teaching approaches/strategies, but this journal of science learning article doi: 10.17509/jsl.v5i3.46743 538 j.sci.learn.2022.5(3).531-539 perspective has been halted or postponed by teachers and circumstances. conversely, the pandemic process has had a positive impact on the understanding of some sdgs. science teachers think that human beings let nature breathe and renew itself. all people understand how health and wellbeing, power of economics, and quality of education are important for people to leave a nice world to the next generation. the governments should make big steps to reach higher education quality and equal education standards for their students. the conclusion of this study is that science teacher should carefully analyze curriculum based on objectives and main goals of science education and apply them to their lessons by considering students' active involvement ensuring analyzing, researching, reasoning, and scientific thinking. this way, we can achieve the next generation's effective public participation in sd and reach the sdgs. besides, government effort is necessary to afford technological devices and internet access for all students references albareda-tiana, s., vidal-raméntol, s., & fernández-morilla, m. (2018). implementing the sustainable development goals at university level. international journal of sustainability in higher education, 19(3), 473-497. doi:10.1108/ijshe-05-2017-0069 andersen, k. n. (2018). evaluation of school tasks in the light of sustainability education: textbook research in science education in luxembourgish primary schools. environmental education research, 24(9), 1301–1319. https://doi.org/10.1080/13504622.2017.1384798 annan-diab, f., & molinari, c. (2017). interdisciplinarity: practical approach to advancing education for sustainability and for the sustainable development goals. international journal of management education, 15(2), 73–83. http://dx.doi.org/10.1016/j.ijme.2017.03.006 ateş, h. (2019). analysis of the science curriculum in terms of education for sustainable development. yyu journal of education faculty, 16(1), 101-127. https://dergipark.org.tr/tr/pub/yyuefd/issue/50700/660343 aytar, a., & özsevgeç, t. (2019). the effect of interdisciplinary science education on sustainable development of 7th-grade students. hacettepe university journal of education, 34(2), 324-357. doi:10.16986/huje.2018045282 blanc, d. l. (2015). development goals as a network of targets. sustainable development, 187, 176-187. https://doi.org/10.1002/sd.1582 borg, c., gericke, n., höglund, h. o., & bergman, e. (2014). subject and experience-bound differences in teachers’ conceptual understanding of sustainable development. environmental education research, 20(4), 526–551. https://doi.org/10.1080/13504622.2013.833584 brito, l., (2012). analyzing sustainable development goals. science 336 (6087), 1396. doi: 10.1126/science.1224531 çobanoğlu, o., & türer, b. (2015). the awareness of science and social science prospective teachers regarding to sustainable environment. international journal of turkish education sciences, 235– 247.https://dergipark.org.tr/tr/pub/goputeb/issue/34517/3850 12 davidson, k. (2014). a typology to categorize the ideologies of actors in the sustainable development debate. sustainable development, 22(1), 1–14. https://doi.org/10.1002/sd.520 delors, j. (1996). report to unesco on education for the 21st centurylearning: the treasure within. paris:unesco dovletmurzaeva, m.a., magomadova, t.l., & barzaeva, m.a. (2021). pandemic impact on sustainable education. in proceedings of the international scientific and practical conference on sustainable development of regional infrastructure (issdri 2021), vol(1) 685-689. doi:10.5220/0010595906850689 esquivel, v., & sweetman, c. (2016) gender and the sustainable development goals. gender & development, 24(1), 1-8. doi: 10.1080/13552074.2016.1153318 fleming, a., wise, r.m., hansen, h. & sams, l. (2017). the sustainable development goals: a case study. marine policy, 86, 94-103. doi: 10.1016/j.marpol.2017.09.019 griggs, d., smith, m, s., rockström, j., öhman, m.c., gaffney, o., glaser, g., kanie, n., noble, i., steffen, w. & shyamsundar. p. (2014). an integrated framework for sustainable development goals. ecology and society, 19(4), 49-72. doi:10.5751/es-07082190449 harman, g. (2017). fen bilgisi öğretmen adaylarının sürdürülebilirlik kavramı hakkındaki farkındalıkları: benim okulum [awareness of pre-service science teachers about the concept of sustainability: my school]. necatibey eğitim fakültesi elektronik fen ve matematik eğitimi dergisi, 11(2), 243-262. doi: 10.17522/balikesirnef.373383 hungerford, h. r. (2009). environmental education (ee) for the 21st century: where have we been? where are we now? where are we headed?. the journal of environmental education, 41(1), 1–6. ttps://doi.org/10.1080/00958960903206773 jimenez, j. d., lerch, j., & bromley, p. (2017). education for global citizenship and sustainable development in social science textbooks. european journal of education, 52(4), 460–476. doi: 10.1111/ejed.12240 kaya, n., çobanoğlu, m. t., & artvinli, e. (2011). environmental education studies in the world and turkey for sustainable development. in 6. ulusal coğrafya sempozyumu (pp. 407–418), ankara. kemp, s., bellingham, l., & longhurst, j. (2014). education for sustainable development: guidance for uk higher education providers. https://www.qaa.ac.uk/docs/qaa/quality-code/educationsustainable-development-guidancejune14.pdf?sfvrsn=1c46f981_8. lawale, s., & bory-adams, a. (2010). the decade of education for sustainable development: towards four pillars of learning. development, 53(4), 547–550. doi: 10.1057/dev.2010.76 leal filho, w., tripathi, s. k., andrade guerra, j. b. s. o. d., ginégarriga, r., orlovic lovren, v., & willats, j. (2019). using the sustainable development goals towards a better understanding of sustainability challenges. international journal of sustainable development & world ecology, 26(2), 179-190. doi: 10.1080/13504509.2018.1505674 moyer, j.d., & hedden, s. (2020). are we on the right path to achieve the sustainable development goals?. world development, 127, 104749. doi:10.1016/j.worlddev.2019.104749. omole, c.o., & ozoji, b.e. (2014). science education and sustainable development in nigeria. american journal of educational research, 2(8), 595-599. doi:10.12691/education-2-8-6. özsevgeç, t., & artun, h. (2005). views of science and technology course teachers about "human and environment unit". uludağ üniversitesi eğitim fakültesi dergisi, 27(1), 67–84. https://dergipark.org.tr/tr/download/article-file/153519 öztürk, m. (2018). response of educational research in turkey to the un decade of education for sustainable development. asia pacific education review, 19(4), 573–586. http://dx.doi.org/10.1007/s12564-018-9554-5 rauch, f. (2002). the potential of education for sustainable development for reform in schools. environmental education research, 8(1), 43–51. doi: 10.1080/13504620120109646 sachs, j.d., schmidt-traub, g., mazzucato, m., messner, d, nakicenovic, n., & rockström, j. (2019). six transformations to journal of science learning article doi: 10.17509/jsl.v5i3.46743 539 j.sci.learn.2022.5(3).531-539 achieve the sustainable development goals. nature sustainability, 2, 805–814. doi:10.1038/s41893-019-0352-9 selçuk, z., palanci, m., kandemir, m., & dündar, h. (2014). tendencies of the researches published in education and science journal: content analysis. egitim ve bilim, 39(173). doi: 10.15390/eb.v39i173.3278 shulla, k., voigt, b-f., cibian, s., scandone, g., martinez, e., nelkovski, f., & salehi, p. (2021). effects of covid 19 on the sustainable development goals (sdgs). discover sustainability, 2(15). doi:10.1007/s43621-021-00026-x simsekli, y. (2015). an implementation to raise environmental awareness of elementary education students. procedia-social and behavioral sciences, 191, 222-226. smith, m. u., & siegel, h. (2004). knowing, believing, and understanding : what goals for science education ?. science & education, 13, 553–582. https://doi.org/10.1023/b:sced.0000042848.14208.bf teixeira, s. r. (2013). the environmental education as a path for global sustainability. procedia social and behavioral sciences, 106, 2769–2774. https://doi.org/10.1016/j.sbspro.2013.12.318 tuncay-yüksel, b., yılmaz-tüzün, ö., & teksoz, g. t. (2011). moral reasoning patterns and influential factors in the context of environmental problems. environmental education research, 18(4), 485–505. https://doi.org/10.1080/13504622.2011.630576 unesco. (1978). intergovernmental conference on environmental education, tbilisi (ussr), 14-26 october 1977: final report (p. 96). united nations conference on environment and development (unced). (1992). results of the world conference on environment and development: agenda 21 (p. 351). rio de janerio, brazil. united nations conference on sustainable development. (2012) outcome of the conference. rio de janerio, brazil. https://rio20.un.org/sites/rio20.un.org/files/a-conf.216l1_english.pdf.pdf united nations development programme. (2016). undp support to the implementation of the 2030 agenda for sustainable development. undp policy and program brief. https://www.undp.org/content/dam/undp/library/sdgs/sdg implementation and undp_policy_and_programme_brief.pdf vladimirova, k., & le blanc, d. (2016). exploring links between education and sustainable development goals through the lens of un flagship reports. sustainable development, 24(4), 254-271. https://doi.org/10.1002/sd.1626 wang, x-y., li, g., malik, s., & anwar, a. (2021). impact of covid19 on achieving the goal of sustainable development: e-learning and educational productivity. economic research-ekonomska istraživanja, doi: 10.1080/1331677x.2021.1927789 yiu, l.s., & saner, r. (2014). sustainable development goals and millennium development goals: an analysis of the shaping and negotiation process. asia pacific journal of public administration, 36(2): 89–107. https://doi.org/10.1080/23276665.2014.911487 yu, j., & cooper, h. (1983). a quantitative review of research design effects on response rates to questionnaires. journal of marketing research, 20(1), 36. https://www.jstor.org/stable/3151410 yücel, e. 0., & özkan, m. (2014). evaluation of science and technology teachers’ views about ecosystem, biological diversity and environmental problems in terms of the instructional design. millî eğitim, 201, 165–182. https://dergipark.org.tr/tr/pub/milliegitim/issue/36164/406526 a © 2022 indonesian society for science educator 540 j.sci.learn.2022.5(3).540-549 received: 19 january 2022 revised: 03 august 2022 published: 27 november 2022 chemistry teachers’ level of scientific explanation about change of state and their beliefs about scientific explanation emre harun karaaslan1* 1naci topçuoğlu vocational high school, gaziantep university, gaziantep, turkey *corresponding author: ehkaraaslan@gmail.com abstract the purpose of this study is to examine the capability of chemistry teachers’ scientific explanation on the subject of change of state related to daily life and their beliefs about scientific basis. the sample of this study constituted forty-six chemistry teachers working at different high schools in gaziantep. the study was conducted following the phenomenographic research method. within the scope of the study, a test consisting of open-ended questions was conducted to determine the teachers’ capability to make scientific explanations and their beliefs about scientific explanations. study findings showed that the chemistry teachers participating in this study had inadequacies in making scientific explanations and that their beliefs about scientific explanations were unsatisfactory. keywords chemistry education, chemistry teacher, science education, scientific explanation 1. introduction chemistry is one of the disciplines that we use in our daily life throughout our lives. from the agents used in cleaning and hygiene works to the making of clothing, from the food industry to the manufacture of plastics, and from the fuel we use to elucidate criminal acts, we witness the use of chemistry in many fields. in addition, we frequently use chemical terms to explain the formation of acid rain and natural phenomena like snowfall. besides, we use chemistry to describe some events we encounter daily. for example, it is possible to explain why the soda we buy from the supermarket tastes different when it is cold through the solubility of gases. therefore, it is possible to say that chemistry is intertwined with our lives and has become indispensable for us; and that most of the subjects in chemistry lessons are related to daily life. one of the missions of educational institutions is to present the relationship between chemistry and everyday life to students from primary school to high school and university. one of the essential objectives of science education is to teach science lessons making associations with daily life (cajas, 1999). besides, the students need to relate the phenomena to their knowledge for a meaningful learning process (yilmaz & çavas, 2006). for significant and permanent learning in chemistry lessons, it is known that it would be helpful for the students to use their chemistry knowledge when explaining the events and phenomena related to daily life (yadigaroglu & demircioglu, 2012). therefore, it is vital to encourage the students to make explanations. as a result, students use different information and concepts when explaining various phenomena, and this helps students go through a meaningful learning process (mcneill & krajcik, 2008). however, it is difficult for students to make appropriate explanations by themselves. so that students can improve their capability of explaining, teachers must be good role models for them at school. therefore, it is considered that the explanations of teachers are important in order that the subjects of science lessons are comprehensible and permanent. hence, especially in science lessons, teachers must explain the subjects related to daily life with causal reasons (saglam, karaaslan & ayas, 2014), and students must be allowed to make explanations. in order to ensure this, it can be stated that teachers first must know what a scientific explanation is and how to make a good scientific explanation. no study was found in the literature that determines the teachers’ beliefs about scientific explanations. in a study by saglam, karaaslan & ayas (2014), primary school teacher candidates’ beliefs about scientific explanations were determined. thus, it is possible journal of science learning article doi: 10.17509/jsl.v5i3.43141 541 j.sci.learn.2022.5(3).540-549 to say that the study is unique as it has determined high school teachers’ beliefs. scientists and other people ask questions “why?” and “how?” throughout their life (lombrozo, 2006). people answer such questions based on their positions. also, science aims to explain a phenomenon or event by answering how and why it occurs (mcneill & krajcik, 2008). besides, the explanation is one of the main concepts frequently used in scientific research (rockse'n, 2016). what is meant by this concept is a scientific explanation. but what is a scientific explanation? unfortunately, it is impossible to give an exact answer to this question. although some consider the scientific explanation as explaining the phenomena through several various methods (grunberg & grunberg, 2011), and some as information structured by scientists (köseoğlu, tümay & budak, 2008), it appears that there is not only one definition in the literature regarding scientific explanation (braaten & windschitl, 2011; saglam, karaaslan & ayas (2014). for example, according to hempel (1965), a natural phenomenon must be explained based on specific laws. on the other hand, salmon (1984) emphasizes cause and effect relationships in an explanation to understand the world around us. on the other hand, friedman (1974) and kitcher (1989) emphasize that many other phenomena must be explained to explain a phenomenon. rockse’n (2016) points to three potential meanings of the word ‘explanation’. these are casual meaning, pedagogical/competence-related purpose, and scientific meaning. in the scientific meaning of explanation, there must be conditional and causal relationships between sequential events. at the same time, braaten & windschitl (2011) suggested a model that projects that observed natural phenomena, particularly, be explained by forming a logical causal relationship between established scientific facts. although the importance of scientific explanation, as well as the need for improving the students’ capability of making scientific explanations, were emphasized (american association for the advancement of science [aaas], 2009; meb 2013; national research council [nrc], 1996), in various studies, it was observed that students, teacher candidates, and teachers had difficulty in explaining the chemical phenomena related to daily life (akgün, tokur & duruk, 2016; dindar, bektaş & çelik, 2010; yıldırım & birinci konur, 2014;).moreover, with advancing technology, although visual and technological tools such as computerized animations, simulations, graphic materials, and laboratory studies are frequently used, these are not focused on scientific explanations (gilmanshina, gilmanshin, sagitova & galeeva, 2016). it is observed that students are mostly asked to provide causal details and build relationships when explaining the questions asked when the studies on science education are reviewed, especially chemistry education (herman, owens, oertli, zangori & newton, 2019). in other words, these explanations are expected to answer questions of “why” and “how” (dindar, bektaş & çelik, 2010). therefore, teachers expect students to build a causal relationship while explaining any event or phenomenon (yadigaroglu & demircioglu, 2012). but on the other hand, it is observed that students mostly make short definitions and cannot explain why they are asked to explain an event related to daily life (demirbaş & pektaş, 2009; saglam, karaaslan & ayas, 2014). as stated above, various studies showed that the students could not make adequate explanations. therefore, it is thought that teachers first need to have sufficient background on this subject to fill this deficiency in the students. this study aims to examine the high school chemistry teachers’ capability of making scientific explanations on the subject of change of state related to daily life and their beliefs about scientific explanation. therefore, answers were sought for this study's research questions: i) what is the high school chemistry teachers’ capability level of making scientific explanations on chemistry subjects related to daily life (change of state)? ii) how are the beliefs of high school chemistry teachers about a scientific explanation? 2. method the phenomenographic research method, one of the qualitative research methods, was applied in this research. people can perceive and interpret the same events differently, even living in the same environment. phenomenographic research is interested in people's different views and perceptions (christensen, johnson & turner, 2011; çepni, 2007; trigwell, 2006). phenomenographic research is focused on the phenomena we are aware of but do not have a detailed understanding of. in this research type, phenomena can appear as events, perceptions, beliefs, concepts, or situations (yıldırım & şimşek, 2013). the phenomenographic research method was used as the purpose of this study is to examine the high school chemistry teachers’ capability of making scientific explanations on the change of state related to daily life, as well as their beliefs about scientific basis. 2.1. participants the study participants were determined by the criterion sampling method of purposeful sampling. purposeful sampling allows for an in-depth study of situations that are thought to have rich information (patton, 1997). the criterion sampling method studies situations that meet a predetermined set of criteria. researchers can create the mentioned criterion or criteria. (yıldırım & şimşek, 2013). the criteria used in determining the sample in this study are as follows; teachers working from different high schools, experience (year), genders, and teachers must have taught the subject of change of state in the chemistry journal of science learning article doi: 10.17509/jsl.v5i3.43141 542 j.sci.learn.2022.5(3).540-549 course during the year the study took place. in this respect, 46 chemistry teachers working at different high schools in gaziantep participated in the study. male teachers were coded as mt1, mt2..., and female teachers were coded as ft1, ft2…. the teachers' genders and experience (year) were given by frequency and percentage in table 1. it can be said that the numbers of male (22) and female (24) teachers are very close and that the teachers with different years of experience were selected (table 1) 2.2. data collection the study aims to determine the high school teachers’ beliefs about scientific explanation and their capability of making scientific explanations on chemistry subjects related to daily life. it was decided to prepare questions on change of state, which is related to daily life and has been taught since the first stage of education, to assess the capability of teachers. then, teachers were asked questions about melting, one of the changes of state phenomena, to determine their capability of making scientific explanations. for this purpose, the teachers were asked two open-ended questions to collect data. in preparing open-ended questions, turkey's high school chemistry curriculum and contents were considered. to determine whether openended questions are suitable for teachers, the opinions of two associate professors and professors, who are science educators, were consulted. a study by karaaslan (2014) was used to prepare the questions. it took approximately 6-10 minutes for the teachers to answer the questions. the questions are given below. 1. what do you think “scientific explanation” means? can you explain? 2. uğur saw his mother melt a piece of butter in the pan while cooking. uğur is curious about knowing how this change in the butter occurs. could you please scientifically explain to uğur how this phenomenon occurs? 2.3. data analysis the first question was analyzed to determine teachers’ beliefs about scientific explanations, and the second was analyzed to identify their capability to make scientific explanations. when analyzing the first question, the statements emphasizing the cause-and-effect relationship in teachers' explanations were regarded as the expected explanation. on the other hand, when analyzing the second question, the teachers were expected to almost make a story out of the events within the frame of cause & effect relationship, and such explanations were regarded as the accepted scientific explanation (braaten & windschitl, 2011; saglam, karaaslan & ayas, 2014). in the analysis of the first question, the deductive analysis method (patton, 2002) was used. this study compiled and used the codes and categories created by saglam, karaaslan & ayas (2014). from this point of view, teachers' answers were read a few times, and similar statements were gathered under the same code. then, these codes were gathered under the defined categories (miles, huberman & saldaña, 2018). after the codes and categories were creat ed, the definitions of each code (table 2) were created using the teachers' explanations (saglam, karaaslan & ayas, 2014). the code definitions are given in table 2. the analysis determined two categories and seven codes, as seen in table 2. to establish the reliability of the codes, the definition table and the explanations of 20 random teachers were given to two chemistry academicians. they were asked to code the teachers' explanations according to the definition table. at the end of the coding by the chemistry academicians, the reliability coefficient was calculated using the formula by miles, huberman & saldaña, (2018). reliability coefficient = number of agreements total of agreements and disagreements 𝑥 100% as a result of the calculations, the reliability coefficient was determined as 87%. therefore, a consistency percentage of 80% between the investigators’ and coders’ assessments is regarded as reliable (cicchetti, 1994; miles, huberman & saldaña, 2018). table 1. demographics of the study sample experience (years) gender 1-5 6-10 11-15 16-20 21 years and more f % female 6 ft2, ft10, ft11, ft17, ft20, ft24 3 ft7, ft19, ft21 2 ft22, ft23 7 ft1, ft3, ft9, ft12, ft13, ft15, ft18 6 ft4, ft5, ft6, ft8, ft14, ft16 24 52,2 male 2 mt17, mt18 2 mt6, mt12 5 mt1, mt7, mt8, mt9 mt20 5 mt10, mt13, mt16, mt19, mt21 8 mt2, mt3, mt4, mt5, mt11, mt14, mt15, mt22 22 47,8 f 8 5 7 12 14 46 % 17,4 10,9 15,2 26,1 30,4 journal of science learning article doi: 10.17509/jsl.v5i3.43141 543 j.sci.learn.2022.5(3).540-549 in the second question of the study, to determine the teachers’ capability of making scientific explanations, the analysis was made using the rubric prepared by saglam, karaaslan & ayas (2014). accordingly, the rubric, which was prepared by taking the expert opinions of two science academicians, was given in table 3. the rubric and the explanations of 20 teachers were given to two chemistry academicians, who were asked to grade the teachers' explanations to establish the grading reliability. for grading, the reliability was calculated by calculating the kappa coefficient within the spss package. if the kappa coefficient is between 0.81 and 1.00, then there is a perfect consistency (landis & koch, 1997). the fact that the calculated kappa coefficient is 0.93 shows that there is an ideal consistency. 3. result 3.1. teachers’ beliefs about scientific explanation to determine the teachers’ beliefs about scientific explanation, they were asked ‘what do you think “scientific explanation” means? can you explain?’ in the definition table, the codes indicating each teacher's explanation and examples of teachers’ explanations for the codes are presented in table 4. looking at table 4, it is seen that most of the explanations of the teachers are included in the evidencebased code. for example, scientific explanation is the “explanations based on scientific data, observations, and experiments,” according to ft3. thus, the explanation of this teacher was included in the evidence-based code. again, as mt12 perceived scientific explanation as “explaining an existing phenomenon based on scientific facts”, the explanation of this teacher was included in the same code. table 2. table of definitions of teachers’ beliefs about scientific explanation 1 codes of scientific knowledge category (teachers’ explanation) definations of code’s (teachers’ explanation) a) evidence based this code includes following expression explanations based on experiment or/and observations; should be done in labs; based on data/proof/reasons; explaining with examples; proved information; b) knowladge level this code includes following expression listeners’ suffcient knowledge level; explaining according to level of listeners’; presenter should have sufficient level of knowledge; 2 codes of explanation category (teachers’ explanation) definations of code’s (teachers’ explanation) a) authorized this code includes following expression explaining by using specific terms/expressions/cocepts; explaining with information based on literature; concepts are explained; explaining terminologically b) scientific explanation this code includes following expression explaining the phenomenons/problems in reason-result relationship; explaining by expressing the reasons and whys c) reasonable this code includes following expression explaining the phenomenons reasonably; explaining persuasively d) intelligible this code includes following expression; using clear and understandable expressinos; explaining objectively e) other this code includes following expression explaining systematically table 3. rubric for determining the teachers’ capability of making scientific explanations points not explanation (0 point) inadequate explanation (1 point) scientific explanation (2 point) properties of explanation a description made without providing any theoretical component or an incorrect explanation made or related premises presented tangentially without forming a story. explaining made from the beginning and causative links made to the subsequent premises, but at least one premise is missing in the story. -explaining made from the beginning and causative links made to the subsequent premises and the premises presented within a complete story. journal of science learning article doi: 10.17509/jsl.v5i3.43141 544 j.sci.learn.2022.5(3).540-549 the teachers' explanations of the cause & effect relationship were included in the scientific explanation code. looking at the table above, it is observed that eight of the explanations of the teachers were included in the scientific explanation code. for example, it is seen in table 4 that explanations such as “it means explaining the events and phenomena that occur in nature within the scope of cause & effect relationship suggested by science. (mt8)”, “it means explaining the causes of the events around us with their reasons (ft23)” point to the cause & effect relationship. figure 1 shows how many times each code is repeated. looking at figure 1, it is seen that there are 69 codes in total. the reason for this is the fact that some teachers’ explanations are included in more than one code. according to the figure, 53% (37 statements) of all the codes (69 codes) are included in the evidence-based code, 7% (5 statements) are included in the knowledge level code, 12% (8 statements) are included in the authorized code, 12% (8 statements) are included in the scientific explanation code, 6% (4 statements) are included in the reasonable code, 9% (6 statements) are included in the intelligible code, and 1% (1 statement) are included in the other code. according to this information, it is seen that only 12% of all the codes have statements to be included in the scientific explanation code. viewing the explanations of the teachers (table 4), which were included in the scientific explanation code, based on the demographic properties (table 1), it is seen that, of female teachers, ft1 has 16-20 years of experience, ft14 has more than 21 years of experience, ft21 has 6-10 years of experience, and ft23 has 11-15 years of experience; of male teachers, mt1 and mt8 have 11-15 years of experience, mt10 has 16-20 years of experience, and mt17 has 1-5 years of experience 3.2. teachers’ level of scientific explanations to determine the teachers’ level of scientific explanations, they were asked the following question; “uğur saw his mother melt a piece of butter on the pan while cooking. uğur is curious about know how this change in the butter occurs. could you please scientifically explain to uğur how this phenomenon occurs?” the scientific explanation expected from the teachers for this question must be; ✓ the heat flows from the pan, the temperature of which is higher, to the butter, the temperature of which is lower. ✓ as the kinetic energy of the particles of the heated butter increases, they start to move faster. ✓ fast-moving particles get away from one another, and the distance between the particles starts to increase. ✓ in this way, the butter in the solid state melts, changing into a liquid state. or a similar explanation (karaaslan, 2014). according to the rubric, the level of the teacher's explanations and the example explanations of the teachers for each level are given in table 5. figure 2 shows the total number of teachers included in each grading type based on the teachers' explanations. looking at figure 2, it is seen that 25 teachers (55%) were given 0 points as their explanations were not regarded as scientific, 14 teachers (30%) were given 1 point as their explanations were scientifically inadequate, and seven teachers (15%) were given 2 points as their explanations were regarded as scientific. reviewing the teachers with scientific explanations (table 5) based on their demographics (table 1), it is seen figure 1. teachers’ beliefs about scientific explanation 37 5 8 8 4 6 1 0 10 20 30 40 evidence based knowladge level authorized scientific explanation reasonable intelligible other n u m b e r o f c o d e s codes figure 2. distribution of the explanations of the teachers based on rubric 25 14 7 0 5 10 15 20 25 30 0 point 1 point 2 point n u m b e r o f t e a c h e rs points journal of science learning article doi: 10.17509/jsl.v5i3.43141 545 j.sci.learn.2022.5(3).540-549 that, of the female teachers, ft16 has more than 21 years of experience, ft17 and ft24 have 1-5 years of experience, and ft18 has 16-20 years of experience; and of the male teachers, mt8 has 11-15 years of experience, and mt16 and mt21 have 16-20 years of experience. considering the demographic properties of the teachers (table 1), it is seen that eight male teachers with more than 21 years of experience did not make any explanations e ligible for being included in the scientific explanation code (table 4) and could not make any complete scientific explanations (table 5). 4. discussion the first question of the study aims to determine the teachers’ beliefs about scientific explanations. viewing the results obtained from the analysis of this question, it was concluded that most teachers perceived scientific explanations as explanations based on scientific experiments or evidence. f or example, it was determined that ft3 perceived scientific explanation as “explanations based on scientific data, observations, and experiments”, and likewise, mt2 perceived scientific explanation as “explanations based on specific foundations and facts”. furthermore, in a study on candidate form teachers by saglam, karaaslan & ayas (2014), it was determined that the majority of the candidate teachers perceived scientific explanations as explanations based on specific evidence, data, experiments, and observations. considering the studies in literature, it is stated that subjects or events especially related to daily life must be explained by emphasizing cause & effect relationships (braaten & windschitl, 2011). however, in the study, it is table 4. examples of teachers’ explanations 1codes of scientific knowledge category examples a) evidence based ft1, ft3, ft4, ft5, ft6, ft7, ft8, ft9, ft10, ft11, ft12, ft13, ft14, ft15, ft17, ft18, ft19, ft21, ft22, ft24, mt1, mt2, mt3, mt4, mt5, mt6, mt7, mt9, mt11, mt12, mt13, mt14, mt16, mt18, mt19, mt20, mt22 explaining based datas, observations and experiments…(ft3) …are explanations based on experienced informations…(ft5) …explanations with some examples in science. explanations based on evidences…(ft11) …explaining based on certain reasons…(mt2) …explaining a phenomenon based on reasons…(mt12) b) knowladge level ft2, ft3, mt1, et2, mt11 …while making scientific explanation, the level of knowledge should be sufficient (ft2) …listeners’ knowledge level should be sufficient, too…(ft3) 2codes of explanation category examples a) authorized ft16, ft20, ft22, mt1, mt6, mt14, mt20,mt21 explaining process by using the expressions which science uses to explain phenomenon happening in nature. (ft16) …explaining by using appropriate concepts and terms…(mt1) …explanations based on information in literature…(mt6) …terminologically explaining procces…(mt14) first, cocept are defined…(mt20) b) scientific explanation ft1, ft14, ft21, ft23, mt1, mt8, mt10, mt17 explaining the reasons of any phenomenon (ft14) …explanations that totally based on reason – result relationship (ft21) explaining the incidents around us with their reasons. (ft23) explaining the incidents and phenomena happinig in the nature in scientific reason – result relationship (mt8) c) reasonable ft1, ft4, ft14, ft16 ...persuasive, satisfactory explanations…(ft1) explaining the incidents happinig in the nature in a reasonable way (ft4) d) intelligible ft7, ft14, ft16, ft21, mt15, mt16 …explaining without any personal comment…(ft7) …explanations that are different from daily speech. (ft21) simple explanations are important…(mt16) it is a kind of explanation that your listener can understand (mt15) e) other mt21 rule and laws have been explored. when we explain an incident scientifically, we express this rule and law. (mt21) journal of science learning article doi: 10.17509/jsl.v5i3.43141 546 j.sci.learn.2022.5(3).540-549 seen in table 4 that only eight teachers emphasize cause & effect relationships when making scientific explanations. thus, it is possible to say that the teachers’ beliefs about scientific explanations are inadequate. therefore, it is thought that the fact that the studies made, especially on the national level, regarding what scientific explanation is and how it should be made are limited, and that the exams for transition to secondary education and higher education are made in the form of tests, and that teachers teach lessons aimed at these exams (demir & demir, 2012) affects their insufficient understanding of scientific explanation. the second question of the study aims to determine the teachers’ level of scientific explanations. in the explanations, it was observed that some teachers mostly made definitions when scientifically explaining the melting phenomenon. for example, the explanation of ft22, “matter changes into the liquid state from the solid state when heated.” is regarded as the definition of the melting phenomenon. again, the explanation of mt15 is as “solid matter increases its temperature and changes its state when it gets heat from outside. in other words, it changes into a liquid state from a solid state when it reaches a definite temperature. that is why the butter changes into a liquid state from a solid state.” can be regarded as a definition. the terms description and explanation must not be confused because description and explanation are two different terms (horwood, 1988). the difference between these two terms can be exemplified using the freezing phenomenon. it is possible to state that saying liquid matter changes into the solid matter by losing heat (emphasizing the observed qualities) is making a definition while explaining the freezing phenomenon by expressing various theories and laws, such as gravitation between particles and particle movements of liquid matter losing heat within the frame of cause & effect relationship is making a scientific explanation. again, considering the results of the second question of the study, it was determined that some teachers made incorrect explanations. for example, ft5 said, “the butter rapidly gives electrons to the pan due to the heat. and it activates and melts the electrons inside the butter”, explaining the melting phenomenon as giving electrons. she was given 0 points due to her non-scientific explanation. therefore, it was concluded that the teachers were incapable of making scientific explanations regarding the change of state related to daily life. reviewing the conducted studies, it is seen that teacher candidate or teachers make incorrect or inadequate explanations in chemistry subjects related to everyday life (alameh, elkhalick & brown, 2022; boyraz, hacıoğlu & aygün, 2016; saglam, karaaslan & ayas, 2014; uluçınar sağır, tekin & karamustafaoğlu, 2012; yadigaroglu & demircioglu, 2012). most of the studies in the literature point to incorrect explanations or misconceptions of events rather than their scientific explanations (yakmaci-güzel, 2013; yavuz & büyükekşi, 2016) it is seen in table 5 that some teachers were given 1 point as they had missing parts in their explanations to the table 5. teachers’ capability level of making scientific explanations and example explanations points not explanation (0 point) inadequate explanation (1 point) scientific explanation (2 point) properties of explanations fm2,ft3, ft4, ft5, ft6, ft7, ft8, ft10, ft12, ft13, ft14, ft15, ft19, ft20, ft22, ft23, mt2, mt3, mt4, mt7, mt10, mt15, mt17, mt18, mt20 ft1, ft9t, ft11, ft21, mt1, mt5, mt6, mt9, mt11, mt12, mt13, mt14, mt19, mt22 ft16, ft17, ft18, ft24, mt8, mt16, mt21 teacher’s examples first of all, butter changes from solid to liquid. at the same time there is chemical changes in the form of the butter. since unsaturated fat rate falls, moleculer structure have been spoiled. (ft3) thanks to heat, it gives the container of the butter electrons fast. heat melts the butter by triggering the electrons that butter contains. (ft5) butter melts and the water in it evaporate. (ft8) matter turns from solid to liquid by getting heat. (ft22) because of heat, butter starts to melt. (mt18). solid matter changes into liquid when they get heat. when we put the butter in a pan and heat it, interaction among moleculs decreases and the matter changes its form. (ft21) when the solid matters get heat, the gravitational force decreases and the form changes. (mt6) butter is a solid matter, and when the solid matter gets heat, the gaps among molecules increases. (mt12) the energy of the molecules in butter increases. hance the molecules get away from each other, and the butter turns into liquid. (ft17) between the substances which has different temperatures, there is a heat enrgy flow from the hotter one to the colder one. the heat energy increases the speed of particles of matters, so the gravitational force among the particles of the matter decreases. the paricles whose the gravitational force decreases get away from each others and becomes liquid then gas. (mt8). journal of science learning article doi: 10.17509/jsl.v5i3.43141 547 j.sci.learn.2022.5(3).540-549 second question of the study. for example, as mt6 did not point to the kinetic energy and particle movements in his explanation, “when the solid matter is heated, gravitation between molecules decreases and the change of state occurs”, he was given 1 point. on the other hand, seven teachers' explanations were regarded as complete scientific explanations, so they were given 2 points. for example, as mt8 said, “between the objects with different temperatures, energy called thermal energy is transferred from the hot object to the cold object, and the thermal energy increases the movement speed of the particles which form the matter; therefore, gravitation between the particles of the heated matter is weakened. the particles with weak gravitation move away from one another to change into a liquid state, and then a gaseous state.”; it is possible to say that he made a complete scientific explanation. because mt8 explained the phenomenon within the frame of the cause & effect relationship, it is considered that teachers must adequately present the differences between the definition and scientific explanation. it can be suggested that teachers mostly make definitions when making scientific explanations because they lack knowledge of how to make a proper explanation (karaaslan, 2014). the inability to adequately associate scientific concepts and explanations when explaining a phenomenon makes it difficult for teachers to construct scientific explanations (polat, 2022). moreover, it is considered that teachers teaching a lesson based on rote learning rather than explaining an event or phenomenon in detail (nakiboğlu, 2009) may prevent them from making scientific explanations. for teachers or candidate teachers to make adequate scientific explanations in chemistry subjects related to daily life, they first should know what a scientific explanation is and how it must be made because an effective scientific basis makes it easier for the students to learn the concepts and enables them to understand these concepts thoroughly (coleman, 1998). moreover, the literature emphasizes that providing students with subjects related to daily life increases their motivation (i̇lhan, yıldırım & sadi-yılmaz, 2016; kutu & sözbilir, 2011). therefore, it is assumed that addressing chemistry subjects related to daily life following scientific explanation rules may support their success in chemistry lessons. it is proposed that teachers’ detailed explanations, including causal relationships and encouraging students in this respect, may increase students' success in international exams such as pisa and timss. because, it was observed that the success level of our students was relatively low, especially in the questions requiring explanation in the said exams (meb, 2016). similarly, the studies conducted in our country showed that the students were incapable of making scientific explanations; in other words, their explanations for the open-ended questions were short and inadequate answers that did not contain any causal relationships (akgün, tokur & duruk, 2016; saglam, karaaslan & ayas, 2014; yadigaroglu & demircioglu, 2012). considering the relationship between the teachers’ opinions of scientific explanation and their years of experience, out of 14 most experienced (21 years and above) teachers, only ft14 pointed to the cause & effect relationship. therefore, the explanation of ft14, “explaining the causes of any event,” was included in the scientific explanation code. thus, it is possible to say that there is not any positive relationship between the teachers’ years of experience and their beliefs about scientific explanations. again, looking at the relationship between the teachers’ capability of making scientific explanations and their years of experience, it was determined that, of the most experienced teachers, only ft16 made a complete scientific explanation and that the other teachers could not make a thorough scientific explanation. from this point of view, as the explanation of fm16, “…there is little space between fat molecules in the solid state. molecules apply gravitational force on one another, and the matter exists in a solid state. when the same matter receives thermal energy from outside, the kinetic energy of the molecules increases and the bond that maintains the gravitation between the molecules is broken, so the movement of the molecules increases. therefore, molecules move away from one another, and the matter begins to change into a liquid state…” this was a complete scientific explanation. therefore, she was given 2 points. from this point of view, it may be concluded that there is no positive relationship between the teachers’ capability to make scientific explanations and their years of experience. when both international and national studies were reviewed, no study determined the teachers’ beliefs about scientific explanations. however, international literature decided that there was just one study on candidate teachers (saglam, karaaslan & ayas, 2014). thus, it is thought that it would be useful to research to determine chemistry and other science teachers’ beliefs about this notion and to make up for their deficiencies. 5. conclusion as mentioned above, various studies demonstrated that students could not make scientific explanations. in this study, it was concluded that teachers' competence levels in this subject were unsatisfactory. from this perspective, it is considered that giving teachers some courses or training regarding scientific explanation would be useful both for themselves and the students. references akgün, a., tokur, f., & duruk, ü. (2016). associating conceptions in science teaching with daily life: water chemistry and water treatment. adıyaman university journal of educational sciences, 6(1), 161-178. alameh, s., el-khalick, f.a., & brown, d. (2022). the nature of scientific explanation: examining the perceptions of the nature, quality, and “goodness” of explanation among college students, https://scholar.google.com/scholar?oi=bibs&cluster=12966575049075695033&btni=1&hl=tr https://scholar.google.com/scholar?oi=bibs&cluster=12966575049075695033&btni=1&hl=tr https://scholar.google.com/scholar?oi=bibs&cluster=12966575049075695033&btni=1&hl=tr journal of science learning article doi: 10.17509/jsl.v5i3.43141 548 j.sci.learn.2022.5(3).540-549 science teachers, and scientists. journal of research in science teaching [jrst], 1-36. https://doi.org/10.1002/tea.21792 american association for the advancement of science [aaas]. (2009). benchmarks for science literacy. washington, dc: author. boyraz, d.s., hacıoğlu, y., & aygün, m., (2016). argumentation and concepts confusion: melting and dissolving. gazi university journal of gazi educational faculty (gujgef). 36 (2), 233-267. braaten, m., & windschitl, m. (2011). working toward a stronger conceptualization of scientific explanation for science education. science education, 95(4), 639-669. cajas, f. (1999). public understanding of science: using technology to enhance school science in everyday life. international journal of science education, 21(7), 765-773. çepni, s. (2007). arastirma ve proje calismalarina giris. trabzon: celepler publications, 22-28. christensen, l. b., johnson, b. & turner, l. a. (2011). research methods, design, and analysis. boston: pearson. cicchetti, d. v. (1994). guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. psychological assessment, 6:284-290. coleman, e. b. (1998). using explanatory knowledge during collaborative problem-solving in science. journal of the learning sciences, 7 (3/4), 387 – 427. demir, s., & demir, a. (2012). new high school instructional programs in turkey: problems, expectations, and suggestions. elementary education online, 11 (1), 35-50. demirbaş, m., & pektaş , h. m. (2009). elementary students’ levels of realization of basic concepts related with environment problem. necatibey faculty of education electronic journal of science and mathematics education, 3(2), 195-211. dindar, a. c., bektaş, o., & çelik, a. y. (2010). what are the pre-service chemistry teachers’ explanations on chemistry topics?. the international journal of research in teacher education, 1(3), 32-41. friedman, m. (1974). explanation and scientific understanding. journal of philosophy, 71, 5-19. gilmanshina, s. i., gilmanshin, i. r., sagitova, r. n., & galeeva, a. i. (2016). the feature of scientific explanation in the teaching of chemistry in the environment of new information of school students' developmental education. international journal of environmental and science education, 11(4), 349-358. grunberg, t., & grunberg, d. (2011). bilim felsefesi [philosophy of science]. i̇. taşdelen (ed.), bilimsel açıklama (scientific explanation) (p. 5284). 1. press, eskişehir, turkey: anadolu university press. hempel, c. g. (1965). aspects of scientific explanation (vol. 1). new york: free press. herman, b. c., owens, d. c., oertli, r. t., zangori, l. a., & newton, m. h. (2019). exploring the complexity of students’ scientific explanations and associated nature of science views within a placebased socioscientific issue context. science & education, 28(3), 329366. horwood, r. h. (1988). explanation and description in science teaching. science education, 72(1), 41-49. i̇lhan, n., yıldırım, a. & sadi-yılmaz, s. (2016). the effect of contextbased chemistry course on high school students’ learning about chemical equilibrium, motivation to learn chemistry and constructivist learning environment. international journal of environmental and science education, 11(9), 3117-3137. doi: 10.12973/ijese.2016.919a karaaslan, e. h. (2014). sınıf öğretmeni adaylarının genel kimyadaki bilimsel kavramları açıklama becerilerinin geliştirilmesi [an investigation on enhancing elementary school teachers candidates’ capability of explaining chemical concepts]. (unpublished doctoral dissertation). karadeniz teknik üniverstesi, eğitim bilimleri enstitüsü, trabzon. kitcher, s (1989.) explanatory unification and the causal structure of the world. p. kitcher and w.c. salmon (eds.), minnesota studies in the philosophy of science; vol, xiii, scientific explanation (410-499), minneapolis: university of minnesota press. köseoğlu, f., tümay, h., & budak, e. (2008). paradigm changes about nature of science and new teaching approaches. journal of gazi education faculty, 28(2), 221-237. kutu, h., & sözbilir, m. (2011). teaching chemistry in our lives unit in the 9th grade chemistry course through context-based arcs instructional model. ondokuz mayıs university education faculty journal, 3(1), 29-62. landis, j. r., & koch, g. g. (1977). the measurement of observer agreement for categorical data. biometrics, 159-174. lombrozo, t. (2006). the structure and function of explanations. trends in cognitive sciences, 10(10), 464-470. mcneill, k. l., & krajcik, j. (2008). inquiry and scientific explanations: helping students use evidence and reasoning. science as inquiry in the secondary setting, 121, 34. miles, m. b., huberman, a. m., & saldaña, j. (2018). qualitative data analysis: a methods sourcebook. sage publications. milli eğitim bakanlığı [meb]. (2013). ortaöğretim (9, 10, 11 ve 12. sınıflar ) kimya dersi öğretim programı [secondary education (9, 10, 11 and 12th grades) chemistry course curriculum], ankara, turkey. milli eğitim bakanlığı [meb]. (2016). pisa 2015 ulusal raporu [pisa 2015 national report.]. ankara, turkey. nakiboğlu, c. (2009). examination on expert chemistry teachers’ secondary school chemistry textbook usage. journal of ahi evran university kırşehir faculty of education, 10(1), 1-10. national research council [nrc]. (1996). national science education standards. washington, dc: the national academies press. patton, m. q. (1987). how to use qualitative methods in evaluation (no. 4). sage. patton, m.q.(2002).qualitative research & evaluation methods (3th ed.). thousand oak: sage publications. polat, m. (2022). examination of evaluation scores of science teacher candidates in relation to new generation question as per determined variables. nevşehir hacı bektaş veli üniversity journal of iss, 12(1), 355-380. rocksén, m. (2016). the many roles of “explanation” in science education: a case study. cultural studies of science education, 11(4), 837-868. saglam, y., karaaslan, e. h., & ayas, a. (2014). creating a taken-asshared understanding for scientific explanation: classroom norm perspective. international journal of education in mathematics, science and technology, 2(2). salmon, w. c. (1984). scientific explanation and the causal structure of the world. princeton university press. trigwell, k. (2006). phenomenography: an approach to research into geography education. journal of geography in higher education, 30(2), 367-372. uluçınar sağır, ş., tekin, s., & karamustafaoğlu, s. (2012). the levels of prospective elementary school teachers’ understanding of some chemistry concepts. dicle university ziya gökalp education faculty journal, 19, 112-135. yadigaroglu, m., & demircioglu, g. (2012). the level of chemistry student teachers of relating their chemistry knowledge to events in daily life. journal of research in education and teaching, 1(2), 165-171. yakmaci-güzel, b. (2013). sınıf öğrencilerinin bazı temalardaki kimya kavram yanılgılarının belirlenmesi ve bu bulguların etkili kullanımına dair öneriler [determining chemistry misconceptions of primary school students in some themes and suggestions for effective use of these findings]. boğaziçi üniversitesi eğitim dergisi, 31(2), 5-26. yavuz, s., & büyükekşi, c. (2016). günlük yaşamdaki kimya kavram yanılgılarının atasözleri ile tespit edilmesi [identifying chemistry misconceptions in daily life with proverbs]. karaelmas fen ve mühendislik dergisi, 6(1), 182-186. yıldırım, a., & şimşek, v. h. (2013). sosyal bilimlerde nitel araştırma yöntemleri [qualitative research in social science]. ankara: seckin publishing yıldırım, n., & birinci konur, k. (2014). fen bilgisi öğretmen adaylarının kimya kavramlarını günlük hayatla https://scholar.google.com/scholar?oi=bibs&cluster=12966575049075695033&btni=1&hl=tr https://doi.org/10.1002/tea.21792 journal of science learning article doi: 10.17509/jsl.v5i3.43141 549 j.sci.learn.2022.5(3).540-549 ilişkilendirebilmelerine yönelik gelişimsel bir araştırma [a developmental research for science teachers candidates’ associating chemistry concepts with everyday life]. the journal of academic social science studies, 30, 305-323. yilmaz, h., & çavas, p. h. (2006). the effect of the 4-e learning cycle method on students' understanding of electricity. journal of turkish science education, 3(1), 2. a © 2022 indonesian society for science educator 488 j.sci.learn.2022.5(3).488-499 received: 23 august 2021 revised: 29 mei 2022 published: 27 november 2022 the development of foodivity interactive as an interactive multimedia to improve students’ understanding of food nutrition topic sulistinayah suwaka putri1*, ari widodo1, yaya wihardi2 1department of science education, faculty of mathematics and science education, universitas pendidikan indonesia, bandung 40154, indonesia 2department of computer science education, faculty of mathematics and science education, universitas pendidikan indonesia, bandung 40154, indonesia *corresponding author: sulistinayahsp@upi.edu abstract the pandemic of covid-19 has affected the education sector. all school access is restricted, transforming education into distance learning. this encourages educators to explore alternative methods for improving students' understanding. inter active multimedia is a method of learning media that can promote efficiency, motivation, and student understanding of subjects. this research aims to measure the students' understanding of nutrition by developing festivity interactive as interactive multimedia. a preexperimental one-group pretest-post-test design was adopted. developing multimedia using the instructional design process addie (analysis, design, develop, implement, and evaluate). the research subject was 53 junior high school students in west java, more precisely in two locations, sukabumi and kuningan. the instrument consisted of an expert judgment rubric, a student questionnaire, and a multiple-choice objective test with pretest and post-test. the v aiken index shows an average validation of 0.8125, indicating that the media is valid and can be used. students responded positively to the questionnaire by a percentage of 93.01 percent. based on the analysis, the n-gain value on students' understanding is 0.35. this indicates increased students’ expert judgments in the moderate improvement category between the pre-and post-tests. the hypothesis test establishes that hypothesis h1 is acceptable, showing a significant difference between pre-and post-tests. these results indicate that using foodivity interactive as an interactive multimedia can enable students’ understanding of nutrition topics. keywords foodivity interactive, multimedia interactive, students understanding, food nutrition topic 1. introduction due to the covid-19 pandemic, many schools and colleges are closed. according to unesco data, nearly 160 countries have implemented national closures, affecting more than half of the world's student population (abidah, hidaayatullaah, simamora, fehabutar & mutakinati, 2020). with the restrictions on interaction, the indonesian ministry of education also issued a policy of closing schools. during an outbreak, most students go to school by distance learning. technology is used to deliver all subjects to students. educators must explore and apply various theories, approaches, and learning design principles to create an innovative learning environment for their students ( bhavya, gautam & sumedha 2021). in line with the rapid development of science and technology in the early 21st century, which demands an increase in the quality of superior human resources. the learning model uses media such as smartphones, desktop pcs, laptops, or other devices connected to the internet. on the other hand, consumers prefer smartphones over other devices because they are easier to carry and offer lower prices than others (sahlström, tanner & valasmo, 2019). educators are directly or indirectly required to follow these developments by innovating and creatively modifying learning. however, in reality, certain challenges are faced in the online learning system. for example, when doing online learning activities at home, students still lack the desire and encouragement to take the initiative to learn independently (godwin, 2012). therefore, distance learning problems during the covid-19 pandemic must be addressed immediately to maximize learning outcomes and significantly impact student behavior (churiyah, sholikhan, filianti & sakdiyyah, 2020). journal of science learning article doi: 10.17509/jsl.v5i3.38032 489 j.sci.learn.2022.5(3).488-499 the low level of understanding and student learning outcomes is one of the reasons for updates in learning strategies and methods of delivering material (transfer of knowledge) to students (huryah, sumarmin & effendi, 2017). one of the factors that determine the quality of educational outcomes is the approach used by teachers in the learning process. the accuracy in using the learning approach taken by the teacher not only arouses students' motivation, interest, and learning achievement but also increases students' understanding of the material provided by the teacher. apparently, in this case, many lessons can be learned from the problems and complaints in indonesia, and students and teachers will be better prepared to interact with new technologies so that online learning can be improved (putri et al., 2020). in this era of increasingly sophisticated technological disruption, students and teachers are expected to be equally skilled in learning technology. students and instructors have various kinds of skills in using learning technology. a good lesson is a lesson the learner has learned, and the multimedia material presented is a method of obtaining answers to the questions posed. it is essential to enhance enjoyment, creativity, willingness, and motivation in the learning process by utilizing digital media (zheng, li & chen, 2018). so, one of the efforts to improve the understanding of science material, especially in biology, is to use interactive multimedia. interactive multimedia will present nutrition topics in junior high schools related to health issues. learning by utilizing interactive multimedia in learning, especially science learning, is expected to be an alternative to improve student learning outcomes. unfortunately, nutrition material in schools rarely gets the attention it deserves. because of national curriculum, the chapter focuses more on the digestive system (roswati, rustaman & nugraha, 2019). while nutrition is an essential component of human existence, it is also necessary for health and well-being during various stages of life. to ensure that students are healthy, they should first have an essential awareness of nutrition and be given a balanced diet with a basic understanding. this conception is essential for proper development into adulthood and the later stages of life (perera, frei, frei, wong & bobe, 2015). interactive multimedia can be further developed to transfer knowledge to students effectively, focuses students' attention during the learning process, concretizes information, and is an endless learning tool. therefore, this study aims to develop interactive multimedia, festivity as a learning aid to increase students' understanding of food nutrition in junior high schools. 2. method the method used in this research is pre-experimental. the pre-experimental method has no control or comparison groups (dawson, 2007). pre-exhowever, preexperiments a cos this research has used pretest post-test designs. the research design utilized in this study is one group pretest and post-test, which is a research design that includes a pretest before treatment and a post-test after treatment. the assessment method in this research is to develop interactive multimedia. the development model used to develop interactive multimedia products is the addie model. addie model is an instructional design process. this model has five stages: analyze, design, develop, implement and evaluate. this design requires knowing the students’ understanding or testing hypotheses about the presence or absence of the effect of the action after implementing the application. after material experts and multimedia experts validate the multimedia, the multimedia is ready to be implemented into the learning process. it is expected that the learning outcomes obtained by students increase after using interactive multimedia, and multimedia can be said to be effective. then it will distribute the table 1 the indicator and aspect for expert judgement indicator aspect content quality accuracy, balanced presentation of ideas, an appropriate level of detail, and reusability in contexts learning goal alignment alignment among learning goals, activities, assessments, and learner characteristics. feedback and adaptation adaptive content or feedback is driven by differential learner input or learner modeling. motivation ability to motivate and interest an identified population of learners. presentation design design of visual and auditory information for enhanced learning and efficient mental processing. interaction usability ease of navigation, predictability of the user interface, and quality of the interface help features. accessibility design of controls and presentation formats to accommodate disabled and mobile learners. standards compliance adherence to international standards and operability on commonly used technical platforms. (source: nesbit, giel, rose & kiley, 2009) journal of science learning article doi: 10.17509/jsl.v5i3.38032 490 j.sci.learn.2022.5(3).488-499 question pretest and post-test, questionnaires, and expert reviews regarding multimedia content. this research aims to obtain data using the instrument to measure the appropriateness of interactive media and how their designs will affect the student’s understanding of food nutrition topics. based on table 1, the instruments will be used on the rubric for expert judgment. the rubric was adopted from lori (learning object review instrument). expert judgment will assess indicators based on predetermined aspects. these aspects have represented the achievement of the media. media can be feasible if it has been through a judicial process. some questionnaires are another way to collect interest data, and they should be viewed as one of several options relevant to a given situation (sinclair, 1975). so that the respondent may create, articulate, and relay the responses effectively, questions must be posted in a clear and understandable. the questionnaire used a likert scale of 1 up to 5 for students, shown in table 2. the questionnaire aims to see students' impression of using interactive foodivity as multimedia. so, the more students agree with the statement, the higher the score. the research instrument in this study is a test item question in the form of an objective test in the form of multiple choice. this research instrument for test item questions in the form of an objective test used the form of multiple-choice with a total of 20 queries. the question is only used to table 2 the category and statement of student questionnaire category statement mobile interface the interactive media design "foodivity interactive" used is attractive. the shape, model, and size font of the media used are simple and easy to read. the use of interactive media, “foodivity interactive” is very easy. material content the video on the interactive media "foodivity interactive" supports understanding the nutritional material. animations in this interactive media are easy-to-understand nutrition material. games in this interactive media help to understand nutrition material. the material contained in this interactive media is related to everyday life. improve understanding the material presented in interactive media is easy to understand. presentation of material in this media helps in answering questions motivation the existence of interactive media can motivate to learn nutrition material. table 3 blue print of cognitive test items concept cognitive process dimension and number of test item total c1 c2 c3 c4 c5 nutrition 1 2 14 17 4 diet balance 7 16 9 3 carbohydrate 20 8 2 fat 5 10 13 3 protein 4 19 2 vitamin 3 11 18 3 food testing 6 12 15 3 total 2 5 6 4 3 20 table 4 validity result question number validity score validity criteria decision 1 0.350 low directly used 2 0.566 enough directly used 3 0.471 enough directly used 4 0.545 enough directly used 5 0.500 enough directly used 6 0.391 low directly used 7 0.332 low directly used 8 0.367 low directly used 9 0.373 low directly used 10 0.363 low directly used 11 0.315 low directly used 12 0.308 low directly used 13 0.305 low directly used 14 0.370 low directly used 15 0.463 enough directly used 16 0,328 low directly used 17 0.428 enough directly used 18 0.336 low directly used 19 0.357 low directly used 20 0.313 low directly used table 5 reliability statistic result cronbach's alpha n of items 0.70 20 journal of science learning article doi: 10.17509/jsl.v5i3.38032 491 j.sci.learn.2022.5(3).488-499 measure the cognitive domain using bloom's taxonomy measurement indicators c1 to c5. based on table 3, the test items are grouped into several concepts according to the cognitive level adopted by the revised bloom taxonomy. the idea includes nutrition, diet balance, carbohydrates, fat, protein, vitamins, and food testing. a trial was conducted on students who had learned about food nutrition before the data collection began. instrument testing was conducted to establish the instrument's validity and reliability in retrieving the relevant data. the validity of each item can be determined through the validity interpretation. by looking at the correlation coefficient, we can determine the validity category. table 4 states the results of the validity of 20 multiple choice questions, and it turns out that all of them can be said to be valid, and the questions can be used. the instrument reliability test in this study was carried out with the help of the spss version 23 application. the results of the reliability of the test item instrument can be seen in table 5. based on the results of data analysis using spss 23, the instrument was obtained with cronbach's alpha coefficient, and it is known that r count = 0.70, so the instrument is said to be reliable in the high category and can be used for data collection. this instrument can be used several times to measure the same object at different times and produce similar data. 3. result and discussion the development of this research resulted in the application of foodivity interactive as interactive multimedia to increase students' understanding. this product can be accessed via an android-based application and html5. explanation of the results of each stage of development is as follows: 3.1 the development of interactive multimedia the results describe the research findings on data collection and analysis procedures. the development of this application uses the addie as instructional design, which includes analysis, design, development, implementation, and evaluation stages. 3.1.1 analysis stage at this stage, the researcher analyzes the existing interactive media applications implemented for students at school. the analysis is carried out to gather the strengths and weaknesses of each existing application. using instructional media can not only raise students' interest and motivation, but it can also make data presentation more appealing and reliable, help data interpretation, summarize information, and improve students' knowledge (priyanto, 2009). many interactive media models are used for learning purposes, including tutorials, drills, practice, simulations, instructional games, inquiry, and information. there were eight applications regarding interactive multimedia that have been analyzed. most of these applications discuss the topic of the digestive system, and only a few cover the issue of food nutrition. also, most of the interactive media on nutrition topics already exist through information and tutorials. only a few interactive media are available in the form of instructional games and simulations. based on each of the weaknesses and strengths of the analyzed application, it can be seen in general in table 6. the weaknesses of existing applications include application design images that are not clear and not hd images. besides that, the font selection and font size must be adjusted. although the parts of the interactive multimedia layout have a harmonious color composition, clarity, and contrast, interactive multimedia has become an excellent medium. a visually appealing display with clear and bright pictures might help students focus on learning to grasp topics (puspitasari, indriyanti & nugrahaningsih, 2019). this was consistent with the belief that the essential aspect of creating visual design goods was the use of color. maharani & asyhari (2020) said that multimedia presentations increase students' interest in a subject and improve students' comprehension, excitement, and satisfaction. according to bennett & brennan (1996), assessment emphasizes self-evaluation. the media maintains a record of their chosen response, and thorough feedback is provided on each topic following the quiz's completion. the quiz requires students to self-assess and holds them accountable for their learning. this is contrary to the table 6 strength and weaknesses of existing interactive media no weaknesses strengthness 1 the text is too long which makes it difficult for the reader to understand content material about nutrition consists of guidelines for balanced nutrition, energy requirement, and facts about nutrition. 2 quiz questions are still too easy and not very deep the placement of each composition is neat 3 some symbols or images are more petite hd and unclear so that it makes the reader confused and difficult to read there is a glossary about the scientific vocabulary. 4 there are still features that are not running the language is relatively easy to understand 5 there are only one or two hierarchies in the media there are simulations and mini games so that users do not get bored journal of science learning article doi: 10.17509/jsl.v5i3.38032 492 j.sci.learn.2022.5(3).488-499 existing media that has already been used. thus, if it is just a single quiz will be insufficient to assess students' understanding. besides that, to better understand the phenomenon of interactive multimedia, novianto, degeng & wedi (2018) discovered that interactive multimedia is multimedia equipped with a controller and may be managed by the user, which gives the user the power to determine exactly what subsequent processing steps should be used. there are several weaknesses that researchers must reduce by considering the strengths of each application. according to ahmizar (2008), the more senses are involved in the learning process, and the more successful the learning process will be. explicitly, this idea implies the utilization of many human senses. therefore, researchers can develop better interactive media applications by analyzing their strengths and weaknesses. 3.1.2 design stage the second stage of developing the addie model is the design stage. in making the design, flowcharts and storyboards produce a visual representation of the media to be created. the design is essential before writing code because media takes much time to create information, links, and organize content to create visual appeal and test media in its appearance and features. in designing this interactive foodivity application, the design structure used is a composite, free navigation structure. the advantage of this navigation structure is that an application can provide a better linkage of information. 3.1.3 development this is the design stage of media development after making flowcharts and storyboards. at the foodivity interactive development stage, construct two software is used because various kinds of behavior have been provided, which can be applied to objects in the media that we create and do not need to understand programming languages. however, they still require knowledge of programming algorithms. when developing the media, inserting objects that will be included in the application, adding action, giving button effects, making navigation, adding games, inserting music and video, creating animation, and showing pop up are all part of the application development process. this aligns with munir (2020), who stated that multimedia is utilized with supporting features, such as buttons, to make it easier for students to select which section to study for themselves to encourage students to learn independently. figure 1 splashscreen view figure 2 plugin object journal of science learning article doi: 10.17509/jsl.v5i3.38032 493 j.sci.learn.2022.5(3).488-499 in construct 2, when we first open construct two, we will find the start page. the start page is the main page or page where the page will appear when we open complete 2. on this page, there are commands for creating and opening projects. the next thing to do is change the project name, description, and window size. this is shown in figure 1 on the left of the screen. next, we put the objects that are needed. these objects do not require placement in the layout because they are hidden and can automatically be used later. now all layouts in our project can use mouse and keyboard input. for all of these objects, we will use sprite objects. sprite objects only display textures or images that can be moved, rotated, or resized. for example, we can see the sprite button in figure 2. all objects, such as text, videos, images, etc., can be accessed on that page. furthermore, the event consists of conditions that test whether a criterion is met. if all needs are met, the action event will be executed. once the action is executed, all of the sub-events will be executed, testing other conditions, then running another action, another sub-event, and so on. such a system allows us to create functional or sophisticated capabilities for media and apps. we can see an example of an even sheet from figure 3, which contains conditions and functions so that they can run according to the command. the event sheet is where we create simple code to run the objects we have created in the layout. the material presented is as easy as possible for users to understand so that they can learn independently (wiana, barliana & riyanto, 2018). because interactive multimedia is designed for self-learning, the user maintains complete control throughout its application. interactive multimedia built employing the drill and practice method is meant to teach users new skills or to reinforce their understanding of a concept. this quiz must be done quickly because there is a countdown timer. the quiz has a countdown to test students' understanding of each material. according to sun, hsieh, sun & hsieh (2018), when highly interactive, challenging, and competitive tools are used, the students will be more focused. that way, having a countdown on the quiz can make students more prepared and engaged in each material. the system working on this quiz is a knockout system. if we answer the question incorrectly, the game will be over, and we will repeat it. therefore, it is required that users study the material seriously. additionally, this quiz gives some questions that are often displayed randomly, ensuring that each time it is used, the questions shown will always be unique, or at least in unique combinations. because pujawan's (2018) study indicates that drill and practice is essentially a learning strategy that aims to provide a more concrete learning experience through the provision of exercise questions to assess students' performance skills and understanding as measured by their complete the learning questions provided. quizzes are an essential aspect of mobile figure 3 event sheet view figure 4 export project view journal of science learning article doi: 10.17509/jsl.v5i3.38032 494 j.sci.learn.2022.5(3).488-499 applications since they can assist students in understanding more about the topics. construct two saves every project as a file with the extension.capx. the application will be exported into html 5 and android applications if it is felt that the application has been completed. there are many more android users than ios users among the youth population. the html5 export project lets bring the game to many platforms. when exporting via apk, we have the option to wait for the game to compile while exporting. these programs will be installed next, with other programs that must be installed. it can be seen in figure 4. as the compilation process for the game is finished, the game has been successfully built into an android-based application file. to return to the first, make the second platform html5 by clicking export and selecting html5 as the export destination. wait for the program to finish, then launch it via the web by opening the index format. 3.1.4 implementation stage foodivity interactive as an interactive media is implemented in several schools in west java. the data collection process involved 53 students at 8th-grade junior high school. the method of collecting data online through whatsapp groups is due to pandemic conditions that do not allow face-to-face meetings. this research targets 8thgrade students who have not studied the digestive system chapter and only focus on a nutrition topic. this research aims to implement the foodivity interactive application as an interactive media learning. 3.1.5 evaluation stage at this stage, the researcher evaluates interactive media products. the first evaluation was performed by material and media experts to determine the product's validity. after the evaluation, the researcher revised till the product was declared valid by media and material experts. the next stage is for the researcher to evaluate the confined interactive learning media products in schools. after utilizing the foodivity interactive application, students were invited to complete a student response form. 3.2 the experts’ judgment media three experts and one practitioner validated this application. expert validation is an expert of material contained in science and an expert of media. the practitioner is a science teacher working at a junior high school. this validation is regarding the weakness of the media and learning materials. the data were analyzed and used to revise the material in the learning media to improve the quality of the learning media used in the research. the scale from 1 to 5 has categories and definitions for each indicator. thus, the expert can choose the most suitable point based on media conditions and perspectives. the results of four experts, media background, and teachers were selected to assess based on the indicator criteria as tabled in table 7: based on table 7, the expert recapitulation results can determine whether or not an application is feasible. learning media before being widely used needs to be evaluated and validated first, in terms of material content, educative aspects, and technical aspects of media, so that when used, media meet the requirements of excellent educational media. this is important to note and ensure figure 5 export the android based application table 7 recapitulation of expert judgement indicator judges rating v expert 1 (science) expert 2 (science) expert 3(media) expert 4 (teacher) content quality 4 5 4 4 0.8125 learning goal alignment 5 4 5 4 0.875 feedback and adaptation 4 5 4 4 0.8125 motivation 4 5 4 4 0.8125 presentation and design 4 5 5 3 0.8125 interaction usability 4 5 5 3 0.8125 accessibility 4 5 4 4 0.8125 standard compliance 3 5 3 5 0.75 average v index 0.8125 journal of science learning article doi: 10.17509/jsl.v5i3.38032 495 j.sci.learn.2022.5(3).488-499 that what is conveyed to students is correct and reasonable. the average value given by the expert is 3 to 5 on a scale of 5. the aspects assessed are content quality, learning goal alignment, feedback and adaptation, motivation, presentation and design, interaction usability, accessibility, and standard compliance. according to bolger & wright (1992), expert judgment is an essential input for decision support. if the judgment obtained from the experts is poor in some respects, then the results of any decision support system will also be flawed. ideally, we want to know whether the judgment is correct or valid. therefore, it is necessary to have objective external criteria to assess validity. 3.3 students response data on student responses to interactive media were obtained from the questionnaire results. the scale consists of 5 scales, including strongly disagree, disagree, neutral, agree, and strongly agree. a score of 5 has a strongly agree, while a score of 1 has a strongly disagree meaning. they used these five scales to find out the context of respondents' impressions and judged by students of the response item (joshi, kale, chandel & pal, 2015). these aspects are in the form of the mobile interface, content material, improved understanding, and motivation students give a positive response to the interactive foodivity application. because students who responded strongly agreed reached 70.75%. in addition, 22.26% gave a coordinated response; the rest, only a few percent, stated that the reaction was less agree or negative. however, those who gave disagreed opinions did not exceed 5%. suppose it is translated into graphic form. based on figure 6, it can be seen that the percentage is high in each aspect. first, the aspect of motivation has the highest rate, 77.37%. this shows that the interactive foodivity application can motivate almost all students. this is supported by the absence of students who gave a response that did not agree with the motivational aspect. this motivation can be the initial foundation of students' interest in studying the material to understand the topic. then in the second position, there is an aspect of improving understanding, reaching 74.53% of students strongly agree. this certainly has a positive impact on interactive media. this shows that using interactive multimedia is to increase awareness of the material. primamukti & farozin's (2018) research on using interactive multimedia on learning interest shows a significant difference in students' learning interest when taught with interactive multimedia. then there is the mobile interface aspect. this aspect focuses more on the design and appearance of an application, such as video, audio, text, and animation, that can support understanding concepts in learning materials. in addition, the display can mean attractive, clear, and easyto-read text, images, and other media accessible, and the last aspect is material content. as many as 68.39 strongly agree, and 25.47 choose to agree. the content material relates to the materials presented in the application. videos or other elements such as simulations can be used to explain the content so that students can understand the content independently. hadibin, purnama & kristianto (2012) discovered that learning through interactive multimedia-based media increases students' attention and responsiveness to the subject matter. that corresponds to the author's development findings. this is consistent with the findings of studies wang (2008) conducted that resulted in figure 6 students questionnaire result table 8 students understanding result component pretest post test descriptive test analysis n 53 53 average score 46.23 66.04 standard deviation 16.696 12.262 highest score 80 90 lowest score 15 40 ngain 0.35 score percentage (%) 35.49 normality test signification (sig α = 0.05) 0.200 0.066 information normality distribute (sig. > 0.05) hypothesis test (paired sample ttest) signification (sig < 0.05) 0.000 information h0 is rejected conclusion there is a significant difference journal of science learning article doi: 10.17509/jsl.v5i3.38032 496 j.sci.learn.2022.5(3).488-499 developing multimedia-enabled learning programs. the findings indicate that the programs are effective in student learning outcomes and recommend that instructors consider incorporating them into their student learning. according to lee & tseng (2008), using digital content instruments resulted in substantial variations in student accomplishment compared to traditional learning. 3.4 students’ understanding skills student's cognitive abilities were analyzed through an objective test of 20 questions. the question aims to explore students' understanding of the topic of nutrition. the questions are categorized into bloom’s taxonomy, which is c1 (remembering), c2 (understanding), c3 (applying), c4 (analyzing), and c5 (evaluating). the questions that will be tested on students are in the form of pretest and posttest. before being tested, this test was tested, and the results were processed using spss version 23 to see the validity and reliability results. from the table, it can be seen that the number of students for pretest and postest is 53 students. the results of the score between the pretest and post-test showed an increase after the interactive media application. the mean score of the pretest was 46.23, and the mean score of the post-test was 66.04. the post-test value is higher than the pretest value. it means the mean score of the pretest indicates that the students have less prior knowledge about the material. although some students had difficulty answering the questions, some got high scores from the pretest. therefore, it can be shown that students with high pretest scores have good prior knowledge about the material. data distribution can be normal if the data has met the minimum normality threshold. meanwhile, it cannot be normal if the data does not meet the normality limit. when referring to the assumption of normality, it states that the sample is drawn from a normally distributed population. kolmogorov smirnov is used when the sample size is less than 100. meanwhile, shapiro wilk can be used when the sample size is less than 50 (joshi, kale, chandel & pal, 2015). this can be seen in the significance level of two test instruments is more than 0.05 (0.200 for pretest and 0.066 for post-test). the data examined using kolmogorovsmirnova may be determined that it is regularly distributed, with a 0.05 significance level. the table shows that the significance value for normality is greater than 0.05 (sig. > 0.05). as a result, the data appears to be distributed normally. it suggests that parametric analysis can be used to analyze the data. because the data were analyzed using parametric analysis, the data will be assessed using a paired sample t-test. paired t-test is a parametric test and the test used to compare the difference between two means of two paired samples with the assumption that the data is normally distributed (potochnik, colombo & wright, 2018). suppose the significance value of 2 tailed is less than 0.05 (sig. (2 tailed) < 0.05), it can be said that there is a significant difference. this test requires that the data be normally distributed (mishra et al., 2019). the paired samples t-test table is the output's main table showing the results of the tests. this can be seen from the significant value (2-tailed) from the table 4.8 can be seen that, from pretest (m= 46.23 , sd= 16.69 ) to postest (m= 66.04, sd= 12.26 ), t (52)= -11.666 , p< 0.05 (two-tailed). the mean increase in students' understanding was 19.81 ranging from 23.21 to 16.40. these results show the significant value of sig. (2-tailed) is 0.000, it means the significance value for the simple test pair is less than 0.05 (.000 < 0.05). so the results of the initial and final tests experienced a significant change. based on descriptive statistics, the initial and final tests proved to be higher in the final test. it can be said that there is a significant difference in students' table 9 the result of n-gain based on each topic aspects topic aspects average score of pre-test average score of post test n-gain score category nutrition 50 70.75 0.71 high balanced nutrition 44.02 70.44 0.89 high carbohydrate 35.84 63.20 0.74 high fat 47.80 64.15 0.46 medium protein 54.70 54.71 0.02 low vitamin 54.70 71.06 0.57 medium food testing 34.60 61.63 0.70 medium table 10 the result of n-gain score cognitive aspects average score of pre-test average score of post-test n-gain score category c1 (remembering) 37.74 66.04 0.45 medium c2 (understanding) 49.43 66.42 0.34 medium c3 (applying) 41.51 63.52 0.38 medium c4 (analyzing) 48.58 69.81 0.41 medium c5 (evaluating) 52.83 64.41 0.27 low journal of science learning article doi: 10.17509/jsl.v5i3.38032 497 j.sci.learn.2022.5(3).488-499 understanding of nutrition topics by using foodivity interactive as an interactive media. after that, a normalized gain test was carried out to analyze the increase in cognitive abilities. the n-gain value is 0.35, which is categorized as a medium increase. suppose the score is 0.30 <g≤0.70, it is categorized as a moderate improvement (hake, 1999). it means a moderate increase from the pretest to the posttest, and the score percentage is 35.49%. data were also analyzed using a normalized gain test based on hake's rule. this test analyzes the level of improvement in students' understanding. based on the analysis of each topic in table 9, balanced nutrition and carbohydrates increase in the high category on the topic of food. this indicates that the increase in student understanding is good and that students understand the material concept in interactive multimedia. meanwhile, on the topic of fat, vitamins, and food testing, it has a medium improvement category. on the topic of content in multimedia, some simulations can encourage students to understand more quickly and motivate students to learn more deeply. this is in line with wilkinson, dafoulas, garelick & huyck (2020) stated that if quiz games improve memory of fundamental facts through visual stimuli and repetition, this may allocate more classroom time for application and thus comprehension of the subject. but on, the topic of protein has a low category. it means that there is no improvement in that topic for understanding. therefore, there may be a need for a more in-depth study of this topic. to analyze the overall results of student understanding, each aspect of understanding is based on the bloom taxonomy, especially in c1 (remembering), c2 (understanding), c3 (applying), c4 (analyzing), and c5 (evaluating) is also analyzed. table 10 shows the average pretest and post-test scores of each aspect of understanding depicted in the table: based on table 10, almost all aspects show the category of medium improvement. the increase in the medium and low categories indicates that more effort is needed to increase understanding. the results of the students' average pretest in each aspect varied. the pretest results show that the c1 (remembering) aspect has the lowest average score compared to other aspects. this can happen due to several factors, one of which is remembering. some students do not know the material, and the topics have never been given before. however, at c5 (evaluating), the average score obtained has the highest. this shows that some students already understand the concept of the material. after that, the average pretest and post-test scores were compared. the n-gain of each aspect can be seen in the table. in the aspect of c5 (evaluating) has the lowest improvement category. this is because, in bloom's taxonomy, c5 is a high level of high-order thinking. in addition, students are required to be able to judge from an event or activity. so, students need a deep understanding. hoque (2016) said that teachers and students must comprehend the cognitive domain's hierarchy of processes and abilities to grasp prerequisite skills for learning and how these skills must be modified to master more sophisticated aspects of discipline-specific concept inventories. in teaching or learning new content, the development of learning skills should never be taken for granted. while the c1-c4 aspects have a medium improvement category, comparing the pretest and post-test, the c1 (remembering) aspect has the highest n-gain score among other aspects. this is because c1 is the most basic level in the taxonomy bloom. c1 is also low-order thinking, so it is often categorized as an accessible item question and only requires the ability to remember without understanding the material. in aspects c2, c3 and c4 have almost identical n-gain scores. the c4 aspect also has a fairly high n-gain, so it can be said that the use of interactive media can affect students' understanding of the material. however, the level of ngain scores in students' overall understanding has a medium category, meaning that efforts are still needed to improve students' understanding to get a high category. 4. conclusion this research was conducted using a pre-experimental method and the addie learning design (analysis, design, develop, implement and evaluate). this research was conducted in public schools in west java, in two locations, sukabumi and kuningan junior high school, in grade 8th. the results of this research are the development of foodivity interactive, which is used to assess students' understanding. the findings of this study indicate that students' understanding increases as a result of using foodivity interactive as interactive multimedia on food nutrition topics. the development of foodivity interactive as interactive multimedia has been carried out using software construct 2. addie instructional design is used in developing foodivity interactive and exported as html 5 and android-based applications. the expert assessment of interactive foodivity is based on the results of index v which was carried out using a likert scale. the mean value found for index v is 0.8125. this shows that the media is valid and can be used as a learning medium. the data questionnaire data is used to determine the results of student responses. 70.07 percent of students strongly agree, while 22.26 percent agree. based on these findings, it can be concluded that more than half of the students stated positive opinions about the application. so, it can be concluded that developing foodivity interactive multimedia can help students better understand nutrition topics. this can be proven by the n-gain results, which show a moderate increase between the pre-and posttreatment tests. the hypothesis test stated that hypothesis journal of science learning article doi: 10.17509/jsl.v5i3.38032 498 j.sci.learn.2022.5(3).488-499 h1 was accepted, which indicated that significant difference between the pretest and post-test references abidah, a., hidaayatullaah, h. n., simamora, r. m., fehabutar, d., & mutakinati, l. (2020). the impact of covid-19 to indonesian education and its relation to the philosophy of “merdeka belajar.” studies in philosophy of science and education, 1(1), 38–49. https://doi.org/10.46627/sipose.v1i1.9 ahmizar, f. (2004). mengoptimalkan multimedia sebagai sarana mencerdaskan bangsa [optimizing multimedia as a tool to educate the nation]. pesantren on line. bennett, s. j., & brennan, m. j. (1996). interactive multimedia learning in physics. australasian journal of educational technology, 12(1). https://doi.org/10.14742/ajet.2031 bhavya, b., gautam, g., & sumedha, m. (2021). impact of covid-19 pandemic on education system. epra international journal of environmental economics, commerce and educational management, june, 6–8. https://doi.org/10.36713/epra6363 bolger, f., & wright, g. (1992). reliability and validity in expert judgment. in expertise and decision support (pp. 47-76). springer, boston, ma. churiyah, m., sholikhan, s., filianti, f., & sakdiyyah, d. a. (2020). indonesia education readiness conducting distance learning in covid-19 pandemic situation. international journal of multicultural and multireligious understanding, 7(6), 491. https://doi.org/10.18415/ijmmu.v7i6.1833 dawson, t. e. (2007). a primer on experimental and quasiexperimental design a primer on experimental and quasiexperimental design. southwest educational research association, january 1997, 1–15. https://eric.ed.gov/?id=ed406440 godwin, j. r. (2012). emerging technologies: challenging hegemonies in online learning. language, learning and technology, 16(2), 4–13. hadibin, m. m., purnama, b. e., & kristianto, g. (2012). pembangunan media pembelajaran teknik komputer jaringan kelas x semster ganjil pada sekolah menengah [development of learning media for networking computer engineering for class x odd semesters in middle school]. indonesian journal on networking and security (ijns), 9330(1), 1–6. hake, r. r. (1999). analyzing change/gain scores. american educational research association, 1(1), 16–22. https://doi.org/10.24036/ekj.v1.i1.a10 huryah, f., sumarmin, r., & effendi, j. (2017). analisis capaian literasi sains biologi siswa sma kelas x sekota padang [analysis of biological science literacy achievements for class x senior high school students in padang]. jurnal eksakta pendidikan (jep), 1(2), 72. https://doi.org/10.24036/jep.v1i2.70 joshi, a., kale, s., chandel, s., & pal, d. (2015). likert scale: explored and explained. british journal of applied science & technology, 7(4), 396–403. https://doi.org/10.9734/bjast/2015/14975 lee, s. h., & tseng, h. c. (2008). investigation of technology integrated instruction in art education: a case study of exploring learning achievement. in journal of educational multimedia and hypermedia, 17(3), 337–361). http://www.editlib.org/p/23648 maharani, p., & asyhari, a. (2020). construct 2 interactive multimedia for temperature and heat topic: a multimedia development for senior high school learning. indonesian journal of science, 03(11), 336–346. https://doi.org/10.24042/ijsme.v4i1.8673 mishra, p., pandey, c. m., singh, u., gupta, a., sahu, c., & keshri, a. (2019). descriptive statistics and normality tests for statistical data. annals of cardiac anaesthesia, 22(1), 67–72. https://doi.org/10.4103/aca.aca_157_18 munir. (2020). multimedia konsep & aplikasi dalam pendidikan [multimedia concepts & applications in education]. in antimicrobial agents and chemotherapy, 58(12). nesbit, a. d., giel, j. l., rose, j. c., & kiley, p. j. (2009). sequencespecific binding to a subset of iscr-regulated promoters does not require iscr fe–s cluster ligation. journal of molecular biology, 387(1), 28-41. novianto, l. a., degeng, i. n. s., & wedi, a. (2018). pengembangan multimedia interaktif mata pelajaran ipa pokok bahasan sistem peredaran darah manusia untuk kelas viii smp wahid hasyim malang [development of interactive multimedia for natural science subjects of the human circulatory system for class viii smp wahid hasyim malang]. jurnal kurikulum teknologi pendidikan (jktp) universitas negeri malang, 1(3), 257–263. http://journal2.um.ac.id/index.php/jktp/article/view/5770 perera, t., frei, s., frei, b., wong, s. s., & bobe, g. (2015). improving nutrition education in u.s. elementary schools: challenges and opportunities. journal of education and practice, 6(30), 41–50. www.iiste.org potochnik, a., colombo, m., & wright, c. (2018). statistics and probability. recipes for science, 2, 167–206. https://doi.org/10.4324/9781315686875-6 primamukti, a. d., & farozin, m. (2018). utilization of interactive multimedia to improve learning interest and learning achievement of child. jurnal prima edukasia, 6(2), 111-117. priyanto, d. (2009). pengembangan multimedia pembelajaran berbasis komputer. 14(1), 1–13. pujawan, k. a. h. (2018). the development of interactive multimedia with drill and practice model on multimedia ii (two dimention animation) course in politeknik ganesha guru. journal of education research and evaluation, 2(1), 22–27. https://doi.org/10.23887/jere.v2i1.13142 puspitasari, d. n., indriyanti, d. r., & nugrahaningsih, w. h. (2019). development of interactive multimedia for human reproduction system in junior high school. journal of biology education, 8(2), 238245. https://doi.org/10.15294/jbe.v8i2.28016 putri, r. s., purwanto, a., pramono, r., asbari, m., wijayanti, l. m., & hyun, c. c. (2020). impact of the covid-19 pandemic on online home learning: an explorative study of primary schools in indonesia. international journal of advanced science and technology, 29(5), 4809–4818. roswati, n., rustaman, n. y., & nugraha, i. (2019). the development of science comic in human digestive system topic for junior high school students. journal of science learning, 3(1), 12–18. https://doi.org/10.17509/jsl.v3i1.18120 sahlström, f., tanner, m., & valasmo, v. (2019). connected youth, connected classrooms. smartphone use and student and teacher participation during plenary teaching. learning, culture and social interaction, 21(january), 311–331. https://doi.org/10.1016/j.lcsi.2019.03.008 sinclair, m. a. (1975). questionnaire design. applied ergonomics, 6(2), 7380. sun, j. c., hsieh, p., sun, j. c., & hsieh, p. (2018). international forum of educational technology & society application of a gamified interactive response system to enhance the intrinsic and extrinsic motivation , student engagement , and attention of english learners published by : international forum of educational technology & society linked references are available on jstor for this article : application of a gamified interactive response system to enhance the intrinsic and extrinsic motivation , student engagement , and attention of english learners. 21(3). wang, l. (2008). developing and evaluating an interactive multimedia instructional tool: learning outcomes and user experiences of optometry students. in journal of educational multimedia and hypermedia 17(1), 43–57). http://www.editlib.org/p/22939 wiana, w., barliana, m. s., & riyanto, a. a. (2018). the effectiveness of using interactive multimedia based on motion graphic in concept mastering enhancement and fashion designing skill in digital format. international journal of emerging technologies in learning, 13(2). https://doi.org/10.3991/ijet.v13i02.7830 wilkinson, k., dafoulas, g., garelick, h., & huyck, c. (2020). are quiz‐ games an effective revision tool in anatomical sciences for higher https://doi.org/10.46627/sipose.v1i1.9 https://doi.org/10.36713/epra6363 https://doi.org/10.3991/ijet.v13i02.7830 journal of science learning article doi: 10.17509/jsl.v5i3.38032 499 j.sci.learn.2022.5(3).488-499 education and what do students think of them?. british journal of educational technology, 51(3), 761-777. zheng, l., li, x., & chen, f. (2018). effects of a mobile self-regulated learning approach on students’ learning achievements and selfregulated learning skills. innovations in education and teaching international, 55(6), 616–624. https://doi.org/10.1080/14703297.2016.1259080. a © 2020 indonesian society for science educator 89 j.sci.learn.2020.3(2).89-98 received: 18 august 2019 revised: 27 february 2020 published: 18 march 2020 the development of web-based learning using interactive media for science learning on levers in human body topic lia astuti1*, yaya wihardi2, diana rochintaniawati1 1international program on science education, faculty of mathematics and science education, universitas pendidikan indonesia, bandung, indonesia 2department of computer education, faculty of mathematics and science education, universitas pendidikan indonesia, bandung, indonesia *corresponding author liastuti084@student.upi.edu abstract integrated curriculum is a popular way to develop 21st-century skills, but most of the materials are written on the books separately. furthermore, web-based learning is an online learning media that can be accessed using an internet connection anytime and anywhere. however, many educational websites do not apply the principles of effective learning. besides, the traditional learning methods tend to be bored for the students. in order to solve this problem, this study designed a website education that uses interactive content to assist students in learning levers in the human body topic as one of the integrated science materials. the process of developing an educational website consists of three steps: (1) analysis, (2) design, (3) construction making. this research method used descriptive method, and the experts' judgment will evaluate it on content, language, and media/it. the questionnaires used the technology acceptance model (tam) and five-dimensional interactivity to investigate the readability of the subjects' perception responses. the research subject was three science teachers and 31 students on private junior high school in bandung. according to the result, generally, it has a good evaluation of each aspect. but, t he website education needs a strong signal to access for not taking load time consumer. keywords website education, levers in the human body, five-dimensional interactivity, technology acceptance model (tam) 1. introduction education in the 21st century has been and is being profoundly influenced by technology and globalization (voogt, erstad, dede, & mishra, 2013). integrated curriculum is one of the effective ways to resolve some challenges associated with developing 21st-century skills (drake & savage, 2016). as a response to 21st century needs, fostering students’ creativity has been explicitly included in the school curriculum of indonesia that use integrated science curriculum to encourage students to see the interconnectedness and interrelationships between the subjects (asrizal, amran, purwani, & sudargo, 2018; lykke, coto, mora, vandel, & jantzen, 2014). the ministry of education and culture of indonesia realized that the integration of science literacy education is important. for this purpose, the 2013 curriculum requires science subjects in junior high school to be implemented with the integrative approach to improve the literacy of students in the school (yuliati, 2013). however, the real condition shows that the implementation of integrated science learning and integration of literacy in the learning process cannot be implemented well because the instructional material still write as separated topic in books (ardianto & rubini, 2016; asrizal, amran, purwani, & sudargo, 2018; pursitasari, nuryanti, & rede, 2015). besides, science is one subject that students often find confusing and mistakenly judged as a difficult subject, especially for physics. hesti, maknun, & feranie (2017) stated that students feel indifferent toward physics lessons because of the complexity of the concepts. in order to visualize the complex concepts, appropriate instructional material is needed to apply for the teachinglearning process (asrizal, amran, purwani, & sudargo, 2018; cook, 2007). instructional materials are an important part of the learning process. educational website technology can mailto:liastuti084@students.upi.edu journal of science learning article doi: 10.17509/jsl.v3i2.19366 90 j.sci.learn.2020.3(2).89-98 develop properly to facilitate students to learn a certain material and to support or facilitate the acquisition of knowledge, competency, and skills (bolkan, goodboy, & kelsey, 2016). the young generation is fully aware of the benefit of personal computer and tablet devices, and almost every student has for accessing the internet, mainly for e-learning and playing games (goh, bay, & chen, 2015). also, the annual survey done in 2007 by indiana university's high school survey of student engagement (hssse), shows that more than 81,000 students in 110 secondary schools in 26 states dropped out of school because they experienced boredom of teaching-learning process on the class. according to seifert (2004), boredom occurs because of the way of the instructional material is presented, and the students are not fully involved in a class, so they lose the motivation to learn. this condition indicates that many traditional lessons are boring and make the lesson ineffective. the other way to deliver instructional materials rather than a traditional lesson is through technology with an interactive application that works with smartphones or personal computer, and it is in line with the 21st-century purposes that is marked by the usage or the integration of technology in daily activities (walsh, 2014; chawinga & zozie, 2016). keengwe & anyanwu (2007) argue that technology in education focuses on being a tool for creating the process of learning alive, concrete, and interesting. web-based technology is often the technology of choice for distance education, given the ease of use of the tools to browse the resources of the web from any devices, and the relative affordability of accessing the ubiquitous web. web-based learning is an online learning media or a website that has educational goals, and many institutions develop instructional material of science as a media for a source of integrated science learning material (kenny, 2000; wang, cheng, chen, mercer, & kirschner, 2017). the advantages of web-based learning from others' learning models is one of the learning models that used the technology of the web and using the internet. it can be accessed anytime and anywhere from any device using any operating system like android, windows, and others (zaiane, 2001). even though most sites meet the criteria for general information websites, but many educational websites do not apply the principles of effective learning, only 17% have all components of the learning paradigm which are critical thinking, independent learning, evidence-based learning, and feedback; and less than 50% do not meet any criteria because of the content inside the web that does not encourage the readers (chou, 2003; cook & dupras, 2004; dogan & dikbiyik, 2016). the solution to visualize the effective website education is by using the interactive' media for the content as the instructional material that will overcome the student boredom to make the materials easier to understand and to encourage student motivation. also, one of the materials of integrated science is levers in the human body, which is a combination of physics and biology subject, and that is the sub-topic of the simple machine materials. holzinger, emberger, wassertheurer, and neal (2008) described that the human body, the chemical substance others are hard to imagine without any multimedia that will show how complex the materials because we cannot see directly with our eyes. while the example of the simple machine can be used in daily life, and the levers mechanism example exists on the human body too that includes skeletal system material. so, it needs interactive multimedia through the video and animation to make it easy to understand and learn. this research aimed to develop an interactive website education for web-based learning. the interactive website education will use the five-dimensional interactivity of the website (chou, 2003). the five-dimensional interactivity is (1) playfulness, (2) choice, (3) connectedness, (4) information collection, (5) reciprocal communication. those used to measure the interactivity of the website education with the existence of the feedback from the quiz games to measure the readers' understanding after reading the content on the website education. other than that, to investigate the perceptions of subjects, this study uses the technology acceptance model (tam) towards the behavioral intention to use the website education (davis, bagozzi, & warshaw, 1989). this research will be conducted by designing and producing interactive website education for web-based learning that used two ways of communication between the readers and the website itself by giving the feedback. norman (2000) stated that the interactive website education include: (1) a central role of language (ask for things even if not visible), (2) richer internal representation of data objects including user history of interaction with documents, applications, web pages, (3) a more expressive interface added more video and others, (4) shared control proactive computers and agents without human commands. the instrument of the website education will be assessed to the experts' judgment. after that, it will be applied to science teachers and students in junior high school to get the perceptions to respond to the use of the website education. 2. method descriptive research was used for this research method. according to fraenkel, wallen, & hyun (2013), descriptive studies describe a given state of affairs as fully and carefully as possible. 2.1 research design the researcher develops website education use interactive media using html 5, and it can be accessed from a browser on a mobile phone or personal computer and use the internet connection. the website education journal of science learning article doi: 10.17509/jsl.v3i2.19366 91 j.sci.learn.2020.3(2).89-98 was judged by the experts' judgment on content (physics and biology), language (english), and media or design (it). 2.2 research subject the location of this research was held in one international school in bandung in the school period of 2018/2019. the school used cambridge and the 2013 curriculum. science teachers accessed the website education, and the population in this research was 7th, 8th, and 9th-grade students have accessed the media to know their perception response. 2.3 content analysis the analysis of content is very important for developing website education as the analysis stages. the need to develop student 21st-century competencies such as creative problem solving and communication to prepare the young generation for the complex global world. an integrated curriculum is a popular way to develop these skills (drake & savage, 2016). in this research, the content of the simple machine used in this research is limited by the indonesian curriculum 2013 that stated on basic competence 3.3 and 4.3. levers cover these competencies in the human body topic, and the determination of content is suitable for junior high school students due to encouraging their interests towards science. the analysis of curriculum about core competency and basic competence indicates the subtopics that will be investigated by students such as (1) identify the types of a simple machine, (2) identify how levers as a type of simple machine through the picture and video, (3) identify the mechanism of levers and the correlations with the human body. the terms of levers in the human body in this research are explained more about the application of the levers as a part of a simple machine able to apply in the human body, such as ligaments, bones, and muscle that exists on skeletal system topic. the structures form levers in the human body create a movement. in simple terms, a joint (where two or more bones join together) forms the pivot (or fulcrum), and the muscles crossing the joint apply the force to move a weight or resistance. all three types of levers are found in the body, depending on the position of the fulcrum, load, and effort. 2.4 research instrument there are three types of instruments used in this research, which are the characteristic development of the website education, the validation of the website education from three experts, and the readability of the science teachers’ and students’ perception responses. first, the development of website education consists of three stages. firstly are the analysis stages, which consisted of: (1) the material analysis, whereas the material was chosen based on the field of study result on junior high school science materials; (2) the user analysis is the suitability whether they were interested in using new technology that applies for learning media or no; (3) the right software necessity analysis is needed for developing the website table 1 experts' judgment validation rubric result aspect indicator total checklist reject revise accept learning goal alignment the content is in line with learning goals 1 5 the learning goals are clear are significant 2 4 the relevance of the item content the topic is delivered clearly 2 4 the combination of the relations between physic and biology subject is the relevance 2 4 the figure/video/ caption was appropriate and relevance with the topic 6 the balance of coverage of the items in relation to the content the content and the design of the web is balance, and there is no barrier to each other 1 5 follows the norms and conventions of scientific writing accurate each other 1 5 the arrange of the word and sentence in the website the choice of the verb from the web is appropriate 6 the form and choice of nouns, pronouns, adjectives, and adverbs from the web are appropriate 6 the word order from the web is appropriate 6 the choice of prepositions from the web is appropriate 6 the formation of tag questions and elliptical responses from the web are appropriate 6 motivation the ability to motivate and attract many users of the web 1 5 presentation design visual design and sound of the web can enhance interactive for the users 2 4 all of the hyperlink/buttons work smoothly 2 4 interaction usability the web has ease navigation 2 4 the web has proportional display interface 6 the web has a good quality of the interface features help the user understand 6 journal of science learning article doi: 10.17509/jsl.v3i2.19366 92 j.sci.learn.2020.3(2).89-98 education (4) the hardware necessity analysis to make sure that the personal computer is compatible enough to access the software. secondly are the design stages, which consisted of (1) the learning material design, the materials contained in the website education is levers in the simple machine and levers in the human body; (2) the flowchart design is the illustration of story flow in the development process of the website education; (3) the storyboard design is the plan of multimedia that will be developed based on the flowchart in order to make development stage easier. thirdly is the development stages that consist of two-step, which consisted of: (1) the construction of interface making that was made based on the storyboard in the design stage. some elements should be done which are collect the user necessities and define the design conceptually, and then validate it, after that, the problems that arise should be fixed before it can be developed; (2) the construction of coding making, while in interface making stage, the features are still not working yet, and the coding process stage should be functioned in order to make all the features work properly. second, the experts' judgment validation of the website education, the experts consisted of three experts in content, language, and media. the rubric for the expert judgment was given in the same aspect and indicator, and it is used using accept, reject, and revise. there is not differentiated for judging the materials' content, language, and it. the reason was for knowing the experts' perspective between all of the indicators and aspects. while, all the content of the items must be carefully evaluated, and the determination of the content validity evidence is often judged by expert judgment (fraenkel, wallen, & hyun, 2013). third, the science teachers' and students' perceptions respond to the website education, they gave the perception respond after using the website education, and in general, it is called a readability test. the readability level of the website education can exert an influence on the science teacher and students' perception response result, e.g., difficulty in reading the website content or the clearness of the website education might distract from the purpose of a test (cohen, manion, & morrison, 2011). the questionnaire will be used five-dimensional interactivity that fulfills the communications need and engage the student motivation to operate and use the web, ha & james (1998) proposed it and technology acceptance model (tam) will be applied extensively for studying information technology due to its effectiveness in assessing the degree of users' acceptance which are the perceived ease of use and perceived usefulness on the behavioral intention that proposed by davis, bagozzi, & warshaw (1989). the scale that will be used in this research is the 5-likert scale because a few researchers have reported higher reliabilities for five-point scales. while, the questionnaire for science teachers and students will be measured that indicating an acceptable level of internal consistency for a short research survey with nonhomogenous items of this kind (fraenkel, wallen, & hyun, 2013). the reliability for the internal consistency of the students and science teacher questionnaire were measured and evaluated by cronbach’s alpha. in general, the minimum acceptable value of cronbach’s alpha is 0.6 (warmbrod, 2014). the results show quantitative and qualitative; the quantitative measurement analyzes the questionnaire for science teachers' and students' perspective responses that use the 5-likert scale and the internal consistency. figure 1 the example of the interface of website education using an interactive media for integrated science, with the link address http://medialengka.com/inscience http://medialengka.com/inscience journal of science learning article doi: 10.17509/jsl.v3i2.19366 93 j.sci.learn.2020.3(2).89-98 qualitative analysis will describe the validation of the experts' judgment questionnaire and the readability of science teachers' and students' perspective response results. 3. result and discussion the results show quantitative and qualitative data. the experts’ judgment validation of website education was conducted before the website education implemented to the science teachers’ and students’ to know the perception respond that use technology acceptance model (tam) to know the user behavioral intention and fivedimensional interactivity to know the interactivity and effectiveness of the website. qualitative analysis will describe the experts’ judgment validation result and the science teachers' and students’ results towards the development of the website education. 3.1 the characteristic of the website education the characteristics of the website education that has been developed have two ways communication with the existence of the feedback on the finding keywords games and quiz games, and the content that really suitable for secondary high school students, which are the colorful content, the existence of animation, video, and interactive multimedia. for example, the human body and the chemical substance others are hard to imagine without any multimedia that will show how complex the materials because we cannot see directly with the eyes. while the simple machines can be used in daily life, and the levers mechanism example exists on the skeletal body system. so, it needs interactive multimedia through the video and animation to make it easy to understand and learn. the interface example is shown in figure 1, and the link to the website education address is http://medialengka.com/inscience. 3.2 experts’ judgment validation the website education will be implemented for science teachers and students. while the content of the items must be carefully evaluated, and the determination of the content validity evidence has to be judged by expert judgment (fraenkel, wallen, & hyun, 2013). therefore, the experts judgment will be judged by the lectures as the expert to judge according to the field of the content (physics and biology), language, and media or design (it) towards the website education whether it can be rejected, accept or there is a revision and should be revised based on the comment and suggestion from the experts with the rubric that has been prepared. the rubric for the expert judgment was given in the same aspect and indicator; there are not differentiated for judging the materials' content, language, and it. the reason was for knowing the experts' perspective between all of the indicators and aspects. based on the rubric given to the experts, the result is described in table 1. the table was showed the results' total of the experts that choose or checklist on the option of reject, revise, and accept of the indicator and the aspect. first, for the reject, it showed the total of experts' judgment suggests to delete the content indicator of the website education. second, for the accept, it showed the total of the experts judgment that accepts the content indicator on website education, and there is no revision, third, for the revise, it meant there was a revision for the website education and showed the total of expert judgment that gives a suggestion of the indicator table 2 result from the experts' judgment on the content comment experts description first experts' judgment on the content 1. dominated by physics 2. add the source of video and animation second experts' judgment on the content 1. the content needs to be detailed, the explanation from the video only covers some of the content 2. the answer why for learning goal is not for the student, it is for the teacher, make it for a student, the objective need to be measurable, so the word "how" no need in objective and no need to capitalized each word 3. it needs a more physic component or explanation for the lever in the human body 4. need more videos experts' judgment on language good variety (games, videos, text, and graphics, etc) first experts' judgment on design the 6 types of simple machines should input the explanation table 3 result from the experts' judgment on the language comment experts description experts' judgment on language 1. the second learning goal is okay, but somewhat "poorer" than the rest because the overall focus, levers, is only a small past. 2. good except for a small section in the human body section, need consistency between concepts of joints/ligaments and fulcrum. 3. you just need to change the content in the "levers in the human body" section to move clearly describe ligaments/joints as equivalent to a mechanical fulcrum, also in the text change "axis" to fulcrum (or add "or fulcrum") second experts' judgment on the content re-check the grammar on the learning goals http://medialengka.com/inscience journal of science learning article doi: 10.17509/jsl.v3i2.19366 94 j.sci.learn.2020.3(2).89-98 whether in a language, content, or the design to make a better website education and make more valid, and the result of the website education revision will be described, as follows: the result of experts’ judgment on content the content itself is integrated science that combined of physics subject and biology subject. the aspects that judged are the contents' objective in line with the necessity of the students in junior high school, the accuracy of content, and the interest in the content. based on table 2, the result of the questionnaire, the experts suggested enriching the materials with some more explanation on physics and biology materials, whether it is from the video, animation, text, etc., several approaches can be used to develop and deliver web-based learning. khalifa & lam (2002) is interpreting the term to include any technology which requires the user to make a response to the information or ideas presented. as well as interactive video, multimedia, hypermedia, computer simulation, and expert systems. this website education as the instructional materials, it can enhance understanding of the concepts easily, especially the existence of game and quiz game. it can encourage the students’ motivation to learn the materials by the picture with gif format, video, the colorful interface, and especially for the existence of the game. the game structure a course with a focus on feedback, goals, and interaction as game attributes and the integration of leaderboards as game elements (alaswad & nadolny, 2015). the result of experts’ judgment on language the aspects that judged by the expert on language shown in table 3; there are contextual spelling, grammar, punctuation, sentence structure, and style for delivering the materials. at first, there is a lot of grammatical error, and the term of the sentence placed that used in this website education. but, for the objective and the materials of the website is already well-implemented. since this website education is in english, the writing of this website education should be correct in all aspects. the grammar assessment should be recognized both in large-scale and classroom-based contexts (purpura, 2013). in addition, the website education should be consistent to explain the term. the flexibility and potentially provided in webbased learning made it possible to enable high structure and high dialogue (course design and delivery) at the same time (huang, chandra, depaolo, & simmons, 2016). the result of experts’ judgment on design the background, fonts, compatibility, sounds, graphics, images (accessibility), layout, navigation, and load time are applied for the aspects of this website education. the experts especially noted on table 4 about the load time to access the website education because there is a lot of animation and video that have proposed to make the website become colorful and interactive, it is important to make the load time shorter, and smooth transition between each feature are strongly advised to be fixed. srivastava, cooley, deshpande, & tan (2000) also stated the web needs time for prefetching documents in order to reduce user-perceived latency when loading a page from a remote site. table 4 result from the experts' judgment on the design comment experts description first experts' on media 1. fill the blank should be input to what? (for the blank, user can click any button without fill the question) 2. video and animation should input the source 3. the true or false page should be direct to biology material second experts' on media some features feel long for loading, and i think that is because of too much animation and videos second experts' on content 1. the indicator of the ability to motivate and attract many users of the web need to be improved 2. it can be attractive if there is a comment or live chat on the web 3. it takes time to load the web first experts' on content better to add the indicator of "motivation" aspect third experts' on content the navigation is confusing the experts' on language 1. i think it is fine for a preliminary student project possibility a little slow to load 2. once a "continue" button did not lead anywhere, but the system did not hang figure 2 comparison score between science teachers and students readability perspective respond the result journal of science learning article doi: 10.17509/jsl.v3i2.19366 95 j.sci.learn.2020.3(2).89-98 3.3 the readability result of science teachers’ and students’ respond the website education is given to the science teachers and students to know the impression of the perception respond towards using the website education, the explanation of the impression from the science teachers’ and the students’ respond explained as follow: science teachers’ perception respond the website education is judged by science teachers who teach in junior high school before implementing it to the students. there are three science teachers from pelita nusantara junior high school in bandung that tried the website education and gave their review, comment, and suggestion. the result was shown in table 5. the questionnaire resulted in most of all; the items are categorized on the "very good" scale; it is described in table 5 that has a score range from 80 to 100. the score of the acceptability of the web, which is perceived ease of use, is 96, and the perceived usefulness is 96. the score of the interactivity of the content that uses five dimensions, which is the playfulness aspect is 82, and the choice aspect is 93, the connectedness aspect is 91, the information collection aspect that means the materials score is 84, and the last is reciprocal communication that reaches the highest score which is 96. the overall response from the teachers is positive, which is to support the website education to be used in the teaching-learning process. in addition, paulsen (2003) described that web education could be a strategy of teaching that used to provide the materials and interactions between the teachers and the students. chart figure 2 shows that all the aspects include in the website education are generally fair and square. the website acceptance, which is represented by perceived ease of use and perceived usefulness aspect, and the interactivity of the website education that is represented by playfulness, choice, connectedness, information collection, and reciprocal communication aspect, has table 5 recapitulation of science teachers' responds regarding the website education construct item sd d n a sa score result average score perceived ease of use 1 3 100 very good 96 2 3 100 very good 3 2 1 87 very good perceived usefulness 4 1 2 93 very good 96 5 3 100 very good 6 1 2 93 very good playfulness 7 1 1 1 80 very good 82 8 2 1 87 very good 9 3 80 very good choice 10 1 2 93 very good 93 11 3 100 very good 12 2 1 87 very good connectedness 13 1 2 87 very good 91 14 1 2 93 very good 15 1 2 93 very good information collection 16 2 1 87 very good 84 17 2 1 87 very good 18 3 80 very good reciprocal communication 19 3 100 very good 96 20 1 2 93 very good 21 1 2 93 very good overall score 91 very good notice: item 1: the web is easy to use; item 2: the web can provide clear guidance; item 3: the web is useful to assist me in learning lever in human body topic; item 4: the web can increase my efficiency to more understand for learning lever in human body topic; item 5: the web can provide useful media like the picture, video, and its' explanation for learning lever in human body topic; item 6: using web is a good idea to learn lever in the human body; item 7: the content and the game of the web made me satisfy; item 8: the content and the game of the web made me have fun when using the web; item 9: the content and the game of the web made me curious to know more about the topic; item 10: the web provides a choice of links or button; item 11: the choice button in the start (home) help me to operate the web; item 12: the choice of navigation button is useful to guide me to operate the web; item 13: the hypertext or the button help me to operate the web; item 14: the navigation button of the web has a relationship with each other; item 15: the navigation button of the web is working smoothly; item 16: the web ask my prior knowledge, it helps me to assist my previous understanding before i learn lever in human body topic; item 17: the web content about lever in the human body adds my insight; item 18: the web content is useful to help me learn lever in human body topic; item 19: the web providesa link button to give feedback and comment; item 20: the contact page of the web help me to give comment; item 21: the contact page of the web help me to give feedback; sa = strongly agree, a = agree, n = netral, d = disagree, sd = strongly disagree journal of science learning article doi: 10.17509/jsl.v3i2.19366 96 j.sci.learn.2020.3(2).89-98 achieved the requirement of a proper website education. the website education as an instructional material aims to help the teacher to give feedback and review to the students (chang, 2007). moreover, to engage the student's interest and excitement, the interactivity of the content on the website is supposed to be suitable for junior high school students. the science teachers noted that this website education creates a better-personalized learning experience for students; the evaluation shows 91 out of 100 that categorized as "very good". junior high school students’ perception respond the application is implemented to junior high school students from grade 7th, 8th, and 9th. the students are gathered in the formal class to try the website education to use interactive media, and it does not matter for the students that have been learned about a simple machine or not yet. the number of students who tried the website is from pelita nusantara junior high school in bandung that consist of 31 students, 7th grade consist of 3 students, 8th grade consists of 5 students, and 9th grade consists of 23 students. the result of students respond is showed in figure 2 the overall aspects got "very good" in scale. the score of the website for web-based learning is 82, which includes the acceptance and the interactivity of the website to be accessed. the questionnaire is consists of the technology acceptance model (tam) (davis, bagozzi, & warshaw, 1989) and using five dimension aspects of interactivity (ha & james, 1998). for the acceptance, the score perceived ease of use of the website is 84, which declares the website is easy to operate, and the score perceived of the usefulness of the website is 84, which is the quality having utility and the website is applicability to be accessed. the technology acceptance model, the questionnaire is used 5-dimensional interactivity. first is the score of the playfulness of the website is 82 which means the website can encourage the students besides the quiz and the question exist on the game that in-line with the concept of levers in the human body, so the students were playing to reach the higher score because the game has a leaderboard to know who is the winner and the students can know their score after playing the game. second, the score of choice of the website is 80 of the student's excitement because of the animation and the color of the website and also engenders an internal emotional sense of satisfaction. third, the score of connectedness is 80, which enhances the feeling of the connection between one hyperlink/button/navigation to another button that will display the material, and it consists of the video, picture, game, animation that have a correlation to one another. the forth of the five-dimensional interactivity is information collection. in this website, education can be called the material content, and its' score is 81, which declares the clarity of materials is quite understandable and generally helps students to give them more meaningful learning. the last is reciprocal communication after the students tried all the navigation and learn the content they can give a comment and feedback about the website or if they have a question so they can contact the developer through contact navigation button and it has 84 scores. the students give positive responses to the website for web-based learning. the overall evaluation score of a questionnaire filled by students is 80 out of 100 scale that indicates "very good" evaluation. students found this website education is easily accessible and helps them to study at home. du et al. (2013) stated that web-based learning had been pleasing effects in making a better participants' knowledge and capability of the performance, and in increase self-efficacy in performing capability, with a high gratification rate uttered by participants. students found this website is exciting and helping them to understand simple machines with sub-topics levers in human body topic. especially of the interactivity of the web and the way to deliver the content, it is proven by the score with 81.4 that indicates very good; students learn how to understand the content easily and more encouraging because the existing of the game and the animation to make them not bored while they read and scrolling the content. aside from the technology acceptance model aspects, science's teacher responses, and junior high school students' responses, the prospect of this website education is generally welcomed. they gave positive evaluations towards this website education as it is seen in their answer to the questionnaire and comment and feedback. students generally responses that website education can make their study easier while inside and outside the class. website education is defined as one teaching strategy 'in which the web is used to provide the materials and interactions between the students and teachers' (zapalska & brozik, 2006) the internal consistency of science teachers’ and students’ respond the result of the internal consistency is showed in table 6 and table 7. it is measured by cronbach’s alpha table 6 internal consistency result of science teacher questionnaire instrument reliability statistics cronbach's alpha n of items .825 21 table 7 internal consistency result of students questionnaire instrument reliability statistics cronbach's alpha n of items .863 21 journal of science learning article doi: 10.17509/jsl.v3i2.19366 97 j.sci.learn.2020.3(2).89-98 that estimates the internal consistency reliability of an instrument by assigning how all items in the instrument relate to all other items and to the total instrument (croasmun & ostrom, 2011). the result shows the cronbach’s alpha is 0.825, which means the science teachers’ questionnaire instrument is sufficient reliability. in addition, based on table 7, the result of cronbach’s alpha shows that the student's questionnaire instrument is 0.863, which means sufficient reliability. therefore, the questionnaire instrument for science teachers and students is approved to be very reliable because it exceeds more than 0.6 as reliable standards. in this way, the web-based learning platforms are increasingly used, and the resources supporting students' academic writing is doubtless a developmental area (åberg, 2016). working with website education, like the presented one, gives opportunities for continuous development while the results of this study show that students and science teachers realize the potential of webbased learning resources.). 4. conclusion website education can be made by developing it in several steps: content analysis, material source analysis, user analysis, software necessity analysis, hardware necessity analysis, the design stage of website education that consists of the analysis of learning materials, flowchart, and storyboard. then, the development stage which consists of interface construction and coding to make the website learning. after that, it is evaluated by experts in three main aspects, which are content, language, and media. the revisions are taken from experts' suggestions; therefore, website education is evaluated by science teachers and junior high school students. based on the validation by the experts' that analyze using the acceptability of the website education, the evaluation overall indicates that there are indicators that need to be evaluated, and there is some indicator too that has been accepting by the experts. based on the science teachers' review, the applicable score is 91 out of 100, which is very good. the impression of junior high school students toward this mobile learning application based on questionnaire analysis gains a score of 82 out of 100, which means very good. and based on the internal consistency result using cronbach's alpha, the result for the science teacher questionnaire is 0.825, and the result for students' questionnaire is 0.863, both of the results shows that the questionnaire is approved to be very reliable because it exceeds more than 0.6 as reliable standards. finally, most of the students like learning science more after using the website education using an interactive content that contains the game as the instructional materials to learn levers in the human body topic because they previously known that learning science using a book and on way direction or lecturing method is boring, feel unmotivated. by using website education, the students feel motivated, fun, and able to assist the students in learning levers in the human body. yet, there are some recommendations for a future study regarding the development of the website education, such as it will be better to increase the generalizability of the finding, to make the subject that is not in the same schools it makes the generalizability of the findings questionable. acknowledgment the authors acknowledge dr. eka cahya prima for stimulating discussion about the appropriate technology that can be used for junior high school students. references åberg, e. s., ståhle, y., engdahl, i., & knutes-nyqvist, h. (2016). designing a website to support students' academic writing process. turkish online journal of educational technologytojet, 15(1), 33-42. alaswad, z., & nadolny, l. (2015). designing for game-based learning: the effective integration of technology to support learning. journal of educational technology systems, 43(4), 389-402. ardianto, d., & rubini, b. (2016). comparison of students' scientific literacy in integrated science learning through model of guided discovery and problem based learning. jurnal pendidikan ipa indonesia, 5(1), 31-37. asrizal, a., amran, a., ananda, a., & festiyed, f. (2018). effectiveness of adaptive, contextual learning model of integrated science by integrating digital age literacy on grade viii students. in iop conference series: materials science and engineering (vol. 335, no. 1, p. 012067). iop publishing. bolkan, s., goodboy, a. k., & kelsey, d. m. (2016). instructor clarity and student motivation: academic performance as a product of students’ ability and motivation to process instructional material. communication education, 65(2), 129-148. chang, m. m. (2007). enhancing web‐based language learning through self‐monitoring. journal of computer assisted learning, 23(3), 187196. chawinga, w. d., & zozie, p. a. (2016). increasing access to higher education through open and distance learning: empirical findings from mzuzu university, malawi. the international review of research in open and distributed learning, 17(4), 1-20. chou, c. (2003). interactivity and interactive functions in web‐based learning systems: a technical framework for designers. british journal of educational technology, 34(3), 265-279. cohen, l., manion, l., & morrison, k. (2011). planning educational research. research methods in education. new york: routledge editors. cook, d. a. (2007). web-based learning: pros, cons, and controversies. clinical medicine, 7(1), 37-42. cook, d. a., & dupras, d. m. (2004). a practical guide to developing effective web-based learning. journal of general internal medicine, 19(6), 698-707. croasmun, j. t., & ostrom, l. (2011). using likert-type scales in the social sciences. journal of adult education, 40(1), 19-22. davis, f. d., bagozzi, r. p., & warshaw, p. r. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35(8), 982-1003. dogan, b., & dikbıyık, e. (2016). opcomits: developing an adaptive and intelligent web-based educational system based on the concept map model. computer applications in engineering education, 24(5), 676-691. drake, s. m., & savage, m. j. (2016). negotiating accountability and integrated curriculum from a global perspective. international journal of learning, teaching, and educational research, 15(6), 127-144. journal of science learning article doi: 10.17509/jsl.v3i2.19366 98 j.sci.learn.2020.3(2).89-98 du, s., liu, z., liu, s., yin, h., xu, g., zhang, h., & wang, a. (2013). web-based distance learning for nurse education: a systematic review. international nursing review, 60(2), 167-177. fraenkel, j. r., wallen, n. e., & hyun, h. h. (2013). how to design and evaluate research in education. new york: mcgraw-hill humanities/social sciences/languages. goh, w. w., bay, s., & chen, v. h. h. (2015). young school children’s use of digital devices and parental rules. telematics and informatics, 32(4), 787-795. ha, l., & james, e. l. (1998). interactivity reexamined: a baseline analysis of early business web sites. journal of broadcasting & electronic media, 42(4), 457-474. hesti, r., maknun, j., & feranie, s. (2017). text-based analogy in overcoming student misconception on simple electricity circuit material. in journal of physics: conference series (vol. 895, no. 1, p. 012146). iop publishing. holzinger, a., emberger, w., wassertheurer, s., & neal, l. (2008). design, development and evaluation of online interactive simulation software for learning human genetics. e & i elektrotechnik und informationstechnik, 125(5), 190-196. huang, x., chandra, a., depaolo, c. a., & simmons, l. l. (2016). understanding transactional distance in web-based learning environments: an empirical study. british journal of educational technology, 47(4), 734-747. keengwe, j., & anyanwu, l. o. (2007). computer technology-infused learning enhancement. journal of science education and technology, 16(5), 387-393. kenny, a. (2000). untangling the web; barriers and benefits for nurse education; an australian perspective. nurse education today, 20(5), 381-388. khalifa, m., & lam, r. (2002). web-based learning: effects on learning process and outcome. ieee transactions on education, 45(4), 350356. lykke, m., coto, m., mora, s., vandel, n., & jantzen, c. (2014). motivating programming students by problem-based learning and lego robots. in 2014 ieee global engineering education conference (educon) (pp. 544-555). ieee. norman, k. l. (2000). desktop distance education: personal hosting of web courses. in web-based learning and teaching technologies: opportunities and challenges (pp. 117-134). igi global. paulsen, m. f. (2003). on-line education and learning management. global e-learning from a scandinavian perspective. purpura, j. e. (2013). assessing grammar. the companion to language assessment, 1, 100-124. pursitasari, i. d., nuryanti, s., & rede, a. (2015). promoting of thematic-based integrated science learning on the junior high school. journal of education and practice, 6(20), 97-101. seifert, t. (2004). understanding student motivation. educational research, 46(2), 137-149. srivastava, j., cooley, r., deshpande, m., & tan, p. n. (2000). web usage mining: discovery and applications of usage patterns from web data. acm sigkdd explorations newsletter, 1(2), 12-23. voogt, j., erstad, o., dede, c., & mishra, p. (2013). challenges to learning and schooling in the digitally networked world of the 21st century. journal of computer-assisted learning, 29(5), 403-413. walsh, a. (2014). seek!: creating and crowdfunding a game-based open educational resource to improve information literacy. insights, 27(1), 63-67. wang, m., cheng, b., chen, j., mercer, n., & kirschner, p. a. (2017). the use of web-based collaborative concept mapping to support group learning and interaction in an online environment. the internet and higher education, 34, 28-40. warmbrod, j. r. (2014). reporting and interpreting scores derived from likert-type scales. journal of agricultural education, 55(5), 3047. yuliati, l. (2013). efektivitas bahan ajar ipa terpadu terhadap kemampuan berpikir tingkat tinggi siswa smp. jurnal pendidikan fisika indonesia, 9(1), 53-57. zaiane, o. (2001). web usage mining for a better web-based learning environment. retrived from https://era.library.ualberta.ca/items/0a182195-ce39-4b5d-a1c1291ed91a0f36/view/336c2a34-b149-4d9d-95a1ad78c08ee35c/tr01-05.pdf. zapalska, a., & brozik, d. (2006). learning styles and online education. campus-wide information systems, 23(5), 325-335. a © 2022 indonesian society for science educator 79 j.sci.learn.2022.5(1).79-90 received: 5 july 2021 revised: 9 august 2021 published: 1 march 2022 development of the stem-pedagogical content knowledge scale for preservice teachers: validity and reliability study behiye akçay1*, filiz avcı1 1istanbul universty cerrahpaşa, turkey *corresponding author. behiye.akcay@iuc.edu.tr abstract this study aims to develop a valid and reliable scale to determine pre-service teachers' stem-pedagogical content knowledge (stem-pck) levels. this study was conducted in the 2018-2019 academic year with 322 pre-service teachers in turkey. in the study, one of the mixed method typologies, exploratory sequential design, was applied. the scale was submitted for evaluation by four experts to determine the content and face validity. exploratory factor analysis (efa) and confirmatory factor analysis (cfa) was performed to determine the scale's construct validity. as a result of efa, the scale had five facto rs: stem pedagogical knowledge, pedagogical knowledge, engineering pedagogical knowledge, mathematics pedagogical knowledge, and science pedagogical knowledge. as a result of cfa χ²/df = 2.71, rmsea = .07, rmr = 0.04, srmr = .07, nfi = .94, nnfi = .96, cfi = .96, ifi = .96, rfi = .94 values were reached and the factor structure determined to be suitable. to determine the reliability of the scale, internal consistency and test-retest reliability analyzes were made. the internal consistency reliability value of the scale was found as .98. the final form of the stem-pck scale is a 5-point likert type that includes 57 items and five factors. keywords stem, pedagogical content knowledge, validity, reliability 1. introduction science, technology, engineering, and mathematics (stem) has been a growing area since president barack obama shared the united states national dialogue in 2011 (epstein & mille, 2011). they were improving stem research studies focused mainly on preparing students for stem-related careers and improving stem literacy for all students. the studies on stem is showed that teachers' effectiveness has a substantial effect on students’ achievement in stem subjects (knezek, christensen & tyler-wood, 2015; york, 2018). in addition, wahono, lin, & chang (2020) explained that stem enactment positively affects asian students’ learning outcomes in academic learning achievement, higher-order thinking skills, and motivation. science teachers should know how to combine many different types of knowledge from science disciplines for effective science instruction for stem areas (schmidt & fulton, 2015). each science discipline requires important content of scientific knowledge and subject-specific pedagogy (york, 2018). because teachers’ knowledge of a particular subject matter, which includes pedagogical content knowledge (pck) and content knowledge (ck), are vital components of teacher education programs (kleickmann et al., 2013). stem-related research showed that pre-service teachers are not experts in stem subjects because of inadequate content knowledge and low selfconfidence to teach stem (epstein & miller, 2011; york, 2018). therefore, pre-service teachers need a more robust stem education (srisawasdi, 2012). most teacher education programs include content courses for chemistry, biology, physics, astronomy, technology, earth sciences, and science teaching methods systems. however, york (2018) argued that to know theoretical knowledge of different disciplines is not enough to understand how to integrate and teach in actual classroom instruction. therefore teacher education programs must prepare pre-service teachers about how to lead a specific content knowledge at the level of their students in terms of pedagogy and inquiry. also, teacher journal of science learning article doi: 10.17509/jsl.v5i1.36293 80 j.sci.learn.2022.5(1).79-90 education programs should support teachers’ pck for stem (srikoom, faikhamta & hanuscin, 2018). srikoom, faikhamta & hanuscin (2018) explained that the components of pck for stem include how teachers comprehend the place of stem in science education curriculum and educational materials, how teachers conceptualized stem education. integrating stem into science education and teaching practices in return the complex multi-dimensional process with many dimensions such as scientific knowledge, mathematical knowledge, technological knowledge, engineering knowledge, content knowledge, pedagogical knowledge become important (sarkim, 2020). initially, pedagogical content knowledge (pck) propose by shulman in 1986. shulman explained three categories of content knowledge, including subject-matter (content) knowledge, subject-matter pedagogical knowledge, and curricular knowledge (kind, 2009). magnusson, krajcik, & borko (1999) explained that pck requires “…teacher’s understanding of how to help students understand the specific subject matter. it includes knowledge of how particular subject matter topics, problems, and issues can be organized, represented, and adapted to the diverse interests and abilities of learners, and then presented for instruction.” (p. 96). pck is unique to each subject, context, and teacher because they have different knowledge about teaching and how to teach the subject matter knowledge to their students (park, suh & seo, 2018). studies showed that qualified science subjects significantly affect students’ attitudes and career choices toward science-related fields (knezek, christensen & tyler-wood, 2015). developing teachers’ stem pedagogical knowledge and proficiency with stemspecific pedagogical strategies would increase their selfconfidence to teach stem and, in turn, would positively affect their students’ career choices in stem-related fields (york, 2018). in recent years, studies on stem education (corlu, capraro, & corlu, 2015; derin, aydın & kirkiç, 2017) have focused mainly on teachers and pre-service teachers. eldeghaidy & mansour (2015) argued that to promotes effective stem education, we need to identify teachers' content knowledge and pedagogical content knowledge to enact stem education in class. therefore, this study aims to develop a valid and reliable scale that can determine the pedagogical content knowledge of pre-service teachers about what, when, why, and how they teach stem subjects and contributes to the field. 2. method 2.1 research model the study aimed to develop a valid and reliable scale to determine teacher candidates' science, technology, engineering, and mathematics-pedagogical content knowledge (stem-pck) levels. to achieve this aim, exploratory sequential design, one of the mixed-method research typologies in which qualitative and quantitative research methods were used together, was used in the study (tashakkori & creswell, 2007). in the exploratory sequential design, after the qualitative data are collected and analyzed, the data are tested and diagnosed with quantitative methods (creswell & clark, 2011). the pattern scheme of the research design shows in figure 1. figure 1 scale development process collection of qualitative data • determining the need • literature research • creating an item pool analysis of qualitative data • taking expert opinion • first form the scale collection of quantitative data • applying the pilot study for item selection • determining the sample analysis of quantitative data • performing statistical analysis for the application of the rearranged scale and for item selection • exploratory factor analysis (spss) confirmatory factor analysis(lisrel) reliability calculations interpretation • creating the final form of the scale journal of science learning article doi: 10.17509/jsl.v5i1.36293 81 j.sci.learn.2022.5(1).79-90 2.2. scale development process in this study, initial scale development steps explained by seçer (2015) were used. these stages are(1) determining the need; (2) literature search and creating an item pool; (3)taking expert opinion; (4) first form of the scale; (5) applying the pilot study for item selection; (6) determining the sample group; (7) performing statistical analysis for the application of the rearranged scale and item selection; (8) creating the final form of the scale. 2.2.1. determining the need when the literature examined, it is seen that in recent years, scale development studies in the fields of pck and tpck (technological pedagogical content knowledge) for teachers and pre-service teachers are frequently used (graham et al., 2009; schmidt et al., 2009; landry, 2010; marks, 1990; sun & strobel, 2014). however, when the literature was examined, a limited number of studies were found that measure the pedagogical content knowledge of teachers and pre-service teachers towards stem (rigelman, 2014; yildirim & sahin-topalcengiz, 2019). while rigelman (2014) developed a rubric to assess teachers’ stem pedagogical content knowledge, yildirim & sahin-topalcengiz (2019) developed a 5-point likert type stem pck scale for pre-service teachers. it has six factors, including 21st-century skills knowledge, pedagogical knowledge, mathematical knowledge, science knowledge, engineering knowledge, and technology knowledge. however, this scale doesn’t include a specific dimension to assess pre-service teachers' stem pck. instead, it has subject matter knowledge dimensions for science, mathematics, engineering, and technology. this study aimed to develop a likert-type scale to measure the stem pedagogical knowledge (stem knowledge for teaching), pedagogical knowledge, engineering pedagogical knowledge, mathematics pedagogical knowledge, and science pedagogical knowledge levels pre-service teachers. i hope that the stem-pck scale, which aims to develop in this study, will play a leading role in determining the stem pck levels of pre-service teachers. 2.2.2. literature research and creating an item pool while creating the scale items developed to determine pre-service teachers' stem pck levels, first examined the theoretical structure with a wide-ranging literature review in pck and tpck. as a result of the literature research (aydin-gunbatar, boz & yerdelen-damar, 2017; schmidt et al. 2009; tiryaki, 2018; pamuk, ulken, & dilek 2012; graham et al., 2009; canbazoglu-bilici, yamak, kavak & guzey., 2013; tyler-wood, knezek & christensen 2010; srikoom, hanuscin & faikhamta, 2017) an item pool consisting of 87 items was created. the draft form of the stem-pck scale was prepared as 5-likert (1: strongly disagree, 2: disagree, 3: undecided, 4: agree, 5: strongly agree) type presented to expert opinion. the item pool prepared to ensure the content and face validity was submitted to the evaluation of three field experts working in the science education department at three different universities in turkey and a turkish language expert who teaches turkish in a public secondary school in turkey to be examined in terms of compliance with the spelling rules (table 1). experts ask to indicate in table 2 their answers regarding the eligibility of the items for the scale. next, experts ask to mark the "appropriate " section on the table if the thing is appropriate and the "not appropriate " area if it is not applicable and write their explanations in the "explanation " section. finally, it states that experts could add the new items suggested to the scale to the table. 2.2.3. first form of the scale in line with the experts, two items (6th and 53rd items) were excluded from the item pool as it was stated that they were not directed to the feature intended to measure. items excluded from the scale: item 6: “i have sufficient opportunities to work on each of the stem fields.” item 53: "i can apply/teach a lesson plan that includes (combines) science, technology, engineering and mathematics subjects." in the feedback from experts, it was also determined that some items were not clearly expressed and understood, some items could not measure the behavior they wanted to measure adequately, and some things could not focus on the behavior they wanted to measure, and 2, 4, 7, 8, 11, 12 ., 13., 14., 15., 16., 17., 18., 23., 24., 25., 26., 27., 28., 29., 30., 31., 32., 33., 34., 44., 54., 58., 75. and 79. i̇tems were rearranged. examples of mixed items: table 1 demographic characteristics of the expert group gender title department university female associate professor science education marmara university female associate professor science education boğaziçi university male assistant professor science education kafkas university female teacher turkish mone turkish teacher table 2 form prepared for expert opinion scale items appropriate not appropriate explanation 1 2 journal of science learning article doi: 10.17509/jsl.v5i1.36293 82 j.sci.learn.2022.5(1).79-90 item 2: the item is written as "i know the importance of stem in science education" was arranged as "i know the importance of stem in education." item 58: the item is written as "while evaluating the stem teaching process, i can develop measurement/evaluation tools appropriate to the subject." it was arranged as "i can prepare the appropriate stem measurement tools for the evaluation process." the draft scale form with 85 items was made ready for pilot application after the expert turkish teacher checked the scale in terms of spelling and language. 2.2.4. applying the pilot study for item selection a pilot study carries out to determine whether there were any unnoticed expressions or format problems on the form and items of the draft scale created before the general applications. the pilot application was carried out by the researchers on a group of 15 senior pre-service science teachers. according to the application results, necessary corrections made on the items decide that 20 minutes are required to implement the scale. 2.2.5. determining the sample group the study population consists of pre-service teachers from 1., 2., 3. and 4. grades that's a study in turkey during 2018-2019. the sample of the study consisted of 322 teacher candidates in different grade levels (1., 2., 3. and 4. grades) from six (6) other teaching departments in seven (7) different universities in turkey using a purposeful sampling method. the scale was prepared online, for which the first draft form create after the pilot study. in the study, the stempck draft scale form link was sent as an e-mail to the preservice teachers in the sample group. pre-service teachers who received the invitation were provided with access to the online scale form to fill out on demand. as a result of the study, 322 teacher candidates filled the scale. the teacher candidates' 274 (85.1%) are female, and 48 (14.9%) are male. the information about the university and departments of the teacher candidates, who constitute the research study group, is given in table 3. 3. result and discussion 3.1. validity studies 3.1.1. content validity within the scope of the content validity, three field experts consulted for the quantitative and qualitative evaluation of the items in the scale. as a result of an assessment of the field experts, the things they found appropriate, unsuitable, and suggested review examines. intercoder reliability among experts is according to miles & huberman (1994). it calculates with the formula (reliability = number of agreements / (number of the accords + disagreements) x 100), and this ratio determines as .87. 3.1.2. face validity for the face validity of the scale, the scale evaluates by three experts. relevant items remove from the scale according to their feedback. after the necessary corrections, an explanation section was prepared on the front page of the hierarchy about what purpose the ranking will be used for, what it aims to measure, and how many items it consists of. later, the scale was applied to pretable 3 descriptive information about the students in the study group f % university i̇stanbul university -cerrahpasa 169 52.5 marmara university 49 15.2 yıldız teknik university 55 17.1 kafkas university 28 8.7 boğaziçi university 12 3.7 recep tayyip erdoğan university 5 1.6 uludağ university 4 1.2 gender female 274 85.1 male 48 14.9 department science education 185 57.5 mathematics education 33 10.2 primary education 66 20.5 social studies education 5 1.6 physics education 25 7.8 chemistry education 8 2.5 grade freshman 1 0.3 sophomore 16 5 junior 218 40.3 senior 87 16.1 total participant 322 journal of science learning article doi: 10.17509/jsl.v5i1.36293 83 j.sci.learn.2022.5(1).79-90 service teachers within the scope of a pilot study, and it was aimed to determine the incomprehensible items from the items in the hierarchy. 3.1.3. construct validity (factor analysis) factor analysis was carried out to reveal the implicit structure of the measuring tool and to determine the subdimensions of the scale by bringing together the related items. in scale development studies, exploratory factor analysis (efa) is performed to reveal the implicit structure of the scale, while confirmatory factor analysis (cfa) is committed to verifying this implicit structure. exploratory factor analysis (efa). exploratory factor analysis (efa) was carried out to determine the sub-dimensions of the items related to each other in the scale and the relationship between them. developing the scale first determines whether the sample size to which the scale is applied is sufficient for factor analysis and whether multivariate normality is achieved (buyukozturk, 2007). the efa result calculated the kaiser-meyer-olkin (kmo) test result as .94 (table 4). according to kaiser (1974), values between .00 and .49 are unacceptable, .50 and .59 are miserable, .60 and .69 are mediocre, .70 and .79 are middling, .80 and .89 are meritorious, and .90 and 1.00 are marvelous. for this reason, the kmo test result obtained shows that the sample size is suitable for factor analysis of the available data. because as the value obtained approaches 1.00, the adequacy of the sample size increases (seçer, 2015). also, the significance of barlett's test of sphericity (bartlett's test of sphericity) indicates that the data set provides multivariate normality (14118.21, p <.05) (seker & gencdogan, 2014). there are 85 items in the first form of the stem-pck scale, creates due to the pilot study. as a result of the factor analysis, five factors were found with an eigenvalue more significant than one and explaining 60.38% of the total variance. the first factor explains 22.52%, the second factor 12.44%, the third factor 9.39%, the fourth factor 8.60%, and the fifth factor 7.43% (table 5). table 4 kmo and bartlett’s test values for stem-pck kaiser-meyer-olkin measure of sampling adequacy (kmo) bartlett's test of sphericity approx. chisquare df p .94 14118.21 1596 .00 table 5 total variance explained results for stem-pck scale factors total eigenvalues rotation sums of squared loadings (% of variance) rotation sums of squared loadings (cumulative %) f1 12.84 22.52 22.52 f2 7.09 12.44 34.96 f3 5.35 9.39 44.36 f4 4.90 8.60 52.96 f5 4.23 7.43 60.38 *extraction method: principal component analysis table 6 rotated component matrix factors item number f1 f2 f3 f4 f5 q46 .80 q53 .79 q47 .78 q45 .77 q48 .77 q43 .76 q57 .74 q49 .74 q56 .72 q52 .72 q50 .72 q61 .71 q55 .70 q59 .69 q54 .69 q62 .69 q42 .68 q60 .65 q58 .61 journal of science learning article doi: 10.17509/jsl.v5i1.36293 84 j.sci.learn.2022.5(1).79-90 the scree plot chart (figure 2) examines five vertical breaks with an eigenvalue greater than 1. therefore, according to these results, it evaluates that the scale had five factors. the resulting component matrix table was examined at the end of the analysis with a factor load value of at least .32 and above for each item. since the sub-dimensions in the measurement tool were thought to be unrelated to each other, varimax rotation was performed, which is one of the orthogonal rotation techniques. items with a difference of less than .10 between factor loading values were evaluated as overlapping items, and as a result of the analysis, 1-2-34-5-6-7-9-12-16-21-22-25-28-30items 33-37-41-74-7577-78-79-81-82-83-84 and 85 were excluded from the scale. when the rotated component matrix table (table 6) was table 6 rotated component matrix (continued) factors item number f1 f2 f3 f4 f5 q51 .59 q63 .57 q44 .57 q80 .52 q8 .49 q70 .80 q71 .79 q73 .74 q66 .72 q65 .72 q64 .70 q72 .70 q69 .69 q67 .65 q68 .63 q76 .59 q29 .78 q17 .75 q32 .75 q24 .72 q13 .71 q36 .70 q40 .69 q20 .67 q15 .80 q18 .76 q27 .73 q34 .72 q11 .71 q39 .70 q31 .66 q23 .64 q14 .77 q19 .77 q10 .76 q26 .68 q35 .65 q38 .55 figure 2 scree plot chart of the scale journal of science learning article doi: 10.17509/jsl.v5i1.36293 85 j.sci.learn.2022.5(1).79-90 examined, it was seen that the load value was .32 and above, and the items did not have overlapping properties. when the factor load values in the sub-dimensions are examined, they were determined that the load values in the first subdimension are between .49 and .80, the load values in the second sub-dimension are between .80 and .59, the load values in the third sub-dimensions are between .78 and .67, the load values in the fourth sub-dimensions are between .80 and .64, the load values in the fifth sub-dimensions are between .77 and .55. when the data obtained as a result of efa are evaluated, the stem-pck scale, the first version of 85 items, revealed its implicit structure consisting of 57 items and five sub-dimensions (table 7). confirmatory factor analysis (cfa). confirmatory factor analysis (cfa) is an analysis performed to test and verify sub-dimensions determined by efa (seçer, 2015). it is seen that the x2/df value of the model obtained as a result of the cfa analysis is less than 3, so it provides the acceptable fit value. in addition, nnfi (non-normed fit index), ifi (incremental fit index), rmsea (root mean square error of approximation), and srmr (standardized root mean square residual) values meet the criteria of good fit; nfi (normed fit index), rfi (relative fit index), cfi (comparative fit index) and rmr (root mean square residual) values were evaluated to be in acceptable compliance ranges. these values show that the model tested for the five-factor structure of the stem-pck scale examined within the scope of cfa fits well (table 8). table 8 data has been interpreted according to the eligibility criteria of seçer (2015). in figure 3, t values of the model are shown. the t values of the model obtained from the stem-pck scale as a result of cfa were examined. since all the values are higher than 1.96, all variables determined that path in the model are significant. therefore, it evaluates that no item was incompatible with other things and had an excellent validity (figure 3). in figure 4, standardized solutions of cfa shows. in this figure, the factor load value of each item has a load value of at least .30 or more was examined. the path diagram given in figure 4 below review shows that the scale factor load values are at the desired level, between .32 and .73. therefore it concludes that standard factor loadings have an effect on each factor on variable’s variance. 3.2. reliability studies to determine the reliability of the stem-pck scale, the internal consistency coefficient was calculated, and testretest reliability analysis was performed. the test-retest method applies a scale to the same subject group twice under the same conditions and within a specific time interval. the correlation coefficient of the measurement values obtained from the two applications is the reliability coefficient of the scale. for the test-retest reliability study, the scale was applied to 34 pre-service teachers twice with an interval of 21 days. the cronbach alpha for the total stem-pck scale was .98. the coefficients for the factors ranged from .89 for science pedagogical knowledge and .97 for stem pedagogical knowledge. cronbach alpha reliability analysis results obtained after the application are given in table 9. table 7 distribution of stem-pck scale items by factors at the end of efa analysis factors sub-dimensions scale items after efa number of items f1 stem pedagogical knowledge (stem knowledge for teaching) 8,42,43,44,45,46,47,48,49,50,51,52,53,54, 55,56,57,58,59,60,61,62,63,80 24 f2 pedagogical knowledge 64,65,66,67,68,69,70,71,72,73,76 11 f3 engineering pedagogical knowledge 13,17,24,29,32,36,40,20 8 f4 mathematics pedagogical knowledge 15,18,27,34,11,39,31,23 8 f5 science pedagogical knowledge 14,19,10,26,35,38 6 total 57 table 8 fit indices values for stem-pck scale model fit indices acceptable fit value perfect fit value research findings interpretation nfi =.90 and above =.95 and above 0.94 acceptable fit nnfi =.90 and above =.95 and above 0.96 perfect fit ifi =.90 and above =.95 and above 0.96 perfect fit rfi =.90 and above =.95 and above 0.94 acceptable fit cfi =.95 and above =.97and above 0.96 acceptable fit rmr =.05 and <.05 =.08 and<.08 0.04 acceptable fit rmsea =.05 and <.05 =.08 and <.08 0.07 perfect fit srmr =.05 and <.05 =.08 and <.08 0.07 perfect fit x2/df 2-3 0-2 2.71 acceptable fit journal of science learning article doi: 10.17509/jsl.v5i1.36293 86 j.sci.learn.2022.5(1).79-90 in this study paired sample t-test is used to determine whether the mean difference between two repeated measures is zero. also, according to the results of paired samples t-test performed to determine whether there is a significant difference between the two applications, it was observed that there was no significant difference between the first application and the last application (t = -0.91, p> 0.05). this result shows that the stem-pck scale gives similar results when applied to the same group at different times (table 10). 3.3. creating the final form of the scale the cronbach alpha (α) internal consistency reliability value of the stem-pedagogical content knowledge (stem-pck) scale was determined as .98. the internal consistency coefficients of the sub-dimensions of the scale were respectively .97 for stem pedagogical knowledge dimension, .92 for pedagogical knowledge dimension, .91 for engineering pedagogical knowledge dimension, .90 for mathematics pedagogical knowledge dimension, and .89 for the science pedagogical knowledge dimension. test-retest reliability founds to be .97 for the scale total. in addition, test-retest reliability of the sub-dimensions of the scale were respectively .97 for the stem pedagogical knowledge dimension, .93 for the pedagogical knowledge dimension, .90 for the engineering, .92 for the mathematics pedagogical knowledge dimension, and .83 for the science pedagogical knowledge dimension. cronbach alpha reliability coefficient value of .70 and above is considered sufficient for the reliability of test scores (buyukozturk, 2007). based on this, it can be said that the results to be obtained from the scale are highly reliable. also, correlations between factors were calculated, and the results are given in table 11 when table 11 examines, it determined that there is no correlation between f2 (pedagogical knowledge) and f3 (engineering pedagogical knowledge) (r = .29, p> .05). also, it seems that the lowest correlation is between f4 (mathematics pedagogical knowledge) and f2 (pedagogical knowledge), and the highest correlation is between f3 (engineering pedagogical knowledge) and f4 (mathematics pedagogical knowledge). 3.4 discussion in our age, developments in information and technology cause different needs along with it. thereservice teachers are expected to have knowledge about science, technology, and engineering, to be able to use the knowledge they have learned and to integrate it with other subjects, as well as to support their technology knowledge table 9 internal consistency and test-retest reliability analysis factors factors’name the final version of scale items number of items internal consistency reliability test-retest reliability f1 stem pedagogical knowledge (stem knowledge for teaching) 1,24,25,26,27,28,29,30,31, 32,33,34,35,36,37,38,39,40 ,41,42,43,44,45,57 24 .97 .97 f2 pedagogical knowledge 46,47,48,49,50,51,52,53,54 ,55,56 11 .92 .93 f3 enginering pedagogical knowledge 4,7,10,12,15,17, 20,23 8 .91 .90 f4 mathematics pedagogical knowledge 3,6,8,11,14,16,18,22 8 .90 .92 f5 science pedagogical knowledge 2,5,9,13,19,21 6 .89 .83 total of scale 57 .98 .97 table 10 test-retestresults of the stem-pck scale application n mean sd df t p 1. application 34 3.93 .59 33 -.91 .37 2. application 34 4.02 .46 33 table 11 correlations between factors (pearson correlation) (n=34) f1 f2 f3. f4 f5 f1 correlation 1 .63** .56** .64** .44* f2 correlation .63** 1 .29 .34* .71** f3 correlation .56** .29 1 .77** .41* f4 correlation .64** .34* .77** 1 .41* f5 correlation .44* .71** .41* .41* 1 ** correlation is significant at the level (2-tailed) *correlation is significant at the 0.005 level (2-tailed) journal of science learning article doi: 10.17509/jsl.v5i1.36293 87 j.sci.learn.2022.5(1).79-90 with content knowledge and field-specific pedagogical method knowledge (mishra & koehler, 2006). for this purpose, studies focused on developing a scale on stem knowledge (corlu, capraro, & corlu, 2015; derin, aydın & kirkiç, 2017) and pedagogical content knowledge (graham et al., 2009; schmidt et al., 2009; landry, 2010; marks, 1990; sun & strobel, 2014). figure 3 t-values of stem-pck scale journal of science learning article doi: 10.17509/jsl.v5i1.36293 88 j.sci.learn.2022.5(1).79-90 for this reason, integrating stem into the teaching of teacher candidates has become an academically valuable issue. therefore, the study aimed to develop a valid and reliable scale to determine pre-service teachers' stempedagogical content knowledge (stem-pck) levels. therefore, a validity and reliability study of the draft scale consisting of 85 items carried out with 322 teacher candidates in different grade levels (1., 2., 3. and 4. grades) figure 4 standardized solution of stempck scale journal of science learning article doi: 10.17509/jsl.v5i1.36293 89 j.sci.learn.2022.5(1).79-90 from six (6) other teaching departments in seven (7) different universities in turkey. the likert-type scale is preferred chiefly among the scale types because it is practical, increases the grading level, and gives measurement results in an equal-interval scale (tezbasaran, 2008). the items of the stem-pck scale prepare as 5-likert type. as a result of the efa, it determined that the kmo value for the stem-pck scale was .94, and bartlett's test result was significant. these results showed that the sample size was suitable for factor analysis. the data set provided multivariate normality as a result of the varimax rotation, the implicit structure of the stem-pck scale consisting of 57 items and five subdimensions reveal. the stem-pck scale's subdimensions determined are stem pedagogical knowledge, pedagogical knowledge, engineering pedagogical knowledge, mathematics pedagogical knowledge, and science pedagogical knowledge. yildirim & sahin-topalcengiz (2019) developed a stempck scale consist of six factors. this scale has only one subdimension, which is pedagogical knowledge, similar to the stem-pck scale. this study aimed to measure pre-service teachers’ pedagogical knowledge in the area of science, mathematics, engineering, and stem, while yildirim & sahin-topalcengiz’s (2019) stempck scale only one dimension to measure the pedagogical knowledge level of pre-service teachers. the 5-factor structure of the scale was confirmed by cfa analysis (χ ² /df=2.71, rmsea = .07, rmr = 0.04, srmr = .07, nfi = .94, nnfi = .96, cfi = .96, ifi = .96. rfi = .94). internal consistency and test-retest reliability analyze conducted to determine the reliability of the scale. as a result, the cronbach's alpha (α) internal consistency reliability value of the scale was .98, and the test-retest reliability value was .97. correlation between 5 factors was calculated in the stem-pck scale, and it was determined that the highest correlation was between engineering pedagogical knowledge and mathematics pedagogical knowledge. conclusion it can be said that the scale, which is developed as a result of validity and reliability analysis, consists of 57 items and five factors in 5-point likert type, is a valid and reliable measurement tool for determining the stem-pedagogical content knowledge (stem-pck) levels of pre-service teachers. with the developed scale, it is thought that it will help measure the pedagogical content knowledge toward stem education, which is spreading rapidly all over the world, of teacher candidates who are educated in national and international fields. references aydin-gunbatar, s., boz, y., & yerdelen-damar, s. (2017). a closer examination of tpack-self-efficacy construct: modeling elementary pre-service science teachers’ tpack-self efficacy. elementary education online, 16(3), 917-934. buyukozturk, s. (2007). sosyal bilimler için very analizi el kitabı (7. baskı).[data analysis handbook for social sciences (7th edition)] pegem a yayıncılık. canbazoglu-bilici, s., yamak, h., kavak, n., & guzey, s. s. (2013). technological pedagogical content knowledge self-efficacy scale (tpack-ses) for pre-service science teachers: construction, validation, and reliability. eurasian journal of educational research, 13(52), 37-60. creswell, j. w., & clark, v. l. p. (2017). designing and conducting mixed methods research. sage publications. corlu, s., capraro, r.m., & çorlu, m. a. (2015). investigating the mental readiness of pre-service teachers for integrated teaching. international online journal of educational sciences, 7 (1), 17-28. derin, g., aydin, e., & kirkiç, k. a. (2017). stem (fen-teknolojimühendislik–matematik) eğitimi tutum ölçeği. [stem (sciencetechnology-engineering-mathematics) education attitude scale].el-cezeri journal of science and engineering, 4(3), 547-559. el-deghaidy, h., & mansour, n. (2015). science teachers’ perceptions of stem education: possibilities and challenges. international journal of learning and teaching, 1(1), 51-54. epstein, d., & miller, r. t. (2011). slow off the mark: elementary school teachers and the crisis in science, technology, engineering, and math education. education digest: essential readings condensed for quick review, 77(1), 4–10. graham, r. c., burgoyne, n., cantrell, p., smith, l., st clair, l., & harris, r. (2009). measuring the tpack confidence of in-service science teachers. techtrends, 53(5), 70-79. kleickmann, t., richter, d., kunter, m., jürgen, e., besser, m., krauss, s. & baumert, j. (2013). teachers’content knowledge and pedagogical content knowledge: the role of structural differences in teacher education. journal of teacher education, 64(1), 90-106. kaiser, h. f. (1974). an index of factorial simplicity. psychometrika, 39(1), 31-36. knezek, g., christensen, r. & tyler-wood, t. (2015). teacher dispositions toward science, technology, engineering, and mathematics (stem). in liu, l., & gibson, d.c. (ed.), research highlights in technology and teacher education (p.45-53). aace. kind, v. (2009) pedagogical content knowledge in science education: perspectives and potential for progress. studies in science education, 45(2), 169-204. landry, g. a. (2010). creating and validating an instrument to measure middle school mathematics teachers’ technological pedagogical content knowledge (tpack) (unpublished ph.d. thesis). university of tennessee, knoxville. magnusson, s., krajcik, j., & borko, h. (1999). nature, sources, and development of pedagogical content knowledge for science teaching. in j. gess-newsome & n. g. lederman (eds.), examining pedagogical content knowledge: the construct and its implications for science education (pp. 95-132). kluwer academic publishers. marks, r. (1990). pedagogical content knowledge: from a mathematical case to a modified conception. journal of teacher education, 41(3), 311. miles, m. b., & huberman, a. m. (1994). qualitative data analysis: an expanded sourcebook (2nd ed.). sage publishers. mishra, p., & koehler, m. j. (2006). technological pedagogical content knowledge: a framework for teacher knowledge. teachers college record, 108(6), 1017-1054. pamuk, s., ulken, a., & dilek, n. (2012). the investigation of preservice teachers' technology integration competencies from technological pedagogical cont. mustafa kemal üniversitesi sosyal bilimler enstitüsü dergisi [journal of mustafa kemal university institute of social sciences] , 9(17), 415-438. park, s., suh, j. & seo, k. (2018). development and validation of measures of secondary science teachers’ pck for teaching photosynthesis. research in science education, 48, 549-573. journal of science learning article doi: 10.17509/jsl.v5i1.36293 90 j.sci.learn.2022.5(1).79-90 rigelman, n. m. (2014). using rich assessment and the atem pedagogical content knowledge rubric to examine teacher learning. paper presented at oregon's teacher of teachers of mathematics (totom) conference. https://www.slideshare.net/nicolerigelman/usingrich-assessments-and-the-pck rubric sarkim, t. (2020). developing teachers’ pck about stem teaching approach through the implementation of design research. journal of physics: conference series, 1470, 1-8. schmidt, d. a., baran, e., thompson, a. d., mishra, p., koehler, m. j., & shin, t. s. (2009). technological pedagogical content knowledge (tpack) the development and validation of an assessment instrument for pre-service teachers. journal of research on technology in education, 42(2), 123-149. schmidt, m. & fulton, l. (2015). lessons learned from the creation of an exemplary stem unit for elementary pre-service teachers: a case study. in liu, l., & gibson, d.c. (ed.), research highlights in technology and teacher education(p. 53-60). aace (association for the advancement of computing in education). seker, h & gencdogan, b. (2014). psikolojide ve egitimde ölcme aracı gelistirme (2nd ed.). nobel yayıncılık. [developing a measurement tool in psychology and education (2nd ed.). nobel publishing.] srisawasdi, n. (2012). fostering pre-service stem teachers’ technological pedagogical content knowledge: a lesson learned from case-based learning approach. journal of korea association science education, 32(8), 1356-1366. srikoom, w., hanuscin, d. l., & faikhamta, c. (2017)). perceptions of in-service teachers toward teaching stem in thailand. asia-pacific forum on science learning and teaching,18, (2), 1-23. srikoom, w., faikhamta, c. & hanuscin, d.l. (2018). dimensions of effective stem integrated teaching practice. k-12 stem education, 4(2), 313-330. sun, y., & strobel, j. (2014). from knowing-about to knowing-to: development of engineering-pedagogical content knowledge by elementary teachers through perceived learning and implementing difficulties. american journal of engineering education (ajee), 5(1), 41-60. tashakkori, a., & creswell, j.w. (2007). the new era of mixed methods. journal of mixed methods [editorial]. journal of mixed methods research, 1(1), 3-7. tezbasaran, a. (2008). likert tipi ölcek hazırlama kılavuzu. [likert type scale preparation guide. turkish psychological association publications]. ankara: turk psikologlar dernegi yayınları tiryaki, s. h. (2018). öğretmenlerin teknolojik pedagojik alan bilgisi (tpab) ve eğitim bilişim ağı’nı kullanmalarına yönelik özyeterlik algılarının düzeylerinin incelenmesi. [examining the levels of self-efficacy perceptions of teachers for their use of technological pedagogical content knowledge (tpack) and educational information network.] (unpublished master’s thesis). balıkesir üniversitesi fen bilimleri enstitüsü, balıkesir. tyler-wood, t., knezek, g., & christensen, r. (2010). instruments for assessing interest in stem content and careers. journal of technology and teacher education, 18(2), 345-368. wahono, b., lin, p.l., & chang, c.y. (2020). evidence of stem enactment effectiveness in asian student learning outcomes. international journal of stem education, 7(36), 1-18. york, m. k. (2018). stem content and pedagogy are not integrated. https://grandchallenges.100kin10.org/assets/downloads/stemcontent-and-pedagogy-are-notintegrated/grandchallengeswhitepapers_york.pdf. yildirim, b., & sahin-topalcengiz, e. (2019). stem pedagogical content knowledge scale (stempck): a validity and reliability study. journal of stem teacher education, 53(2), 1-20. https://www.slideshare.net/nicolerigelman/using-rich-assessments-and-the-pck%20rubric https://www.slideshare.net/nicolerigelman/using-rich-assessments-and-the-pck%20rubric https://grandchallenges.100kin10.org/assets/downloads/stem-content-and-pedagogy-are-not-integrated/grandchallengeswhitepapers_york.pdf https://grandchallenges.100kin10.org/assets/downloads/stem-content-and-pedagogy-are-not-integrated/grandchallengeswhitepapers_york.pdf https://grandchallenges.100kin10.org/assets/downloads/stem-content-and-pedagogy-are-not-integrated/grandchallengeswhitepapers_york.pdf a © 2020 indonesian society for science educator 57 j.sci.learn.2020.3(2).57-66 received: 15 october 2019 revised: 11 january 2020 published: 5 march 2020 development of eighth grade students’ epistemological beliefs through writing-to-learn activities şengül atasoy1*, osman küçük2 1faculty of education, recep tayyip erdogan university, çayeli-rize, turkey 2department of animal nutrition, faculty of veterinary, university of fırat, 23100 elazıǧ, turkey *corresponding author. sengulatasoy@hotmail.com abstract the aim of the present study was to develop the epistemological beliefs of eighth grade students through writingto-learn (wtl) activities. a one-group pretest-posttest quasi-experimental research design was utilized in the study. the sample group consisted of a total of 18 eighth grade students, attending a secondary school in the rural part of trabzon in turkey. to improve the epistemological beliefs of students, wtl activities were developed and utilized in the "matter and its structure" unit of the science subject. each wtl activity focused on one dimension of epistemology, such as source of knowledge, organization of knowledge, certainty of knowledge, speed of learning, and learning control. the wtl activities in the study were conducted throughout a total of 24 lessons. semi-structured interviews were employed to collect data. beliefs regarding each dimension of epistemology were identified to be at the level of absolutist, multiplist or evaluatist by means of the "epistemological belief levels rubric". the findings of the study revealed that the wtl activities increased students’ levels of the epistemological beliefs. hence, students’ epistemological beliefs can be developed further by dwelling more on the history of science unit within the subject of science by raising students’ awareness. keywords epistemological beliefs, writing-to-learn, science instruction 1. introduction the beliefs that individuals have underlie all their decisions and conducts (hofer & pintrich, 1997; pajares, 1992). as for epistemological beliefs, they play an active role in individuals’ attribution of meaning to what they experience and the information they receive. epistemological beliefs can be defined as individuals’ subjective beliefs regarding what the concept of “knowledge” means, and how knowing and learning take place (schommer, 1990). schommer (1994) put forward a model called epistemological belief system which includes dimensions such as source of knowledge, organization of knowledge, certainty of knowledge, speed of learning, and learning control that reveal the multi-dimensional and independent structure of epistemological beliefs. according to this system, individuals with naive epistemological beliefs have an intense belief that knowledge is simple, that knowledge is innate, that learning takes place immediately, and that knowledge is unalterable, whereas individuals with sophisticated epistemological beliefs have an intense belief that knowledge is complex, that knowledge is formed through experience and effort, that learning is over time and that knowledge changes (buehl & alexander, 2001). hence, having sophisticated epistemological beliefs is of crucial importance since students with sophisticated epistemological beliefs make use of a higher number of and higher-order cognitive information processing strategies throughout their learning activities. furthermore, these students display a higher level of academic performance, a positive attitude towards their school, and can establish diversified and profound thoughts and ideas (deryakulu & büyüköztürk, 2002). çavuş (2013) has reported that, compared to students possessing naive epistemological beliefs, those who have sophisticated epistemological beliefs establish more sophisticated cause-effect relationships in socio-scientific subjects, produce responses including more comprehensive explanations and can foresee possible different results and present suggestions. equipping students with sophisticated epistemological mailto:sengulatasoy@hotmail.com journal of science learning article doi: 10.17509/jsl.v3i2.20573 58 j.sci.learn.2020.3(2).57-66 beliefs necessitates instructional implementations in which particularly students are active, knowledge is constructed by the students themselves, and learning is based on the student’s participation (kuzgun & deryakulu, 2014). writing-to-learn (wtl) contributes to students’ ability to comment on their own thoughts, think critically and express their own feelings and experiences, produce rational responses, provide real-life-related responses and develop higher order cognitive functions including solving problems (atasoy, 2012; mason & boscolo, 2000). it is determined that wtl activities (journal writing and explanatory writing) had a positive effect on conceptual understanding because of allowing the writer to clarify his or her knowledge, organize the ideas to be written, and reflect on the learning experience. on grounds that these abilities of students are developed together with a high level of epistemological beliefs, it seems that the impact of learning through writing on epistemological beliefs is an important area that needs to be investigated. 1.1. writing-to-learn (wtl) writing is an important learning mechanism in expressing our opinions about a certain topic, in reorganizing our opinions about a topic, and in partly discovering our ideas and dreams (graham, 2008). if we want our students to use wtl, students should be prevented from copying the information written by their teachers; rather, they should be given the opportunity to make use of their ability to think and reason so that they can write using their own voice of expression (hand, prain, lawrence & yore, 1999; mason & boscolo, 2000). teachers also need to establish an appropriate learning environment and must encourage students to become engaged in wtl activities (kieft, rijlaarsdam & bergh, 2006). wtl extends the retention period of scientific knowledge and reinforces initial knowledge and new ideas (rivard & straw, 2000). writing is a higher order cognitive activity which entails the ability to coordinate one’s own knowledge and fundamental abilities by various means (walker, shippen, alberto, houchins, & cihak, 2005). it is reported that wtl activities encourage students to access scientific information, and enable them to adapt to a learning environment where epistemological beliefs and reasoning strategies are exercised (prain & hand, 1999; yore, bisanz & hand, 2003). therefore, in this study, it was investigated how wtl activities affect epistemological beliefs. 1.2. literature review on epistemological beliefs numerous studies on epistemological beliefs have been identified in the literature of the last 30 years. among these studies, the ones that have an important place in the literature are those studying the relationship between individuals’ epistemological beliefs and such factors as problem solving abilities (aksan & sözer, 2007; charoula & valanides, 2012; schommer & dunnell, 1997), academic achievement (schommer & dunnell, 1997; schommer-aikins, mau, brookhart, & hutter, 2000; schommer-aikins, duell & hutter, 2005; topçu & tüzün, 2009; youn, yang, choi, 2001), learning/teaching strategies (chan, 2004; deryakulu, 2004; hashweh, 1996; mahasneh, 2018; uslu, 2018), gender, area of study and grade level at school (aydemir, aydemir & boz, 2013; chen, xu, xiao & zhou, 2019; gürol, altunbaş & karaaslan, 2010; meral & çolak, 2009; topçu & tüzün, 2009). according to the findings of these studies, individuals with a high level of epistemological beliefs have the tendency to persist in their efforts to learn and, as a result, achieve a higher level of academic achievement. it has been revealed that students who believe that learning is not an innate ability have a high level of academic achievement (schommer-aikins, mau, brookhart & hutter, 2000). in a study by schommer and dunnell (1997), it was revealed that students who were of the belief that learning is an innate and rapid process and that knowledge is absolute had a low level of academic achievement and problem solving ability. according to youn, yang and choi (2001), there is a positive correlation between students’ levels of academic achievement and their epistemological beliefs of knowledge. furthermore, there is also a relationship between epistemological beliefs and understanding of teaching/learning (chan, 2004). teachers with sophisticated epistemological beliefs are more sensitive towards various views of students (hashweh, 1996). in other studies, a positive correlation has been reported between epistemological beliefs and the ability to solve problems, establish associations with daily life, reason and think critically (aksan & sözer, 2007; belet & güven, 2011; charoula & valanides, 2012). based on all these findings, it seems clear that in order to reach the expected academic achievement in educational environments, it is essential to develop students’ epistemological beliefs. it is put forward that epistemological beliefs have an impact on individuals’ approach to knowledge, their academic achievement, their levels of success in their courses and their motivation, and that these beliefs start to form substantially as of primary school level (aksu, demir & sümer, 2002). thus, it is evident that students’ epistemological beliefs should be developed during their primary education. however, when the related literature is examined, it is observed that the sample groups of studies are comprised of individuals within higher age groups, such as teachers, undergraduate students and high school level students (e.g. aksan & sözer, 2007; aydemir, aydemir & boz, 2013; chan, 2004; charoula & valanides, 2012; cheng, chan, tang & cheng, 2009; deryakulu & büyüköztürk, 2002; deryakulu, 2004; meral & çolak, 2009; oksal, şengerci & bilgin, 2007; özkan-hıdıroğlu & hıdıroğlu, 2016; topçu & tüzün, 2009; trakulphadetkrai, 2012; schommer, 1990; schommer, crouse & rhodes, journal of science learning article doi: 10.17509/jsl.v3i2.20573 59 j.sci.learn.2020.3(2).57-66 1992; youn, yang & choi, 2001). however, there seems to be limited amount of research conducted with secondary school students (schommer-aikins, mau, brookhart & hutter, 2000; schommer-aikins, duell & hutter, 2005). these studies were primarily conducted to identify epistemological beliefs and to examine the relationship between these beliefs and some demographic factors. a literature review also yields studies in which the effect of implementations of different teaching methods and techniques on epistemological beliefs was examined (for example, cognitive coaching method--demir, 2009; peer teaching--gok, 2018; nature of science--johnson & willoughby, 2018; classroom discussing-king, levesque, weckerly & blythe, 2000; research-based laboratory activities--deniz, 2011; may & etkina, 2002). studies on the effect of writing on epistemological beliefs are quite limited. these are also related to reflective journal writing (brownlee, petriwskyj, thorpe, stacey & gibson, 2011; dart, boulton-lewis, brownlee & mccrindle, 1998; güven, 2013). reflective journal writing has included student teachers' beliefs about learning and teaching (dart, boulton-lewis, brownlee & mccrindle, 1998), reflecting their own field experiences (brownlee, petriwskyj, thorpe, stacey & gibson, 2011) and laboratory experiences (güven, 2013). they concluded that reflective journal writing enabled epistemological beliefs to progress in a positive way. 1.3. schommer’s epistemological beliefs system according to schommer (1994), epistemological beliefs are individuals' beliefs about the source, certainty and organization of knowledge and the speed and control of learning. schommer (1992), proposed a five-factor epistemological beliefs system. the current study, in which students’ epistemological beliefs were examined, is based on the five dimensions of this system. these dimensions are explained as follows: the first dimension (organization of knowledge) is the simple versus complex nature of knowledge. the shallow or easily comprehensible aspect of knowledge results in epistemological beliefs related to whether or not content is formed with the combination of different concepts. the second dimension (certainty of knowledge) is the unchanging nature of knowledge, that is its certainty. in this dimension, beliefs are put forward as regards knowledge being unchangeable, regardless of whether or not knowledge is unconditionally accurate and whether or not it changes. the third dimension (source of knowledge) involves presenting the view regarding whether knowledge is derived from an authority or a subjective source. individuals holding naive epistemological beliefs think that source of knowledge is the authorities and experts whereas those holding sophisticated beliefs content that knowledge is produced through extensive observation, reasoning, and judgements (schommer-aikins, 2004). in the fourth dimension (speed of learning), the speed of acquiring knowledge is addressed. in this dimension, the answer to the question of whether knowledge is acquired instantly or step by step with experience is sought. it expresses the dimension of belief based on whether knowledge is formed instantly through learning. the fifth dimension is learning control. this dimension entails the notion that the learning process is changeable and is developmental, as opposed to the belief that the learning phenomenon is something that exists as of birth and is unchangeable. 1.4. the aim of the study the aim of the present study was to identify the impact of wtl activities on eighth grade students’ levels of epistemological beliefs. 2. method a one-group pretest-posttest quasi-experimental design was employed in the present study. this model was chosen owing to the lack of a class that could be assigned as a control group in the school where the study was being conducted. in this model where the implementation was applied in a single group, measurements were made before and after the implementation. the progress between the preand post-measurement was accepted as the impact of the intervention (karasar, 2012). in the present study, the initial epistemological belief levels of the secondary students were identified and subsequently, the wtl activities to develop these levels were implemented. then, the final interviews were conducted on whether or not the implementation was effective and, thus, the effectiveness of the wtl activities was evaluated. 2.1. sample the study group was composed of 18 eighth grade students (10 males and 8 females). convenient sampling strategy was used in recruitment of students. this was because the second researcher was working at the school in which data were collected and this eased access to participants as well as the process (büyüköztürk, kılıççakmak, akgün, karadeniz, & demirel, 2018). according to the science teacher, the overall success level of these students was either low or moderate. the science teacher explained the science lessons to this class by lecturing and frequently had students do multiple-choice tests to prepare them for the high school entrance exam. these students, who were in their final year at secondary school, spent most of their time preparing for the high school entrance exam. according to science teachers, these students do not have a sufficient level of reasoning skills since they are focused more on solving multiplechoice tests to prepare for the high school entrance exam. hence, in exams including open-ended questions, they journal of science learning article doi: 10.17509/jsl.v3i2.20573 60 j.sci.learn.2020.3(2).57-66 have difficulty in writing what they know or making inferences. 2.2. the preparation and implementation of wtl activities in this section, the identification of the topic, and the development and the implementation of the writing activities are addressed under separate headings. 2.3. the identification of the topic the wtl activities were prepared based on the topics within the unit of “matter and its structure”. the topics were as follows: the particle structure of matter (views proposed on the structure of the atom from history to today), pure matter, mixtures, separation of mixtures, home wastes-recycling, and chemical industry. these topics were believed to overlap with the contexts of the dimensions of epistemological beliefs and consist more of theoretical explanations at the conceptual level. by taking into consideration the entirety of the unit, the writing activities were prepared to include all the topics. 2.4. the development of the wtl activities explanatory writing involving question prompts to solve problem cases was used as wtl activity.the activities were designed in the format of worksheets (see appendix a for a sample of an activity). during the preparation of the worksheets, special attention was paid to preparing questions that would enable students to think while they wrote, to question, and lay the foundation to assist them in structuring arguments in their minds. in addition, it was ensured that the contexts established in the writing activities would lead to discussions of the epistemological belief dimensions. the prepared activities were examined by an expert who had conducted research on writing and epistemological beliefs, and based on the recommendations made, the necessary modifications were made. the validity of the language used in the activities was examined by a language expert and the necessary modifications were made based on the feedback. the prepared activities were then piloted on nine students. the results were evaluated by one researcher and an expert and finally, the decision regarding its appropriateness was made. 2.5. the implementation of the wtl activities the wtl activities were implemented in the science classes for six weeks (a total of 24 lessons) – one activity each week – by the second researcher. the schedule presented in table 1 was followed in the implementation of the writing activities. the science lessons were initially taught theoretically by the teacher. subsequently, the writing activity related to the topic was distributed in a worksheet format to the students, who were asked to complete the blanks with their opinions regarding the questions. subsequently, the students were asked to exchange the worksheet they had completed with that of their peer to read what their peer had written and then they were required to hold a class discussion. this enabled the students to assume the responsibility of completing their own worksheet completely. the students were also given the opportunity to revise what they had written by thinking about the questions again after class. 2.6. data collection semi-structured interviews based on epistemological beliefs were prepared and used as a means to collect data. a question for each dimension of the epistemological belief was prepared for the interview (appendix b). in the preparation of these questions, interview forms (güven, 2013) and scales (deryakulu & büyüköztürk, 2002; evcim, 2010) that existed in the related literature and similarly aimed to identify students’ epistemological beliefs were benefitted from. the questions prepared for the interviews were modified based on the suggestions made by two experts in terms of comprehensibility, scope and content. subsequently, the questions were piloted on three students who were not part of the sample group. as a result of the pilot study, the interview questions were evaluated based on comprehensibility and the retrieval of possible expected responses. the interviews were held on one-on-one basis with 18 eighth grade students. 2.7. data analysis for the data analysis of the study, the “epistemological belief levels rubric” was utilized. this rubric was prepared based on the five-factor epistemological belief levels defined by schommer (1990) and those defined by khun (2001) as absolutist, multiplist and evaluatist. it is stated that individuals holding “absolutist”, that is naive, epistemological beliefs believe in the certainty of knowledge, believe that knowledge is derived from external sources, that one must confide in these sources (authorities) and that knowledge is accumulated. it is argued that for individuals with “multiplist”, or moderate level of, epistemological beliefs, knowledge lacks certainty, knowledge cannot be known directly and knowledge can change from person to person. in addition, it is stated that claims are views that are freely chosen subjective, personal views and that each view possesses equal rights. finally, it is argued that individuals with “evaluatist”, that is high, epistemological table 1 the implementation schedule implementation week title of the implemented activity 1st week historical development of atomic models 2nd week definition of element 3rd week mixtures 4th week separation of mixtures 5th week recycling 6th week the chemical industry journal of science learning article doi: 10.17509/jsl.v3i2.20573 61 j.sci.learn.2020.3(2).57-66 beliefs, believe that knowledge lacks certainty. it is asserted that the claims put forward by these people are their own beliefs or judgments, and that when these judgments are presented, proof is also used and discussed so that those with the highest validity are accepted as true knowledge. furthermore, it is maintained that the individuals at this level use critical thinking to strengthen their arguments and the find it useful in reinforcing the meaning conveyed in their arguments. to increase the reliability of the rubric developed in the study, the sample performances of the scales developed by education scholars were examined. the feedback of education scholars working in this area was received for the criteria in the rubric and their descriptors, and the results of the implementation were scored by two separate raters. to increase the validity of the rubric prepared, initially a pilot study was conducted with nine students. in this way, the predetermined criteria were found to be addressing the students’ explanations. the data obtained by means of the interviews were analyzed via the “epistemological belief levels rubric” (table 2). in this way, the students’ epistemological belief levels were categorized as absolutist, multiplist and evaluatist within each epistemological belief dimension (source of knowledge, organization of knowledge, certainty of knowledge, speed of learning, and learning control). the data obtained from the interviews were evaluated simultaneously by the researchers. subsequently, the compatibility between the evaluations was checked, and a discussion was held on the incompatible data to reach a common decision. 3. result and discussion in this section, the data regarding epistemological beliefs identified via the preand post-interviews of the wtl activities are presented. the levels identified for each epistemological belief dimension were compared in percentages between the preand postinterviews. figure 1 displays the comparative percentages of the epistemological belief levels based on the students’ interview responses to the question on the source of knowledge dimension. as can be observed in figure 1, when the students’ responses given to the questions on the dimension of source of knowledge are examined, the number of students with epistemological beliefs at the absolutist level has decreased by 50%. a student response at the absolutist level was as follows: “…may have learned it at school in the science lesson.” based on this statement, it can be deduced that the student considers the source of knowledge as books of authority and expert teachers. the number of students at the multiplist level of epistemological beliefs increased from 14% to 50%. a student response at this level was as follows: “may have researched and learned it. may have asked the electrician or saw it elsewhere. may have even searched it on the table 2 epistemological belief levels rubric dimensions absolutist multiplist evaluatist source of knowledge sources of knowledge are authorities and experts the claims are subjective, personal views freely chosen by the individuals themselves and each view has equal rights. individuals interpret and question events based on objective and subjective tools and, thus, make inferences to arrive at conclusions. organization of knowledge the belief that knowledge is segmental the belief that knowledge is partially related with other knowledge the belief that knowledge is complex and is integrated with each other certainty of knowledge the belief in the certainty, absoluteness, and clarity of knowledge the belief in the subjectivity and the lack of certainty and directness of knowledge the belief in the continuous change, renewal and active organization of knowledge speed of learning the belief that either there is rapid learning or no learning at all the belief that if learning has not occurred instantly, it will occur by means of reviews and exercises the belief that learning is a staged process learning control the belief that the learning ability is innate, constant and unchangeable the belief that the learning ability is not innate, but acquired the belief that learning occurs with experience and that learning is an ability that can undergo change figure 1 student responses to the question on the source of knowledge dimension journal of science learning article doi: 10.17509/jsl.v3i2.20573 62 j.sci.learn.2020.3(2).57-66 internet and read comments.” it can be understood from the student’s explanation that s/he considers that source of knowledge can be learnt by making use of different sources and that doing research is one way of obtaining knowledge. the number of students at the evaluatist level of epistemological beliefs increased by 14%. a sample student response at this level was as follows: “may have tried and made observations. in this way s/he may have decided which one is the brightest.” this statement implies that the source of knowledge for these individuals is their own objective and subjective interpretations and questioning of events and, thus, they make inferences to arrive at conclusions. figure 2 portrays the comparative percentages of epistemological levels based on students’ responses to the interview question based on the organization of knowledge dimension. as can be observed in figure 2, there is a decrease in the percentage of student responses at the multiplist level of epistemological beliefs in the organization of knowledge dimension, while there is an increase in the percentage of responses at the absolutist and evaluatist levels. a sample student response reflecting the absolutist level is as follows: “i also believe that not learning the knowledge in a lower class does not have an effect on learning the new knowledge. we are learning new knowledge every year. learning new knowledge does not really necessitate knowing other knowledge.” it can be deduced from this explanation that the student considers knowledge in segments. the number of students at the multiplist level of epistemological beliefs decreased by 21%. a sample student response of a student at this level of epistemological beliefs is as follows: “…some knowledge, especially in mathematics, if your mathematics in the lower class was not good, you still cannot do it because it is related to the topic learnt in the previous year. the teacher reviews us the topic but it is not sufficient for me. but science is not like that. i mean, to understand the topics you explain, that don’t include computations you explain, i don’t need other knowledge and i am more successful. i mean depending on the subject, or even topic, some are related and others are easy, simple; they can be learnt without knowing anything else.” it can be inferred from this explanation that the student considers knowledge to be partly related with each other. a sample student response at the evaluatist level is as follows: "i think all subjects are interrelated. for example, while we are solving a problem on heat and temperature in your lesson, we use our mathematics knowledge. while drawing the structure of dna in the notebook, we add aesthetics by remembering the charcoal drawing technique of our visual arts teacher. in the writing activity we did, we used the knowledge we learned in turkish. we learned the movements of the earth both in science and in social sciences.” based on this explanation, it can be understood that the student sees knowledge as integrated with one another. figure 3 displays the comparative percentages of epistemological levels based on students’ responses to the interview question based on the dimension of accuracy of knowledge. as can be observed in figure 3, in the certainty of knowledge dimension, almost all the students produced responses in the post-interview at the evaluatist level. the number of students at the evaluatist level increased from 26% to 86%. a sample student response at the evaluatist level is as follows: “i think it’s associated with technology. as time proceeds, technology develops, new devices emerge; they seem more profound with these devices. you know it is also in the writing activity. there were views related to the structure of the atom throughout history. like this. there’s no end to it. for the time being, this is the best model that can explain it. in the future, a new technology can develop. they will use that. a completely different model will appear. and i think that’s what happened with the white light.” it can figure 2 the student responses to the question on the organization of knowledge dimension figure 3 the student responses given to the question on the certainty of knowledge dimension journal of science learning article doi: 10.17509/jsl.v3i2.20573 63 j.sci.learn.2020.3(2).57-66 be inferred from this explanation that the student considers knowledge to be changing continuously, to be renewing itself, and to have an active structure. a sample student response at the multiplist level is as follows: " i think scientists looked at the white light from different perspectives and then decided whether or not it was pure. in the past science was not developed. books and libraries were limited. their preliminary knowledge and experiment equipment at the time they examined the white light were probably insufficient. considering that it says ‘later’, probably facilities in newton’s times were more varied. so he arrived at the correct [result]. perhaps in the future they will even say that light is not white.” this explanation shows that this student regards knowledge as not absolute and as changeable over time. a sample student response at the absolutist level is as follows: “newton found that light is not pure through research and experiment. so he did the previous experiments inaccurately because the result of the experiment was that white light is not pure. in fact, last year we did a colour wheel in class and saw the white. i mean when you set up the experiment accurately, the result is always the same.” this explanation shows that knowledge is observed as precise, absolute, and clear by the student. figure 4 displays the comparative percentages of epistemological levels based on students’ responses to the interview question based on the speed of learning dimension. as can be observed in figure 4, in the speed of learning dimension, all the students made explanations at the evaluatist level in the post-interview. a sample student response at the evaluatist level is as follows: “there is never 100% learning in class. i always have to review it at home and do exercises so that i can learn it. for example, last sunday, i went to the bazaar with my mum, [so] i couldn’t come to school. i read the summary you sent us. i looked at the example in the book. then i could solve the [questions in the] photocopy. can i solve all the questions? of course not. but in time, with your assistance as well, i think i can catch up.” based on this response, it can be understood that the student sees learning as a staged process. a sample student response at the absolutist level is as follows: "exactly, if i learned it in class, it means i learned it. otherwise, i can never learn it later on. i have to hear it from someone else.” this explanation implies that the student regards learning as a rapid and instant event. figure 5 displays the comparative percentages of epistemological levels based on students’ responses to the interview questions based on the learning control dimension. as can be observed in figure 5, while a majority of the students were at the multiplist level in the learning control dimension in the pre-interview, they reached the evaluatist level in the post-interview. a sample student response at the evaluatist level is as follows: “…for example, my science was always bad in previous years. this year i paid more attention, i studied regularly. in the exam during the first term, i did the best in science. when that was the case, i studied science harder. now, i am able to understand the topics more easily. while writing (in the activities) aziz sancar was mentioned as being at moderate level of intelligence but he always studied hard and won the nobel prize. who knows? maybe we will create a more updated version of the atom model we wrote on that paper.” it can be deduced from this explanation that the student considers learning as a process acquired through experience and as an ability that can change. the number of students at the multiplist level of epistemological beliefs decreased by 50%. a student explanation at this level is as follows: "rather than ability, it is studying in the right and disciplined way. if ahmet changes his study habits, he will probably be successful. when he finds the right way to study, it will be much easier for him to be successful.” it can be understood from this explanation that the student believes that the figure 4 the student responses given to the question on the speed of learning dimension figure 5 the student responses given to the question on the learning control dimension journal of science learning article doi: 10.17509/jsl.v3i2.20573 64 j.sci.learn.2020.3(2).57-66 learning ability is not innate rather it can be acquired by studying. a sample student response at the absolutist level is as follows: “… i study mathematics more than anyone else. still i can’t get a very good grade. i’ve got no talent for numbers. i am trying to pass the exam. otherwise, what i have got to do with mathematics?” it can be inferred from this explanation that learning control is associated with intelligence, which is considered innate and fixed and unchangeable. in summary, it was revealed that the number of students at the absolutist level of epistemological beliefs decreased in all the dimensions (except the organization of knowledge dimension). while the number of students at the multiplist level of epistemological beliefs increased in the source of knowledge dimension, it decreased in the other dimensions, except for learning speed. finally, there has been an increase in the number of students at the evaluatist level of epistemological beliefs across all the dimensions. 4. conclusion in this section, how the wtl activities impacted the five dimensions (source of knowledge, organization of knowledge, certainty of knowledge, speed of learning and learning control) of 8th grade students’ epistemological beliefs is discussed in light of the findings. in the present study, it was revealed that the science lessons conducted by means of wtl activities had a positive impact on all dimensions of epistemological beliefs. other studies investigating the impact of writing on epistemological beliefs have also found a positive interaction (brownlee, petriwskyj, thorpe, stacey & gibson, 2011; güven, 2013). güven (2013) found that reflective journal writing provides a development of student teachers' source of knowledge, organization of knowledge, speed of learning and learning control dimensions, but no change in their beliefs about certainty of knowledge. some research has shown that interventions have developed all dimensions of epistemological beliefs (deniz, 2011), while others have shown that a few dimensions have improved and a few have remained the same (chai, teo & lee, 2009; chen & chang, 2008). it is understood that the different interventions influenced the different epistemological belief dimensions. in the study, the observed positive development in the dimensions of certainty of knowledge, source of knowledge and organization of knowledge is attributed to the fact that the historical development of atom models and the definition of element were addressed in the worksheet and the variation in the models was examined because when students were explaining their opinions, they frequently mentioned the samples in the writing activities. with discussions held on such questions as “what do you think ‘developing different models of the atom’ means?, can a new atom model emerge in the future?, why?, is the current definition of element sufficient in defining ‘element’?” in the writing activities, an attempt was made to raise students’ awareness to the fact that knowledge is a subject to change. in addition, some activities with a positive impact on the development of epistemological beliefs among the wtl activities used in the study can be listed as follows: directing students towards establishing their own atom models, having them question how the mixtures need to be separated, directing them towards developing a green and economic new detergent and subsequently having them make inferences. the reason why these activities were beneficial was that while they were writing their responses, they got the opportunity to make interpretations about their own opinions, and expressed their own feelings and experiences. by doing so, they were able to develop higher order cognitive functions, such as thinking critically, providing reasoned responses, relating their responses to daily life (albert, 2000; dicamilla & anton, 1997; stonewater, 2002). other studies also reported the positive impact of various interventions in class on students’ epistemological beliefs (chai, teo & lee, 2009; chen & chang, 2008; deniz, 2011; king, levesque, weckerly & blythe, 2000). for example, in a study conducted in the year 2000 by king, levesque, weckerly & blythe, it was reported that class discussions in which students were required to talk on the topics of knowledge and learning influenced students’ epistemological beliefs positively. it is believed that the progress observed in the dimension of learning control could be attributed to the activity of questioning by whom and how the new knowledge in the wtl activities was discovered and the ability of the human being to discover and produce new things. the discussions on such questions as “how could the scientists who developed these models have obtained information about atoms?, if you were in pursuit of a model based on the structure of the atom, with which activity and from which point would you start your endeavour?” are believed to influence students’ epistemological beliefs positively by encouraging them to make interpretations and inferences, build empathy, put themselves in the shoes of the person mentioned in the activity, question the decisions they take and reflect on their learning process. ensuring that students reflect their views on the nature of knowledge and learning affects their epistemological beliefs (may & etkina, 2002). based on the work, we recommend some points: first, more time should be spent on the topics in the history of science unit in science lessons, in which students should be made aware of the fact that knowledge can change over time, and depending on whether or not the topic is convenient, students should be encouraged to design their own models of a pre-determined topic. second, as in the wtl activities, instead of providing students with journal of science learning article doi: 10.17509/jsl.v3i2.20573 65 j.sci.learn.2020.3(2).57-66 information directly, in-class research activities and discussions can be used effectively to enable students to draw their own inferences and, thus, construct knowledge; in this way, their epistemological beliefs can be developed. third, similar implementations can be done in a longer period of time and the implementation period can be evaluated to reflect both teacher and student experiences. references aksan, n. &sözer, m (2007). the relationships among epistemological beliefs and problem solving skills of university students. ahi evran university kırşehir education faculty journal, 8(1), 31-50. (in turkish) aksu, m., demir.c. &sümer, h. z. (2002). students’ beliefs about mathematics: descriptive study. education and science, 27(123), 7277. albert, l. r. (2000). outside-in – inside-out: seventh-grade students’ mathematical thought processes. educational studies in mathematics, 41(2), 109-141. atasoy, ş. (2013). effect of writing-to-learn strategy on undergraduates’ conceptual understanding of electrostatics. the asia-pacific education researcher, 22(4), 593-602. atasoy, e. (2012). the examination of writing based activities in mathematics from the perspectives of learning and teaching. unpublished phd thesis, trabzon, turkey. (in turkish). aydemir, n., aydemir, m. & boz, y. (2013). high school students’ epistemological beliefs. kastamonu education journal, 21, 1305-1316. (with an english extended abstract) belet, d. &güven, m. (2011). meta-cognitive strategy usage and epistemological beliefs of primary school teacher trainees. educational sciences: theory & practice, 11(1), 31-57. brownlee, j., petriwskyj, a., thorpe, k., stacey, p. & gibson, m. (2011). changing personal epistemologies in early childhood preservice teachers using an integrated teaching program. higher education research & development, 30(4), 477-490. buehl, m. m. & alexander, p. a. (2001). beliefs about academic knowledge. educational psychological review, 13(4), 385-418. büyüköztürk, ş., kılıççakmak, e., akgün, ö. e., karadeniz, ş. &demirel, f. (2018). scientific research methods in education (24th edition). ankara: pegem academy. (in turkish) çavuş, r. (2013). epistemological beliefs with different of 8th grade students' perspectives on socio-scientific issues. unpublished master’s thesis. sakarya university graduate school of educational sciences, sakarya. (in turkish) chai, c. s., teo, t. & lee, c. b. (2009). the change in epistemological beliefs and beliefs about teaching and learning: a study among pre-service teachers. asiapacific journal of teacher education, 37(4), 351-362. chan, k.w. (2004). preservice teachers' epistemological beliefs and conceptions about teaching and learning: cultural implications for research in teacher education. australian journal of teacher education, 29(1), 1-14. charoula, a. &valanides, n. (2012). epistemological beliefs and illstructured problem-solving in solo and paired contexts. educational technology & society, 15(1), 2-14. chen, c. c. & chang, c. y. (2008). the effects of a teaching program on changing students’ epistemological baliefs and learning. the international journal of learning, 15(3), 161-168. chen, l., xu, s., xiao, h. & zhou, s. (2019). variations instudents’ epistemological beliefs towards physics learning across majors, genders, and university tiers. physical review physics education research, 15(1), 1-11. cheng, m. m. h., chan, k. w., tang, s. y. f. & cheng, a. y. n. (2009). pre-service teacher education students' epistemological beliefs and their conceptions of teaching. teaching and teacher education, 25(2), 319-327. dart, b. c., boulton-lewis, g. m., brownlee, j. m. &mccrindle, a. r. (1998). change in knowledge of learning and teaching through journal writing. research papers in education 13(3), 291-318. demir, ö. (2009). the effect of metacognitive strategies instructed through cognitive coaching on students? epistemological beliefs, metacognitive skills, academic achievement and the retention of these in 6th grade social studies course. unpublished phd thesis. çukurova university, adana. (in turkish) deniz, h. (2011). examination of changes in prospective elementary teachers’ epistemological beliefs in science and exploration of factors meditating that change. journal of science education and technology, 20(6), 750-760. deryakulu, d. &büyüköztürk, ş. (2002). validity and reliability study of epistemological beliefs questionnaire. educational research journal, 8, 111-125. (in turkish) deryakulu, d. (2004). the relationships between university students' learning and study strategies and their epistemological beliefs. educational administration: theory and practice, 10(38), 230-249. (in turkish) dicamilla, f. j. & anton, m. (1997). repetition in the collaborative discourse of l2 learners: a vygotskian perspective. the canadian modern language review, 53, 609–633. evcim, i̇. (2010). the relation between 8th grade primary school students? epistemological beliefs, science lesson acquisition, level of use of science in daily life and academic success. unpublished master's thesis. marmara university institute of educational sciences, i̇stanbul. (in turkish) gok, t. (2018). the evaluation of conceptual learning and epistemological beliefs on physics learning by think-pair-share. journal of education in science, environment and health (jeseh), 4(1), 69-80. graham, s. (2008). research on writing development, practice, instruction, and assessment: introduction to a special issue of reading and writing. reading and writing, 21(1), 1-2. gürol, a., altunbaş, s. &karaaslan, n. (2010). a study of self efficacy and epistemological beliefs of prospective teachers. e-journal of new world sciences academy, 5(3), 1395-1404. (in turkish) güven, g. (2013). the examination of preservice elementary teachers' reflection journal writing and epistemological beliefs in the science and technology laboratory applications. unpublished master's thesis. muğlasıtkıkoçman university graduate school of educational sciences, muğla. (in turkish) hand, b., prain, v., lawrence, c. & yore, d. l. (1999). a writing in science framework designed to enhance science literacy. international journal of science education, 21(10), 1021-1035. hashweh, m. z. (1996). effects of science teachers’ epistemological beliefs in teaching. journal of research in science teaching, 33(1), 4763. hofer, b. &pintrich, p. (1997). the development of epistemological theories: beliefs about knowledge and knowing and their relation to learning. review of educational research, 67, 88-140. johnson, k. & willoughby, s. (2018). changing epistemological beliefs with nature of science implementations. physical review physics education research, 14, 1-14. karasar, n. (2012). scientific research methods (24th edition). ankara: nobel publishing house. (in turkish) khun, d. (2001). how do people know? psychological science, 12, 1-8. kieft, m., rijlaarsdam, g. & bergh, h. b. (2006). writing as a learning tool: testing the role of students’ writing strategies. european journal of psychology of education, 11(1), 17-34. king, c. a., levesque, m. j., weckerly, s. j. & blythe, n. l. (2000). the effects of a course in educational psychology on pre-service teachers’ epistemological beliefs. poster presented at the annual meeting of the american educational research association, new orleans, la. kuzgun, y. &deryakulu, d. (2014). individual differences in education (3rd edition).ankara: nobel publications. (in turkish) mahasneh, a. m. (2018). investigation of the relationship between teaching and learning conceptions and epistemological beliefs journal of science learning article doi: 10.17509/jsl.v3i2.20573 66 j.sci.learn.2020.3(2).57-66 among student teachers from hashemite university in jordan. european journal of contemporary education, 7(3), 531-540. mason, l. &boscolo, p. (2000). writing and conceptual change. what changes? instructional science, 28(3), 199-226. may, d. b. &etkina, e. (2002). college physics students’ epistemological selfreflection and its relationship to conceptual learning. american journal of physics, 70(12), 1249-1258. meral, m. &çolak, e. (2009). an investigation of student-teachers’ scientific epistemological belief. ondokuzmayıs university faculty of education journal, 27(1), 121-165. (in turkish) oksal, a., şengerci, e. &bilgin, a. (2007). determination of central epistemological beliefs forming student teachers’ life theories. elementary education online, 6(3), 411-421. özkanhıdıroğlu, y. &hıdıroğlu, ç. n. (2016). examining epistemological beliefs in explaining mathematics teachers’ approaches in mathematical modeling. journal of theory and practice in education, 12(1), 244-268. pajares, m. f. (1992). teachers’ beliefs and educational research: cleaning up a messy construct. review of educational research, 62(3), 307-333. prain, v. & hand, b. (1999). students’ perceptions of writing for learning in secondary school science. science education, 83(2), 151162. rivard, p. l. & straw, b. s. (2000). the effect of talk and writing on learning science: an exploratory study. science education, 84, 566593. schommeraikins, m., duell, o. &hutter, r. (2005). epistemological beliefs, mathematical problem-solving beliefs, and academic performance of middle school students. elementary school journal, 105, 289-304. schommeraikins, m., mau, w., brookhart, s. &hutter, r. (2000). understanding middle students’ beliefs about knowledge and learning using a multidimensional paradigm. journal of educational research, 94, 120-127. schommer, m. &dunnell, p. a. (1997). epistemological beliefs of gifted high school students. roeper review, 19, 153-156. schommer, m. (1990). effects of beliefs about the nature of knowledge on comprehension. journal of educational psychology, 82(3), 498-504. schommer, m. (1994). synthesizing epistemological belief research: tentative understandings and provocative confusions. educational psychology review, 6(4), 293-319. schommer, m., crouse, a. & rhodes, n. (1992). epistemological beliefs and mathematical text comprehension: believing it is simple does not make it so. journal of educational psychology, 84(4), 435–443. schommer-aikins, m. (2004). explaining the epistemological belief system: introducing the embedded systemic model and coordinated research approach. educational psychologist, 39(1), 1929. stonewater, j. k. (2002). the mathematics writer’s checklist: the development of a preliminary assessment tool for writing in mathematics. school science and mathematics, 102(7), 324-334. topçu, m. s. &tüzün, ö. (2009). elementary students’ metacognition and epistemological beliefs considering science achievement, gender and socioeconomic status. elementary education online, 8(3), 676-693. trakulphadetkrai, n. v. (2012). relationship between classroom authority and epistemological beliefs as espoused by primary school mathematics teachers from the very high and very low socio-economic regions in thailand. journal of international and comparative education, 1(2), 71-89. uslu, o. (2018). analysis of variables that affect teaching learning approaches and epistemological beliefs of pre-service teachers by structural equation model. international journal of research in education and science, 4(1), 237-251. walker, b., shippen, m. e., alberto, p., houchins, d. e. &cihak, d. f. (2005). using the expressive writing programme to improve the writing skills of high school students with learning disabilities. learning disabilities research and practice, 20(3), 175-183. yore, l., bisanz, g. l. & hand, b. m. (2003). examining the literacy component of science literacy: 25 years of language arts and science research. international journal of science education, 25(6), 689725. youn, i., yang, k. m. & choi, i. j. (2001). an analysis of the nature of epistemological beliefs investigating factors affecting the epistemological development of south korean high school students. asia pacific education review, 2(1), 10-21. a © 2022 indonesian society for science educator 478 j.sci.learn.2022.5(3).478-487 received: 04 january 2022 revised: 10 may 2022 published: 26 november 2022 the effect of engineering design-based stem activities on the refugee students’ sense of school belonging zehra çakır*1, sema altun yalçın2, şeyma güvenç günsel2 1erzincan binali yıldırım üniversitesi, turkey 2eğitim fakültesi, erzincan binali yıldırım üniversitesi, turkey *corresponding author: zehracakir.29@hotmail.com abstract in the research, the effect of engineering design-based stem activities on the school belonging to seventh-grade refugee students was examined. a mixed-method was used in the research. the research sample consists of 32 syrian refugee students studying in the 7th grade in the fall semester of the 2019-2020 academic year. the study lasted for ten weeks, with two lessons per week. in addition, the “school attachment scale for children and adolescents” was used as a quantitative data collection tool. on the other hand, qualitative data collection tools were used as “stem education reflective diary form, stem education interview form, and teacher interview form,” which were prepared based on quantitative scale items. statistical methods were used to analyze the quantitative data obtained in the research, and content analysis was used in the qualitative data analysis. the quantitative data showed a significant difference between the pre-test and post-test scores. as a result, it was concluded that the engineering designbased stem activities carried out contributed positively to the individual and social development of syrian refugee students; their positive attitudes towards the lesson and the teacher impacted their sense of belonging to the school. keywords school belonging, refugee, engineering design-based, stem 1. introduction approximately 6 million people were forced to leave their homeland with the escalation of the civil war in syria in 2011. according to the united nations high commissioner for refugees (unhcr) data, turkey has been the country hosting the highest number of refugees in the world since 2016 (unhcr, 2018). about 63% of the one and a half million school-aged refugees continue their unfinished education (duman, 2019). since they were initially thought to return to their countries, arabic courses were taught in the short-term target by the curriculum in syria to educate the refugee children (uysal & dinçer, 2012). the continuation of the war and instability in syria in 2013 led to increased immigration to turkey and its neighboring countries. although it was not taken into consideration in the education problem much in the beginning years of the immigration, day after day, like other basic needs, education inevitably came to the fore (saidi, 2014). the ‘project on promoting integration of syrian kids into the turkish education system’ (pictes) was started in cooperation with unicef and the ministry of national education (cemalcılar,2010). education, training, and schooling play a leading role in refugee children’s avoiding the negative effects of the war. herewith, education is the most significant factor for them to reestablish social relations, cope with the poor conditions that refugee children encounter, and have an environment in which they feel safe against the traumas they experienced (özdemir, sezgin, şirin, karip & erkan, 2010). education is one of a nation's fundamental sources of achievement and wealth (özden & turan, 2013). the countries must integrate stem, which is the current field of education, into their curricula to compete with current and future technological developments, meet their needs, and raise well-equipped individuals (dugger, 2010). stem is an educational approach, formed by the combination of the initials of the disciplines of science, technology, engineering, and mathematics written in english, aiming to provide individuals with knowledge and skills and including in-school and out-of-school educational activities at each grade level (çakır & altun yalçın, 2021a; gonzalez journal of science learning article doi: 10.17509/jsl.v5i3.39846 479 j.sci.learn.2022.5(3).478-487 & kuenzi, 2012). figure 2 shows integrated stem education (akgunduz, 2016). according to the united states national security council, stem has three primary aims increasing the number of students who will continue their careers in stem fields in higher education, expanding the participation in the stem workforce, and raising individuals with stem literacy. stem education is important because of its contribution to scientific and technological progress and the development and sustainability of innovation (altun yalçın, 2019). one method that is applied in the stem approach is the engineering design process (edp). engineers solve the design problems with the patterns they develop and handle them within the scope of some criteria and restrictions to conclude this process successfully. edp is a cyclic, creative and dynamic process consisting of producing various solutions ways to solve an engineering problem and determining the most appropriate course of solution, and designing a product by using the best way of a solution that has been determined (çepni, 2017; altun yalçın & yalçın, 2018). as a result of successfully implementing edp in a teaching environment, students may have the opportunity to solve the design problems as engineers by applying their creativity. in edp, individuals need proper materials, technology, and mathematics knowledge to analyze the data obtained for the scientific concepts to be understandable and for an innovative product to be produced accordingly (aydın & karslı baydere, 2019). the edp training approach focuses on real-life problems relevant to several disciplines. individuals should plan, design, and apply the activities they implement to solve these problems (çepni, 2017). at the same time, it contributes to the child's socialization and positive feelings toward the school in the activities created in cooperation (samur & altun yalçın, 2021). thus, the students' feelings toward school and their sense of belonging to school may play a significant role in forming their perceptions of themselves and their environment (balak, 2017). in addition to the fact that individuals who feel happy and peaceful, a school society with positive emotions, and the school climate that develops with the continuation of this situation increases the quality of education, the students' positive attitudes towards school will be practical in fulfilling their duties and responsibilities willingly (yildiz, alkan & cengel, 2019). the sense of belonging is the individual's feeling of being accepted, loved, approved by their social environment, and feeling like a member of the society in which s/he lives in connection with them, seeing themselves as a valuable asset (özgök, 2013). the students' feeling of belonging to school is also significant in terms of academic development as well as their social and psychological development (arıkan, 2015). therewithal, it was revealed that the sense of belonging to the school is a significant factor not only in terms of achievement and development at school but also in terms of development that will continue throughout life (bellici, 2015). considering this information for the thousands of young, dynamic syrian refugee students living in turkey not to be a lost generation and not to grow up society that is ignorant, illiterate, unaware of technology, engineering, science, and knowledge which damages the country they live in, even if they do not benefit their country in the long run, it was mainly focused on the education of refugee children and emphasizing primarily on reinforcing their sense of belonging to the school. in this scope, as the stem applications for the current curricula were explored, it was noticed that there were no studies in which the refugee students' sense of belonging to school was studied. stem can produce effective results due to its properties. stem activities improve students' adaptation to school and motivation for science, technology, mathematics, and other courses. in addition, it contributes to concrete and permanent learning in science education, using technology and engineering design skills. with group work and handson activities, it contributes to students’ developing positive feelings towards school and teacher (çakır & yalçın, 2022). the research was conducted to determine whether the engineering design-based stem activities created with simple materials influence the sense of belonging to the school of the 7th-grade secondary school syrian refugee students. 2. method 2.1. research model a mixed method was employed in the research. the mixed-method research studies are not simply a combination of the qualitative and quantitative methods but the integration studies in which the strengths of these methods are covered by supporting research (creswell, 2006). the mixed research method is applied when no answer was found to the quantitative and qualitative research questions alone. one of the essential features of the mixed-method is that it does not limit the researcher's options throughout the research, but broader and more figure 1. integrated stem education (akgündüz, 2015) journal of science learning article doi: 10.17509/jsl.v5i3.39846 480 j.sci.learn.2022.5(3).478-487 comprehensive answers are obtained from the research problem (baki & gökçek, 2012). the consolidation design of the mixed method was applied in the research. a simultaneous triangulation model, equivalent to creswell’s (2003) mixed-method, was employed. this pattern collects and analyzes the quantitative and qualitative data simultaneously. data analysis is generally performed separately, and combining occurs during the interpretation of the data. consolidation is the triangulation of data, that is, the discussion of how close they are to each other. this pattern is proper when it is aimed to validate, strengthen and cross-validate the research findings. 2.2. study group the sample of the research consists of 32 syrian refugee students (22 female, ten male) who were students in the 7th grade of a secondary school of the ministry of national education in a province located in the southeast anatolia region in the fall term of the 2019-2020 education year. the students were aged between 13 and 15 in general. after the syrian refugees came to turkey in 2011, they were included in the pictes project following the education in gem established primarily in the camps. after the turkish proficiency examination, they could continue their education in turkish schools. besides, were students directly registered in turkish schools from the camps without being involved in the pictes project. most of the students also lost their siblings, father, and close relatives, in addition to being exposed to the negative effects of the war in syria they faced. it was determined that the syrian refugee students who participated in the study had not previously taken any training in stem education. then, stem activities were applied to the 32 syrian refugee students over ten weeks. the most significant reason that the 7th graders as the sample group of the study were that the teachers giving the stem education entered into the lessons of the students for two years before and had no problem in terms of communicating and knowing the students, that the students were willing for the in-class activities. 2.3. data collection tools the school attachment scale for children and adolescents (sas-ca) to collect qualitative data in the research to evaluate the school attachments of children and adolescents in the usa in 2005 hill (2002). the scale consists of 15 items describing the students' attitudes towards the teacher, friends, and school. for instance, the cronbach alpha internal consistency coefficient was found as 0.84. the teacher interview form, stem education interview form with simple materials, and stem education reflective journal form with simple materials were applied in collecting the quantitative data of the research. the forms developed by the researcher in parallel sub-dimensions of the quantitative scales and considering the characteristics of the stem education aimed to be acquired. stem education interview form with simple materials: it consists of 5 open-ended questions describing the purpose of the stem activities, the students' belonging to stem activities, and the school after the activity. teacher interview form: in this form, the opinions of the teacher who carried out the activities were given about whether the actions affected the students after the stem activities and about the changing and developing student behaviors. in contrast, the activities were being carried out. it consists of 2 open-ended questions. the questions were created within the basis of the teacher attachment and peer attachment sub-dimension items of the sas-ca. stem education reflective journal form with simple materials: after each activity, six open-ended questions are prepared to reveal the thoughts of the students about the activities, to determine the situations in which they have problems while doing the exercises, to enable the students to think and dream after the activities and support the subdimensions of the sas-ca were included in this form, 2.4. implementation process the research was carried out by a teacher who is a professional in stem education and has entered the lessons of the syrian refugee students for two years as a teacher at the ministry of national education. the research continued for ten weeks with 2 hours per week in the science applications course. "the school attachment scale for children and adolescents" was applied with the help of arabic translators before its application as a pretest. sample applications relevant to the engineering design-based stem education were shared with the students by determining as a result of the literature reviews. the process of students’ solving the engineering problems was started with a problem situation for them to acquire scientific knowledge, use their design skills by actively participating in the design process, use their creativity, and make the design testable. the problem situation was presented with a problem scenario in which the students will need information within the scope of the science topic discussed so that they can encounter and relate to the subject in their daily lives, directing them to research and questioning. during the parachute egg activity applied within the scope of the study, the achievements for this activity were determined under the name of "science, engineering, mathematics and technology outcomes". in the applications activities, the students were asked to create groups with three and four primarily for them to study in comfort, and interventions were made to ensure a balanced distribution to the groups in case of need. the study by çepni (2017) was considered in the implementation stages of performing stem activities with syrian refugee students. they were expected to describe the problem by associating it with a problem situation we encounter in daily life after giving theoretical information journal of science learning article doi: 10.17509/jsl.v5i3.39846 481 j.sci.learn.2022.5(3).478-487 appropriate to the course of the course. for instance, “an emergency coded meeting is held in the turkish armed forces. sending supplies to children stuck in the war zone is required. in the meeting, you are asked to design material that provides a safe landing for aid to land in the war zone. “determine the problem situation, and what are your solution recommendations for the problem situation?” discuss with your group members and write down what are your ways of solution.” then, the teacher asks the students to design their products according to their determined solutions. in this stage, they were asked to decide which simple materials they could encounter daily and which could be easily found to design the solution to the problem. (for example, garbage bags in different sizes, grocery bags, rope, tape, scissors, ruler, pet cup, egg, cotton aluminum foil, and paint materials). afterward, each student group was enabled time to design their standard product and create a product with the selected simple materials. finally, the students presented their products to their peers after testing them. in this stage, each group expresses ideas about the product the other groups develop. " share the product that you have developed with your friends." the students examine their designs considering the recommendations of their teachers and peers in other groups and complete the activity. after each exercise, the refugee students were asked to reply to the stem reflective journal form with simple materials appropriate to the activity. the applied stem activities are convenient in terms of cost, time, and readily available materials. in this process, the learning processes of building, testing, recording the data and evaluating the results that enable to solve the problems, as in real life of engineers and scientists, to the students. with the expression of lachapelle et al. (2011), the engineering design is as ask, imagine, plan, design, and improve. in the asking step, the students are asked to describe the problem situation of that week given to them in the group. in the stage of imagining, they are provided to create ideas related to the product that will be created by researching the problem. they are expected to develop possible solutions with these ideas in the plan step. in the design step, they are expected to make group designs by selecting the best among the solutions. in the improvement phase, they are expected to configure the prototype and test the product they have created. after the eight activities, the students were asked to reply to the stem interview form with simple materials to learn the general thoughts of the students relevant to the activities and determine their effects. finally, the opinions of the teacher constituting the activities were taken in the teacher interview form. some of the products (car with balloon, car with ping pong, catapult) that the students produce are presented in figure 2. 2.5. data analysis the quantitative data obtained in the research were analyzed with statistical analysis methods. as the kolmogorov–smirnov (since the sample number was above 30) significance value was found below the 0,05 at the end of the analysis of the sas-catable data, it was realized that the data were insufficient in providing normal distribution. therefore, the decision was reached by considering the data's kurtosis, skewness, and median values. according to can (2016), it can be accepted that the data show a normal distribution as the kurtosis and skewness numbers are divided by their standard errors if the values are between (+1,96) and (-1,96). it considered to have a normal distribution when the median and mean values of the data are close to each other or the same. as a result of the analyses, the paired sampling t-test was applied to determine whether a significant difference was observed between the pre-test and post-test results of the scale data. qualitative data analysis was carried out on the stem education reflective journal form with simple materials, stem education interview form with simple materials and teacher opinion form after the stem education was used as the qualitative data collection tools. the data obtained figure 2. the visuals of stem student products of car with balloon, car with ping pong and catapult table 1. the order of application of the questionnaires and activities in the study pre-test application (sas-ca) 30/10/2019 my dream car 07/11/2019 jiggling ant 14/11/2019 car with ping-pong 21/11/2019 car with baloon 28/11/2019 art robot 05/12/2019 parachute egg 12/12/2019 catapult 19/12/2019 vane 26/12/2019 post-test application (sas-ca) 14/01/2020 stem interview form with simple materials 14/01/2020 teacher interview form 15/01/2020 journal of science learning article doi: 10.17509/jsl.v5i3.39846 482 j.sci.learn.2022.5(3).478-487 within the scope of the research were analyzed as appropriate to the content analysis method. the content analysis technique is an iterable and systematic technique that enables to summarise in smaller categories by using some words of a topic with codes, following specific rules (büyüköztürk, kılıç-çakmak, akgün, karadeniz & demirel, 2016). the stages that should be followed in using the content analysis can be defined as identifying the concepts, determining the units of analysis, determining the location of the data related to the subject, determining the coding categories, evaluating and interpreting the obtained results in the table. spending time with syrian refugee students for ten weeks and having the activities performed by the teacher, who had entered the lessons of the students in the sample group for two years, spending 6 hours a week with the students in the sample group support the validity of the study. the reliability formula to calculate the reliability of the research suggested by miles & huberman (1994) as reliability = consensus / (agreement + disagreement) was applied. the reliability of the study was 85% due to the content analysis depending on the consensus of the two experts. that the reliability calculations have been found as over 70% is regarded that the study is reliable (miles & huberman, 1994). so, the result reached was accepted as reliable for the research. 3. result and discussion considering this information, as a result of the data analysis, the kurtosis pre-test value of the data was found to = -1,165, the pre-test skewness value = -1.012, the mode pre-test = 4.44, the arithmetic mean pre-test x=4.4438, the median pre-test = 4.34. mode post-test was found as =4.66 arithmetic mean post-test x=4.6678, median post-test 4.61, post-test kurtosis=1.837 and post-test skewness=1.302. it was assumed that the data were normally distributed as the kurtosis and skewness pre-test and posttest values were divided by their standard errors. the resulting values were (+1,96) and (-1,96), and the values (arithmetic mean, mode, median) were close to each other. the general pre-test and post-test scores related to the ‘school attachment scale for children and adolescents’ to determine the refugee students’ school attachment skills with the stem activities applied with simple materials are presented in table 2. the pre-test and post-test scores of the school attachment paired samples t-test applied to the syrian refugee students are presented in table 2. paired sample ttest determines the effect of stem activities and specifies the differences in pre-test and post-test results. the test results show a significant difference between the pre-test score averages obtained before the application of the stem activities =55.096 and the post-test score averages after the activity =57.871 (t30: 2.13; p<0.05). thanks to this result, it can be claimed that the applied stem activities affect the school attachment (belonging) of the syrian refugee students. the paired samples t-test pre-test and post-test results related to the peer attachment, which is the sub-dimension of the school attachment scale applied to the syrian refugee students, are presented in table 3. before the t-test was analyzed in the research carried out with a single group, it was determined to have a normal distribution. the pre-test kurtosis value for the peer-attachment sub-dimension was 1.614, and the post-test kurtosis value was 1.98. the peer attachment sub-dimension pre-test is =-1.355 and post-test is =1.313 (p<0.05). the activities were applied to determine the relationship between the refugee students and their peers. the paired samples t-test was applied to determine the significant difference between the pre-test and post-test scores. in the test results, the pre-test score average was as =21.093; the post-test score average obtained after the activity = was 22.093, and no significant difference was encountered (t31:1.478, p>0.05). table 2. the paired sample t-test results for the school attachment scale for children and adolescents measurements n x̄ ss t sd p pre-test 32 55.096 4.307 2.13 30 .041 post-test 32 57.871 6.179 p<0.05 table 2. the paired samples t-test results related to the sub-dimensions of the school attachment scale for children and adolescents measurements n ss t p 1 pre32 21.09 2.319 1.478 .150 1 post32 22.09 3.108 2 pre 32 18.03 1.379 .732 .470 2 post32 18.31 2.054 3 pre32 24.05 2.113 3.039 .005 3 post32 25.58 2.029 1st sub-dimension: peer attachment, 2nd sub-dimension: school attachment, 3rd sub-dimension: teacher attachment journal of science learning article doi: 10.17509/jsl.v5i3.39846 483 j.sci.learn.2022.5(3).478-487 no significant difference was found between the pretest average obtained before the application of the stem activities as =18.031 and the scoring average received after the activity post-test as =18.312 in the pre-test and posttest results of the applied stem activities related to the school attachment situations of the syrian refugee students (t31: 732, p>0.05). therefore, it can be stated that the test results were insignificant and necessitated more activities that would provide syrian refugee students’ attachment to school. at the end of the stem activities, a significant difference was found between the pre-test=16.00 before the activities and the post-test mean score obtained after the activity=17.58 in the results of the paired samples t-test pre-test and post-test to determine the refugee students’ attitudes towards the teacher conducting the activities (t30: 3.039, p<0.05). therefore, it can be stated that the test results were significant and demonstrated that the applied stem activity applications influenced the syrian refugee students' attitudes towards the teacher. 3.1 the findings and interpretations related to the stem education reflective journal form with simple materials in the qualitative part of the research, six items take place in the reflective journals. the numbers given in table 1 represent these items. the number 1 refers to the item “were the stem activities you did with simple materials appropriate for your level? why?”; the number 2 refers to the thing “what problems did you encounter while doing stem activities? how did you solve it?”; the number 3 “what did you learn with the stem activities”; the number 4 “what did you like most about stem activities? why?”; the number 5 “how did stem activities contribute to you?”; the number 6 “what else would you like to do with these simple materials from the stem activity? why?". these question items coded with numbers were analyzed, constituting the collective and cooperative codes for each jiggly ant, ping-pong ball car, balloon moving car, art robot, parachute egg event, catapult, wind vane activity and interpreted in table 4. general and standard codes for the whole activities were placed in table 4. code 1 refers to the refugee students claiming that all activities were appropriate. some students stated that the parachute activity was challenging and claimed it was less suitable. code 2 refers that the students did not have problems while doing the exercises and had some difficulty. the students having problems referred that the engine used in the jiggly ant event did not work, so it could not walk, but this problem was resolved by mounting a new engine. in the car with ping pong ball activity, some students claimed that they could not provide the balance with the art robot. they fixed the problem by removing the balls and pencils and reinserting them to ensure balance. they also stated that they had difficulty cutting the wind vane's wings. some of them even claimed they had problems as the wheel did not turn because they stuck the wheels too close to the bottle. the car moving with balloon was appropriate to the students’ levels, but they had difficulty in their first attempts while doing the activity as they did it randomly. they applied their knowledge of measurement from mathematics and engineering to fit the covers and balloon to overcome the problem correctly. code 3 refers to learning to work collaboratively in the activities, build circuits, do new activities, and air resistance in parachute activity. in code 4, the students claimed that they enjoyed the movement of their car, which turned into a product, the waddling ant walking, the rotation of the flag they attached to the car with ping-pong, the collaborative work in the group, the art robot drawing circles, the wind vane's turning most among the activities. in code 5, the students claimed that the lesson was fun, they learned a new activity, it was beneficial to work with the group, and they learned that the catapult device was used in wars. we can refer that the stem activity provides positive development in students, as there is new learning, the lesson is entertaining, there is a problem, and there is cooperation while doing the activity. in code 6, the students referred to some expressions, including what else they wanted to do with these materials. as we consider, they stated that wished to design activities such as a house with a stick, robot, ship, car, plane, floating plane-engine, flying ship-house-carballoon-elephant-lion, speaking car, floating plane-train, swing, helicopter, brace, rocket, make fire with sun rays, clock, volcano, color wheel, flying lion and watch. as it is understood from these wishes, we can state that students are generally interested in moving objects. in addition, it can be claimed that the stem activity here influenced the students’ world of thought and creativity. 3.2. findings and interpretations related to the stem education with simple materials interview form the stem education with simple materials interview form applied to the syrian refugee students as the post-test consists of 5 items. the numbers written in italic in the table 5 represent the items. for instance, the number 1 represents the question ‘do you want to get stem education with simple materials?’. while applying the school attachment scale for children and adolescents used table 4. the analyses of reflective journal form with simple materials code name frequency(f) 1. appropriate-partly appropriate 193-9 2.no matter-a bit difficult 202-4 3. making robots 73 new activity 114 cooperation 20 4. loved-enjoyed 203 5. entertaining and good lesson 194 6. designing authentic products 385 journal of science learning article doi: 10.17509/jsl.v5i3.39846 484 j.sci.learn.2022.5(3).478-487 in the research as the pre-test and post-test, the students were helped to understand the items in the questionnaire better by translating them by the arabic teachers. the latter was working in the school and ensured to reply to them. the students’ answers to the interview questions were coded in table 5 after the stem activities with the syrian refugee students. for example, according to the students' solutions, the first question, “do you want to get stem with simple materials?” was coded as activity code: 1. as the students loved the activities much, they claimed that they wanted to take stem education and that they wanted to be engineers. therefore, they thought of going to another place for a more comprehensive stem education, and the school was better with the activities. “yes, i want to get a stem education because i want to be an engineer.” “i want to get stem education to make a talking and walking robot.” “i liked the activities much. so i want to get a stem education to learn new activities.” code:2 ‘activity’ code was created for the item “do you want to get advanced stem education?” it was stated that the students wanted to get advanced education as the activities in this code attracted their attention; some students were interested in engineering, loved the teacher who gave the stem education, and learned new information in this code. “yes, i want to get advanced stem education. because i want to be an engineer, and i loved those activities much.” “i want to get advanced stem education. i want to travel to space. i want to learn new things. the course is entertaining. i want to make robots.” code:3 46% of the students replied, "did the stem education affect your career choice?” however, that did not have any effect 53.3% claimed that they decided to change their ideas related to their future jobs after the stem education. at the same time, the students who changed their minds about their job preferences referred that they wanted to be mathematics teachers or engineers in general, and those who did not claim that they wanted to choose jobs such as a doctor, nurse, or lawyer. code: 4 to the item “did the stem education with simple materials influence your school belonging?" all of the students gave that it had an effect. the students also stated that they loved the activities and increased their curiosity. moreover, as the teacher did the activity, they also loved the teacher, and they liked the school more after the activities as they had a good time during the two hours. ”…this school is like my home, my teacher is like my mum, and the activities are enjoyable; i hope the activities will continue….” “…the school is like my home. i learn new things at school….” code: 15 the code "friend" was created for the question " did the stem education with simple materials affects your communication with your friends?" the refugee students said they communicated with friends and helped each other in the group, and the group members listened to each other while doing the activities. “yes, i learned to help my friend, and they helped me, too….” “i learned turkish in this activity. i learned new things….” “yes, i see; because i know turkish a bit, i learned more after the activities….” in general, as the data in the stem education with simple materials interview form were taken into account, it can be claimed that the activities carried out for eight weeks contributed to the student's development and changed the students’ solidarity, communication, attitude toward school, lesson, teacher; influenced the job preferences of the 53% of the students. 3.3. the findings and interpretations related to the teacher interview form aster the stem education the teacher, who carried out the stem activities for eight weeks, was asked, after the activities, the question, "what kind of changes did you observe among the students? were any changes in the students' attitudes towards the school, the science applications course, and you after the stem activities?" the teacher claimed that there was a positive increase in the attitudes of the refugee students towards her, and awareness was observed as few teachers perform such activities at school. she also stated that the students loved the science applications course more after the activities and were more willing to come to school on the days when the activities would be performed. the teacher replied the question, “could you explain how the students communicated with you and their friends in the classroom while they were performing the activities?" by sharing the activities while the students were doing the activities, they interacted in the group and with the other groups; they helped the transfer students while they were doing the activities, and the students who were misbehaving in the classroom warned each other not to make their teachers upset. 3.4. discussion the basic aim of the research is to explore the effect of stem activities on the sense of school belonging of the 7th-grade refugee students. the limitations of the study can be claimed as that the number of students were restricted to 32, that the sample groups only consisted of the refugee students, that the activities were only eight and that only table 5. findings related to the stem education with simple materials ınterview form code name frequency (f) rate (%) 1activity 32 %100 2 activity 32 %100 3-no 14 %46,6 3 -yes 16 %53,3 4 -belonging 32 %100 5 –peer 32 %100 5 -peer 32 %100 journal of science learning article doi: 10.17509/jsl.v5i3.39846 485 j.sci.learn.2022.5(3).478-487 the experimental group exists, that is, there was no control group. the strengths of the study can be stated as getting help from an expert in the field of stem, data collection, analysis and that follow-up of the process were carried out by three experts, that the study was supported by quantitative and qualitative data collection tools by using a mixed method. the students' achievement, students' feeling safe at school, their relationships with their friends, and the teacher's supportive attitude are effective in creating a sense of school belonging. on the other hand, the study's qualitative data consist of the findings determining that the activities developed the students’ sense of school belonging. accordingly, it was found that the stem activities applied to the syrian refugee students cause a significant change in their school attachment skills, productive time at school, and contribution to their development. yildiz, alkan & cengel (2019), supporting the finding that the positive relationship between the school belonging and school, suggested how happy students feel in their group of friends, in the classroom, or at school, how they feel belong to their social environment, the undesirable resistant behaviors will decrease to that extent and accordingly, perceptions of school life quality will increase positively. as mentioned above, it is evident from the students’ attitudes towards their teacher who gave the stem education at the end of the interview forms applied in the study. it is noticed that in addition to the activities enabling to develop of positive behaviors such as effective interaction in the group, listening, and expressing themselves, the students can construct their engineering processes with stem activities and not only gain knowledge but also cause the desire to create new products that they expect. (günalan, 2018), who refers that the communication of students with their teachers in their daily lives at school determines the structure of the social atmosphere, claims that the students' moral and social attitudes are the result of their one-to-one relationships with their teachers. in their study related to the school climate, in their study, ersanlı & koçyiğit (2017) determined that teachers and school managers should establish positive communication with students for students do not feel lonely at school and feel belonging to the school and create a positive school climate. çakır & altun yalçın (2020) stated that stem education for preschool children increased student success, and they liked the lesson. a significant difference was encountered after the activities in the teacher attachment sub-dimension of the sas-ca of the study. alan (2020) claimed that stem applications contribute positively to the children’s cognitive process skills, knowledge, skills, emotions, and tendencies. çakır & altun yalçın (2021a) claimed that the stem education that they applied to the pre-school students increased their achievements and made them love the course. yetkin & aküzüm (2022) stated that stem education developed the students’ understanding of learning, improved their personal and social developments, contributed to acquiring and using knowledge, and positively affected their attiudes related to stem. according to the results reached from the teacher interview form and student stem interview form with simple materials, it was found that the stem activities developed the sense of belonging to the school by attracting the attention of the refugee students, increasing their desires, and being happy to come to school. in addition, students who are more interested in school and lessons are more successful, want to continue their education after finishing secondary school, and want to improve themselves, which are among the positive effects of stem education. in their study, karışan & yurdakul (2017) explored the effect of the stem applications on the attitudes of the 6th graders towards stem. at the end of the study, it was claimed that the students’ attitudes toward the stem field increased positively in the study by gökbayrak & karışan (2017), the 6th graders refer to their opinions on the stem activities as they were entertaining, instructive, and motivating. similarly, in addition to the studies suggesting that in addition that the stem education approaches increased the students' positive attitudes, motivations, and achievements, there are even studies referring that the activities make the process more entertaining (kucuk & sisman, 2017). according to the reflective journal form results of the study, it was concluded that the stem activity applications had a positive effect on students. şanlı & özerbaş (2021) claimed that stem education affected the secondary school students’ general motivation and attitudes toward science positively, provided their socialization. in a study, uğraş (2018) observed that stem activities positively affected the seventh graders’ attitudes towards courses, scientific creativities, and motivation beliefs. piila, salmi, & yhuneberg (2021) claimed in their study that the steam learning model increased students' academic achievements. it was concluded that the students found the application easy, attracted their attention, aroused a sense of curiosity, contributed to their learning, developed their skills of working with the group, sharing, cooperation, and communication, and created a desire to do creative new things. the results demonstrate similarity with the studies in the literature. topsakal & altun yalçın (2020) stated that problem-based stem education positively affects students' emotions, thoughts, and behaviors. in the form results held at the end of each activity, it was stated that the students learned how to conduct scientific research, discussion, and questioning. in addition, the students referred that their imagination and engineering skills were developed and that students wanted to choose stem fields as a profession in the future. in the research carried out with 7th graders, özcan & koca (2019a) stated that stem activities positively affect group work. genç & uğraş journal of science learning article doi: 10.17509/jsl.v5i3.39846 486 j.sci.learn.2022.5(3).478-487 (2018) claimed that stem activities encourage students to collaborative learning. çakır & altun yalçın (2021b) stated that they support cooperative learning in the results of their study, which includes montessori approach-based stem education. karakaya, yantırı, yılmaz & yılmaz (2019) suggested that the students participated actively while performing the stem activities, provided the opportunity to reveal their creativity, and affected the success of the lessons. kahraman & doğan (2020) investigated secondary school students’ opinions towards stem activities. the students generally expressed that they achieved through ingroup cooperation with the activities, produced original products, were willing in the activities, they loved and found the activities interesting. yılmaz baltabıyık & duru (2021) found that the stem applications developed the secondary school students’ scientific creativity and understanding of science concepts. neccar (2019) claimed that the students stated that the course, related to the process, was better during the stem activities and ensured their learning by having fun. özcan & koca (2019b) stated, in their study with 7th-grade students, that the stem applications made the learning environment more fun, contributed positively to learning information, and supported the development of self-confidence. meral & altun yalçın (2019) claimed that stem activities changed their viewpoints on science and created an effective course process by making the science course entertaining. 4. conclusion in conclusion, a significant difference was found between the pre and post-test scores of the quantitative data collected with the school attachment scale for children and adolescents. in other words, it was found that the engineering design-based stem activities that were performed developed the syrian refugee students’ individual and social developments. the students learned with fun during the activities, contributed to their communication skills, created a feeling of curiosity, and positively affected their attitudes towards the course and teacher and their school attachments. the quantitative data of the research included the findings identifying that the activities developed the students’ feelings of affection to school. accordingly, it was found that the stem activities provided significantly to the syrian refugee students’ school attachment skills, spent effective time at school, and contributed to their development. these recommendations are expressed to shed light on future research: stem application can be given to the students not at only one class level but beginning from the pre-school period. the number of activities and groups can be increased. the refugee students’ parents can be invited to school and informed about the importance of stem education and encouragement for their students to get stem education. stem applications can also be given in other courses according to the curricula topics rather than in only one course. references akgunduz, d. (2016). a research about the placement of the top thousand students placed in stem fields in turkey between the years 2000 and 2014. eurasia journal of mathematics, science and technology education, 12(5), 1365-1377. alan, u. (2020). improved stem for pre-school children examining the effects of the education program, phd thesis, hacettepe university educational sciences instrument, ankara. altun yalçın, s. & yalçın, p. (2018). investigation of metaphorical perceptions of science teacher candidates on stem education. the journal of academic social science studies, 70, 39-59. 10.9761/jasss7705. altun yalçın, s. (2019). the effect of ıntegrated stem education on teachers and the opinions of teachers. journal of social humanities and administrative sciences, 5(20), 948-963. arıkan, g. (2015). examination of the sense of belonging to school and quality of school life in sports high and anatolian high school students: the example of the southeastern anatolia region. (phd thesis), çukurova university institute of health sciences, adana. aydın, e. & karslı baydere, f. (2019). seventh grade students' views on stem activities: an example of separation of mixtures. ondokuz mayıs university faculty of education journal, 38(1), 35-52. doi:https://doi.org/10.7822/omuefd.439843 baki, a. & gökçek, t. (2012). an overview of mixed methods research. electronic journal of social sciences (electronic), 11(42), 1-21. balak, d. (2017). examination of the relationship between primary school students' emotional intelligence levels, peer bullying and sense of belonging to school. published master's thesis, marmara university institute of educational sciences, istanbul. bellici, n. (2015). examination of school attachment in secondary school students in terms of various variables. abant i̇zzet baysal university journal of education faculty, 15(1), 48-65. büyüköztürk, ş., kılıç-çakmak, e., akgün, ö. e., karadeniz, s. & demirel, f. (2016). scientific research methods. 21st edition. ankara: pegem publishing15:14 çakır, z. & altun yalçın, s. (2020). evaluation of stem education in preschool education in terms of teacher and parent views . international journal of active learning, 5 (2), 142-178. doi: 10.48067/ijal.823224 çakır, z. & altun yalçın, s. (2021a). the effect of montessori approachbased stem activities on pre-service teachers' attitudes towards science and science teaching. opus international journal of society researches, 17(35), 1895-1924. doi: 10.26466/opus.831879 çakır, z. & altun yalçın, s. (2021b). the investigation of the effect of montessori approach-based stem activities on the problem-solving skills of pre-service pre-school teachers. journal of theoretical educational science, 14 (2) , 93-119. doi: 10.30831/akukeg.824773 çakır, z., & yalçın, s.a. (2022). the effect of the montessori approachbased stem activities on the pre-school pre-service teachers’ lifelong learning. pamukkale university journal of education,1-32, doi:10.9779 pauefd.1022966. can, a. (2016). quantitative data analysis in scientific research process with spss. 6th edition, pegem academy. cemalcılar, z. (2010). schools as socialization contexts: understanding the impact of school climate factors on students' sense of school belonging. applied psychology: an international review, 59, 243–272. çepni, s. (2017). stem (+ a/+ e) education from theory to practice. (1st edition). pegem publishing: ankara. creswell, j. w. (2003). research design: qualitative, quantitative, and mixed methods approaches (2nd ed.). thousand oaks, ca: sage. creswell, m. (2006). a question of balance: how france and the united states created cold war europe (10). harvard university press. dugger, e. w. (2010). evolution of stem in the united states. 6th biennial international conference on technology education research. australia. journal of science learning article doi: 10.17509/jsl.v5i3.39846 487 j.sci.learn.2022.5(3).478-487 duman, t. (2019). the importance of education for social cohesion: the case of syrians in turkey. sefad, (41), 343-368. ersanlı, k. & koçyiğit, m. (2013). psychometric properties of belonging scale. turkish studies, 8(12), 751-764. genç, z. & uğraş, m. (2018). examination of pre-school teacher candidates' stem teaching orientations and views on stem education. bartın university journal of education faculty, 7(2), 724-744. gökbayrak, s. & karışan, d. (2017). examination of sixth grade students' views on stem-based activities. journal of field education research, 3(1), 25-40. gonzalez, h. b. & kuenzi j. (2012). congressional research service science technology, engineering, and mathematics education: a primer. günalan, n. (2018). what affects secondary school students' quality of school life, sense of belonging to school and school climate?. published master's thesis, adnan menderes university institute of social sciences, aydın15:15 hill, m. d. (2002). the effects of integrated mathematıcs / scıence currıculum and instructıon on mathematıcs achıevement and student attıtudes in grade sıx. (doctoral thesis),texas a&m, and. kahraman, e. & dogan, a. (2020). views of middle school students for stem students. anadolu teacher magazine, 4(1), 1-20. doi: 10.35346/aod.728000 karakaya, f., yantırı, h., yılmaz, g. & yılmaz, m. (2019). determination of primary school students' views on stem activities: a case of 4th grade. international journal of turkish educational sciences, 7(13), 114. karışan, d. & yurdakul, y. (2017). the effect of microprocessor assisted science-technology-engineering mathematics applications on the attitudes of 6th grade students towards these fields. adnan menderes university faculty of education journal of educational sciences, 8(1), 37-52. kucuk, s., & sisman, b. (2017). behavioral patterns of elementary students and teachers in one-to-one robotics instruction. computers & education, 111, 31-43. lachapelle, c. p., cunningham, c. m., jocz, j., kay, a. e., phadnis, p., wertheimer, j. & arteaga, r. (2011). engineering is elementary: an evaluation of years 4 through 6 field testing. boston, ma: museum of science. meral, m. & altun yalçın, s. (2019).the effect of stem education on the attitudes of pre-service primary teachers towards their professions and pre-service teachers' views on steme education. international social sciences studies journal, 5(50), 6649-6660. miles, m. b., & huberman, a. m. (1994). qualitative data analysis: an expanded sourcebook. sage. neccar, d. (2019). it addresses the views of stem education in science education from secondary school education to science education education and stem. master's thesis, gazi university, ankara. özcan, h. & koca, e. (2019a). adaptation of the attitude scale towards stem into turkish: validity and reliability study. hacettepe university journal of the faculty of education, 34(2) , 387-401. özcan, h. & koca, e. (2019b). the effect of stem approach and pressure subject teaching on secondary school 7th grade students' academic achievement and attitudes towards stem. education and science, 44(198), 201-227. özdemir, s. , sezgin, f. , şirin, h. , karip, e. & erkan s. (2010). examination of variables that predict primary school students' perceptions of school climate. hacettepe university journal of education, 38(1), 213-224. özden, y. & turan, s. (2013). introduction to educational science. ankara özgök, a. (2013). examination of secondary school students' sense of belonging to school, level of commitment to friends and empathetic classroom atmosphere perception. unpublished master thesis, çukurova university, institute of social sciences, adana. piila, e., salmi, h. & yhuneberg, h. (2021). steam-learning to mars: students’ ideas of space research. education sciences, 11(3), 122. saidi, a. (2014). turkey's policies towards solving the education problem of syrian refugees. süleyman demirel university faculty of arts and sciences journal of social sciences, 2014 (31), 267-305 . samur, e. & altun yalçın, s. (2021). the effect of stem activities on reflective thinking skills of pre-school teachers. bilge international journal of social studies, 5(1), 65-76. doi: 10.47257/busad.946745 şanlı, m. & özerbaş, d. h. s. (2021). the effect of stem activities on students' attitudes towards stem fields and their motivation for science. manisa celal bayar university journal of social sciences, 19 (3) , 139-154 topsakal, i. & altun yalçın, s. (2020). investigating the effects of problem-based stem education on students' learning climates. international journal of scholars in education, 3(1), 42-59. uğraş, m. (2018). the effects of stem activities on stem attitudes, scientific creativity and motivation beliefs of the students and their views on stem education. international online journal of educational sciences, 165-182. unhcr. (2018). syria emergency. https://www.unhcr.org/syriaemergency.html. uysal, h., & dincer, c. (2012). preschool bullying. journal of theoretical educational sciences, 5(4), 468-483. yetkin, n. & aküzüm, c. (2022). examining the relationship between primary school fourth graders' understanding of learning and attitudes toward stem education. journal of human and community sciences research, 11(1), 744-769. doi: 10.15869/itobiad.951520 yildiz, e. p., alkan, a., & cengel, m. (2019). teacher candidates attitudes towards the stem and sub-dimensions of stem. cypriot journal of educational sciences, 14(2), 322-344. yılmaz baltabiyık, d. & duru, m. k. (2021). impact of stem applications on conceptual understanding and creative of secondary school students. journal of research and experience, 6 (1), 22-33. https://www.unhcr.org/syria-emergency.html https://www.unhcr.org/syria-emergency.html a © 2019 indonesian society for science educator 97 j.sci.learn.2019.2(3).97-107 received: 1 july 2019 revised: 28 july 2019 published: 31 july 2019 the effect of documentary films on preservice science teachers’ views of nature of science hakan şevki ayvacı1, dilek o� zbek1 1department of mathematics and science education, trabzon university, turkey *corresponding author. ozbekdilek86@gmail.com abstract understanding the nature of science, one of the most important dimensions of scientific literacy, is regarded as an absolute necessity in science education. to teach students the nature of science, science teachers should emphasize the nature of science in the classrooms. this is possible through the training of science teachers with knowledge of the nature of science. in this study, documentary films were used to teach preservice science teachers about the nature of science. this study aims to investigate the effect of nature of science course conducted with documentary films on preservice science teachers’ views of nature of science. the study, in which the experimental design was used, was conducted with 30 preservice teachers in nature of science and history of science courses. throughout the courses, documentary films were watched, and nature of science aspects of the documentary films was discussed. the views of nature of science questionnaire (vnos-c) was used as pre-test and post-test, and the data were analyzed with spss. as a conclusion, preservice science teachers’ views regarding the nature of science were enhanced after the implementation. keywords nature of science, preservice science teachers, documentary films 1. introduction the developments in science and technology have affected the education policies of countries, and these alter education systems necessary. in parallel to those alterations, significant education institutions like the american association for the advancement of science (aaas) and the national research council (nrc) emphasize that educating each individual as scientifically literate is one of the main goals of science education (american association for the advancement of science, 1990; national research council, 1996). in this context, understanding the nature of science (nos), which is one of the key components of scientific literacy, is accepted as an absolute need in science education (meichtry, 1992; lederman, 2007). nature of science is typically described as an epistemology of science, values, and beliefs inherent to the scientific knowledge, science as a way of knowing, and also it includes the historical, philosophical and sociological aspects of science (lederman, 1992). nature of science involves understanding what science is and what role it plays; who scientists are and what roles they play; the nature of scientific evidence, observations, facts, rules, laws, and the scientific method; and how science is done (taşar, 2003). since nature of science has a complex structure, which includes different disciplines like sociology of science, philosophy of science and history of science, researchers focus on specific features to make definitions and put emphasize on these features. these features, which are called aspects of nos, are: scientific knowledge is tentative (subject to change); science is empirically based (based on or derived from observation of the natural world); science is inferential, imaginative and creative; science is subjective and theory-laden; science is socially and culturally embedded (lederman, abd-el-khalick, bell, & schwartz, 2002). because technology has become an essential part of education, videos, movies, and documentaries have started to play a role in the education process. videos have become an important teaching tool in terms of visualizing abstract terms and appealing to multiple senses. numerous research studies about the effects of using documentaries in teaching nos, which is a part of science education, are mailto:ozbekdilek86@gmail.com journal of science learning article doi: 10.17509/jsl.v2i3.17998 98 j.sci.learn.2019.2(3).97-107 encountered in the literature. seçkin-kapucu, cakmakci & aydoğdu (2015) emphasize that documentaries linked to science content would be fruitful for teaching nos to secondary school students about specific topics. moreover, in research about teaching nos, it is stated that students also want to benefit from videos and activities in lessons (dereli, 2016). science teachers have a significant influence on their students for learning nos. a science teacher needs to have information about nos to include activities about nos in his/her lessons, which means science teachers must have a new and updated point of view about nos if the opinions of students about nos are to be improved (sorensen newton & mccarthy, 2012). in a study about science technology society course, it is stated that introducing scientific videos to preservice science teachers can be useful especially help them understand that science is affected by the social and cultural values of the society. when the studies regarding this subject are reviewed, it is implied that the interests of students in science and the scientific literacy level of students would be increased if science concepts were taught via science-fiction movies (efthimiou & llewellyn, 2007). in a two-week study with 11 teachers that was conducted by bloom, binns & koehler (2015), the effects of documentaries in the subject of teaching nos were researched and it is stated that these effects are limited. in the suggestions of the same study, it is stated that these effects should be investigated by applying these studies over a more extended period. in this study, documentaries about scientific events and the lives of scientists were watched by preservice science teachers (pst) who attended the nos course for 12 weeks and the effects of these documentaries on the perceptions of pst were researched. research questions; (1.) is there a statistically significant difference between posttest scores of experimental and control groups on understanding nos? (2) is there a statistically significant difference between pretest and posttest scores of the experimental group on understanding nos? 2. method 2.1 research design in this research, quasi-experimental design with the pretest-posttest control group was used to determine the effects of documentary films on pst’ understanding of nos as seen in table 1. this design refers to the application of the experiment and the interpretation of data without random assignment (cook, campbell & shadish, 2002). the independent variable of the study was documentary films as a way of teaching nos, and the dependent variable is preservice teachers’ understanding of nos. before and after the course, vnos questionnaire was applied to pst, and the effect of the documentaries on the views of pst’ nos understanding was evaluated with a comparison of preand post-tests of the experimental and control group. 2.2 sample / participants this study was conducted in the “nature of science and history of science” course in the 2017-2018 spring term. participants of this study consisted of third-grade preservice teachers who were studying in the science teaching program and taking this course. the preservice teachers had a similar background; they took the same courses in the first two years of the college and had no experience about nos. sixty preservice teachers were included in this study who continued to take this course and participate in pre and post-tests. the experimental group consisted of 30 preservice teachers (12 females and 18 males), and the control group consisted of 30 (14 females and 16 males) preservice teachers. the age of the sample was about 20 years old. the experimental group and the control group were chosen randomly. 2.3 intervention the study was implemented in science teaching program students at a state university in turkey in the “nature of science and history of science” course in the 2017-2018 spring term. the course was taught for two hours per week and spanned 14 weeks (one semester). pre and post-tests were applied at the first and the last week of the semester. at the remaining 12 weeks, documentary films were used as a way of teaching nos at the experimental group, regular content was used for teaching science at the control group. both groups were taught by the same instructor and held explicit reflective discussions about nos. all the activities and documentary films were addressed the seven targeted aspects of nos. experimental group. documentaries about scientific events and the lives of scientists were watched by experimental group and discussions about these documentaries were made via the questions asked by the instructor. eleven documentaries were chosen carefully by the researcher and instructor which would be suitable to highlight nos aspects like the characteristics of scientific knowledge, progress and historical periods of science, the science-society relation and the lives of scientists. table 2 shows the content of documentary films and nos aspects emphasized within each documentary films. documentary films have been watched in each lesson, and questions have been directed to pre-service science teachers by pausing the video at the points which gives the possibility to relate aspects of nos and discussions were provided by specifying the ideas of the pre-service science teachers. for table 1 design of the research group pre-test treatment post-test experimental group vnos-c documentary films vnos-c experimental group vnos-c regular content vnos-c journal of science learning article doi: 10.17509/jsl.v2i3.17998 99 j.sci.learn.2019.2(3).97-107 example, before the documentary film about galileo “does beliefs and culture of a society affect science?” was asked to pst and they were let to discuss it briefly. at the end of the documentary, the same question was asked and after pst had stated their opinions, the instructor stressed the social and cultural aspect of nos. the length of each documentary was about 45-60 mins and watching each documentary and the discussion took two lesson hours. control group. the control group was received regular content which was determined by the higher education table 2 introduction of documentaries and nos aspects introduction of documentaries nos aspects 1.mankind-the story of all of us: inventors 1&2: in this documentary, the adventure of mankind’s survival is told. we witness mankind challenging dangers while trying to find new places and resources on earth. this documentary unrolls the story of mankind existing on earth. beginning with the first man's existence, human beings will continue with the adventure of hunting and gathering, learning about the use of agricultural activities, which is a great revolution, and how to learn to use various tools of human beings and learn to survive by developing them. after that, the first battles emerge as a result of the negative aspects of agricultural activities and the emergence of disputes across borders. when it puts up a fight to survive in its early days, mankind uses his effort to destroy his own kind in time and mankind now consumes the resources on earth rapidly. the development of scientific information, the sciencetechnology interaction, the processes of science history and the interaction of science with the social and cultural environment that it belongs to are mentioned. 2. avicenna ‘’the emperor of medicine’’: the documentary, which takes place in the documentary series named ‘the candles of asia’ that highlight scientific works in the islamic world in the middle ages, is about avicenna’s life. this documentary gives information about avicenna’s life as it gives information about his scientific study understanding. avicenna, a muslim persian scientist and philosopher, had a good education from a famous scientist of his age. with the help of his intelligence and strong memory, he reached his teachers’ levels and had in-depth knowledge in various subjects like philosophy, literature, mathematics and medicine at the age of 14. he focused on medicine and improved new treatments. he got the ‘doctor’ title at the age of 19. avicenna, who was accepted as one of the best doctors of his age, carried out his work without sleeping at night and revealed many works at a young age. ‘kitabu’ş şifa’, one of the biggest works of avicenna, is an encyclopedia which gathers the information of his age in areas of logic, physics, geometry, astronomy, mathematics, music and metaphysics. ‘kanun fi’t-tıbbise, maybe avicenna’s most famous work, has been taught as a textbook in europe for 600 years and is named as the bible of medicine. the subjective aspect of science, the characteristics of scientists and their personal lives, and the interaction of science with the social and cultural environment that it belongs to were mentioned. 3. al-farabi “philosophy courses in transoxiana”: the documentary, which handles the famous islamic philosopher al-farabi’s scientific works in science, art, and philosophy without isolating them from the cultural, social and political atmosphere of the era, can be interpreted as a productive and extraordinary trip to culture and humanity, as well as being a biographical documentary. al-farabi, known as the first muslim philosopher, produced works about philosophy and walked from city to city by following in aristotle’s footsteps. he made an effort to ensure that philosophy and religion did not interfere with each other. in this documentary, the subjective and experimental factors of nos are mentioned. in addition to this, science and society relation is also mentioned. 4. the life of albert einstein: the documentary, which starts with einstein’s youth years, mentions einstein’s personal life as well as his academic life. his close relation with theoretical physics started when he moved to switzerland after quitting his school in germany because of its education style and family problems. einstein, who had a lively personal life, started his patent officer job when he broke up with his great love and student milena maric. the science environment, which became very political because of the world wars, deeply affected einstein’s life. einstein, who was known as anti-war, went to the usa as a refugee after he had certified the theory of relativity and settled there. einstein, who did not lose his hope and inspiration, said: “the important thing is not to stop questioning; curiosity has its own reason for existing.” einstein, who kept on having relationships in his late years, found himself in a big challenge. in this documentary, the relation of science with the society that it is produced in, the place of imagination in science, and the relation of scientific theory and law are mentioned. in addition to this, the subjectivity of scientific information and the private lives and characteristics of scientists are also mentioned. continued journal of science learning article doi: 10.17509/jsl.v2i3.17998 100 j.sci.learn.2019.2(3).97-107 council (hec, 2018). the content was “philosophy of science (content and purpose of philosophy, paradigms, philosophical thoughts and its effect on development of science), nature of knowledge (ontology, epistemology, nature of scientific concepts, scientific knowledge and it’s characteristics), history of science, scientific literacy and nos aspects , the role of nos in curriculum and teaching. nos (nos teaching approaches and nos assessment), nos activities (integrated and non-integrated nos activities), nos and science, technology and society relation” according to science teaching undergraduate program (hec, 2018). lessons were taught mostly by table 2 introduction of documentaries and nos aspects (continued) introduction of documentaries nos aspects 5. who is galileo galilei? what contribution has he made to science and technology?: galileo, who shook the understanding of science in the medieval age, caused a revolution by questioning the church belief which had the control of the era, and aristotle’s knowledge. galileo was born in italy in the 16th century and studied there in areas of medicine, mathematics, and philosophy. galileo, who had been doing experiments about physics since his early years, got his professorship at the age of 25. in 1609, he invented a more advanced telescope by examining a telescope, and he had a chance to make observations no one had done before. according to his observations, he suggested the heliocentric model and he published his studies about it. since he collided with the church which advocated the geocentric model, he was judged more than once by the vatican. teaching his theories and publishing his books was banned and he was sentenced to lifetime house confinement. galileo became a symbol for the science-religion conflict due to his challenge with the church in his lifetime. in this documentary, the importance and difference of observation and implication while science is produced, the interaction of science with social and cultural values and especially religion-science are mentioned. in addition to these, the sciencetechnology relation, the experimentality of science, the change of scientific information in time, and the paradigm change in science are also mentioned in this documentary. 6. nikola tesla documentary: nikola tesla undoubtedly is one of the most distinctive personalities of science history. trying to understand his ideas took many years. tesla, who said that his photographic memory and creative genius were a heritage from his mother, started to make inventions in his early years. despite illnesses and hardships in his childhood, he never quit and made numerous inventions including earthquake machines, death rays, and universal wireless energy. we get closer to the mind and ideas that are behind those strange inventions. in this documentary, the subjectivity of science, the place of creativity and imagination in science, and the personal lives of scientists are mentioned. 7. thomas edison documentary: in this documentary, which gives us the possibility to know edison better, we witness edison, whose life affected our lives. edison, who was born in 1847 in the usa, started to be interested in science from his early years. he started to do his own experiments at the age of 10 by creating his own lab at home. edison made many inventions in his life and his patents were accepted in many countries. the inventor, who has many inventions like the gramophone, carbon microphone and the cinema machine, also founded his own company by showing entrepreneurship and he succeeded in making his inventions commercialized. in this documentary, the effect of scientific information on the society that it is produced in and the effect of society on science, the place of imagination and creativity in science, the science-technology relation and the importance of experiments in science are mentioned. 8. time bending beyond the universe: a fact which is accepted as stable and accurate anywhere in the universe means it is universal. but according to albert einstein’s theory of relativity, this is not true. this means that time is not the same in all the universe, it changes from observer to observer, and it is relative. as a result of mass which creates warps in space-time, time bends and time flows more slowly from the observer who is at that place to another one. moreover, time is not universal right there. let me explain time bending with a more accessible example. lay a sheet on a bed without leaving wrinkles. here, the sheet without wrinkles represents two a dimensional space-time plane. when we put an iron marble on this sheet, the marble will sink into the sheet a little bit. just like in the example, time also can be bent by a mass represented by a marble. when the mass gets bigger, the bending will be greater. if the mass is more than the spaceplane can carry, the space plane will collapse and a black hole will exist. in this documentary, the importance of observation and deduction in the production of scientific knowledge, the paradigm change and the scientific laws and the theories that scientific information can change and develop over time are mentioned. 9. isaac newton documentary: isaac newton, who was born in the uk in 1642, spent most of his childhood with his grandmother because of family problems. since he was curious about science, he read many books and designed various tool models. he had to work in various jobs besides his education life because of his poor family conditions. he started to study in cambridge in 1661 but he had to pause because of the plague epidemic. in that period, he moved to a farm and continued to study there. after the epidemic, he came back to cambridge university and he continued to work there as a mathematics professor. he focused on mechanics and he published an important work named principia. in this documentary, the personal characteristics of scientists, the subjective side of scientific information, the importance of imagination and creativity in science, science-technology and the interaction of science with the society that it is produced in are mentioned. 10. encounter with pluto: pluto, which is billions of miles away, is one of the mysterious celestial bodies in our solar system. a space vehicle named new horizons was taken into service to investigate pluto more closely and sweep the mysteries away. the space vehicle, whose duty is to study pluto and its satellites, has been traveling in the solar system since january 2006 in order to reach pluto. new horizons, which has been traveling for 10 years to complete its duty, will only have a few minutes to view pluto. if it ends its duty, we will have the chance to see the deepest parts of the solar system for the first time. this documentary emphasizes the importance of science experiments, observations and inferences and changeable scientific knowledge. journal of science learning article doi: 10.17509/jsl.v2i3.17998 101 j.sci.learn.2019.2(3).97-107 powerpoint presentations. both integrated and nonintegrated nos teaching activities helped pst to learn nos. for example, for explaining the distinctions between observation and inference, both tricky tracks activity developed by lederman and abd-el-khalick (1998) and atom models were used. after the activities, reflective discussions were guided by the instructor by asking questions about emphasized nos aspects. 2.4 data collection instruments the questionnaire (vnos-c), which consists of openended questions, was applied twice as preand postapplication in order to evaluate the effect of the documentaries watched in the “nature of science and science history” course on the views of preservice teachers about nos. vnos (views of nature of science) questionnaires, whose most basic form was created by lederman and o’malley (1990), were designed to reveal students’ thoughts about one or more than one nos fact and they consist of open-ended questions which give students the chance to state their thoughts without being under the effect of individual options. in their studies about the reliability and validity of vnos questionnaires, which were conducted in-depth, lederman, abd-elkhalick, bell, & schwartz (2002) stated that vnos had been a valid measuring tool in order to understand the perspectives of students about nos. in this study, a vnos-c questionnaire which was suitable for preservice teachers was used and there are different versions that are suitable for different age group levels. the survey, which evaluates the opinions of preservice teachers about nos, consists of 10 open-ended questions. 2.5 data analysis primarily, the answers which were given to the questions in the vnos-c questionnaire by preservice teachers were classified as naive, transitional and informed, similar to khishfe and lederman’s (2006). each aspect of nos is not questioned in only one item in this questionnaire which sought the opinions of preservice teachers about the aspects of nos. so, the answers given to all questions in the questionnaire were examined carefully in order to classify each element of nos. the answers of preservice teachers were classified as naïve when a preservice teacher didn’t demonstrate any meaningful understanding related to the element in question and informed when a preservice teacher provided evidence of meaningful understanding related to the aspect of each question. in addition to this, if a preservice teacher demonstrated a useful understanding of an aspect but also had some misconceptions, the answers were classified as transitional. after this, answers of preservice teachers for each aspect were scored from 0 to 2 for naive, transitional and informed. some examples of preservice teachers’ answers are shown in table 3. pst’ responses to the vnos-c were scored by two researchers (one studied nos on her master degree, the other on his ph.d.) independently. then the analyses were compared, and reliability was calculated as %90 (miles and huberman, 1994). the inconsistencies between researchers were resolved through discussion. data which were obtained at the beginning and end of the course were compared with the help of spss 16.0 statistical program. 3. result and discussion 3.1 results in this study, the vnos-c questionnaire was applied twice as pre-test and post-test to determine the effect of documentary films on the pst’ nos understandings. results obtained from the analysis are given below in the tables according to the research questions. preliminary tests as a preliminary analysis, all the data obtained was tested for normality in order to decide if the data had parametric value or not as shown in table 4. because the sample was less than 50, the shapiro-wilk test was applied to determine if the data exhibit a normal distribution (büyüköztürk, 2011). the test results showed that the data was normally distributed therefore, parametric tests were chosen to analyze the effect of documentary films. according to the test results given in the table, significance values are higher than 0,05. furthermore, kurtosis and skewness values are between the range of -1 to +1. these results indicate normal distribution (hair, black, babin & anderson; 2013). thus, independent samples t-test was used to compare the group scores, and paired samples t-test was used to determine the effect of the documentary films on understanding nos of the experimental group. before comparing means with independent samples ttest, homogeneity of variance was tested as tabulated in table 5. the assumption on homogeneity between the control and experimental groups was checked by investigating levene’s test of equality of error variances. according to the test results, the significance value is greater than 0,05. therefore, the homogeneity of regression assumption was met. table 4 test of normality measurement group shapirowilk statistic df sig. skewness kurtosis pre-test experimental ,947 30 ,144 ,280 -,684 post-test experimental ,940 30 ,094 ,397 ,427 pre-test control ,947 30 ,144 -,002 -,992 post-test control ,941 30 ,096 -,504 -,459 journal of science learning article doi: 10.17509/jsl.v2i3.17998 102 j.sci.learn.2019.2(3).97-107 results and discussion on comparison between the experimental and control groups for testing the effectiveness of documentary films on pst’ nos understanding, vnos-c scores were analyzed. the comparison results of understanding of the pst’ nos understandings were presented in table 6. as seen in table 6, the pre-test mean scores of experimental and control groups were 4,13 and 4,27 respectively. after the intervention, the mean scores of the control group were 6,27 while the mean scores of the experimental group which was taught with documentary films were 8,53. the results show that there was an increase in the mean scores of the preservice teachers in both groups. independent samples t-test was conducted to determine whether the post-test scores showed a significant difference. as a result of the analysis, it was found that the scores between post-tests of experimental and control group differed meaningfully in favor of the experimental group. this means that using documentary films had a significant effect on pst’ nos understandings positively. table 3 examples of pst’ answers nos aspects naive views transitional views informed views the empirical nos science is an empirical-based discipline. so scientists conduct experiments on their labs to get solid evidence and without an experiment, you cannot prove your theory. scientists cannot always perform experiments to proof scientific claims, sometimes their studies are based on observation of the world. doing an experiment is a way of scientific investigation, but not the only way. you can’t always rely on experiments, for instance, einstein didn’t conduct an analysis to explain gravitational waves. knowledge can be derived from observations, imagination as well as experiments. the tentative nos theories can change over time since they are not proven. but once they’ve proven they become laws and laws are specific and cannot change. yes, theories can change over time. finding new evidence can change theories. for example, from democritus’s days till these days, atom theories had changed due to new evidences. since the scientific information we know about the natural world increases, we develop better tools to understand it or we gain a different perspective on the phenomenon we investigate. because of that, scientific knowledge can change. the theory-laden nos because of the missing evidence, scientists had to guess the missing parts to complete the theories. when the truth comes out, they will reach a consensus on one of the theories. because of the lack of certain knowledge, it is difficult to explain why and scientists have different opinions because they are different. even if scientists use the same data, they make different experiments, observations, and they can reach different conclusions. their imagination, beliefs or their pasts can affect their investigation. distinctions between observation and inference scientists are not certain about the atom because they cannot see it and they are still investigating how an atom looks like. the evidence they gathered with the help of electron microscopes and the experiments such as rutherford’s gold foil experiment, scientist have an idea what atom looks like. even if they cannot directly observe the atom, based on the observations, experiments they conduct and using their creativity they can make a model of an atom that explains to us how an atom supposed to look like. the creative and imaginative nos no, they don’t. because if they used their imagination then the scientific knowledge they’ve derived would not be reliable. that’s the difference between science and other disciplines. i think scientists use their creativity in their work. they think different than normal people and they are curious about the natural world. so they come up with interesting ideas, questions or inventions by their creative minds. sometimes phenomenon cannot be observed directly, like atoms, genes etc. creativity and imagination play a big role in these situations. for example einstein’s thought experiments. he used his imagination at all stages of the experiment. distinctions and relationship between scientific theories and laws theories are like predictions; they haven’t been proven by enough data. when they are proven by experiments they become absolute and called laws. the theory is not a certain scientific knowledge that is agreed by all scientists like a law. but they are different kinds of scientific knowledge and can’t become one and another. theories are explanations of natural phenomenon whilst laws are descriptions of the relationships among this phenomenon. both of them are well supported by evidence. there is no hierarchical relation. social and cultural influences on scientific knowledge science is universal; the rules and facts are valid and same all around the world. science should not be influenced by cultures and beliefs. otherwise it wouldn’t be certain. science can be affected by society’s ideas or belief yet it still remains universal. science is influenced by cultural and social values. for example, galileo’s ideas about the solar system were forbidden because it was against the church. this shows us how can scientific progress can be effected easily by beliefs of a society table 5 levene's test for equality of variances details levene's test for equality of variances f sig. experimental & control group pretests equal variances assumed ,652 ,423 equal variances not assumed journal of science learning article doi: 10.17509/jsl.v2i3.17998 103 j.sci.learn.2019.2(3).97-107 as vnos-c scores of experimental and control groups were compared by nos aspects, it is seen that there was no statistically significant difference between experimental and control groups regarding “distinctions and relationship between scientific theories and laws” and “social and cultural influences on scientific knowledge” aspects. as it is seen in table 7, there was a statistically significant difference between experimental and control groups regarding other five aspects which are “the empirical nos”, “distinctions between observation and inference”, “the tentative nos”, “the theory-laden nos” and “the creative and imaginative nos”. considering mean scores, at all aspects experimental group scores were higher than control group scores. however, according to the independent samples t-test, there was a statistically significant difference between groups in five of the aspects in favor of the experimental group. results and discussion on comparison between the experimental and control groups under this title, pst’ nos understandings in the experimental group before and after the intervention are presented. as it is seen in table 8, majority of the pst’ views were transitional in terms of “the empirical nos”, “distinctions between observation and inference”, “the tentative nos”, “the theory-laden nos” and “the creative and imaginative nos” aspects in the experimental groups before the intervention. in addition to this pst’s views regarding “distinctions and relationship between scientific theories and laws” and “social and cultural influences on scientific knowledge” aspects were naïve at the beginning. the results presented that after the intervention, the majority of pst’s views about “the empirical nos” and “the tentative nos” aspects have changed from transitional to informed and “distinctions and relationship between scientific theories and laws” and “social and cultural influences on scientific knowledge” aspects have transformed from naïve to transitional. moreover, only one pst held naïve views on “distinctions between observation and inference” and “the creative and imaginative nos” aspects and no pst left who held naïve views on “the theory-laden nos” aspect after the intervention. the results presented in table 8 have shown that teaching with documentary films is an effective way to enhance understandings of pst on certain nos aspects. in order to determine whether this increase is statistically meaningful or not, paired-samples t-test was conducted. according to table 9, paired-samples t-test results showed that there is a statistically significant difference between the pre-test and post-test. as it is seen in the table, mean scores of the experimental group have increased from 4,13 to 8,53 after the intervention. in order to explain the results more detailed, paired-samples t-test was conducted for each nos aspect. paired samples t-test results of pre-test and post-test scores of pst has shown that there is a statistically significant difference between pre-test and post-test scores of pst on nos understanding in each aspect except for one; the tentative nos. even though there was an increase according to the mean scores, there is no statistically significant difference regarding the tentative nos aspect. to overall, results of the tests have shown that using documentary films is effective to teach pst nos. 3.2 discussion in this study, the effects of documentary films on pst’ views about nos was investigated. according to the table 7, it was found that there was a significant difference table 6. mean and standard deviation values of the experimental and control groups tests groups n x sd df t p pretest experimental 30 4,13 1,852 58 -,260 0,796* control 30 4,27 2,116 posttest experimental 30 8,53 1,852 58 3,972 0,000* control 30 6,27 2,518 *p<0,05 table 7. mean and standard deviation values of nos aspects between groups nos aspects groups n x sd df t p the empirical nos control 30 1,23 0,568 29 2,062 0,044 experimental 30 1,57 0,679 distinctions between observation and inference control 30 0,90 0,758 29 2,443 0,018 experimental 30 1,33 0,607 the tentative nos control 30 0,83 0,521 29 2,723 0,009 experimental 30 1,27 0,699 distinctions and relationship between scientific theories and laws control 30 0,73 0,629 29 0,894 0,375 experimental 30 0,87 0,521 the theory-laden nos control 30 0,93 0,466 29 3,101 0,003 experimental 30 1,30 0,450 the creative and imaginative nos control 30 0,77 0,365 29 2,432 0,019 experimental 30 1,07 0,568 social and cultural influences on scientific knowledge control 30 0,87 0,571 29 1,516 0,136 experimental 30 1,13 0,776 journal of science learning article doi: 10.17509/jsl.v2i3.17998 104 j.sci.learn.2019.2(3).97-107 between posttest scores of the experimental and control groups in favor of experimental group regarding “distinctions between observation and inference” and “the empirical nos” aspects. in addition to that, there was a significant difference between pretest and posttest scores of the experimental group regarding these aspects (table 10). as we examine table 8 for the development of pst understandings on these aspects, after the intervention no pst left with naive views on “the empirical nos” aspect and only one pst held a naive view on “distinctions between observation and inference” aspect. these reveal that using documentary films had a positive effect on pst’ nos understandings on “the empirical nos” and “distinctions between observation and inference” aspects. observations are explanations of natural phenomena which are accessible to senses and can be observed by other individuals. but scientists do not have direct success to the most natural phenomena, they use scientific instruments and conduct experiments to generate empirical evidence and based on observations and experiments, scientists interpret and derive a conclusion which is called inference (lederman, abd-el-khalick, bell, & schwartz, 2002; lederman, 2007). since engaging students with laboratory activities; try to work as a scientist, conduct scientific investigations and experiments help them to improve their understanding about nos (wardani & winarno, 2017; prima, utari, chandra, hasanah, & rusdiana, 2018), watching scientists on work and discussing might have provided a development on pst understanding of nos. for instance, on the 5. documentary film galileo documented the experiments and observations he made and observed the movement of jupiter's satellites, and was seen to be influenced by the fact that there are objects that do not revolve around the world, and thus there are examples of such things that contradict the idea of the world as the center of the universe. historical science stories help pst to connect with scientists and have the opportunity to see the examples of how scientific researches are conducted. (laçin-şimsek, 2019). according to the results of the study, a significant difference was found on the theory-laden aspect of nos between posttest scores of the experimental and control groups in favor of the experimental group and between pretest and posttest scores of the experimental group. as science is a human activity, it shouldn’t be expected to be objective. scientists’, prior knowledge, philosophical perspectives, training, experiences, backgrounds beliefs, bias, values, and expectations can affect their studies. these may influence scientists’ researches; what they observe, how they observe, and how they interpret their investigations. since scientists aren’t objective, their observations cannot be expected to be objective; their views depend on their theoretical perspectives (lederman, 2007). pst had the chance to look at the lives of scientists like newton, teslathe, and einstein more closely. they could observe that scientists are involved in their work discipline, scientific study and scientific knowledge as well as their personal lives and personal characteristics, and recognize that the prejudices of the scientists, their beliefs and their experiences could affect their decision-making processes and their scientific viewpoints, in the light of this aid to understanding theory-laden aspect of science (lederman, abd-el-khalick, bell, & schwartz, 2002; table 8 percentages and frequencies of naive, transitional and informed views of nos aspects of the experimental group before and after the intervention nos aspects tests naive transitional informed f % f % f % the empirical nos pre-test 8 26 21 70 1 3 post-test 14 46 16 53 distinctions between observation and inference pre-test 14 46 15 50 1 3 post-test 1 3 20 66 9 30 the tentative nos pre-test 7 23 16 53 7 23 post-test 5 16 10 33 15 50 distinctions and relationship between scientific theories and laws pre-test 24 80 6 20 post-test 8 26 18 60 4 13 the theory-laden nos pre-test 9 30 20 66 1 3 post-test 21 70 9 30 the creative and imaginative nos pre-test 14 46 16 53 post-test 1 3 26 86 3 10 social and cultural influences on scientific knowledge pre-test 20 66 10 33 post-test 3 10 20 66 7 23 table 9 mean and standard deviation values of the experimental group tests n x sd df t p pre-test 30 4,13 1,852 29 11,162 ,000 posttest 30 8,53 1,852 journal of science learning article doi: 10.17509/jsl.v2i3.17998 105 j.sci.learn.2019.2(3).97-107 vanderlinden, 2007). owing to that experiences pst might have developed a better understanding on the tentative aspect of nos. the results indicated that on “the creative and imaginative nos” aspect, a significant difference was found between posttest scores of the experimental and control groups in favor of the experimental group and between pretest and posttest scores of the experimental group. even though science is based on observations of the natural world, creativity and imagination play a significant role in science. science is not rational and orderly; scientists don’t always follow specific method to conduct their researches while generating scientific knowledge, this process contains creativity and scientists can use their imagination to fill the gaps in their studies (lederman, 2007). with the documentary film of tesla, the importance of creativity and imagination in science was mentioned by discussing exciting ideas and incomprehensible inventions. it was emphasized that scientists use their imagination and creativity to produce new ideas and that imaginative power occupies an exceptional place beside the methods and rules, showing that science is not only methodological (lederman, abd-el-khalick, bell, & schwartz, 2002). similarly, it can be said that the emphasis on the imagination that lies behind the theories of einstein's behavior on macro dimensions influenced the development of the pst’ views in this aspect of science. the results showed that there was no statistically significant difference between experimental and control groups on “distinctions and relationship between scientific theories and laws” “social and cultural influences on scientific knowledge” aspects. when the literature is examined, studies which have been conducted with samples at every level show that there are many conceptual errors about theory and law concepts, and that there are difficulties in understanding these concepts (mccomas, 1998; buaraphan, 2010; deng, chen, tsai, & chai, 2011; göksu aslan, murat & zor, 2016). it is a common misconception of individuals that there is a hierarchical relationship between theories and laws; with more supporting proof, scientific theories become scientific laws. according to this misconception, scientific laws are more secure than scientific methods and have a higher status than scientific methods (lederman, 2007; mccomas 1998). contrary to this common belief, scientific theories and laws are a different kind of scientific knowledge and don’t change one into another. a scientific theory is an explanation of observable phenomena whereas a scientific law is a description of the relationship between observable phenomena (lederman, 2007). when the table 10 examined, it was found that there was a significant difference between pretest and posttest scores of the experimental group on “distinctions and relationship between scientific theories and laws” aspect. this significant difference can be the effect of the following documentary films. this scientific knowledge were discussed especially in the time bending beyond the universe documentary, paradigms and theories were explained, the points that theories and laws are different and that they will not convert into each other were underlined, in the newton and einstein documentaries, newton’s laws and einstein’s theories were explained to describe this scientific knowledge. as it is mentioned before one of the two aspects that there was no statistically significant difference between experimental and control groups was “social and cultural influences on scientific knowledge”. according to the literature, these two aspects, the social and cultural influences on scientific knowledge and the distinction between theories and laws, were more challenging to teach table 10 mean and standard deviation values of nos aspects before and after the intervention groups tests n x sd df t p the empirical nos pre-test 30 0,77 ,092 29 -4,942 ,000 post-test 30 1,57 ,104 distinctions between observation and inference pre-test 30 0,57 ,568 29 -5,887 ,000 post-test 30 1,27 ,521 the tentative nos pre-test 30 1,00 ,695 29 -1,904 ,067 post-test 30 1,33 ,758 distinctions and relationship between scientific theories and laws pre-test 30 0,20 ,407 29 -5,525 ,000 post-test 30 0,87 ,629 the theory-laden nos pre-test 30 0,73 ,521 29 -4,958 ,000 post-test 30 1,30 ,466 the creative and imaginative nos pre-test 30 0,53 ,093 29 -5,757 ,000 post-test 30 1,07 ,067 social and cultural influences on scientific knowledge pre-test 30 0,33 ,479 29 -6,595 ,000 post-test 30 1,13 ,571 journal of science learning article doi: 10.17509/jsl.v2i3.17998 106 j.sci.learn.2019.2(3).97-107 to pst than the other five aspects (bell, lederman, & abdel-khalick, 2000). when the pretest and post-test scores of the experimental group are compared, there was a statistically significant difference in favor of posttest on “social and cultural influences on scientific knowledge” aspect. contrary to common belief, science affects and is affected by the culture in which it is embedded. because science is a human enterprise and scientific knowledge is generated by humans; it can affect and be affected by the ethical, social, political, moral and religious aspects of a culture (lederman, abd-el-khalick, bell, & schwartz, 2002). at the intervention the period of the scientists' lives was revealed with the documentary films, making it possible for the pst to become aware of the social and cultural values of the society in which science is produced. for example, the world was at war during the einstein era, and in the period of galileo, there was a prohibition of the view of the heliocentric model due to the dominant influence of the church. besides this, it can be said that one of the factors influencing the change in the opinions of the pst is the mention of religion and science in the avicenna and al-farabi parts, in particular, emphasizing that religious belief made a great contribution to the progress of scientific research at that time. similar to this study, ayar (2007) stated that science-technology-society lessons were used to study the influence of science teachers' opinions on the nos, and it was concluded that the use of film sections and newspaper news in the lessons was successful in the relationship between science and social values. the results of the study also showed that there is a statistically significant difference was found between posttest scores of the experimental and control groups in favor of experimental group on the aspect of “the tentative nos”. even though scientific knowledge is considered as absolute or certain, this knowledge can change through time, by new data or interpreting data with the light of new theoretical ideas (lederman, abd-elkhalick, bell, & schwartz, 2002). scientists based their growing understanding on empirical data that becomes more extensive with each new wave of technology (sterling, 2009). tentative nos was emphasized by comparing newton’s physics and einstein’s physics and explaining why pluto wasn’t considered a planet anymore. it is also revealed in almost every documentary film that whatever the type of scientific knowledge is, it will not be specific, and that science can change in the light of new findings or reinterpretation of old data. 4. conclusion in this study, the effects of documentary films on pst’ views about nos was investigated. according to the results, there is a statistically significant difference between posttest scores of experimental and control groups (table 6). therefore, it can be concluded that teaching nos with documentary films is an effective way to teach nos. when the aspects of the nos are examined, the fact that the nos is included in more than one documentary for each aspect, and that the same aspects are spread over a long period by giving the opportunity for them to be discussed through different examples in different weeks, is effective in changing the opinions of pst. according to the literature before the intervention, pst’ opinions were resistant to change and therefore, long term applications are needed (çil, 2010; mulvey, chiu, ghosh & bell, 2017). the pretest scores of pst showed that they have many misconceptions about nos. in order to prevent this, education about nos should begin in students’ early years and supporting it with documentary films and sections from science history until undergraduate degree level will help to get rid of misconceptions. since nos is based on different disciplines like science philosophy and science history, lessons at undergraduate level need to be given that is enriched with sections from science history and documentary films which focus on scientific works and scientists instead of theoretical ones. it can be suggested that documentaries and scientific videos like these should be used in not only nos courses but also science lessons integrated with topics, and they should emphasize nos in this way. presenting science in a way which includes stories, documentary films and the conditions that it has been made in, and criticizing it, will help to improve students’ opinions about nos. making an effort to help them understand how problems exist instead of how to solve them will increase their curiosity about science, and it will make learning science easier. references american association for the advancement of science (aaas) (1990). science for all americans. new york: oxford university press. ayar, m.c. (2007). fen-teknoloji-toplum dersinin fen bilgisi öğretmeni adaylarının bilimin doğasına ilişkin görüşlerine etkisi, yayınlanmamış yüksek lisans tezi, marmara üniversitesi, i̇stanbul. bell, r.l., lederman, n.g., & abd-el-khalick, f. (2000). developing and acting upon one’s conception of the nature of science: a follow-up study. journal of research in science teaching, 37, 563–581. bloom, m., binns, i. c., & koehler, c. (2015). multifaceted nos instruction: contextualizing nature of science with documentary films. international journal of environmental and science education, 10(3), 405-428. buaraphan, k. (2010). prospective and in-service science teachers’ conceptions of the nature of science, science educator, 19(2), 35-47. büyüköztürk, ş. (2011). sosyal bilimler için veri analizi el kitabı: i̇statistik, araştırma deseni spss uygulamaları ve yorum. pegem a yayıncılık. çil, e. (2010). bilimin doğasının kavramsal değişim pedagojisi ve doğrudan yansıtıcı yaklaşım i̇le öğretilmesi: işık ünitesi örneği yayınlanmamış doktora tezi, karadeniz teknik üniversitesi, trabzon. cook, t. d., campbell, d. t., & shadish, w. (2002). experimental and quasi-experimental designs for generalized causal inference. boston, ma: houghton mifflin. deng, f., chen, d. t., tsai, c. c., & chai, c. s. (2011). students' views of the nature of science: a critical review of research. science education, 95(6), 961-999. journal of science learning article doi: 10.17509/jsl.v2i3.17998 107 j.sci.learn.2019.2(3).97-107 dereli, f. (2016). sınıf dünya ve evren konu alanına uyarlanmış bilimin doğası kazanımlarının akıllı tahta etkinlikleri ile öğretimi. yayımlanmamış yüksek lisans tezi, mehmet akif ersoy üniversitesi, burdur. efthimiou, c. j., & llewellyn, r. a. (2007). cinema, fermi problems and general education. physics education, 42(3), 253. göksu, v., aslan, o., murat, ö., & zor, t. ş. (2016). açık-düşündürücü ve tarih temelli öğretimin fen bilimleri öğretmen adaylarının bilimin doğası anlayışları üzerindeki etkisi. mustafa kemal üniversitesi sosyal bilimler enstitüsü dergisi, 13(34), 313-327. hair, j. f., black, w. c., babin, b. j., anderson, r. e., & tatham, r. l. (2013). multivariate data analysis: pearson education limited. higher education council (hec) (2018). science teaching undergraduate program. ankara. khishfe, r., & lederman, n. (2006). teaching nature of science within a controversial topic: integrated versus nonintegrated. journal of research in science teaching: the official journal of the national association for research in science teaching, 43(4), 395-418. laçin-simsek, c. (2019). what can stories on history of science give to students? thoughts of science teachers candidates. international journal of instruction, 12(1), 99-112. lederman, n. g. (1992). students' and teachers' conceptions of the nature of science: a review of the research. journal of research in science teaching, 29(4), 331-359. lederman, n. g. (2007). nature of science: past, present, and future. in abell, s. k. and lederman, n. g. (eds.), handbook of research on science education (pp. 831879). london: lawrence erlbaum associates. lederman, n. g., & o'malley, m. (1990). students' perceptions of tentativeness in science: development, use, and sources of change. science education, 74(2), 225-239. lederman, n. g., abd-el-khalick, f., bell, r. l., & schwartz, r. (2002). views of nature of science questionnaire: toward valid and meaningful assessment of learners’ conceptions of nature of science, journal of research in science teaching, 39(6), 497-521. mccomas, w. f. (1998). mccomas, w. f. (1998). the principal elements of the nature of science: dispelling the myths. in the nature of science in science education (pp. 53-70). springer, dordrecht. meichtry, y.j. (1992). influencing student understanding of the nature of science: data from a case of curriculum development, journal of research in science teaching, 29(4), 89-407. miles, m. & huberman, m. (1994). an expanded sourcebook qualitative data analysis. (2th edition), america: person education. mulvey, b. k., chiu, j. l., ghosh, r. & bell, r. l. (2017). making learning last: teachers’ long-term retention of improved nature of science conceptions and instructional rationales. international journal of science education, 39(1), 62-85. national research council (nrc) (1996). national science education standards, washington, dc. national academic press. prima, e. c., utari, s., chandra, d. t., hasanah, l., & rusdiana, d. (2018). heat and temperature experiment designs to support students’ conception on nature of science. jotse: journal of technology and science education, 8(4), 453-472. seckin-kapucu, m., cakmakci, g., & aydoğdu, c. (2015). the influence of documentary films on 8th grade students' views about nature of science. educational sciences: theory and practice, 15(3), 797-808. sorensen, p., newton, l., & mccarthy, s. (2012). developing a science teacher education course that supports student teachers’ thinking and teaching about the nature of science. research in science & technological education, 30(1), 29-47. sterling, d. r. (2009). from aristotle to today: making the history and nature of science relevant. science scope, 32(5), 30. taşar, m. f. (2003). teaching history and the nature of science in science teacher education programs. pamukkale university journal of education, 1(13), 30-42. vanderlinden, d. w. (2007). teaching the content and context of science: the effect of using historical narratives to teach the nature of science and science content in an undergraduate introductory geology course. unpublished doctoral dissertation, iowa state university, ames, ia. wardani, t. b., & winarno, n. (2017). using inquiry-based laboratory activities in lights and optics topic to improve students' understanding about nature of science (nos). journal of science learning, 1(1), 28-35. a © 2022 indonesian society for science educator 386 j.sci.learn.2022.5(3).386-397 received: 03 july 2022 revised: 16 october 2022 published: 27 november 2022 the impact of stem applications on problem-solving skills of 4th-grade students raşit zengin1*, tuğçe kavak1, gonca keçeci1, fikriye kırbağ zengin1 1department of mathematics and science education, faculty of education, fırat university, elazığ, turkey *corresponding author: rzengin@firat.edu.tr abstract this study examined the impact of stem applications realized with 4th-grade elementary school students on their problem-solving skills. the study was conducted within the convergent parallel design framework, a mixed research method. the research was carried out with 42 4th-grade primary school students, 21 in the experimental group and 21 in the control group, studying in a public school in turkey. in the study, which lasted for ten weeks, stem applications were carried out with the students in the experimental group. the control group applied the current science curriculum without performing stem applications. the research data consists of the developing problem solving inventory for children at the level of primary education (psic) and the semi-structured interview conducted with the students in the experimental group. quantitative data were analyzed using an independent sample t-test, while qualitative data were analyzed with content analysis. when the psic post-test scores of the 4thgrade primary school students in the experimental and control groups were examined, a statistically significant difference was found in favor of the experimental group. according to the qualitative data analysis of the study, it was determined that the stude nts encountered various problems in the process and developed solutions for them. the students stated that using stem activities in the classes is fun, so they learn more permanently and easily, and they want the classes to be delivered in this way from now on. keywords stem education, primary school 4th grade students, problem solving skills 1. introduction in the 21st century, the race between developed countries in the fields of invention, production, and technological development is accelerating. thus, in such a competitive environment, governments have started to invest more in science, engineering, and innovative technologies (ministry of national education [mne], 2018b). the age of technology, in which we live and where breathtaking developments are experienced, expects individuals to be able to think critically and analytically, solve daily life problems, be creative, research and question, and make effective decisions (pekbay, 2017). for this reason, individuals now have new responsibilities. one of these responsibilities is to find solutions to the problems they will encounter in their daily life. with the increasing technology, people face more and more problems every day. in the past years, jobs that required muscle power and labor were expected from individuals, but now that there are machines that do these jobs, muscle power has been replaced by machines and technological devices. the new responsibility of individuals is to design, manufacture and market these machines. for these reasons, developed and developing countries aim to abandon education systems with only content teaching and turn to stem education, which is a project-based interdisciplinary approach towards research, inquiry, invention, and production (ministry of national education [mne], 2018b). stem education; represents the relationship in which science, technology, mathematics, and engineering are intertwined and interlocked. stem education aims to raise individuals willing to examine stem-related issues, use scientific methods while doing research, and recognize the contemporary and cultural environment created by every discipline in stem (tiryaki & adigüzel, 2021). since students work interdisciplinary in stem activities, they can produce practical solutions to problems and relate the relevant subject to daily life (özçelik & akgündüz, 2018). in addition, students gain multiple perspectives on problems as they actively participate in stem activities (bahar, yener, yılmaz, hayrettin & gürer, 2018). journal of science learning article doi: 10.17509/jsl.v5i3.48182 387 j.sci.learn.2022.5(3).386-397 we may encounter several definitions when we examine the literature on problem definition. dewey (1997) defined it as anything that interferes with the human mind and obscures and challenges beliefs. according to karasar (2005), a problem is any desired problem to overcome. for the individual to want to overcome this difficulty, s/he must feel uncomfortable. therefore, the problem can be defined as any situation that causes discomfort in the individual. when the diagnoses of the problem are analyzed, we can state that to call a case a problem. it must confuse people. the individual has not encountered this situation before, and this problem is new to them. for this reason, a situation that is a problem for one individual may not be a problem for another. for example, suppose the individual encounters a situation that requires a solution strategy and sees the situation but does not immediately think of the strategy. in that case, it can be said that there is a problem (yenilmez, 2010). the ability to do what needs to be done when an individual does not know what to do is called a "problem-solving skill" (üstün & bozkurt, 2003). when we look at the education system in turkey, it is supported by experimental research results that different non-traditional problem-solving techniques improve students' problem-solving skills more compared to traditional problem-solving techniques (gök & sılay, 2008; özkök, 2005). we can use stem education to develop students' problem-solving skills with a non-traditional method because stem education is an educational approach that provides students with problem-solving skills and enables them to think creatively (sangngam, 2021). in this context, turkey has aimed to introduce science, technology, engineering, and mathematics disciplines from an interdisciplinary perspective to students from an early age by switching to stem education. at the same time, it is aimed to provide students with skills such as problemsolving, questioning, research, product development, and aesthetic perspective. our students have the energy, talent, and opportunity to do several things. therefore, we should increase these opportunities for them, encourage them to stem education based on research and questioning, and make them realize their talents and achievements (ministry of national education [mne], 2016; asigigan & samur, 2021). when the studies are examined, doğan, aydın & kahraman (2020); güven (2022); hebebci (2019); kartini, widodo, winarno & astuti (2021); kurt & benzer (2020); tran (2018); yurttaş (2021) conducted stem activities with middle school students and examined the effects of these activities on students' problem-solving skills. akçay (2019); öztürk & çınar (2022); şanlı (2021), and yalçın & erden (2021), on the other hand, examined the effects of stem activities on preschool students' problem-solving skills. acar (2020); alan (2017); öztürk & yalçın (2020) looked at the impact of stem activities on problemsolving skills with teachers and prospective teachers. acar, tertemiz & taşdemir (2020); asigigan & samur (2021); yıldız & özdemir (2022) examined the effect of stem activities on primary school students' problem-solving skills. the study was carried out to examine the effect of stem activities on the problem-solving abilities of young students. 1.1. problem of research the innovation era we live in draws a roadmap for the future generation that will rule the country. our children, called "generation z" or "digital settlers/digital natives", now prefer to interact and socialize through mobile communication devices rather than playing games on the street. unlike us, it is accepted that children of this age are born with information and communication technologies. therefore, to have a more significant say in global competition and to create a sustainable economy, wellplanned stem education should be offered to this generation, which makes up 17% of our population (tekin poyraz, 2018). when we look at the science curriculum, it is stated that the use of science and engineering applications in education has an important place in taking the necessary steps for our countries, such as competitiveness, socioeconomic development, scientific research, and technological development (ministry of national education [mne], 2018a). when we look at the studies on stem education, the number of studies conducted in turkey has been increasing rapidly in recent years. however, it is seen that the majority of the studies are carried out with secondary school students, teacher candidates, and teachers. the study with primary school students is at a lesser level. with this study, it is thought that this need will be met to some extent. this study aimed to determine the effects of stem education on the problem-solving skills of primary school 4th-grade students, and students opinions were obtained regarding the stem applications conducted. 1.2. research focus stem education is an integrated approach that adopts creative problem-solving techniques for our young scientists and engineer candidate students, who are the treasure of our future. stem education is one of our most up-to-date training that aims to enable students to look at problems from an interdisciplinary perspective and gain the required knowledge and skills (şahin, ayar & adıgüzel, 2014; kartini, widodo, winarno & astuti, 2021). for primary school students, the concept of engineering is a concept that has not yet been fully realized, does not attract much attention, and is ignored. engineers have responsibilities in several technological fields, which the age of technology has made indispensable for our daily lives, and the need for this profession is increasing daily. however, younger students are not aware of the study fields and benefits of engineering. therefore, it is very journal of science learning article doi: 10.17509/jsl.v5i3.48182 388 j.sci.learn.2022.5(3).386-397 critical that we introduce this profession to the junior engineer candidates of the future, make it popular, and inspire the profession's importance in the first stages of education. advocates of an integrated approach to stem education in primary education argue that students' interest, success, and motivation can be increased, especially with real-world problems, which will help increase the number of students who want to pursue a career in stem fields (honey, pearson, & schweingruber, 2014). it is important to present various stem applications as examples to our teachers who want to implement them and to bring these applications into the literature to make students interested in stem fields and to disseminate and develop stem education. 2. method in this research, which is based on a mixed-method, a parallel pattern converging from mixed-method practices was used. the purpose of the contemporary parallel design is to collect quantitative and qualitative data to present the problem in the research in a comprehensive and detailed way. the main point is to bring together the mutually supportive aspects of the qualitative and quantitative methods and the non-overlapping elements. for this reason, if desired, data obtained by quantitative methods and data obtained by qualitative methods can be used to support each other (creswell & clark, 2015). 2.1. working process the research was carried out in a public school in elazig in, turkey. the study group the research consists of 42 primary school students from 4th-grade students enrolled at this primary school. among these students, 21 were included in the experimental group, 13 of whom are girls and 8 of whom are boys. there were also 21 students in the control group, 9 of whom were girls and 12 of whom were boys. while determining the experimental and control groups, the year-end average scores of the previous year's branches were taken as a basis. the study covered ten weeks, and two weeks of the process were reserved for pretest and post-test applications. while the science course curriculum and stem applications were carried out with the students in the experimental group, the courses were taught only in line with the current science curriculum with the students in the control group. figure 1 shows flow chart regarding the steps followed during the research experimental and control groups were determined in the first week of the research, and pre-tests were applied to both groups. during the study process, the current program was adhered to in teaching the control group. in addition, the current curriculum, robotic studies, and fun science experiments were applied to the students in the experimental group by integrating them into stem education. in the second week of the research, the "my water is multi-colored" activity was held with the students. the students learned about the foods they use in their daily lives and what the concepts of acid and base contained in various materials are, and the concept of ph, which is even included in water bottles in their daily lives. still, they don't have an idea about what it is. cabbage juice was used as a separator, and different colors were obtained in cabbage juice by adding lemon, shampoo, vinegar, bleach, milk, juice, baking soda, shampoo, and soap to cabbage juice. the students classified the colors and concluded that one of the two groups formed was acid, and the other was base. using microprocessor-supported laboratory sets, they found that pink and similar colors were between 0-7, while purple and blue color-like color dec received values between 7-14. therefore, the ph december of acids and bases is inferred. in the third week of the research, the activity "we design cool cars" was held with the students. students were given materials such as pet bottles, bottle caps, balloons, tires, straws, garbage skewers, rulers, and cardboard, which are waste materials. they were asked to design a car by being told to use the materials they wanted. then, the students were divided into two groups and created their cars using air thrust in a team effort within the specified time. as a result, they have gained awareness and made a design that air can be used as a force. then, the two groups' students completed the cars they designed respectively. figure 1 flow chart regarding the steps followed during the research journal of science learning article doi: 10.17509/jsl.v5i3.48182 389 j.sci.learn.2022.5(3).386-397 in the 4th week of the research, the "whose raft is. stronger" activity was held with the students. students were asked about swimming and sinking events at the start of the event. after discussing swimming and sinking events, the question of why objects float or sink was asked, and swimming and sinking events were questioned with students. finally, students learning about swimming and sinking were asked to design a raft. the students made their raft designs by working as a team within the specified time. in the 5th week, the activity "we designed the 4th bridge to istanbul" was held with the students. students were given various materials such as straws, cardboard, foam, sticks, two cars, play dough, and aluminum foil, and they were asked to design a bridge using the materials they wanted. the students shared ideas within the specified time and created a bridge. two toy cars, one big and one small, were driven on both designed bridges, and the durability of the bridges were tested. in the 6th week of the practices, the activity "balancing the weights" was held. students have designed leverage using lego robotic sets. the students were asked how these levers could be balanced by placing different loads on the two sides of the designed lifter. students have found and interpreted how to balance a light load and a heavy load by trial and error. in the 7th week of the application, the "energy from the sun" event was held. students have designed their solar-powered cars using lego robotic sets. the students raced their cars in the schoolyard with the opportunity to have sunny weather. students have discovered that they can use the sun as an energy source. they deduced that this energy is an inexhaustible energy that exists in nature. furthermore, they have concluded how this energy is stored. in the 8th week of the study, the "don't say that there will be no wind-powered car" activity was held. students have designed wind-powered cars using lego robotic sets. the students were asked whether they had seen the wind turbine before starting the exercise. after the students learned about using and storing wind energy, they designed their vehicles. the students in two groups then had the vehicles they designed raced. in the 9th week of the study, the "eggs don't always break" activity was carried out. students were asked to design, so they were thrown from the specified height and did not break. the students designed according to their ideas, and the eggs were started to be thrown in the determined order. the last two unbroken eggs remain. in the 10th week of the study, post-tests were applied to the students in the experimental and control groups. in addition, interviews were conducted with the students in the experimental group about the practices carried out. figure 2 depicts sample photos of stem application in experimental groups 2.2.data collection tools quantitative and qualitative data tools were used to collect data for the study, which was carried out in line with the mixed-method design. for this purpose, the quantitative data collection tool of the analysis consisted of the developing problem solving inventory for children at the level of primary education (psic). the qualitative data was the semi-structured interview with the experimental group students. the "problem solving inventory for children" developed by serin, bulut-serin & saygılı (2010) was used in the research. the scale measures primary school students' perceptions of their problem-solving skills. the figure 2 sample photos of stem application in experimental groups journal of science learning article doi: 10.17509/jsl.v5i3.48182 390 j.sci.learn.2022.5(3).386-397 cronbach's alpha reliability coefficient of the scale, which consists of a total of 24 items, was found to be 0.80. the researcher developed semi-structured interview questions and examined them by three faculty members at firat university faculty of education, mathematics and science education, science education department. after they were structured in line with expert opinion, five 4thgrade students were asked to read the questions and indicate the points they did not understand. then, the interview questions, which took their final form and consisted of 5 questions, were applied to the students in the experimental group to determine the feelings and opinions of the students in the experimental group where stem applications were carried out. interviews were conducted with 19 students in the experimental group for an average of 30 to 40 minutes. while giving sentences belonging to students in the interview analysis, real student names were not used, but pseudonyms were used. 2.3. data analysis spss 22 statistic package program was used to analyze quantitative data in the study. descriptive statistical analysis was performed on the data belonging to each variable for the experimental and control groups. standard deviation, arithmetic mean, skewness coefficient, kurtosis coefficient, and maximum and minimum values of the data belonging to the groups were calculated. the findings were used to understand the relations and distributions of the variables with each other and to control the assumptions before the analysis. since the group size was smaller than 50 in the study, the shapiro-wilks test was applied to examine the normality of the scores. an independent samples t-test was conducted to test the study's research question. semistructured interviews with primary school 4th-grade students were evaluated with content analysis. the basic process in content analysis is to gather similar data within the scope of specific themes and concepts to organize and interpret them in a way that readers can understand (çepni, 2014). 3. result and discussion this study examined the impact of stem applications on primary school 4th-grade students' problem-solving skills, and student opinions about the applications were analyzed. the students in the experimental group were offered stem applications for eight weeks along with the science course curriculum. 3.1. quantitative data the average distribution of the problem-solving inventory pre-test scores for primary school children (prepsic) and the problem-solving inventory post-test for primary school children (post-psic) applied to the study group was examined. the comparison of the test scores of the 4th-grade primary school students between the groups, the kurtosis and skewness values, standard deviation, shapiro-wilk values, and minimum and maximum values are given in table 1. as seen in table 1, before the application, the pre-psic score mean of the experimental group (/=93.38) was very close to the post-psic mean value of the control group ( =93.47). it is seen that the problem-solving inventory between groups was close to each other before the application. when both groups are examined, there are ( x difference=10.76) a points difference between the pre-psic mean score ( x =93.38) and post-psic mean score ( x =104.14) of the experimental group. in contrast, the difference is more negligible ( x difference=0.57) between the pre-psic mean score ( x =93.47), and post-psic means score ( x =94.04) of the control group. 3.2. inferential statistics results an independent samples t-test was conducted to test the study's research question. the t-test results of the experimental and control groups before and after the application are given in table 2. as shown in table 2, there is no significant difference between the psic scores of the experimental and control groups before the application (t(40)=-.024, p>.05). x table 1 descriptive statistics of primary school 4th grade students' problem inventory test pretest and posttest scores for primary education level children." tests groups gender n x ss skewness kurtosis range min max shapiro-wilk pre-psic experimental female 13 97.53 12.29 -.055 -1.674 33 80 113 .088 male 8 86.62 17.89 -.202 -1.94 45 64 109 .234 total 21 93.38 15.23 -.491 -.719 49 64 113 .143 control female 9 95.15 7.96 -.852 .930 30 77 107 .488 male 12 90.75 13.47 -.622 -1.29 37 69 106 .237 total 21 93.47 10.31 -.958 .168 38 69 107 .043 post-psic experimental female 13 104.76 13.05 .789 .523 39 78 117 .040 male 8 103.12 10.5 -.136 -1.247 29 89 118 .613 total 21 104.14 11.9 -.564 -.760 40 78 118 .046 control female 9 96.15 10.84 -.050 -.552 36 78 114 .801 male 12 90.62 18.81 -.854 .450 59 56 115 .615 total 21 94.04 14.21 -.909 1.307 59 56 115 .270 journal of science learning article doi: 10.17509/jsl.v5i3.48182 391 j.sci.learn.2022.5(3).386-397 however, after the application, a significant difference was found between the psic scores of experimental group and control group (t (40)= 2.225, p<.05). this statistical difference was found in favor of experimental group 3.3. qualitative data semi-structured interviews were conducted with the students in the experimental group to obtain their feelings and opinions about stem activities. in terms of the ethics of the study, pseudonyms were used for the students. the researcher and three field specialists examined the data analyzed by content ana lysis. first, the codes were determined so that the meanings of the answers given by the students were not distorted. then, after the regulations were identified, the answers were grouped within themselves and divided into the necessary categories. finally, in the analysis conducted in the form of induction, the themes that would deliver the most general meaning were determined last. the students were asked, "can you please indicate the contributions that the activities you participated in during the semester have made to you?" the table obtained as a result of the content analysis is given below. as a result of the analysis, three themes and categories were determined. these themes are "contribution to skills", "contribution to attitude," and "contribution to academic achievement". table 2 t-test results of primary school 4th grade students' problem inventory pre-test and post-test scores for primary education level children n x sd df t p pre-pisc experimental 31 93.38 15.23 40 -.024 0,981 control 31 93.47 10.31 post-psic experimental 31 104.14 11.9 40 2,225 0,032 control 31 94.04 14.21 table 3 students' opinions on the contribution of the activities they attended during the semester theme category code f % contribution to skills scientific process skills observation 18 94.7 measuring 10 52.6 classification 7 36.8 using numbers 5 26.3 inference 12 63.1 experimentation 18 94.7 deciding 12 63.1 21. century skills problem-solving skill 6 31.6 creativity and innovation 3 15.8 communication and collaboration 4 21.1 contribution to attitude attitude toward natural sciences increased interest in science 6 31.6 love for science 1 5.3 finding the class fun 1 5.3 attitude towards technology liking technology 4 21.1 increased interest in technology 6 31.6 finding technology important 2 10.5 attitude towards engineering interest in engineering 4 21.1 love for design 2 10.5 feeling like an engineer 8 42.1 attitude towards mathematics increased interest in mathematics 4 21.1 don't like math 2 10.5 contribution to academic achievement contribution to course achievement science course success increase 9 47.4 mathematics course success increase 5 26.3 my success has increased 2 10.5 increase in knowledge better learning 4 21.1 increase in knowledge 4 21.1 learning to experiment 6 31.6 contribution to learning active participation 2 10.5 learning by doing 3 15.8 learning to learn 1 5.3 trial and error 2 10.5 meaningful learning 2 10.5 apply what is learned in real life 2 10.5 intelligence increase 3 15.8 journal of science learning article doi: 10.17509/jsl.v5i3.48182 392 j.sci.learn.2022.5(3).386-397 when table 3 is analyzed, there are two categories under the "skills" theme, namely "scientific process skills" and "21. century skills". among the science process skills, 18 students stated observation skills, ten mentioned measurement skills, seven mentioned classification skills, five mentioned the ability to use numbers, 12 students stated deduction skills, 18 indicated experimentation skills, and 12 mentioned decision-making skills. the answers given by some of the students are given below: pelin: it has contributed to my ability to observe because, in my experiments, i saw that i could understand and learn better by observing. eren: it contributed to measurement; i learned that measurements are used in real life. i learned to make decisions while doing something. it contributed to my classification skills. i learned to separate things into different places. selin: while observing, i observe the first and last states of objects as we do in experiments. büşra: it has contributed to my decision-making skills because we have to make decisions while doing it ourselves. also, it helped my ability to experiment because we did it ourselves. ezgi: it contributed to my ability to experiment because i learned to test. i feel confident when experimenting because experimentation is an excellent thing. hatice: it contributed to my ability to conclude; we would conclude all the activities and write them in our diaries. tolga: it contributed to my classification skills. i used to classify materials in activities. burcu: it has contributed to my ability to experiment because i think i can get better grades in science class by doing experiments and making better decisions about my classes by observing. the 21st-century skills category is based on the skills included in p21 (partnership for 21st-century learning), used by several schools and educators in the united states (p21, 2017). there are three codes parallel to the skills of problem-solving, communication and cooperation, creativity, and innovation in the framework of p21. the answers given by some of the students are given below. selin: i was undecided on some issues. i shared this with my group friends, who helped me too. burcu: i struggled with how we combine things in applications. but i overcame it. selin: i pushed myself to think, and we made new inventions. büşra: like engineers, we also produced something. zuhal: we had difficulties because we did not work as a team. but in the end, we realized we were not working as a team, and we all worked together. mehmet: we overcame the difficulties with teamwork. ezgi: i faced difficulties in many spots, but i succeeded by gaining my trust. sıdıka: i couldn't put the pieces together at first, but now i've learned. eren: we invented bridges and designed eggs that don't crack, which is nice. tolga: i realized that we could come together and do experiments. ezgi: it taught me a lot. that is, it contributed a lot. for example, in engineering, i can build most things. i can make an underpass and an overpass. when table 3 is analyzed, there are four categories under the attitude theme. these are attitude towards science, attitude towards technology, attitude towards engineering, and attitude towards mathematics. in the category of attitude towards science, it was observed that six students' interest in science increased, one student liked science and technology, and one student found the class enjoyable. the answers given by some of the students are given below: mehmet: my interest in science has increased. sıdıka: these activities made me love science and technology. burcu: it increased my interest in science class. selin: doing the activities increased my interest in science classes. in the category of attitude towards technology, it is seen that four students like technology, six students are more interested in technology, and two students find technology important. the answers given by some of the students are given below: eren: i started to find technology important with these activities. pelin: my interest in technology has increased, and my life will be easier with technology. burçak: we can use technology daily, and our interest in technology increases. tolga: i was interested in both technology and its benefits. alperen: it developed my interest in technology because we made a solar-powered car, which really attracted my attention. in the category of attitude toward engineering, it is seen that four students said that their interest in engineering increased, two students liked to design, and eight students felt like engineers. the answers given by some of the students are given below: alperen: we built leverage. i felt like an engineer when i built leverage. burçak: my interest in engineering is increasing. ezgi: i loved designing. kaan: we did a lot of things like engineers. tolga: i felt like an engineer during the events. pelin: my interest in engineering grew. regarding attitude towards mathematics, four students stated that their interest in mathematics increased, and two indicated that they liked mathematics. the answers given by some of the students are given below: sıdıka: the activities contributed to my mathematics class, and my interest increased. emre: i loved math very much with these activities. when table 3 is analyzed, academic success is studied under three categories: contribution to course achievement, increase in knowledge, and learning assistance. in the dimension of contribution to course achievement, nine students stated that they experienced an journal of science learning article doi: 10.17509/jsl.v5i3.48182 393 j.sci.learn.2022.5(3).386-397 increase in their science course achievement, and five students stated that they experienced an increase in mathematics course achievement. in addition, two students stated that they experienced an increase in their general achievement. the answers given by some of the students are given below: selin: my science and technology class knowledge has been updated, and my success level has increased. hatice: these activities increased my math achievement. burcu: i think i can get better grades in science class. tuba: in science class, the same topics come up in our books, and you know this beforehand. it also increased my success in mathematics because i learned to measure. pelin: the activities improved my measurements and estimations in mathematics, and my success level increased in this way. alperen: i am learning and succeeding in something new in science class. for example, i learned about objects that sink in and float on water, and i will use this knowledge in science class. in the category of increase in knowledge, it was observed that six students answered that they learned to experiment, four responded that they learned better, and four answered that their knowledge increased. some of the students' answers are given below: kaan: thanks to the activities, my knowledge increased. eren: i learned to experiment. zuhal: we learned better things. gamze: the activities contributed a lot. for example, i learned to experiment. ezgi: i learned to experiment. i feel confident when experimenting. i learned many things, and my knowledge increased. burçak: we learn about both experiments and science. in the category of contributing to learning, two students stated that they contributed actively to participating, three students mentioned learning by doing, one student mentioned learning to learn, two students mentioned trial and error, two students mentioned meaningful learning, two students mentioned learning by applying what was learned in real life, and three students mentioned increase in their intelligence. the answers given by some of the students are given below: pelin: it helped me a lot. i found ways to learn better. hatice: the activities contributed helped me understand better. tuba: the activities contributed to me because we both did and learned. zuhal: i used to say there is no such thing as an uncrackable egg, but we tried it and saw that it is possible to make uncrackable eggs. the question "do you think that there is a connection between science-technology-engineering-mathematics as a result of the applications carried out?" was asked to the students. the frequency and percentage distribution of their answers to the posed question are given in table 4. in response to the question "do you think that there is a connection between science-technology-engineeringmathematics as a result of the practices performed", it was concluded that 16 students thought that at least one or more of these areas were involved in the activities carried out, and were related. in comparison, three students did not find a connection. among the students who thought that there was a connection, it was concluded that two students believed that the realized applications were only related to science, one student thought that there was a connection between the fields of science and technology, three students thought that engineering and maths were related, five students believed that science, technology, and engineering fields were interrelated, one student believed that there was a connection between science, engineering, and maths, one student thought that science, technology, and maths were related, and three students believed that the fields of science, technology, engineering, and maths were interrelated. answers given by some of the students are provided below. burcu: yes, i think there is a connection. while doing the rafting activity, we learned that water is buoyant in science, and we learned counting in mathematics. i also liked to produce in engineering. alperen: for example, i think i am dealing with technology while making a wind-powered car. when we put sinking and nonsinking objects in a basin full of water, i learn about science. for example, when i was building a bridge, i had a dream as if i was an engineer. selin: i think so. for example, science and technology. we did experiments on science and improved our technology. there is also a relationship between mathematics and engineering, and we do it in some ways in mathematics. in engineering, we invent, as in experiments. table 4 frequency and percentage distribution of students' responses to whether there is a connection between science, technology, engineering, and mathematics as a result of the applications connection dimension name of discipline f % there is connection only science 2 10.5 science-technology 1 5.3 engineering-math 3 15.8 science-technology-engineering 5 26.3 science-engineering-mathematics 1 5.3 science-technology-math 1 5.3 science-technology-engineering-math 3 15.8 there is no connection 3 15.8 journal of science learning article doi: 10.17509/jsl.v5i3.48182 394 j.sci.learn.2022.5(3).386-397 gamze: we designed a solar-powered car with technology. i felt like an engineer. burçak: i think it's just science. because the activities we do look like science. büşra: i think there is a relationship. for example, i learned something about science while making a raft. while building the raft, we did it with math information. i did technology while creating a raft. tolga: there is no connection because there is no relation between them. table 5 shows the frequency and percentage distribution of the answers given by the students to the question, "would you explain if you had difficulties during the implementations?" in response to the question, "would you please explain if you had difficulties during the implementations," which was asked the students, ten students stated that they had difficulties. in comparison, nine students indicated that they did not have any problems. among the students who had difficulties, it was seen that five students answered that they had difficulties in designing, one student mentioned assembling lego pieces, one student mentioned classifying, two students mentioned teamwork, and one student mentioned using the activity materials. the answers given by some of the students are given below: pelin: i did not have any difficulties; even if i did, i managed to cope with them. sıdıka: i had a hard time designing a bridge at first, but then i learned. mehmet: i had a hard time putting the lego pieces together because if we put the lego in the wrong place, that experiment wouldn't work. büşra: i have not had any difficulties so far. tuba: yes, i did. for example, i had a little difficulty making the edges of the raft. table 6 shows the frequency and percentage distribution of the answers given to the students for the question, "which activity did you enjoy the most from the practices? can you explain?" to the question posed to the students, "which activity did you enjoy the most from the practices carried out, please explain", seven students cited "energy from the sun". among the students who enjoyed this activity, 3 of them found it interesting, two said they saw it for the first time, one one student liked the "don't say you can't have a wind-powered car" activity because it caught their attention. six students said they enjoyed the "eggs don't always crack" activity. three students who chose this activity stated that they liked designing, one said they wanted it, and two said eggs always cracked. on the other hand, two students expressed the activity "we are building the 4th bridge to istanbul" with the explanation that "it caught their attention". furthermore, one student stated, "my water is very colorful" activity, and another said that "the color change caught my attention". on the other hand, one student expressed the activity "whose raft is more durable" as the activity they liked the table 5 frequency and percentage distribution of the answers regarding whether the students experience difficulties during the applications student status codes f % having difficulty designing 5 26.3 assembling lego pieces 1 5.3 making a classification 1 5.3 in teamwork 2 10.5 use of event materials 1 5.3 not having difficulty i did not experience any difficulties 9 47.4 table 6 frequency and percentage distribution of students' responses to the applications they liked most name of the activity % codes f % the energy coming from the sun 36.8 interesting 3 15.8 i saw it for the first time 2 10.5 i was intrigued 1 5.3 it works differently from a normal car 1 5.3 don't say you can't have a wind-powered car 5.3 i was intrigued 1 5.3 eggs don't always crack 31.6 i like to design 3 15.8 i loved 1 5.3 i thought eggs always cracked 2 10.5 we are building the 4th bridge to istanbul 10.5 i was intrigued 2 10.5 my water is very colorful 10.5 it felt so weird 1 5.3 the color change caught my attention 1 5.3 whose raft is more durable 5.3 i was intrigued 1 5.3 journal of science learning article doi: 10.17509/jsl.v5i3.48182 395 j.sci.learn.2022.5(3).386-397 most because it attracted their attention. the answers given by some of the students are given below: tolga: solar-powered car, because the solar-powered car is an exciting thing. zuhal: an uncrackable egg, because i used to say there can never be an uncrackable egg, but we tried and saw that it is indeed possible to make uncrackable eggs. ahmet: color-changing water because it was so weird. eren: i loved the uncrackable egg activity, and i liked to design. burcu: whose raft is more durable was the activity i liked the most because it caught my attention. table 7 shows the frequency and percentage distribution of the answers the 4th-grade primary school students gave to the question, "would you like to receive science classes with such activities or similar activities? why?" all students answered "yes, i would like" to the question "would you like the science course to be taught with such or similar activities? why?" seven students said, "the activities are so nice, and they contribute," seven students said, "i am learning by having fun", one student said, "my handicraft is improving", and one student said, "i learn permanently", one student said "i learn easily" and two students said, "it contributes to the science class". the answers given by some of the students are given below: alperen: i would because while making the applications, we both learn something and transfer that information to the science class. tuba: yes, i would because our classes are so much fun this way. burcu: yes, because such activities can contribute to further developing our handicrafts. mehmet: yes, i would, because it contributed and the classes are so good. pelin: i would like to continue doing activities like this because we had more fun, which helped us learn better. ezgi: yes, i would because we both had fun and learned some things. it gets better and better this way. sıdıka: i would like to learn more efficiently, and i do not forget. in this study, which was carried out with 4th-grade primary school students, the effect of stem applications on students' problem-solving skills was examined, and student opinions were obtained. it was observed that the problem-solving skills of the 4th-grade primary school students in the experimental group where stem applications were carried out were significantly higher than those of the 4th-grade primary school students in the control group where stem applications were not performed. based on this result, it can be stated that stem applications effectively improve the problem-solving skills of 4th-grade primary school students. as a result of the findings obtained through the analysis of the interviews with 4th-grade primary school students after the applications, it was seen that the primary school students encountered various problems during the applications, defined the problem, tried to develop a solution, and finally solved the issues during the activities. when the relevant literature is examined, pekbay (2017) conducted stem activities with secondary school 7thgrade students and examined the effects on students' problem-solving skills based on daily life. in the study, it was concluded that stem activities were influential in the development of students' problem-solving skills. ceylan (2014) examined the effect on students' problem-solving skills by applying an instructional design based on stem education on acids and bases. the author concluded that the students in the experimental group were more successful in problem-solving skills compared to the control group. alan, zengin, & keçeci (2019) conducted a stem application to support the integrated teaching knowledge of pre-service science teachers and examined the impact on students' problem-solving skills. it was concluded that the stem applications were influential in developing problem-solving skills in the experimental group pre-service teachers compared to the control group pre-service teachers. in the literature, there are studies on the problem-solving abilities of primary school 4th-grade students in science classes. erden &yalcin (2021) studied the effect of stem education on preschool students' creativity and problem-solving skills by applying stem education prepared according to the design thinking model. as a result of the analysis, it was concluded that there was a significant and permanent increase in the creativity and problem-solving skills of the experimental group. demir (2018) examined whether there is a relationship between primary school 4th-grade students' perceived problem-solving skills in science classes and their routine and non-routine problem-solving skills. in their research, english, king & smeed (2017) asked sixth graders to build earthquake-resistant structures to follow the engineering design processes. table 7 frequency and percentage distribution of students' views and opinions on the realization of science classes with stem activities codes f % yes, i would like the events are very friendly, and they make a contribution 7 36.8 i am learning by having fun 7 36.8 my handicraft is improving 1 5.3 i learn permanently 1 5.3 i learn easily 1 5.3 it contributes to science class 2 10.5 journal of science learning article doi: 10.17509/jsl.v5i3.48182 396 j.sci.learn.2022.5(3).386-397 when the study results were examined, the students stated that their skills in solving engineering problems improved, and the intelligence of stem basic concepts was ensured. parno, yuliati, & ni'mah (2019), in their research, aimed to reveal the influence of problem based learning-science technology engineering mathematics (pbl-stem) on students' problem-solving skills in the topic of optical instruments. the result showed that the learning model influenced students' problem-solving skills. experiment i and ii groups, obtained an increase in problem-solving skills at a level higher than the control group. as a result of the research found a significant relationship between students' ability to solve routine and non-routine problems and their perceived problem-solving skills (acar, tertemiz & taşdemir, 2020). the problemsolving abilities result obtained within the scope of the study coincide with the problem-solving skills results of secondary school students of stem education. it is also compatible with the problem-solving abilities performed by primary school 4th-grade students. it is aimed to raise our students, entrusted to us today, who will be the adults of the future, as individuals with 21stcentury skills who produce, contribute to economic and social developments, and follow the requirements of the world and the age. moreover, the need for students who think, research, question, and invent is increasing daily. therefore, countries are searching for an education model that will prepare students for life as responsible individuals with high problem-solving, critical thinking, and decisionmaking skills. for this reason, stem education, which enables students to transform the theoretical knowledge they have learned in science, technology, engineering, and mathematics disciplines into practice, innovative inventions, and products, is being included in the curriculum of many countries in the world (ministry of national education [mne], 2018b). 4. conclusion when we look at the studies on stem education, the number of studies conducted in our country has been increasing rapidly in recent years. however, most studies are conducted with secondary school students, pre-service teachers, and teachers. due to the scarcity of studies on primary school students, it is recommended that researchers conduct studies with primary school students to overcome this deficiency. for stem educators who will carry out stem applications, it is recommended to prepare booklets containing stem activities following student levels. for a more effective stem education to be implemented, schools should be arranged to provide conditions favorable for stem education. it is recommended that stem activities be carried out with stem educators in the design-skill workshops, which are planned to be opened in the 2023 vision references acar, d. (2020). the role of stem awareness in predicting the contribution of teachers' problem solving skills and behaviors to the development of creative thinking. academia journal of educational research, 5(1), 77-89. https://dergipark.org.tr/en/download/article-file/1094399 acar, d., tertemiz, n. & taşdemir, a. (2020). the relationship between mathematics and science problem solving skills and achievements of students who were being educated with stem. bartın university journal of educational research, 3(2), 12-23. https://dergipark.org.tr/en/pub/bujer/issue/54228/712374 akçay, b. (2019). stem etkinliklerinin anaokuluna devam eden 6 yaş çocukların problem çözme becerilerine etkisi [the effects of stem activiıes on the problem solving skills of 6 aged preschool children] (master's thesis). yildiz technical university, turkey. alan, b. (2017). supporting preservice science teachers' integrated teaching knowledge: preparation training practices to stem. [unpublished master’s thesis]. fırat university, turkey alan, b., zengin, f. k., & keçeci, g. (2019). using stem applications for supporting integrated teaching knowledge of pre-service science teachers. journal of baltic science education, 18(2), 158-170. https://doi.org/10.33225/jbse/19.18.06 asigigan, s. i. & samur, y. (2021). the effect of gamified stem practices on students' intrinsic motivation, critical thinking disposition levels, and perception of problem-solving skills. international journal of education in mathematics, science, and technology (ijemst), 9(2), 332-352. https://doi.org/10.46328/ijemst.1157 bahar, m., yener, d., yılmaz, m., hayrettin, e. & gürer, f. (2018). the changes of standards in the 2018 science curriculum and stem integration. abant i̇zzet baysal university journal of education faculty, 18(2), 702-735. https://doi.org/10.17240/aibuefd.2018..-412111 ceylan, s. (2014). a study for preparing an instructional design based on science, technology, engineering and mathematics (stem) approach on the topic of acids and bases at secondary school science course. [unpublished master thesis]. uludağ university education science institute, bursa, turkey. creswell, j. w., & clark, v. l. p. (2015). pesquisa de métodos mistos-: série métodos de pesquisa [mixed method research-: research methods series.] penso editora. çepni, s. (2014). introduction to research and project work (7. edition). trabzon: celepler printing press. demir, d. (2018). the relationship between students' association with daily life and perceived problem solving skills and their routine and non-routine problem solving skills (master's thesis), kocaeli university, turkey. dewey, j. (1997). how we think? new york: prometheus books. doğan, a., aydın e & kahraman e. (2020). the effect of stem activity applications on problem solving skill perceptions of middle school students. estudam journal of education, 5(2), 123-144. https://dergipark.org.tr/en/download/article-file/1197604 english, l.d., king, d. & smeed, j. (2017). advancing integrated stem learning through engineering design: sixth grade students" design and construction of earthquake resistant buildings. the journal of educational research, 110(3), 255-271. https://doi.org/10.1080/00220671.2016.1264053 erden, s. & yalçın, v. (2021). investigation of the effect of preschool stem activities prepared according to the problem based learning approach on children's problem-solving skills. trakya journal of education, 11(3), 1239-1250. doi: 10.24315/tred.789922 gök, r. & sılay, i. (2008). the effects of problem-solving strategies on students' achievement. on the cooperative learning groups in physics teaching. hacettepe universty journal of education, 34, 116-126. http://www.efdergi.hacettepe.edu.tr/shw_artcl-519.html güven, h. (2022). the effect of stem and stem-based robotic activities on the problem solving and metacognitive skills of secondary school students (master's thesis). erzincan binali yıldırım university, turkey hebebci, m. t. (2019). the impacts of science, technology, engineering and mathematics applications on middle school students' academic achievement, https://dx.doi.org/10.33225/jbse/19.18.06 https://doi.org/10.46328/ijemst.1157 https://doi.org/10.17240/aibuefd.2018..-412111 journal of science learning article doi: 10.17509/jsl.v5i3.48182 397 j.sci.learn.2022.5(3).386-397 scientific creativity and attitudes (phd dissertation). necmettin erbakan university, turkey. honey, m., pearson, g. & schweingruber, h. (eds). national academy of engineering and national research council (2014). stem integration in k-12 education: status, prospects, and an agenda for research. washington d.c. : the national academies press. karasar, n. (2005) scientific research methods (15. edition). ankara: nobel publication distribution. kartini, f. s., widodo, a., winarno, n., & astuti, l. (2021). promoting student's problem-solving skills through stem project-based learning in earth layer and disasters topic. journal of science learning, 4(3), 257-266. https://doi.org/10.17509/jsl.v4i3.27555 kurt, m. & benzer s. (2020). an investigation on the effect of stem practices on sixth grade students' academic achievement, problem solving skills, and attitudes towards stem. journal of science learning, 3(2), 79-88. doi: 10.17509/jsl.v3i2.21419 ministry of national education -mne (2018b). stem education teacher's handbook. ankara: ministry of national education, general directorate of innovation and educational technologies (yeği̇tek). ministry of national education-mne (2016). stem education report. ankara: ministry of national education, general directorate of innovation and educational technologies (yeği̇tek). ministry of national education-mne (2018a). science course (primary and secondary schools 3, 4, 5, 6, 7 and 8th grades) curriculum. national education press, ankara. özçelik, a. & akgündüz, d. (2018). evaluation of gifted/talented students' out-of-school stem education. trakya university journal of education faculty, 8(2), 334-351. doi: 10.24315/trkefd.331579 özkök, a. (2005). effects of interdisciplinary creative problem solving teaching program on creative problem solving skills. hacettepe universty journal of education, 28(28), 159-167. http://www.efdergi.hacettepe.edu.tr/shw_artcl-784.html. öztürk, s. c. & yalçın, s.a. (2020). the effect of stem education on pre-service science teachers' problem solving skills. turkish studies education, 15(4), 2893-2915. https://dx.doi.org/10.47423/turkishstudies.43707 öztürk, z. d. & çınar, s. (2022). the effect of engıneerıng desıgn based stem educatıon on the problem-solvıng skılls of pre-school students. trakya journal of education, 12(1), 34-56. doi: 10.24315/tred.868414 pekbay, c. (2017). effects of science technology engineering and mathematics activities on middle school students. (master's thesis), hacettepe university, ankara. sangngam, s. (2021). the development of early childhood students' creative thinking problem solving abilities through stem education learning activities. in journal of physics: conference series (vol. 1835, no. 1, p. 012008). iop publishing. serin, o., bulut-serin, n., & saygılı, g. (2010). developing problem solving inventory for children at the level of primary education (psic). elementary education online, 9(2), 446-458. https://dergipark.org.tr/en/download/article-file/90752 şahin, a., ayar, m. c., & adıgüzel, t. (2014). stem related after-school program activities and associated outcomes on student learning. education sciences theory & practice, 14(1), 297-322. doi: 10.12738/estp.2014.1.1876 şanlı, z. s. (2021). the effects of stem activities on creative thinking and problem solving skills of 60-72 month-children's in early childhood (phd dissertation). hacettepe university, turkey. tekin poyraz, g. (2018). examination of stem education in kayseri province and feasibility of distance stem education. (master's thesis), anadolu universty, turkey. tiryaki, a., & adigüzel, s. (2021). the effect of stem-based robotic applications on the creativity and attitude of students. journal of science learning, 4(3), 288-297. doi:10.17509/jsl.v4i3.29683 tran, y. (2018). computer programming effects in elementary: perceptions and career aspirations in stem. technology, knowledge and learning, 23(2), 273-29. https://doi.org/10.1007/s10758-0189358-z üstün, a., & bozkurt, e. (2003). some occupational factors affecting the problem solving skills of primary school principals according to their perceptions of themselves. kastamonu education journal, 11(1), 13-20. https://akademiye.org/tr/wpcontent/uploads/2021/05/turk-kulturunde-veazerbaycan-destanlarinda-at.pdf#page=15 yalçın, v. & erden, ş. (2021). the effect of stem activities prepared according to the design thinking model on preschool children's creativity and problem-solving skills. thinking skills and creativity, (41), 1-14.doi: https://doi.org/10.1016/j.tsc.2021.100864 yenilmez, k. (2010). primary school students' determination levels about problem types. the journal of sau education faculty, 19, 124137. https://dergipark.org.tr/en/download/article-file/115623 yıldız, s., & özdemir, m. (2022) i̇lkokul öğrencilerinin problem çözme becerilerinin ve steme yönelik tutumlarının incelenmesi [examınıng of prımary school students’ problem-solvıng skılls and attıtudes towards stem]. milli eğitim dergisi, 51(234), 987-1010. doi: 10.37669/milliegitim.837070 yuliati, l., & ni'mah, b. q. a. (2019, february). the influence of pblstem on students' problem-solving skills in the topic of optical instruments. in journal of physics: conference series (vol. 1171, no. 1, p. 012013). iop publishing. yurttaş, ş. (2021). the effect of group engineering design-based robotics applications on students' daily life-based problem solving skills (master's thesis). bursa uludağ university, turkey. https://doi.org/10.17509/jsl.v4i3.27555 http://www.efdergi.hacettepe.edu.tr/shw_artcl-784.html https://doi.org/10.1007/s10758-018-9358-z https://doi.org/10.1007/s10758-018-9358-z a © 2023 indonesian society for science educator 181 j.sci.learn.2023.6(2).181-193 received: 31 october 2022 revised: 09 december 2022 published: 31 march 2023 ±this study is a part of the unpublished ph.d. dissertation entitled “a case study on pre-service science teachers’ dynamic mental constructs on the concepts regarding the phases of the earth’s moon” approved by the middle east technical university examination of dynamic mental constructs and their change regarding phases of the moon ali sağdıç1*, elvan sahin2 1department of mathematics and science education, dede korkut education faculty, kafkas university, turkey 2department of mathematics and science education, education faculty, middle east technical university, turkey *corresponding author: sagdic.ali@gmail.com abstract an immense body of literature on astronomy studies has provided evidence that individuals perceive the lunar phases concept as difficult. furthermore, many studies have shown erroneous explanations or alternative conceptions of lunar phases. however, there is also a need to understand how individuals construct an explanation of the moon’s phases. this paper aims to explore pre-service science teachers' construction process of their explanations regarding lunar phases through clinical interviews. the data were obtained from fourteen pre-service science teachers and analyzed, identifying their nodes and dynamic mental constructs. the results indicated that most pre-service science teachers did not organize their nodes in a manner consistent with normative scientific explanations. in addition, pre-service science teachers changed their dynamic mental constructs while explaining different lunar phases, utilizing different media such as drawing and three-dimensional models, and responding to prompted questions. it was suggested that different opportunities, including technology-enriched activities regarding phases of the moon, should be provided for pre-service science teachers to reorganize their nodes. keywords phases of the moon, pre-service science teachers, knowledge in pieces, clinical interview 1. introduction astronomy is one of the oldest vehicles for people to understand celestial bodies and the celestial phenomena surrounding them. thirty-thousand-year-old cave paintings show that our ancestors systematically observed celestial bodies (karttunen, kroger, oja, poutanen, & donner, 2007). their observation of lunar phases, eclipses, planets, stars, and other celestial bodies was the initial step of modern astronomy. today, astronomy presents attractive issues for human beings. people consider questions such as how the universe started, how it will end, whether there is life on distant planets and galaxies, and black holes (de pree & axelrod, 2001). astronomy has also been perceived as an indispensable component of educational programs since it is closely intertwined with human life (bailey & slater, 2004). topics dealing with astronomy, such as lunar phases, eclipses, and the solar system, have been integrated into educational programs in various ways (pasachoff & percy, 2009). however, the inclusion of astronomy topics in education programs may not ensure individuals’ sufficient understanding. understanding astronomy topics requires more than knowing appropriate facts; it depends on individuals’ imagination, representation, and transformation abilities (subramaniam & padalkar, 2009). individuals possess a substantial body of astronomy knowledge before instruction, which influences their learning. studies (e.g., venville, louisell, & wilhelm, 2012) showed that even a three-year-old child knows why we observe different lunar phases. determining these existing conceptions is important since learning occurs due to the interaction between existing knowledge and new experiences (posner, strike, hewson, & gertzog, 1982). more specifically, existing knowledge is transformed into new forms that are expected to be consistent with scientific understanding (sherin, 2006), and this process has been entitled conceptual change (disessa & sherin, 1998). different frameworks exist explaining the structure and development of knowledge. among these frameworks, the “knowledge in pieces perspective” proposes that the journal of science learning article doi: 10.17509/jsl.v6i2.51686 182 j.sci.learn.2023.6(2).181-193 knowledge system is constituted by knowledge elements (minstrell, 1982; smith, disessa, & roschelle, 1993). accordingly, knowledge elements are bits of a knowledge system, and individuals explain a natural phenomenon by activating and organizing them in a context (disessa, gillespie, & esterly, 2004). an activated group of knowledge elements should fit together in a context and be meaningful to solve a problem or respond to a question (hammer, scherr, & redish, 2005). researchers studying conceptual change propose a variety of different terminology to clarify knowledge elements as facets (minstrell & stimpson, 1996), phenomenological primitives (disessa, 1993), and node/mode (sherin, krakowski, & lee, 2012). according to the mode-node framework, a node refers to all knowledge elements (e.g., mental images, mental models, general schemas). nodes are connected, and a group of connected nodes is called a mode. in addition, individuals activate nodes and modes during the sensemaking process. therefore, an operational and temporal form called a dynamic mental construct (dmc) is constituted. clinical interviews are one of the rational methods to understand individuals’ conceptual understanding. according to posner and gertzog (1982), clinical interviews portray knowledge systems by identifying existing concepts and interactions among them. participants are exposed to problem situations and attempt to produce feasible solutions during a clinical interview (disessa, 2007). clinical interviews provide a comfortable environment for interviewees to speak freely and reconsider their responses. since this process is dynamic, participants may revise or alter their initial responses without manipulation (disessa et al., 2004; sherin et al., 2012). changes in participants' responses during clinical interviews are perceived as conceptual changes. this change occurs in a short period and is described using various terminologies such as “conceptual dynamics” (sherin et al., 2012) and “crystallization” (blown & bryce, 2006). in this study, pre-service science teachers’ conceptual understanding changed during clinical interviews. the aim was to portray the fragmented understanding of pre-service science teachers and examine how their understandings change in different contexts. to achieve this aim, this research focuses on the following questions. a) how do pre-service science teachers organize their nodes to explain the moon's phases during a clinical interview? b) how do pre-service science teachers change their initial explanations of the phases of the moon during a clinical interview? previous studies on the conceptual change regarding phases of the moon concepts (e.g., trundle, atwood, & christopher, 2002) were generally designed in light of a classical view. accordingly, individuals’ knowledge was perceived as a coherent structure among different contexts. these studies typically examined responses via a coding schema and categorized participants' responses as scientific or alternative explanations. to illustrate, labeling a participant's explanation as "alternative eclipse" means that s/he believes the earth's shadow causes the lunar phases (e.g., hobson, trundle, & saçkes, 2010; trundle et al., 2002; trundle & bell, 2010). however, this method cannot fully reflect the participant's conceptual understanding. for instance, it is not clear whether participants consider the orbit of the moon around the earth or the illumination of the moon by the sun. in addition, these studies presumed that participants propose the same explanation for each lunar phase, and the explanation does not change in different media. the current study followed the knowledge in pieces perspective to predict and understand pre-service science teachers' understanding of the moon's phases. knowledge pieces are activated and modified depending on the context (parnafes, 2007). since they are not coherent, students' responses may be contradictory or unstable from one situation to another due to a change of context (parnafes, 2012). considering these ideas, this study aimed to map pre-service science teachers' conceptual understanding of the moon’s phases in various context. pre-service science teachers' explanations of moon phases using different media were examined, as each moon phase corresponds to a different context. additionally, it tracked how their explanations changed with different media. unlike the literature, this study aimed not to determine pre-service science teachers' misconceptions about the moon’s phases but to map the structure of their conceptual understanding. therefore, the current study's findings present how preservice science teachers activated knowledge elements and how they produced a dmc regarding the moon's phases. 1.1 how do phases of the moon occur? the moon is observed with its eight major phases from earth. these different appearances result from the sun/earth/moon system. a normative scientific explanation of lunar phases covers four mechanisms (parnafes, 2012; trundle et al., 2002). as seen in table 1, the moon revolves around the earth, and half of the moon is illuminated by the sun while revolving. in addition, it should be noted that half of the moon can be seen from earth. finally, individuals should visualize these three conditions together and predict that the illuminated part of the moon is visible from earth to determine lunar phases. a scientific explanation regarding the moon's phases should cover nodes of orbit, illumination, and half. however, individuals may activate unrelated nodes and produce different dmcs. one of the popular nodes activated while explaining the moon's phases is "shadow". more specifically, the phases of the moon concept are often confused with the eclipse concept since they are related phenomena (kanli, 2014; semercioğlu & kalkan, journal of science learning article doi: 10.17509/jsl.v6i2.51686 183 j.sci.learn.2023.6(2).181-193 2021). furthermore, the location of an observer on earth (dunlop, 2000; stahly, krockover, & shepardson, 1999), earth's rotation (baxter, 1989), and the distance between the sun/earth/moon (trundle et al., 2002) are other common conceptions regarding the cause of lunar phases. to arrive at a reasonable conclusion about the normative scientific explanation of lunar phases requires performing spatial abilities. studies (e.g., black, 2005; wellner, 1995) have shown that high abilities in spatial thinking improve students' understanding and explanation of the phases of the moon topic. to be more precise, spatial characteristics of celestial bodies, such as size, position, or motion, should be visualized by the individual for reasoning on phases of the moon (subramaniam & padalkar, 2009; wilhelm, jackson, sullivan, & wilhelm, 2013). 2. method 2.1 research design the qualitative research paradigm shaped the current study to obtain detailed information regarding pre-service science teachers' conceptual understanding of the phases of the moon. this study was designed as a case study. a case study is regarded as examining an issue from one or more cases within a bounded system (creswell, 2007). case studies provide holistic descriptions and explanations of situations, settings, or events (merriam, 2009). pre-service science teachers who participated in the present study are referred to as cases. their conceptual understanding of the phases of the moon is a complex system including different knowledge elements. 2.2 participants a total of fourteen pre-service science teachers participated in the current study. the research participants were selected by purposive sampling and consisted of ten juniors and four seniors. these teacher candidates enrolled in astronomy and physics courses in the science teacher education program. participants were acquainted with the first author of the study. the researcher had assisted in some major undergraduate courses of the participants before the interview. as a result of the rapport between the researcher and participants, they responded to interview questions sincerely without any academic or social concerns. pseudonyms were utilized to protect confidentiality. 2.3 data collection pre-service science teachers attempted to find a feasible way to explain why we observe lunar phases, and they tried to show this mechanism using drawings and models. to ascertain the nature and extent of pre-service science teachers' conceptual understanding of phases of the moon, clinical interviews (posner & gertzog, 1982) with a multimedia approach (blown & bryce, 2006, 2010) were table 1 component of scientific understanding of lunar phases explanation demonstration the moon orbits the earth half of the moon is illuminated by the sun an observer from the earth always sees half of the moon the lighted side of the moon that is seen from the earth determines the phase journal of science learning article doi: 10.17509/jsl.v6i2.51686 184 j.sci.learn.2023.6(2).181-193 utilized. accordingly, pre-service science teachers were asked to respond to different interview questions in various ways (e.g., verbal explanation, drawing, and model demonstration). it was presumed that since the context (e.g., explaining different moon phases) changes, activated nodes, and dmcs change. therefore, clinical interviews were conducted to detect various nodes and different dmcs activated by pre-service science teachers during different parts of the interview. each interview lasted between sixty to ninety minutes. the interview process was flexible, and participants could freely talk and reconsider their responses. interview questions were constituted based on examining previous studies on understanding the phases of the moon. a total of four main interview questions (see table 2) and many follow-up questions were directed to participants. initially, pre-service science teachers verbally responded to the first two questions. then, papers and pencils were provided to pre-service science teachers for the third question. after verbal responses and drawings, they were asked to use three-dimensional models to respond to the fourth question. pre-service science teachers showed the sequence of the sun/earth/moon for each phase. in addition, they explained how the moon orbits, how the sun illuminates, and which part of the moon is visible using a three-dimensional model. obtaining richer and more profound information on the conceptual understanding of pre-service science teachers is a primary goal of the researchers. in addition, researchers aim to elucidate participants' conceptions without any manipulation or instruction. participants generally do not consider different perspectives of an issue and are unaware of inconsistencies in their explanations (sherin et al., 2012). therefore, inconsistencies among their explanations were explicitly pointed out by the researchers and directed as further questions to observe the change in their understanding. for instance, if the waning crescent is explained differently in the model demonstration from their drawing, their drawings are shown again. 2.4 data analysis data were analyzed to reveal knowledge elements and how these elements constitute a dmc. this research followed the node/mode framework and knowledge in pieces perspectives. first, the activated nodes of the preservice science teachers were coded. before the data analysis, previous research on phases of the moon was examined to determine possible nodes, and most of the nodes were adopted from previous studies (nielsen & hoban, 2015; parnafes, 2012; trundle et al., 2002). then, additional nodes emerging from the analysis were added to the node list. a complete list of the nodes and their descriptions are presented in table 3. finally, activated nodes and interactions between them were analyzed to generate dmcs. to establish the reliability of the coding, two additional researchers with experience in phases of the moon and table 2 interview questions 1 in what ways do you see the moon when you look at it from the earth? if you looked at the moon, in what appearances would you observe the moon? 2 what could be the reasons behind the moon's different appearances? how would you explain this for each different appearance? 3 would you draw the explanation of each different appearance? 4 would you demonstrate the occurrence of each different appearance with three-dimensional models? table 3 description of activated nodes nodes description orbit the moon orbits the earth. illumination a star shines on a planet and illuminates half of its surface. half only half of the moon can be observed from the earth. apparent the entire illuminated half of the moon does not always point toward the earth. shadow/blocking the planets cast shadows in the opposite direction to the sun. the earth/moon or other celestial bodies can block the light from the sun. location observers stand in different locations on the earth's surface to observe different lunar phases. distance things that are closer look bigger, and things that are farther away look smaller. angle* earth receives the light from the moon at a different angle. day/night* the moon and its phases are apparent only at night. tilt* earth’s axis is tilted. reflection* different parts of the moon reflect light to the earth to different extents. the direction of sunlight changes before they receive the earth’s surface. (refraction) different wavelengths of sunlight scatter before they receive the earth. (scattering) *nodes emerged in the current study. journal of science learning article doi: 10.17509/jsl.v6i2.51686 185 j.sci.learn.2023.6(2).181-193 qualitative research coded 10% of the data. consistency among the first author and the researchers' coding was 82%. the coders discussed inconsistencies and reached an agreement. the different sources of data (verbal responses, drawings, and demonstrations) were triangulated on a basis to confirm assertions generated. pre-service science teachers' responses in three different ways were analyzed independently. finally, the analysis of each response was compared with others. sample analysis presented in the following excerpt showed how the study's credibility improved by analyzing different sources. for instance, aykut verbally explained the phases of the moon: interviewer: what could be the reasons behind the moon's different appearances? aykut: it is related to being close or away. it moves closer or farther due to the earth's horizontal axis. therefore, there is no stable distance between the moon and the earth. therefore, larger shapes occur when it is closer to the earth. it might be asserted that phases of the moon occur because of the earth's spin on its horizontal axis, considering aykut's expression, "it moves closer or farther due to the earth's horizontal axis". therefore, it was also plausible to infer that the distance between the earth and the moon changes due to the earth's horizontal spin. however, aykut drew figure 1, which shows the orbit of the moon around the earth. a comparison of these different data sources indicated that aykut’s responses correspond to the moon's orbit instead of the earth's spin. 3. findings the analysis of pre-service science teachers' responses showed that they activated eleven nodes, four of which orbit, illumination, half, and apparent are essential for appropriately explaining the phases of the moon. the remaining seven nodes shadow, reflection, location, distance, tilted, day & night, and angle are not directly related to the phases of the moon. a dmc includes between two or four nodes. additionally, fourteen preservice science teachers produced twenty-five dmcs. the total number of nodes detected from pre-service science teachers' dmcs is presented in table 4. the dmcs were classified into three groups based on the number of activated nodes: low-level (two nodes), medium-level (three nodes), and high-level (more than three nodes). the total number of activated nodes is presumed to be an indicator of the quality of the dmc due to two factors. initially, the explanatory power of dmcs increases while the number of activated nodes increases. for instance, a dmc including orbit, illumination, and half explains the moon's phases more clearly than another dmc including only orbit and illumination nodes. the second factor is related to the consistency among the nodes. while the number of nodes increases, ensuring consistency among them becomes a more challenging task. pre-service science teachers should be aware of possible discrepancies among the nodes. the nodes did not contribute equally to the explanations provided by the pre-service science teachers. different medium-level dmcs were named based on the dominant nodes: angle, distance, shadow, reflection, and location. table 5 shows the number of dmcs produced by different pre-service science teachers and the eight groups into which the 25 dmcs were categorized. figure 1 aykut’s drawing of phases of the moon table 4 knowledge elements activated by pre-service science teachers elements *number element *number orbit 25 location 4 illumination 25 distance 3 shadow 8 tilted 3 apparent 4 day & night 2 half 4 angle 1 refraction 4 *total number of nodes detected from all dmc table 5 change of pre-service science teachers’ dmc name first dmc second dmc third dmc aytolun angle aykut distance high high ayça shadow reflection high aygun shadow high ayten low day & night aysil reflection location shadow aynur shadow high aydan low aytac reflection shadow aybuke location shadow aysu high ayfer high aybilge high aysin high journal of science learning article doi: 10.17509/jsl.v6i2.51686 186 j.sci.learn.2023.6(2).181-193 table 5 shows that eight pre-service science teachers suggested at least two dmcs. the remaining six pre-service science teachers produced only one dmc during the interview. on the other hand, fourteen medium-level, two low-level, and nine high-level dmcs were detected from pre-service science teachers' responses on phases of the moon. high-level dmc may not necessarily denote preservice science teachers' scientific explanation. although nine high-level dmcs were detected, only four proposed by aysu, ayfer, aybilge, and aysin were classified as scientific explanations. although aykut, ayça, and aygun produced high-level dmcs, these dmcs do not include the required nodes to explain the phases of the moon consistent with scientific explanations. 3.1 the low-level dmc all the pre-service science teachers activated orbit and illumination nodes in their explanations. however, some tried to explain the phases of the moon only with these two nodes. since the explanatory power of this explanation was low, they were called low-level dmc. finally, there were two pre-service science teachers (ayten and aydan) who explained the phases of the moon with illumination and orbit nodes. as seen in the following excerpt, aydan stated that phases of the moon occur as a consequence of the illumination of the moon and the moon's orbit around the earth. the moon should be illuminated. however, the moon does not have its light. the source of the light is the sun. the sun illuminates both the earth and the moon. therefore, we observe the moon phases since the moon is illuminated while orbiting around the earth. pre-service science teachers produced low-level dmcs at the beginning of the interviews. however, after realizing the shortcomings of their explanations, they changed their dmcs by activating additional nodes. 3.2 the medium-level dmcs in addition to orbit and illumination nodes, pre-service science teachers activated one more node to explain the phases of the moon. shadow, location, distance, reflection, and day & night nodes were activated as additional third nodes for explaining the phases of the moon. since these additional nodes are more dominant than orbit and illumination nodes, detected dmcs were entitled to considering these additional dominant nodes. these medium-level dmcs are presented in the following sections. shadow-dominated dmc shadow-dominated dmc means that the phase of the moon occurs since the moon is in the shadow cone of the earth or the earth blocks the light of the sun. a total number of six shadow-dominated dmcs were detected. for example, aynur gave the following explanation: aynur: it occurs there (the moon is between the earth and the sun) since the moon receives all sunlight. therefore, it is the full moon here. therefore, no obstacle blocks its surface. interviewer: what about the last quarter? aynur: the last quarter, the right side of the moon was blocked. sunlight…the earth blocks the right side of the moon. so the left side of it receives sunlight. as seen in figure 2 and the previous excerpt, aynur explained that the moon should be full between the earth and the sun. according to this viewpoint, the earth does not prevent the illumination of the moon in this position. however, it was also revealed that aynur could not visualize which part of the moon receives sunlight and which part of the moon can be seen from the earth. location-dominated dmc according to this perspective, phases of the moon are mainly determined by the observer's location on earth. pre-service science teachers determined locations on earth and then predicted lunar phases from these points. two pre-service science teachers produced location-dominated dmcs to explain lunar phases. to illustrate, aybuke explained, "it is important from where we observe the moon. simultaneously, someone looking at the moon from america and i (in europe) observe differently since they receive lights from different parts of the moon than me." in addition to this explanation, s/he created a diagram showing that people from different locations can simultaneously observe the moon as the waning gibbous and crescent gibbous, as shown in figure 3. figure 2 aynur’s drawing of phases of the moon journal of science learning article doi: 10.17509/jsl.v6i2.51686 187 j.sci.learn.2023.6(2).181-193 angle-dominated dmc there was one pre-service science teacher, aytolun, who produced angle-dominated dmc. angle-dominated dmc refers to considering the angle of reflected sunlight to earth from the moon. for instance, aytolun asserted: while the moon rotates and orbits, the light-received from the moon most probably spreads out. for example, if we observe the new moon when the angle of the light (between the normal of the surface) is ninety degrees. while reducing this degree, the appearance of the moon changes in the first quarter. as seen from the utterance, s/he activated three nodes as orbit, illumination, and angle to explain the phases of the moon. s/he mainly considered the angle of the lights changing while the moon orbits around the earth. this drawing also showed the perspective of the pre-service teacher on phases of the moon. as seen in figure 5, a normal vector perpendicular to the moon's surface was drawn, and the α symbol denotes the angle between the normal and the lights. distance-dominated dmc the main perspective behind distance-dominated dmc is that the distance between earth and the moon determines the moon’s appearance. aykut developed distance-dominated dmc, activating distance, orbit, and illumination nodes to elucidate the phases of the moon. s/he explained: it is related to being close or away. it moves closer or farther due to the earth's horizontal axis. therefore, there is no stable distance between the moon and the earth. therefore, larger shapes occur when it is closer to the earth. as evidenced by previous statement and figure 5, aykut posited that the distance between the moon and earth determines the moon’s phases. accordingly, as the distance decreases, the visible portion of the moon from earth increases. reflection-dominated dmc pre-service science teachers attempted to explain the moon’s phases using optical principles such as refraction, reflection, or scattering. in brief, reflection-dominated dmc refers to the idea that the moon reflects light while orbiting earth; resulting in the observation of different lunar appearances. for example, aysil drew figure 6 and explained: the sun is here, and we orbit it. the moon is here. since all reflected lights do not reach earth, we only see a part of the moon. i may observe the waxing crescent at this point. scientists know their appearance by calculating. since the specific angle, the direction of light, and the position of the moon and earth are known, the actual phases of the moon can be calculated. i am just forecasting the phases since i do not have data. day and night-dominated dmc pre-service science teachers employed the day and night mechanism to explain lunar phases. two pre-service science teachers (aygün and ayten) activated the day and night nodes. since aygün's explanation comprises more than three nodes, only ayten's explanation was classified as day and night-dominated dmc. s/he generated this dmc figure 3 aybuke’s drawing of phases of the moon figure 4 aytolun’s drawing of phases of the moon figure 5 aykut’s drawing of phases of the moon figure 6 aysil’s drawing of phases of the moon journal of science learning article doi: 10.17509/jsl.v6i2.51686 188 j.sci.learn.2023.6(2).181-193 while displaying three-dimensional models. s/he articulated: ayten: what would happen in this position? do i observe the moon? (silence for a couple of seconds) it is challenging for me to explain. why do i not observe the moon? (silence for 40 seconds). ah, i see. the phases i examine are present here. in this case, earth is illuminated. this side of earth is also in daytime. i need to observe the moon at the night side. so, do i observe the full moon in this position? i think… yes, i observe the full moon in this position. i am sure now. sunlight receives this part (the part facing the earth) of the moon and entirely illuminates this part. this part of earth is day, and i observe full moon from that side of earth. interviewer: what would happen after this point? ayten: then, the moon passed the other side. side of daytime. i could not see the moon when it was daytime on earth. as shown in the excerpt, ayten attempted to explain the lunar phases by considering day and night mechanisms. accordingly, the moon is only visible at night, and its phases change depending on illumination and orbit. after the demonstration, s/he produced figure 7 to show each appearance of lunar phases. 3.3 high-level dmc analysis of the interview data revealed that eight preservice science teachers activated nine high dmcs. a total of four dmcs consisted of the scientific explanation of the moon’s phases, as they included orbit, illumination, half, and apparent nodes. conversely, the other four pre-service science teachers activated four nodes to explain the moon’s phases. for instance, aykut activated tilted, distance, illumination, and orbit nodes to explain the occurrence of lunar phases, as seen in the following statement. aykut: it is the full moon when in a location close to earth. it appears as the first or last quarter when it is far from the earth. we can observe a small portion of the moon due to its distance. but… i have just noticed… why do we observe a small portion rather than viewing the complete moon as small? (silence for about twenty seconds) i think it is related to the orbit and the earth's tilt. interviewer: how does the earth's tilt influence the appearance of the moon? aykut: as the earth has a tilted axis, our angle of view does not cover the whole moon. thus, we observe it as first or lastquarter views. however, in a more suitable position, it can be seen as the full moon. here, aykut activated the tilted node to explain the first and last quarter moon. however, the statement, "why do we observe a small portion rather than viewing the complete moon as small?" revealed that the initial explanation is not enough to explain the first and last quarters. therefore, s/he integrated tilted nodes into the dmc and explained lunar phases. the following excerpts also demonstrated a similar process performed by ayca. interviewer: can you explain why we observe waxing crescents using these models? ayca: this part is light. the rest of it is dark. (silence for a couple of seconds). i think the moon can reflect rays only from this illuminated part. i observe it as a crescent moon since rays reflected from the other part are not enough. for instance, if i vertically expose a lamp to a point, it would be illuminated perfectly. however, if the lamp were tilted, illumination would be reduced in some parts of this point. since the moon is tilted, only the crescent part of it is perfectly illuminated. rays reaching the rest of it are not enough to reach earth. accordingly, s/he articulated that the moon is tilted; therefore, the illumination of the moon changes. as a result, the perfectly illuminated part reflects the sunlight better. considering ayca's explanation, it was revealed that s/he activated illumination, orbit, tilted, and reflection nodes to explain the moon’s phases. 3.4 change in science teachers’ dmc this part of the research aimed to uncover factors influencing pre-service science teachers' activation of different nodes and changing their dmc. analysis showed that pre-service science teachers changed their dmcs due to four factors drawing, three-dimensional models, explaining different phases, and prompted questions. details on the role of these factors in pre-service science teachers' dmcs are presented in this part. three-dimensional models a total of four pre-service science teachers changed their dmc after they were asked to show phases of the moon using a three-dimensional model. one of the critical points is that utilizing three-dimensional models contributed to their spatial abilities. for instance, in the second dmc, aysil articulated: an observer (pointing to figure 8) looks at only a quarter of the moon. however, the observer here has an angle that observes the complete moon. the position of the moon and earth and our location are known. however, the moon is observed differently from different locations on earth. as seen from the previous utterance, aysil explained the moon’s phases by activating location, orbit, and illumination nodes. however, after using the threedimensional model, s/he explained: … since this part of the earth could not receive sunlight, it is night. another side is day. a person from this side (facing the moon) of earth never observes the moon since earth blocks the moon. none of the sunlight reaches the moon. the moon is smaller than earth. earth covers the place where the moon is located. after a slight movement of the moon (s/he moves the moon's right side in its orbit), it receives light. therefore, we observe a crescent. sunlight has an angle. while it moves, i observe an increase in the crescent shape. it becomes thicker. it is more logical. figure 7 ayten’s drawing of phases of the moon journal of science learning article doi: 10.17509/jsl.v6i2.51686 189 j.sci.learn.2023.6(2).181-193 with this response and demonstration in figure 9, aysil seemed to have shifted to a shadow-dominated dmc from a location-dominated dmc. s/he activated orbit, illumination, and shadow nodes while explaining with a three-dimensional model. although pre-service science teachers changed their existing dmc, they could not produce a scientific explanation of the phases of the moon. in other words, the three-dimensional model did not contribute to reaching scientific understanding. pre-service science teachers activated different nodes and produced different dmcs, which still resulted in inconsistent scientific explanations. drawing pre-service science teachers changed their initial explanation of the moon’s phases while drawing lunar phases. it was plausible to consider that drawing allowed the pre-service teacher to see all the moon phases together. in this way, they found an opportunity to compare each lunar phase with one another. for instance, ayca initially explained, "while moving away from the sun and revolving around earth, depending on the sunlight received and shadowing, we observe different phases of the moon". this explanation showed that s/he initially produced shadow-dominated dmc while activating shadow, orbit, and illumination nodes. however, her explanation changed while demonstrating the phases of the moon in figure 10. s/he articulated: interviewer: could you explain how all these different moon appearances occur? ayça: as i have drawn (figure 10), while the moon revolves around earth, we see it (the new moon position) as dark because of earth’s shadow. do we see it as dark since the moon shades it… (silence for a couple of seconds) interviewer: are there any points in your explanation that do not satisfy you? ayça: according to my explanation, if it shades it in that position (new moon), we never see the full moon. the moon completely shades it because earth is bigger, and the moon is smaller when compared to earth. therefore, the moon must be completely dark in its full moon phases. hence, there is no shade of earth on the moon. interviewer: alright, why do we see it as dark in the new moon position? ayça: it may be related to that… sunlight is coming from the sun. it may scatter. because of scattering, earth does not receive any sunlight. so moon's light cannot reach earth. ayça changed their dmc after drawing. s/he considered that a shadow-dominated dmc could not explain both the new and full moon. therefore, s/he shifted to a reflection-dominated dmc (scattering). it is plausible that pre-service science teachers visualize one of the moon phases while responding to why we observe different phases. therefore, the drawing ensured that ayça considered the new moon and full mmoon phases together to explain the rationale behind the phases of the moon. different phases as noted earlier, pre-service science teachers were asked to explain how we observe each moon phase. therefore, they tried explaining the occurrence of the eight moon phases. analysis of responses shows that pre-service science teachers activated different nodes while explaining different lunar phases. to illustrate, we return to the interview conducted with aygün. first, s/he explained phases of the moon concepts using a shadow-dominated dmc. then, considering the drawing in figure 11, the interviewer asked aygün whether we observe the moon in the daytime. as seen from the following utterance, aygün proposed a new model considering the crescent moon. aygün: the lighted part of the moon was limited (referring to the models seen in figure 11). due to its shape, these parts are dark (indicating that part of the moon does not receive sunlight). even in this position (figure on the right). figure 8 aysil’s initial drawing of phases of the moon figure 9 aysil’s demonstration of phases of the moon figure 10 ayca's drawing of phases of the moon journal of science learning article doi: 10.17509/jsl.v6i2.51686 190 j.sci.learn.2023.6(2).181-193 interviewer: how would we observe the moon? aygün: for this (figure on the right), it is mostly half-moon. and for this (figure on the left), it is crescent. interviewer: what would happen at night? aygün: these are for daytime. it is different at night. we observe phases due to the shadow of earth. it is seen in the daytime since the sun illuminates but at night, it is because of the shadow. aygün produced two different models to explain the phases of the moon. first, s/he explained the crescent moon with the illumination of the moon in the daytime while explaining the rest of the phases via shadowdominated dmc. prompted questions pre-service science teachers changed their dmcs after the interviewer prompted questions. these questions were generally directed to pre-service science teachers to capture the details of their explanations. while explaining details, they often realized inconsistencies or lacking points. then they reorganized their dmcs. for instance, aykut explained the full moon and new moon phases using a distance-dominated dmc. however, s/he explained the first and last quarter phases considering location with three-dimensional models. finally, the interviewer directed a prompted question, as seen from the following utterance, and aykut reorganized their dmc. interviewer: is there an equal distance between the moon and earth in your demonstration? aykut: because i had explained them, i did not mention distance again. interviewer: could you explain again, considering the role of distance? aykut: i consider the moon approaches earth in full moon phases. in the case of phases first and last quarter, distance is not close and not away. therefore, crescent moon is far from the earth. interviewer: what would happen if an observer saw a full moon when there is a long distance between earth and the moon? aykut: no way! humm... it would be possible. we sometimes observe the small full moon and sometimes the big full moon. (silence a couple of seconds and then smiles). therefore, it shows me that there is no linkage with distance. interviewer: so, what is the factor or factors that influence the appearance of the moon? aykut: phases of the moon, how can i explain? our viewpoint. the location where we observe the moon. as shown in the previous excerpt, aykut changed activated nodes after the interviewer's question. therefore, aykut reconsidered dmc and reorganized it. 4. discussion this study aimed to understand the process of constructing explanations by identifying nodes and activated dmcs. results showed that pre-service science teachers activated various nodes during the interview process, attempting to produce a dmc by activating these nodes. on one hand, a total of ten pre-service science teachers sought to explain the lunar phases during the interview. in other words, these pre-service science teachers generated one or more dmcs by activating different nodes. these conceptual structures may quickly change into other forms on short time scales of seconds. therefore, in alignment with the studies of sherin et al. (2012), these structures were regarded as dmcs, focusing on their dynamic and temporary features. on the other hand, four pre-service science teachers confidently and immediately responded to interview questions, providing answers consistent with the normative scientific explanations. these responses were also considered dmcs, as individuals may recall an issue based on their experience. their experience and familiarity enabled the activation of appropriate knowledge elements (körhasan, 2021). sherin et al. (2012) noted that a group of activated nodes might constrain dmcs. to cite an example, this situation resembles asking an astronomer to explain the seasons. due to their experience with astronomy-related issues, they can quickly activate appropriate nodes to produce a dmc. in the current study, four pre-service science teachers were fourth-grade (senior) students, which may indicate in-depth and considerable experience with the phases of the moon. the present study investigated pre-service science teachers' dmcs on moon phases and tracked changes in their dmcs during a clinical interview without instructional support. the findings demonstrated that individuals might rapidly change their conceptual understanding, consistent with previous research (parnafes, 2012; sherin et al., 2012). in this study, preservice science teachers adjusted and reframed their explanations during a clinical interview. they attempted to provide a more logical explanation of the phases of the moon after recognizing their shortcomings or researcher’s efforts to highlight inconsistencies. consequently, some suggestions can be made about how to teach moon phases effectively. for example, asking challenging questions and revealing inconsistencies in their responses may help improve their understanding of a specific topic. since researchers’ prompts triggered shifts in participants' conceptions, instructors or teachers can utilize the same technique to enhance their students’ understanding of lunar phases. furthermore, pre-service science teachers should have the opportunity to explore and ask questions regarding their current conceptual understanding, confronting deficiencies and incompatibility of their explanations. undergraduate astronomy courses and figure 11 aygün’s demonstration of phases of the moon journal of science learning article doi: 10.17509/jsl.v6i2.51686 191 j.sci.learn.2023.6(2).181-193 teaching activities should provide pre-service teachers the opportunity to improve their awareness of their understanding of moon phases. the current study showed that 10 out of 14 interviewees did not construct a dmc based on scientific knowledge. this finding is in line with previous studies (e.g., bayraktar, 2009; kanli, 2014; semercioğlu & kalkan, 2021; trumper, 2001; trundle et al., 2002) that indicate individuals harbor misconceptions about phases of the moon. nonetheless, explanations not consistent with normative scientific explanations were not considered misconceptions in this study. following the footstep of the knowledge in-pieces perspective, erroneous interpretations of pre-service science teachers were viewed as a problem of node organization and activation. accordingly, preservice science teachers should be provided with different contexts to activate their nodes to achieve a more sophisticated and stable understanding (disessa, 1993). in other words, pre-service science teachers should develop the skills to present the same phenomena in various contexts. for instance, ucar (2014) introduced phases of the earth context for pre-service teachers. participants attempted to explain how the earth appears when viewed from the moon in that study. this approach might be effective, as the same knowledge elements should be activated for explaining both earth's and the moon's phases. moreover, examining the phases of venus and mercury could be another effective method for fostering a sophisticated understanding. pre-service science teachers can predict the appearance of venus and mercury by considering the mechanism of moon phases. considering some of the nodes (e.g., reflections, angle) activated by pre-service science teachers, it was revealed that their dmcs were influenced not only by their daily life and astronomy courses they took but also by the physics courses they participated in. pre-service science teachers activated these nodes, although they were not directly linked to phases of the moon issues. this situation may suggest that pre-service science teachers do not have a sophisticated understanding of physics subjects, such as the nature of waves. since pre-service science teachers could not organize their mental constructs concerning these physics subjects, they tend to activate these unrelated knowledge elements when explaining the moon’s phases. pre-service science teachers' spatial abilities also shaped their explanations of the phases of the moon. the finding of this study was consistent with previous research (cole, wilhelm, & yang, 2015; plummer, 2014; wilhelm et al., 2013), demonstrating that explaining the moon's phases requires a certain level of spatial ability. furthermore, this study clarified why low spatial ability may hinder understanding of the phases of the moon. it was shown that pre-service science teachers activated unrelated nodes due to their low spatial abilities. for example, they could not visualize how the moon was illuminated and how it could be seen from a location on earth. as a result, they activated location, angle, and reflection dmcs to explain the phases of the moon. therefore, activities aimed at improving pre-service science teachers' spatial abilities should be designed. technology resources such as simulations and augmented reality may effectively enhance visualization of lunar phases. numerous studies on understanding on phases of the moon (e.g., hobson et al., 2010; trundle et al., 2002) have overlooked potential changes in pre-service science teachers' responses during the data collection process. in the current study, pre-service science teachers' understanding was examined through clinical interviews, and changes in their responses during the interviews were investigated. results indicated that pre-service science teachers might produce different explanations for different lunar phases. for instance, they explained full and new moon phases by activating shadow and distance nodes while activating angle, tilted, and location nodes for gibbous and crescent moons. it was revealed that determining activated nodes while considering different phases might be a more appropriate strategy for examining pre-service science teachers' understanding of the moon’s phases. additionally, instructors should design activities that demonstrate the occurrence of all phases, rather than presenting a general mechanism. 5. conclusion this paper explored pre-service science teachers' construction of an explanations for the phases of the moon concept in light of the knowledge in pieces perspective. data were obtained from clinical interviews conducted with fourteen pre-service science teachers. the results indicated that pre-service science teachers produced eight dynamic mental constructs using eleven knowledge elements. each dynamic mental construct contained between two and four knowledge elements. in other words, pre-service science teachers activated both related and unrelated nodes. low spatial abilities and shortcomings in some physics subjects, such as waves and reflection rules, were the chief culprit for activating of unrelated nodes. this study also demonstarted that pre-service science teachers' explanations changed while explaining different lunar phases, utilizing different media such as models, and responding to interviewers' prompted questions. however, although the explanation of the moon's phases changed, they could not construct a normative scientific explanation at the end of the clinical interviews. therefore, pre-service science teachers need different contexts regarding lunar phases to produce a more stable and sophisticated conceptual understanding of the phases of the moon. it is also interesting to note that some pre-service science teachers construct different explanations for different lunar phases. for example, shadow and distance nodes explained journal of science learning article doi: 10.17509/jsl.v6i2.51686 192 j.sci.learn.2023.6(2).181-193 full and new moon phases, and gibbous and crescent moons were explained by angle, tilted, and location nodes. references bailey, j. m., & slater, t. f. (2004). a review of astronomy education research. astronomy education review, 2(2), 20–45. baxter, j. (1989). children’s understanding of familiar astronomical events. international journal of science education, 11, 502–513. doi:10.1080/0950069890110503 bayraktar, ş. (2009). pre-service primary teachers’ ideas about lunar phases. journal of turkish science education, 6(2), 12–23. black, a. a. j. (2005). spatial ability and earth science conceptual understanding. journal of geoscience education, 53(4), 402–414. blown, e., & bryce, t. g. k. (2006). knowledge restructuring in the development of children’s cosmologies. international journal of science education, 28, 1411–1462. doi:1080/09500690600718062 blown, e., & bryce, t. g. k. (2010). conceptual coherence revealed in multi-modal representations of astronomy knowledge. international journal of science education, 32, 31–67. doi:10.1080/09500690902974207 cole, m., wilhelm, j., & yang, h. (2015). student moon observations and spatial-scientific reasoning. international journal of science education, 37, 1815–1833 doi:10.1080/09500693.2015.1052861 creswell, j. w. (2007). qualitative inquiry and research design: choosing among five approaches (2nd ed.). thousand oaks, ca: sage. de pree, c., & axelrod, a. (2001). the complete idiot’s guide to astronomy (2nd ed.). indianapolis: alpha. disessa, a. a. (1993). toward an epistemology of physics. cognition and instruction, 10(2&3), 105–225. disessa, a. a. (2007). an interactional analysis of clinical interviewing. cognition and instruction, 25, 523–565. doi:10.1080/07370000701632413 disessa, a. a., gillespie, n. m., & esterly, j. b. (2004). coherence versus fragmentation in the development of the concept of force. cognitive science, 28, 843–900. doi:10.1016/j.cogsci.2004.05.003 disessa, a. a., & sherin, b. l. (1998). what changes in conceptual change? international journal of science education, 20, 1155–1191. doi:10.1080/0950069980201002 dunlop, j. (2000). how children observe the universe. publications of the astronomical society of australia, 17, 194. doi:10.1071/as00194 hammer, d., scherr, r. e., & redish, e. f. (2005). resources, framing, and transfer. in j. mestre (ed.), transfer of learning from a modern multidisciplinary perspective (pp. 89–120). greenwich, ct: age publishing. hobson, s. m., trundle, k. c., & saçkes, m. (2010). using a planetarium software program to promote conceptual change with young children. journal of science education and technology, 19(2), 165–176. doi:10.1007/s10956-009-9189-8 kanli, u. (2014). a study on identifying the misconceptions of preservice and in-service teachers about basic astronomy concepts. eurasia journal of mathematics, science and technology education, 10, 471–479. doi:10.12973/eurasia.2014.1120a karttunen, h., kroger, p., oja, h., poutanen, m., & donner, k. j. (eds.). (2007). fundamental astronomy (5th ed.). heidelberg: springer. körhasan, n. d. (2021). knowledge elements used by pre-service primary teachers to explain free fall. journal of turkish science education, 18(4), 574–588. doi:10.36681/tused.2021.91 merriam, s. b. (2009). qualitative research: a guide to design and implementation. san francisco, ca: jossey-bass. minstrell, j. (1982). explaining the ’’at rest’’ condition of an object. the physics teacher, 20(10), 10–14. minstrell, j., & stimpson, v. (1996). a classroom environment for learning: guiding students’ reconstruction of understanding and reasoning. in l. schauble & r. glaser (eds.), innovations in learning: new environments for education. (pp. 175–202). mahwah, new jersey: lawrence erlbaum associates. nielsen, w., & hoban, g. (2015). designing a digital teaching resource to explain phases of the moon: a case study of preservice elementary teachers making a slowmation. journal of research in science teaching, 52(9), 1207–1233 doi:10.1002/tea.21242 parnafes, o. (2007). what does ‘fast’ mean? understanding the physical world through computational representations. the journal of the learning sciences, 16(3), 415–450. doi:10.1080/10508400701413443 parnafes, o. (2012). developing explanations and developing understanding: students explain the phases of the moon using visual representations. cognition and instruction, 30(4), 359–403. doi:10.1080/07370008.2012.716885 pasachoff, j. m., & percy, j. r. (2009). teaching and learning astronomy. new york: cambridge university press. plummer, j. d. (2014). spatial thinking as the dimension of progress in an astronomy learning progression. studies in science education, 50(1), 1–45. doi:10.1080/03057267.2013.869039 posner, g. j., & gertzog, w. a. (1982). the clinical interview and the measurement of conceptual change. science education, 66(2), 195–209. doi:10.1002/sce.3730660206 posner, g. j., strike, k. a., hewson, p. w., & gertzog, w. a. (1982). accommodation of a scientific conception: toward a theory of conceptual change. science education, 66(2), 211–227. doi:10.1002/sce.3730660207 semercioğlu, m. g., & kalkan, h. (2021). understanding of teachers on phases of the moon and the lunar eclipse. european journal of education studies, 8(2), 102–131. doi:10.46827/ejes.v8i2.3555 sherin, b. l. (2006). common sense clarified: the role of intuitive knowledge in physics problem solving. journal of research in science teaching, 43(6), 535–555. doi:.1002/tea.20136 sherin, b. l., krakowski, m., & lee, v. r. (2012). some assembly required: how scientific explanations are constructed during clinical interviews. journal of research in science teaching, 49, 166– 198. doi:10.1002/tea.20455 smith, j. p., disessa, a. a., & roschelle, j. (1993). misconceptions reconceived : a constructivist analysis of knowledge in transition. the journal of the learning science, 3(2), 115–163. doi:10.1207/s15327809jls0302_1 stahly, l. l., krockover, g. h., & shepardson, d. p. (1999). third grade students’ ideas about the lunar phases. journal of research in science teaching, 36(2), 159–177. subramaniam, k., & padalkar, s. (2009). visualisation and reasoning in explaining the phases of the moon. international journal of science education, 31(3), 395–417. doi:10.1080/09500690802595805 trumper, r. (2001). assessing students’ basic astronomy conceptions from junior high school through university. australian science teachers journal, 47(1), 21. trundle, k. c., atwood, r. k., & christopher, j. e. (2002). preservice elementary teachers' conceptions of moon phases before and after instruction. journal of research in science teaching, 39(7), 633–658. doi:10.1002/tea.10039 trundle, k. c., & bell, r. l. (2010). the use of a computer simulation to promote conceptual change: a quasiexperimental study. computers and education, 54, 1078–1088. doi:10.1016/j.compedu.2009.10.012 journal of science learning article doi: 10.17509/jsl.v6i2.51686 193 j.sci.learn.2023.6(2).181-193 ucar, s. (2014). the effect of simulation-based and model-based education on the transfer of teaching with regard to moon phases. journal of baltic science education, 13(3), 327–338. doi:10.33225/jbse/14.13.327 venville, g. j., louisell, r. d., & wilhelm, j. a. (2012). young children's knowledge about the moon: a complex dynamic system. research in science education, 42, 729–752. doi:10.1007/s11165-011-9220-y wellner, k. l. (1995). a correlational study of seven projective spatial structures with regard to the phases of the moon [unpublished doctoral dissertation]. university of iowa. wilhelm, j., jackson, c., sullivan, a., & wilhelm, r. (2013). examining differences between preteen groups spatialscientific understandings: a quasi-experimental study. journal of educational research, 106(5), 337–351. doi:10.1080/00220671.2012.753858 a © 2020 indonesian society for science educator 46 j.sci.learn.2020.3(2).46-56 received: 22 october 2019 revised: 25 december 2019 published: 4 march 2020 determination of the factors affecting high school students’ preferences of and satisfaction with choosing biology as an elective course nazihan ursavas1*, adem kesimal2 1recep tayyip erdoğan university, rize, turkey 2çayeli fen lisesi, meb, rize, turkey *corresponding author. nazihan.ursavas@erdogan.edu.tr abstract school subjects divided into compulsory and elective courses. compulsory courses are the basic courses that students must take to graduate, while elective courses give freedom for students. in turkey, biology courses taught as compulsory under the required courses in high schools. but, it is also instructed as elective courses called “advanced biology” in anatolian high schools and “biology applications” in science high schools at grades eleven and twelve. this new development calls for determining high school students’ reasons for choosing and not choosing biology. to elicit their reasons, the sample of the current study consisted of 135 volunteer students drawn from two different schools in northeastern turkey. an open-ended questionnaire used for data collection. the students’ responses to the survey subjected to content analysis. the factors affecting their choices of biology were high due to future-oriented issues and emotional reasons. the factors affecting students’ views of declining biology as an elective course were high due to future-oriented issues, psychological reasons, and external resources. they tended to change their decisions because of related-courses and exam-oriented reasons. future studies should be undertaken with larger samples to identify the role of gender differences in biology preferences. keywords biology, career choice, elective course, satisfaction 1. introduction distinctive personality, abilities, and interests of each individual are the starting point of education. for this reason, in schools, which constitute an essential part of the education process, it is necessary to put programs into practice that will reveal the talents of the individuals and form their future careers (ko, sammons & bakkum, 2016). school subjects divided into compulsory and elective courses. required courses are the basic courses that students must take, while elective courses give freedom for students. allowing students to choose the courses they want is an appropriate approach to the contemporary democratic understanding (genç & kalafat, 2007; movchan & zarishniak, 2017), as well as offering alternatives to help students develop positive attitudes towards school. elective courses allow students to train in multiple ways (demir & ok, 1996). students may be able to discover their hidden talents with the lessons they will choose by their interests, needs, and abilities, and they can use such knowledge and skills in elective courses to solve problems. although elective courses give individuals a democratic freedom in terms of discovering their interests and abilities, some researchers have found out that the increase in the number of elective courses leads to decreasing achievement scores in some international tests, so the hours of elective courses have reduced with the influence of families and educators (hedges, pacheco, & webber, 2014; pascual, 2014). when evaluated in this respect, not only the interests and needs of the students are taken into consideration in the course selection, but also external resources such as families and teachers can be useful in this process. besides, research has revealed that the selection of elective courses in schools depends on the schools’ adequate number of classrooms, a sufficient number of teachers and the required number of the students (at least 10) to offer the course. the application of elective courses may vary for countries. that is, a compulsory course in a country can teach as an elective course in another country. in turkey, mailto:nazihan.ursavas@erdogan.edu.tr journal of science learning article doi: 10.17509/jsl.v3i2.20714 47 j.sci.learn.2020.3(2).46-56 starting from the second stage of elementary school years, there is the possibility to choose courses in high school and undergraduate education (taş, 2004). the regulations related to elective courses in the turkish national education system carried out within the framework of the decisions of the board of education. its foreseen that the teachers' board should determine the elective courses to teach in schools by taking into consideration the conditions of the school and the environment, the interests, wishes and needs of the students and the opinions of the parents at the beginning of the academic year (dergisi, 2005). it decided to teach various sciencebased courses such as physics, chemistry, and biology in secondary education as an elective course starting from the 10th or 11th grade on according to the type of school. at the moment, biology is a compulsory school subject in the 9th and 10th grades in anatolian high schools for three hours a week; but it is renamed as advanced biology and put under the field of mathematics and science as a 3-hour elective course for the 11th and 12th grades. in science high schools, a three-hour biology course is compulsory for the 9th, 10th, 11th, and 12th grades, but another three-hour biology course is elective for the 10th, 11th, and 12th under the science applications for students who choose field of mathematics and science. few studies have conducted regarding elective courses (tuncer & özüt, 2017) at the undergraduate level. for example, program changes in recent years have attracted related research on primary education. factors influencing the choice of courses include gender, attitude, family, teacher, career choice and personal development mainly (ekici & hevedanlı, 2010; essary, 1998; khan & ali, 2012; osborne, simon, & collins, 2003; pascual, 2014; toews, 1998; tuncer & özüt, 2017; uçgun, 2012). early attempts to identify such influential factors are essential since they are the first steps of a long journey of creating a career (nagy, trautwein, baumert, köller, & garrett, 2006). in the secondary education period, students make choices to determine their future professions; therefore, the question of whether they will have professions that will allow them to be hired by an organization immediately (pascual, 2014). at the undergraduate and higher levels like mba or ph.d., as a profession has already decided, social factors are at the forefront while making course choices (uçgun, 2012). family factors and the profession of parents can be effective at secondary education course selection (pascual, 2014); these factors determined as the least effective factors among university students (tezcan & gümüş, 2008). one of the common factors that affect the choice of students at both secondary and university levels is a group of friends (pascual, 2014; uçgun, 2012). many factors can influence the course selection of students at the university level. some of these are students’ views about the faculty members, the method of the course, the instructor's grading system, the personal skills, the course and the academic factors focusing on the characteristics of the instructor, personal/contextual factors related to the student who chose the course, familiar instructors, being in the same class with friends, creating a program that fits the working hours, being flexible in absenteeism and being able to find the instructor in office hours (babad & tayeb, 2003; pass, mehta, mehta, 2012; tezcan & gümüş, 2008; uçgun, 2012). the factors affecting the course choices can be studied generally for a single course or a single section, and the results can use for the improvement of the course or section (toews, 1998). in a study conducted for physics lesson, it stated that there were internal and external factors such as difficulty and lack of necessity at the beginning of the students' not choosing physics course, and the reasons for choosing were plans for the future due to the need for physics courses in their future education and careers (toews, 1998). as for the students studying press and broadcasting, the factors influencing the course choices were found to be as criteria such as personal development, leisure time and graduation, and suggested that different departments should put forward these needs (yüksel, i̇spir & adıyaman, 2016). attitude, which argued to affect many areas, is one of the essential factors affecting course selection. negative attitude towards a course adversely affects the selection of the course (essary, 1998; rogers & ford, 1997). therefore, each course can be considered one by one to reveal the factors affecting course selection more effectively. the factors that affect the course selection vary according to the nature of the course (ekici & hevedanli, 2010; osborne, simon, & collins, 2003; özel, 2016; şeker, 2013; toews, 1998). therefore, it considered essential to find out the factors that affect the choice of each course individually. this study aimed to reveal the factors that affect the preference of biology as an elective course. because biology, despite being divided into subbranches to make its learning more accessible and more meaningful for students, is still seen one of the memorization-based and most painful lessons (bahar, 2003; çimer, 2004; maskiewicz, 2006; özatlı, 2006; öztap, özay & öztap, 2003; saygın, atiboz, & salman, 2006; tekkaya & balcı, 2003; tekkaya, özkan, & sungur, 2001). although there is a negative attitude towards biology in general, it revealed that positive attitudes towards sub-branches exhibited and it reported that the attitudes of girls towards biology are more favorable than boys (kurnaz & alev, 2009; tezcan & gümüş, 2008; uçgun, 2012).; nasr, soltani, and asghar, 2011) have journal of science learning article doi: 10.17509/jsl.v3i2.20714 48 j.sci.learn.2020.3(2).46-56 revealed that only students who find biology fun prove a relationship between biology knowledge and course achievement. the relevant literature in which the factors affecting the selection of the courses examined reveals that research focus more on undergraduate and graduate samples, especially in the national dimension (kurnaz & alev, 2009, uçgun, 2012; yüksel, i̇spir & adıyaman, 2016) and some of them, have recently observed in primary education level (tuncer & özüt, 2017). unfortunately, none of the studies at primary education level contain students. the number of studies investigating the factors affecting the selection of students at high school level is quite low, and even no turkish literature is available in this context while there is only a little english research at the undergraduate degree (nagy, trautwein, baumert, köller, & garrett, 2006). for this reason, this research expected to determine at first the factors affecting the course choices before the university entrance exam, which will guide students’ future. besides; the study can help guide the students properly according to the influential factors, help them make well-informed decisions on courses in the light of their personality and insufficiencies, and effectively use decision-making skills to solve their problems in everyday life through the 21st-century skills, which is one of the main aims of curricula. moreover, the study will light the way for the course teachers to see what criteria the students consider while improving themselves and informing parents about their children's decisions at the same time. also, this study thought to be important in terms of providing information about vocational training in science, technology, engineering, and mathematics (stem) branches and helping to determine effective strategies. the last but not least, it is expected to contribute to the literature studying the high school level, which has been overshadowed by the graduate and undergraduate levels. the fact that course selection stands as a new topic to discover after selection of the branch of study at high school has made it obligatory to conduct this research. in this study, it's aimed to find out reasons for high school students’ choosing or not choosing biology course as an elective course and whether they are content with their choices as a step which will affect their career choices directly and their lives indirectly. for this purpose, the answer sought for the following research questions. 1. what are the factors that affect eleventh and twelfthgrade students’ choices of biology courses as an elective course? 2. what are the factors that influence eleventh and twelfth-grade students’ not choosing biology as an elective course? 3. what are their reasons for changing their decisions after choosing biology as an elective course? 4. what are their reasons for changing their decisions after rejecting biology as an elective course? 2. method 2.1. study design this research is a descriptive case study, which aims to reveal eleventh and twelfth-grade students’ reasons for choosing or discarding biology as an elective course. in descriptive studies, it is generally necessary to clarify a given situation, to make evaluations following standards, and to describe and explain the situation thoroughly without any changes in the environment (cohen & manion, 1994). to make a general judgment about the universe composed of many numbers within the scope of this research, a case study carried out by describing the selected sample according to the world (creswell, 1998). 2.2. sample in choosing what case to study, there is an array of possibilities available for purposeful sampling (creswell, 1998). in this study, the sampling determined by using typical case sampling. according to this sampling method, if its desire to introduce a new application or an innovation, one or more most typical of such situations can study in particular (yıldırım & şimşek, 2008). this research held in a medium-sized city in the northeast of turkey. two schools selected from this province. these schools are anatolian high school and science high school. to be able to register at these schools, students have to be successful in the nationwide test applied at the end of the 8th-grade semester. successful students then choose a school according to their scores. those students were included in the study sample because they experienced in selecting a school and how to make a conscious choice. thus, it assumed that these students have at least a selection experience, they have criteria according to their preferences, they are critical of their choices with good and bad sides not only by themselves but also with people around them. the total number of students is 135. they are in the 11th and 12th grades. the sample included 98 (72%) students in the eleventh grade and 37 (28%) students in the twelfth grade. of the students in the 11th grade, 52 were girls (53.06%), and 46 were boys (46.9 %). of the students in the twelfth grade, 17 were girls (45.9%), and 20 were boys (54.05%). the average biology grade point of all students in the previous year was 62.13. 64 (47.4%) of the students preferred biology as an elective course, while 71 (52.5) did not take the electoral subject. the detailed distribution of the students shown in table 1. 2.3. data collection an open-ended questionnaire used to collect data. the questions prepared according to the opinions of two experts studying biology and science education. the final form gives as a result of the corrections obtained from the applications with a class excluded from the sample. in the journal of science learning article doi: 10.17509/jsl.v3i2.20714 49 j.sci.learn.2020.3(2).46-56 first part of the structure, students asked about their gender, class, and biology grades in the previous year. in the second part, the students were asked questions in two columns. both columns have four questions. in column one, the items included “what are the factors that convinced you to choose biology as an elective course?”, “are you glad to have chosen biology as an elective course? indicate your reasons.”, “i chose biology, but i would like to change my decision because….” and “i chose biology and i would not like to change my decision, because…” the second column included such items as “what are the factors that convinced you not to choose biology as an elective course?”, “are you glad not to have chosen biology as an elective course? indicate your reasons.”, “i did not choose biology, but i would like to change my decision because…”, and “i did not choose biology and i would not like to change my decision, because…” the questions made logically similar so that the respondents could choose their column easily, and data loss could prevent because of the students’ confusion on missing the meaning of the questions. students' names not asked because of repercussions. questionnaires numbered. 2.4. data analysis all answers of the students analyzed by the researchers with content analysis and descriptive analysis methods. the responses individually coded with words or phrases that reflect the generality of the descriptions. then those codes which serve the same purpose were grouped under common categories. each question in the form identified as themes such as choosing the course, not choosing the course, and changing their decisions. the following steps followed throughout the analysis procedures; 1. coding of the data gathered from students, 2. collection of the codes under common categories 3. establishing themes based on the codes, 4. arranging the codes and themes, 5. describing and interpreting the findings (yıldırım & şimşek, 2008). as some student responses corresponded to multiple themes, only the frequencies of the respective codes and themes provided in the result tables and percentages omitted. 2.5. validity and reliability studies the formula proposed by miles & huberman (1994) uses to calculate coding reliability for content analysis. according to this formula, the compliance of the codes elicited by the authors separately checked. coding reliability found to be 93. since this value was above 70, it concludes that the encoding was reliable. after the finalization of the codes, the categories were formed and then finalized by two experts in the field. the findings include frequencies and sample quotations of these codes. 3. result and discussion according to the data obtained, the results examined under three headings. these are the factors affecting students’ preferences of biology as an elective course, the factors that affect students' views of declining biology as an elective course, and students’ satisfaction with their preferences. 3.1. the factors affecting students’ choices of biology as an elective course sixty-four of the students chose biology as an elective course. the factors affecting the students' preference of biology as an elective course expressed with 16 codes, and these codes collected under six categories (table 2). according to this, the factors are (a) future-oriented issues, (b) course relevance, (c) emotional reasons, (d) obligation, (e) external sources, and (f) individual reasons. among these, “future-oriented-issues” and “emotional factors” are the categories with the closest frequencies. they followed by “obligation” and “course relevance,” whose rates are equal to each other. the least frequency categories are “external sources” and “individual reasons.” as seen in table 2, the most significant factor influencing the students’ biology course selection constituted by their plans. the sub-themes of futureoriented issues were career choice and program choice, and their respective frequencies were 24 and 16. the students stated that they needed to know biology to achieve their future career goals and that they selected biology because it was among the fundamental courses of their high school program. furthermore, the students rarely mentioned personal reasons, such as closing their knowledge gap and improving their knowledge base. table 1 distribution of students who chose and not choose biology as an elective course gender students, who choose biology students, who did not choose biology grade f % grade f % girl 11 18 28,12 11 34 47,88 12 10 15,62 12 7 9,85 boy 11 25 39,06 11 21 29,57 12 11 17,18 12 9 12,67 total 64 (%47,4) 100 71 (%52,5) 100 journal of science learning article doi: 10.17509/jsl.v3i2.20714 50 j.sci.learn.2020.3(2).46-56 another important reason for selecting biology was emotional reasons (37), which includes the codes of loving the course (17), interest (13), and curiosity (7). moreover, the least mentioned theme was external reasons (6) that cover teacher (3), teacher’s efforts (1), love of mathematics (1), and the broad range of career choices available to quantitative majors (1). the external reasons theme followed by the least prominent issue titled personal reasons (3) and comprised of closing the knowledge gap (2) and improving one’s knowledge base (1). 3.2. the factors that affect students' views of declining biology as an elective course another essential side of the study was the identifications of the reasons behind the students’ decisions for not choosing biology. the data regarding this branch of inquiry were also analyzed using the content analysis method, and the findings indicated similar as well as different themes to the previous section. of the 135 students participating in the study, 71 did not choose biology as an elective course. hence, the students’ reasons for not selecting biology fallen under five themes named: (a) future-oriented issues, (b) emotional reasons, (c) external sources, (d) failure, and (e) course relevance. as before, each theme consisted of multiple codes. a detailed table of frequencies for the themes and the codes provided in table 3. future-oriented issues, similar to the reasons for selecting biology, were the most prevalent theme (27) for not choosing biology. while career choice was a remarkable factor in choosing biology, its prevalence was somewhat lower as a reason for not selecting biology (9), where program choice (18) became a stronger factor. another factor leading the students to decide not to choose biology was emotional reasons (26), in which dislike of the course (13) was a significant reason. despite its relatively lower frequency, misguidance/lack of knowledge on availability (7) was a noteworthy reason for students’ not selecting biology. the code falls under the theme of external sources (21). it table 2 factors affecting students' choices of biology as an elective course categories codes f sample quotations total futureoriented-issues career choice 24 i chose it (biology) because it is more relevant form my career plans (s38). 40 program choice 16 i am in the quantitative-based program and biology was listed as a field course (s37). emotional reasons loving the course 17 because i liked biology. the half of it belongs to the teacher and the other half to me, but i like biology and, therefore, choose it (s 45). 37 to be interested 13 i like biology. the topics are of my interest, so i will keep taking it. (s58) curiosity 7 first of all, it is curiosity. especially, getting to know myself as a human being. my interest in and willingness to learn about the lives and biological structures of the other living organisms (s99). obligations to be obliged 12 in general, i had been willing to choose the quantitative-based program, but biology was of no interest to me. when i chose the quantitative-based program, biology was mandated to me (s103). 12 course related having interesting topics 8 i chose it because i am interested in the field of biology. the topics are interesting. i find the topics interesting especially when they are from daily life. (s98). 12 quantitativebased content 2 for being in quantitative-based section (s46). requiring memorization 1 for me, 70% of biology is a lesson based on memorization, and i chose biology lesson because i trusted myself (s101). having fun in class 1 i'm interested in biology; the lessons are fun (s33). external sources the teacher 3 i like the lesson and x teacher (s43). 6 the teacher’s effort 1 i love the biology teacher. also, the teacher is working hard for us. (s42) love of mathematics 1 i preferred biology as i was interested in mathematics (s126) broad range of career choices available to quantitative majors 1 because i had the widest course areas in the quantitative-based section. i didn't have an occupation goal, i thought more kind of lesson means wide choice of profession (s117). personal reasons closing the knowledge gap 2 because i am a quantitative-based section student and i have biology subject deficiencies (s60). 3 improving one’s knowledge base 1 i wanted to improve upon the things i learned in 9th and 10th grades (s125). journal of science learning article doi: 10.17509/jsl.v3i2.20714 51 j.sci.learn.2020.3(2).46-56 found out that the respondents were not informed about the new course selection system, so they did not know they could choose various courses, including biology. the students also stated that they did not choose biology because of failure (20) and course-related reasons (16). these two themes are the least frequent ones. it seen that the students did not choose biology because they thought that they would not be able to understand the course (11), could not succeed (6), and would not be able to pass the exams. besides, they mentioned some features inherent in the very nature of biology as reasons for not selecting, such as being a course of memorization (7), being quantitative (4), being complex (2), and being a course including foreign words (2). 3.3. students’ satisfaction with their preferences another aspect of the study examines whether the students feel content about their course selection decisions. in this respect, all of the students answered the question, “are you pleased with your decision to have\not to have chosen biology as an elective? please explain the reasons for your answer.” the findings from this line of inquiry provided in two different tables. among the 64 students who have chosen biology, 13 reported discontents about their decisions, and they table 3. factors affecting students’ not choosing biology as an elective course categories codes f sample quotations total futureoriented issues program choice 18 i did not choose it due to my program, i am a tm [turkishmathematics: i.e.; equally weighted program] student (s5). 27 career choice 9 i did not choose it since my future profession will be an equallyweighted major (s29). emotional reasons dislike of the course 13 [because] i don’t like the course (s24). 26 disinterest 9 i have no interest in biology (s9). preference for verbal courses 2 i like verbal courses more. i will not answer the 11th and 12thgrade biology questions in the higher education entrance exam (hee: ygs in turkish) (s75). finding it boring 1 biology was boring to me and i had difficulty of understanding the lesson. (s20). finding it unnecessary 1 i did not feel any urges to choose since i am in the equally-weighted program (s85). external sources teacher 7 because of the course teacher (s66). 21 misguidance/lack of knowledge on course availability 7 misguidance of the teacher (s82). i am a tm student and i did not know i could choose it (s95). not liking physics and chemistry 3 i did not choose because biology is a branch of science courses and i do not like physics and chemistry (s11). classroom 1 because of the teacher and the course’s classroom (s13). peer influence 1 i did not choose because my friend didn’t choose, either (s64). confidence in self-study skills 1 it was not related to the university major i would study. since hee biology questions require memorization, i can handle them by restudying (s21). lack of quantitative logic 1 my quantitative logic is not quite good (s86). failure learning difficulty 11 i have difficulty (a hard time?) in learning biology (s92). 20 failure to succeed 6 i am not quite successful in that course (s94). failure to pass the course 1 i had a hard time understanding, also had difficulty in passing the exams (s3). negative prior experiences. 1 i had a low score in the 9th grade (previous grade). (s69). lack of knowledge 1 i'm not knowledgeable in biology (s8). course related memorization 7 because it is based on memorization and there are many foreign words (s17). 16 being a quantitative-based content 4 biology is one of the quantitative-based course and the program i want is equally-weighted (s19). complexity 2 i think it is hard and complex (s86). foreign words 2 it requires a lot of memorization and there are lots of foreign words. i have difficulty in that course so i did not choose (s16). being a verbal content 1 i did not choose it because of its verbal emphasis and my difficulty in memorizing (s7). table 3 factors affecting students’ not choosing biology as an elective course categories codes f sample quotations total futureoriented issues program choice 18 i did not choose it due to my program, i am a tm [turkishmathematics: i.e.; equally weighted program] student (s5). 27 career choice 9 i did not choose it since my future profession will be an equallyweighted major (s29). emotional reasons dislike of the course 13 [because] i don’t like the course (s24). 26 disinterest 9 i have no interest in biology (s9). preference for verbal courses 2 i like verbal courses more. i will not answer the 11th and 12thgrade biology questions in the higher education entrance exam (hee: ygs in turkish) (s75). finding it boring 1 biology was boring to me and i had difficulty of understanding the lesson. (s20). finding it unnecessary 1 i did not feel any urges to choose since i am in the equally-weighted program (s85). external sources teacher 7 because of the course teacher (s66). 21 misguidance/lack of knowledge on course availability 7 misguidance of the teacher (s82). i am a tm student and i did not know i could choose it (s95). not liking physics and chemistry 3 i did not choose because biology is a branch of science courses and i do not like physics and chemistry (s11). classroom 1 because of the teacher and the course’s classroom (s13). peer influence 1 i did not choose because my friend didn’t choose, either (s64). confidence in self-study skills 1 it was not related to the university major i would study. since hee biology questions require memorization, i can handle them by restudying (s21). lack of quantitative logic 1 my quantitative logic is not quite good (s86). failure learning difficulty 11 i have difficulty (a hard time?) in learning biology (s92). 20 failure to succeed 6 i am not quite successful in that course (s94). failure to pass the course 1 i had a hard time understanding, also had difficulty in passing the exams (s3). negative prior experiences. 1 i had a low score in the 9th grade (previous grade). (s69). lack of knowledge 1 i'm not knowledgeable in biology (s8). course related memorization 7 because it is based on memorization and there are many foreign words (s17). 16 being a quantitative-based content 4 biology is one of the quantitative-based course and the program i want is equally-weighted (s19). complexity 2 i think it is hard and complex (s86). foreign words 2 it requires a lot of memorization and there are lots of foreign words. i have difficulty in that course so i did not choose (s16). being a verbal content 1 i did not choose it because of its verbal emphasis and my difficulty in memorizing (s7). journal of science learning article doi: 10.17509/jsl.v3i2.20714 52 j.sci.learn.2020.3(2).46-56 related this to the themes of course relevance, personal reasons, and career choice — the topics and codes about the reasons leading the students to change their decision given in table 4. the categories and codes the reasons for changing the preferences of the students detailed in the table below. students who prefer the course want to change their decisions because of several reasons. those reasons categorized into three categories. the most influential factor for the students who wish to improve their selections is memorization (4) code organized under the theme of course relevance (10). the students also stated other reasons for regret, such as finding it hard (3), finding it annoying (1), failure to understand its logic (1), and not regarding it as quantitative (1). the theme of personal reasons also influences the students’ desire to replace the course. under this theme, the students mentioned the feeling of not feeling of belonging (2), liking other courses more (1), realizing the lack of interest (1), and change of interest (1). as to the career choice theme, it is the least often mentioned theme in this student group. similarly, among the 71 students who have not chosen biology, 30 students expressed their feeling of discontent. they wish to choose biology as an elective course. the themes derived from their responses include exam orientation, emotional reasons, personal development, teacher-related, course relevance, and external sources. table 5 below showcases the themes and codes about the students’ reasons for discontent. for the students who have not chosen the course and did not feel content with that decision, exam orientation (14) is the most common reason for wishing for a change, and it followed by emotional reasons (11). some other reasons the student mentioned include finding biology easy to succeed if one studies (3), the belief that it contributes to common knowledge (1), and contributing to personal development (1). these reasons organized within the theme of personal growth. also, other issues, such as teacher-related, course-related, and external sources, were the least frequent ones with two occurrences for each indicated in table 5. 3.3 discussion students who attend high school education have to attend all courses in the 9th and 10th grades. starting from the 11th grade, they choose a section and continue to study with the courses required by this section. selecting this section is usually closely related to the profession students want to study at university. while these areas generally divided into quantitative-based, verbalbased, and equally-weighted (qualitative/quantitative-based), some of them may vary depending on the school’s conditions, including physical education, music, and english-based sections. each section teaches compulsory courses. besides, students can choose some basic subjects as elective courses. with the updated teaching programs, apart from social activitybased courses, some required courses like biology, chemistry, mathematics, and physics can be elected, too. for this reason, this study aimed to determine the extent to which students prefer biology course and what kind of factors are influential in their decision-making process. according to the findings of the study, 47.4% of the participants preferred biology as an elective course, while table 4 students’ reasons for changing the decisions for selecting biology as an elective course category codes f sample quotations total course related finding it memorization-based 4 it is a memorization-based lesson, i don’t think it is quantitative-based (s56). 10 finding the lesson hard 3 because it is hard (s49). finding it boring 1 it is full of memorization and boring (s51). (s51). failure to understand its logic 1 i find it hard and i have a hard time grasping its logic (s36). biology is not quantitativebased 1 i don’t think it is a quantitative-based lesson. (s56). personal reasons do not belonging here 2 i would like to change it because i don’t think i belong here (s99). 5 liking physics much more 1 instead of biology, i would choose physics as a quantitative course because i like it more than biology (s102). realizing the lack of interest 1 i wanted to pursue quantitative studies, and biology was a must-have course. but then i realized i do not like biology (s102). change of interest 1 i realized that i am more inclined to the equally-weighted program (s117). career choice career plans 1 i had no interest in biology. i considered transferring to the equallyweighted class for a while but due to my career choices, i remained in the quantitative program (s103)). 2 career plans in other fields 1 my future career plan is more closely related to the fields of mathematics and physics (s38). journal of science learning article doi: 10.17509/jsl.v3i2.20714 53 j.sci.learn.2020.3(2).46-56 52.5% did not. the percentage of female students who preferred the elective course is 43.74%, and the percentage of others is 57.73%. when considered for male students, the percentage of males who preferred the course is 56.24%; the others constitute 42.24%. when we look at the selection rates of the course, it can say that there is a balanced distribution both in terms of total students and genders. in other words, approximately half of the students participating in the study chose the course, and the other half did not. furthermore, it found out that male students prefer biology more than their female peers. however, a significance test not performed based on gender. although equality of opportunity in education in recent years has improved more than in previous years, it stated that gender still plays a role in the choice of some courses, socialization mechanisms, and rational preference motivations shown as reasons (gabay-egozi, shavit, & yaish, 2014). in the studies on gender preferences, nagy, trautwein, baumert, köller, & garrett (2006) revealed a meaningful difference, and female students mostly prefer biology, where gabay-egozi, shavit, & yaish, (2014) found out no significant difference in the context of a biology course. when examined in terms of attitude; studies are showing that there are no meaningful differences (pehlivan & köseoğlu, 2010; prokop, prokop, & tunnicliffe, 2007) despite studies which show that female students exhibit more positive attitudes towards biology than males (atik & erkoç, 2015; erkol & uğulu, 2013). although there is no statistical analysis in terms of gender within the scope of this study, there are many studies in which there are different situations by gender in the current literature. in the present study, which intended to identify the reasons that affect the students' course choices, it found out that the students prefer biology course mostly for future-oriented reasons (see table 2). this category also ranks first among reasons for not choosing the course, which covers field professions and professional choices of students. it thought to be cultural because medicine, engineering, and mathematics considered prestigious jobs in turkey, and more students guided to these areas for this reason. these areas also are seen as indicators of success. this situation is consistent with international studies. pascual (2014) stated in his research that the students reported course selection with the motivation of table 5 students’ reasons for changing their decisions themes codes f sample quotations total examorientation useful for hee 6 i am not pleased (because of my decision); because, i will need this course at the hee exam and that’s why i am not pleased (s78). 14 classes covering hee topics 2 i would like to choose it if hee topics will be covered (s85). answering extra questions at hee 2 it could have helped me answer a couple more questions in hee (s6). included in hee 1 i will need it in college entrance exams (s88). future usability 1 it may become useful in the future (s5). surpassing other students 1 because i need more science courses to surpass other students of the equal-weighted majors. therefore, i would have liked to take it (s78). feeling the necessity 1 i need that course (s82). emotional reasons loving the course 6 i am not [pleased]. i like biology, but i could not choose it (s91). 11 finding the course enjoyable 4 it was an enjoyable course (s69). being interested in 1 i’d like to change my decision because the topics for the 9th and 10th grades are different. maybe the 11th and 12th grade biology subjects would have attracted my interest (s94). individual development easy to succeed if one studies 3 i'm not satisfied because it is an doable course if one studies (s89). 5 contributing to common knowledge 1 i think biology lesson will contribute to my common knowledge (s62). personal development 1 i think biology is a perfect science for personal development and understanding life, living organisms, and myself better. (s135). teacherbased liking the teacher 1 i like the biology course and the teacher, i find it enjoyable as well. so, i would want to change my choice (s1). 2 changes in temperament 1 mr./ms. x has become nicer, and good-natured (s65). courserelated learning about living organisms 2 it would be nice to learn about living organisms as well (s23). 2 external sources misguidance 1 i didn't know i could. otherwise, of course, i would have selected the course (s83). 2 employment opportunities 1 there are more employment opportunities for quantitative majors (s22). journal of science learning article doi: 10.17509/jsl.v3i2.20714 54 j.sci.learn.2020.3(2).46-56 future employment potentials. besides, remember that the studies in the national literature are of the undergraduate and master's degree levels, it may not be surprising not to explore such a factor in the national research since they have already completed their professional choices. for this reason, professional choice, which includes plans for the future, can be the most crucial reason affecting secondary school students' motivation for course selections. emotional reasons, course-related reasons, external sources, and personal reasons (see table 2), which are the categories that affect the course selection of the students, showed results consistent with the literature. in his study with undergraduate students, uçgun (2012) found out reasons for students' course choices as the enjoyment of qualitative-based and quantitative-based courses and possessing talents for qualitative-based or quantitativebased courses. likewise, secondary education students emphasized the quantitative-based nature of the subject under the course-related category and love for the course in the emotional reasons category. as another category that affects the students' course choices found in this study, obligation/imperative not repeated in the literature. although this category has its place under the factors affecting decisions of biology and is seemingly the desired situation, it has negative implications. as noted in the category of future-oriented issues, it found out that some of the respondents chose the course as an investment in their future profession, some necessarily chose it as a part of the section of the study, and some others had to take the course — those who directly stated the obligation covered in this category. also, the frequency of 12 seems too important to ignore. in a study with undergraduate students, yüksel, i̇spir, & adıyaman (2016) defined one of the factors affecting the course choices of students as personal development. a similar category emerged in this study in which the category is under the factors that change students' minds after rejecting biology as a chosen subject (table 5). it can infer from this category that the students wanted to change their preferences expecting contribution to their general knowledge and personal development as a result. the course-related category is related to both groups of the factors influential on preferring and discarding biology as elective. under this category, students suggested that biology is both quantitative-based and memorizationbased. similarly, students who did not choose the course specify the same factors in higher frequencies as the reason why they did not choose. these two are different codes. however, the sum of the frequencies in both groups indicates that biology is seen memorization-based (f=8) and quantitative-based (f=6) by students. the former is one of the most often problems in the literature (ursavaş, 2014; özatlı, 2006; maskiewicz, 2006; bahar, 2003; öztap, özay, & öztap, 2003). that raises the question of whether biology lessons are taught by the approach that will fulfill the requirements of the program. one of the essential codes which belong to the external sources category is the teacher. although the factor of teacher did not have much effect on students' course choice, it is repeated seven times on the opposite side. that means that the teacher factor is essential for students to exhibit a negative attitude towards a lesson. in similar studies, it stated that the teaching staff was competent in the course selection (babad & tayeb, 2003; pass, mehta, mehta, 2012; tezcan & gümüş, 2008;). it is one of the categories that affect the students in their wish for changing their decision of not choosing the course as elective. çalik and cobern (2017) concluded in their intercultural studies that one of the authors who taught chemistry changed their students' views on chemistry courses and chemists positively. that may be due to the attributions to teachers in a society made up of emotional individuals. in spite of its low frequency in course selection, misguiding/not knowing of the possibility to choose under the category of external sources seems to be significant considering the only recent introduction of the selection procedure. that shows that problems faced in the gradual transition from the status of compulsory to the elective courses and implies that proper guidance has not ensured so far. pamuk and kiraz (2016) found that 53.4% of school principals do not guide students, while 33.3% do the opposite in this process. this process is essential for the future of the students. it is necessary to take into consideration how the students will be affected while making the changes and informing them in advance so that they can evaluate the process thoroughly. the information could spread by school administrators or responsible teachers as well as guidance teachers to reduce the workload of the stakeholders. in this process, it is also possible to enlighten parents so that they can help their children with a professional approach in the decisionmaking process. apart from all those categories and codes, there is another category that is expected to be in external sources but is not expressed by any of the students: family/parent factor, which is frequently mentioned in the literature and reported with varying effects in different studies. it is surprising that these students, who are not still of full age, point out that they are not influenced by their parents, even better to say that they do not count it as a factor for choosing courses. that may be the result of the situation mentioned in the previous paragraph. in other words, the fact that the procedure is recent yet, that even the students themselves are not informed by the authorities, implying that they do not even fully know the process, there may be parents who are unaware of this process and thus, the students may have given decisions by themselves. therefore, the need arises to provide journal of science learning article doi: 10.17509/jsl.v3i2.20714 55 j.sci.learn.2020.3(2).46-56 adequate information in the triangle of school-family and students since the current practice will affect the future preferences of the students). 4. conclusion in this study, the factors affecting students’ elective course choices examined under two headings as choosing and not choosing the course. similar categories found in both groups. in other words, one category may be the reason why the student wants biology as elective, while it hinders another one from choosing it. the reasons why students choose bilogi as their elective courses are future orientation, emotional reasons, obligations/obligations, course relevance, external sources, and personal reasons. meanwhile, the reasons why students do not choose biology as an elective course are future-oriented issues, emotional reasons, external sources, failure, and course relevance. the most popular category affecting students’ preferences for elective courses seems to be their plans. considering that the students are in the 11th and 12th grades, this category expected to be a factor as an essential step in opting for the section and future occupations at university. external sources seem more effective when discarding biology as an elective course compared to the opposite decision. the effect of the teacher as a code under the external sources category is also significant because the teacher seems to be more effective in refusing biology than choosing it as an elective course. this situation partly shows the effect of teachers on the choice of course and the success in the course eventually. for this reason, the results of teachers should take into consideration, and the interaction should build on this effect. one of the most relevant categories that affect students’ choices is obligation/imperative. it found out that students chose biology not for the sake of their interests but due to obligation. that implies that students ignore their interests in informing their future, which leads to bringing up unsuccessful individuals. to eliminate this situation, especially families should give importance to the individual interests of students. in this way, individuals who qualified in their respective fields of work, love their jobs, and appreciate what they do and what they can do better, and gradually the country can develop. although family influence is available in the relevant literature but not existing among our findings, the fact that the family factor not revealed in the course selection of the students in our study suggests that the families do not have enough knowledge about the process and thus, they are not active enough in this process. it must remember that elective courses have just introduced to secondary education, and parents may not know much about it. for this reason, parents can interview to explain this situation, and the main reasons for their absence in this process can reveal. since this process is a new practice, it can ensure that a vital stakeholder such as family becomes aware of this process as well as all the other stakeholders before making the course choices. for this purpose, school management can hold information meetings, and the stakeholders can warn to make the students choose their interests and needs at the forefront. in conclusion, this study thought to be important for determining the reasons affecting course selection of students, who start growing interest in academic and career prospects, to present the findings to their teachers, educational managers, guidance counselors, and families so that the stakeholders can guide students reasonably and effectively in this process. in future research, the effects of the factors can examine in larger samples, and the differences in the factors can be examined by gender. similar studies can carry out through different courses, or the main reasons affecting the selection of various classes can put forward. references atik, a. d., & erkoç. f. (2015). ortaöğretim dokuzuncu sınıf öğrencilerinin biyoloji bilimine ve dersine yönelik tutumlarının çeşitli değişkenler açısından incelenmesi. adıyaman üniversitesi journal of educational sciences, 5 (2), 211-244. babad, e., & tayeb, a. (2003). experimental analysis of students' course selection. british journal of educational psychology, 73(3), 373393. bahar, m. (2003). misconceptions in biology education and conceptual change strategies. educational sciences: theory & practice, 3(1), 55-64. çalik, m., & cobern, w. w. (2017). a cross-cultural study of ckcm efficacy in an undergraduate chemistry classroom. chemistry education research and practice, 18(4), 691-709. çimer, a. (2004). a study of turkish biology teachers’ and students’ views of effective teaching in schools and teacher education. unpublished doctorate dissertation, the university of nottingham school of education, nottingham uk. cohen, l., & manion, l. (1994). research methods in education (4th ed.). london: routledge. creswell, j. w. (1998). qualitative inquiry and research design: choosing among five traditions. london: sage publications. demir, a., & ok, a. (1996). orta doğu teknik üniversitesindeki öğretim üye ve öğrencilerinin seçmeli dersler hakkındaki görüşleri. hacettepe üniversitesi eğitim fakültesi dergisi, 12(12). ekici, g., & hevedanlı, m. (2010). lise öğrencilerinin biyoloji dersine yönelik tutumlarının farklı değişkenler açısından incelenmesi. türk fen eğitimi dergisi, 7(4), 97-109. erkol, s., &uğulu, i̇. (2013). ortaöğretim öğrencilerinin biyoloji ve çevreye yönelik tutumlarının karşılaştırılması. journal of research in education and teaching. 2 (3), 231-239. essary, d. r. (1998). critical incidents influencing students’ selection of elective science. (unpublished doctoral dissertation). texas, a&mcommerce. gabay-egozi, l., shavit, y., & yaish, m. (2014). gender differences in fields of study: the role of significant others and rational choice motivations. european sociological review, 31(3), 284-297. genç, s. z., & kalafat, t. (2007). öğretmen adaylarının demokratik tutumları i̇le problem. pamukkale üniversitesi eğitim fakültesi dergisi, 22(22), 10-22. hedges, m. r., pacheco, g. a., & webber, d. j. (2014). what determines students’ choices of elective modules? international review of economics education. 17, 39-54. journal of science learning article doi: 10.17509/jsl.v3i2.20714 56 j.sci.learn.2020.3(2).46-56 khan, g. n., & ali, a. (2012). higher secondary school students’ attitude toward chemistry. asian social sciences, 8(6), 165-169. ko, j., sammons, p., & bakkum, l. (2016). effective teaching. reading, berkshire education development trust. retrieved from https://www.educationdevelopmenttrust.com/educationdevelo pmenttrust/files/98/98ad6340-0ef6-4e1d-a541db6018afce7d.pdf on december 20, 2019. kurnaz, m. a., & alev, n. (2009). i̇lköğretim ve ortaöğretim lisansüstü öğrencilerinin ders seçimi yaklaşımları ve ilgili sorunları. türk fen eğitimi dergisi, 6 (3), 38-52. maskiewicz, a. l. (2006). rethinking biology instruction: the application of dnr-based instruction to the learning and teaching of biology (doctoral dissertation, uc san diego). miles, m. b., huberman, a. m., huberman, m. a., & huberman, m. (1994). qualitative data analysis: an expanded sourcebook. sage movchan, l., & zarishniak, i. (2017). the role of elective courses in students’ professional development: foreign experience. comparative professional pedagogy, 7(2), 20-26. nagy, g., trautwein, u., baumert, j., köller, o., & garrett, j. (2006). gender and course selection in upper secondary education: effects of academic self-concept and intrinsic value. educational research and evaluation, 12(4), 323-345. nasr, a. r., & soltani, a. k. (2011). attitude towards biology and its effects on student's achievement. international journal of biology, 3(4), 100. osborne, j., simon, s., & collins, s. (2003). attitudes towards science: a review of the literature and its implications. international journal of science education, 25(9), 1049-1079. özatlı, n. (2006). öğrencilerin biyoloji dersinde zor olarak algıladıkları konuların tespiti ve boşaltım sistemi konusundaki bilişsel yapılarının yeni teknikler ile ortaya konması. yayımlanmamış doktora tezi, balıkesir üniversitesi, balıkesir. özel, b. (2016). sınıf öğrencilerinin kimya dersine yönelik tutumlarının incelenmesi. yayımlanmamış yüksek lisans tezi. pamukkale üniversitesi, eğitim bilimleri enstitüsü, denizli. öztap, h., özay, e., & öztap, f. (2003). teaching cell division to secondary school students: an investigation of difficulties experienced by turkish teachers: case studies. journal of biological education, 38(1), 13-15. pamuk, e., & kiraz, z. (2016). ortaokullarda okutulan seçmeli ders uygulamalarında okul yöneticilerinin karşılaştıkları problemler. abant i̇zzet baysal üniveristesi eğitim fakültesi dergisi, 16 (3), 9771003. pascual, n. t. (2014). factors affecting high school students’ career preferences a basis for career planning program. international journal of sciences: basic and applied research. 16 (1), 1-14. pass, m. w., mehta, s. s.,& mehta g. b. (2012). course selection: student preferences for instructor practices. academy of educational leadership journal, 16 (1), 31-38. pehlivan, h., & köseoğlu, p. (2010). attitudes towards biology course and the academic self-concept of the students attending at ankara science high school. hacettepe üniversitesi eğitim fakültesi dergisi, 38, 225-235. prokop, p., prokop, m., & tunnicliffe, s. d. (2007) is biology boring? student attitudes toward biology. journal of biological education, 42(1), 36-39. rogers, w. d., & ford, r. (1997). factors that affect student attitude toward biology. bioscience, 23(2), 3-5. saygin, ö., atilboz, n. g., & salman, s. (2006). yapılandırmacı öğretim yaklaşımının biyoloji dersi konularını öğrenme başarısı üzerine etkisi canlılığın temel birimi hücre. gazi üniversitesi gazi eğitim fakültesi dergisi, 26(1), 51-64. şeker, m. (2013). i̇lköğretim 6, 7 ve 8. sınıflarda din kültürü ve ahlak bilgisi dersinde öğrenci başarısını etkileyen faktörler (tosya örneği). yayımlanmamış yüksek lisans tezi, hitit üniversitesi, sosyal bilimler enstitüsü, çorum. taş, b. s. (2004). i̇lköğretim 6, 7 ve 8. sınıflar seçmeli ders programlarının öğretmen ve öğrenci görüşleri doğrultusunda değerlendirilmesi. çukurova üniversitesi sosyal bilimler enstitüsü,(yayımlanmamış yüksek lisans tezi), adana. dergisi, t. (2005). milli eğitim bakanlığı öğretmenevleri, öğretmenevi ve akşam sanat okulları, öğretmen lokalleri ve sosyal tesisler yönetmeliği. tekkaya, c., & balcı, s. (2003). öğrencilerin fotosentez ve bitkilerde konularındaki kavram yanılgılarının saptanması. hacettepe üniversitesi eğitim fakültesi dergisi, 24, 101-107. tekkaya, c., özkan, ö., & sungur, s. (2001). biology concepts perceived as difficult by turkish high school students. hacettepe üniversitesi eğitim fakültesi dergisi, 21, 145-150. tezcan, h., & gümüş, y. (2008). üniversite öğrencilerinin seçmeli ders tercihlerine etki eden faktörlerin araştırılması/ the investigation of the factors affecting the choice of the elective courses of university students. gü gazi eğitim fakültesi dergisi, 28 (1), 1-17. toews, w. (1988). why take physics in high school̄why plan to teach physics?. the physics teacher, 26(7), 458-460. tuncer, m., & özüt, a. (2017). i̇lköğretim düzeyindeki seçmeli derslerin seçim kriterlerinin öğretmen ve okul yöneticisi görüşlerine göre değerlendirilmesi. journal of educational reflections, 1(1), 51-65. uçgun, d. (2012). türkçe öğretmeni adaylarının seçmeli ders tercihlerini etkileyen etmenler. international journal of human sciences, 9 (2), 1337-1349. ursavaş, n. (2014). egs (dnr) tabanlı öğretim yönergesi kullanılarak öğretmen adaylarının sahip oldukları biyolojik anlam şekilleri ve düşünme yollarının geliştirilmesi (doctoral dissertation, doktora tezi: karadeniz teknik üniversitesi, trabzon). yıldırım, a., & şimşek, h. (2008). sosyal bilimlerde nitel araştırma yöntemleri. ankara: seçkin yayıncılık. yüksel, e., i̇spir, n. b., & adıyaman, h. k. (2016). öğrencilerin ders seçimlerini etkileyen faktörler: basın ve yayın bölümü öğrencileri üzerine bir araştırma. e-kurgu. retriewed from on http://ekurgu.anadolu.edu.tr/assets/upload/pdf/ 20160531142048_ekurgu.pdf, april, 2019. https://www.educationdevelopmenttrust.com/educationdevelopmenttrust/files/98/98ad6340-0ef6-4e1d-a541-db6018afce7d.pdf https://www.educationdevelopmenttrust.com/educationdevelopmenttrust/files/98/98ad6340-0ef6-4e1d-a541-db6018afce7d.pdf https://www.educationdevelopmenttrust.com/educationdevelopmenttrust/files/98/98ad6340-0ef6-4e1d-a541-db6018afce7d.pdf http://ekurgu.anadolu.edu.tr/assets/upload/pdf/%2020160531142048_ekurgu.pdf http://ekurgu.anadolu.edu.tr/assets/upload/pdf/%2020160531142048_ekurgu.pdf a © 2023 indonesian society for science educator 34 j.sci.learn.2023.6(1).34-47 received: 23 september 2022 revised: 19 january 2023 published: 20 march 2023 a bibliometric analysis of biodiversity education mustafa derman1* 1department of secondary science and mathematics education, kâzım karabekir education faculty, atatürk university, erzurum , turkey *corresponding author: dermanmustafa155@gmail.com abstract the importance of biodiversity is one of the most critical issues today, and intense efforts are being made to protect and maintain biodiversity. in this study, biodiversity, which is related to many disciplines, was evaluated in terms of education. a bibliometric analysis was used. the bibliometric method is widely used to reveal the relationship between scientific studies, the effect of the studies, and the effect of the researchers and journals in a particular field. research findings showed that many articles have been published on biodiversity recently. in education, out-of-school learning activities are used more widely today. moreover, the usa, china, and germany are among the countries that publish more articles about biodiversity. based on the research findings, it was determined that combining the formal education process with out-of-school activities will help students exhibit positive behavior about the environment and biodiversity. considering the trend in recent years, studies related to climate change, ecosystem service, sustainability, and citizen science can be carried out. keywords biodiversity, citizen science, nature, outdoor learning 1. introduction biodiversity refers to the diversity of life on earth. millions of creatures live in our world. every living thing has an important function in the ecosystem. the concept of biodiversity started to be widely used after the rio earth summit. at the conference in 1992, the definition of the concept of biodiversity was made. biodiversity education, which is tried to be a part of environmental education, is of an essential effect on a sustainable future (barker & elliott, 2000). environmental education in schools, especially in childhood education, tries to raise awareness and acquire positive attitudes toward the environment (barraza, 2001). therefore, educators have made much effort to integrate environmental and biodiversity education into all levels of education (collins-figueroa, 2012). apart from formal education, the importance of biodiversity is brought to people through informal education; tv programs and civil actions are the best examples of this (slingsby, 2009). because biodiversity is the responsibility not only of educators and scientists, all should question the reasons for the extinction of biodiversity and its threats (dreyfus, wals, & van weelie, 1999). however, people must first be aware of their environment and nature's biodiversity. because the formation of awareness includes having sufficient knowledge on this subject, educators are constantly investigating the awareness and knowledge levels of students, teachers, and other members of society by conducting research in this field (luvison araújo & dos santos alitto, 2021). one of the places that should be recognized as a priority is the immediate environment. individuals need information about the living things in the aquatic and terrestrial ecosystems where they live. with outdoor learning activities, students can interact in these environments (bermudez, pérez-mesa, & ottogalli, 2022); biodiversity is not only the dimension of recognition and awareness. we can see aspects of biodiversity in every part of our lives. it is a concept associated with health, food, cosmetics, engineering, economics, and many other disciplines. therefore, the importance of biodiversity should be well understood in all dimensions. any negativity that may occur in biodiversity, an essential ecosystem component, will negatively affect people in some way (bilali, dambo, nanema, bassole, & calabrese, 2022). in other words, human life will become more difficult due to the extinction of living things that exist in the ecosystem or are endangered species (xu, xiao, li, & wang, 2022). the sustainability of ecosystems depends on biodiversity. conservation of biodiversity is essential for ecosystems to journal of science learning article doi: 10.17509/jsl.v6i1.50831 35 j.sci.learn.2023.6(1).34-47 maintain their dynamic structure. for example, an insect's extinction affects pollen transport to plants, and in this case, it may reduce fruit yield (van oudenhoven & de groot, 2012). the protection of biodiversity significantly impacts people's survival and adequate food supply. therefore, taking action against biodiversity loss is mandatory to leave a sustainable future for future generations (grant rd, 2007). one of the essential natural resources is biodiversity. people consume natural resources (or biodiversity) as raw materials. some researchers express raw material as productivity or ecosystem service. the primary meaning here is nature's benefit to humans (mace et al., 2012; worm & duffy, 2003). different disciplines have carried out research to reach raw materials. therefore, biodiversity can be related to many disciplines (nyaupane & poudel, 2011; worm & duffy, 2003). health, tourism, food, and clothing industry are among the benefits of biodiversity to people (nyaupane & poudel, 2011). in other words, biodiversity also significantly contributes to the world economy and the continuity of healthy ecosystems. benefits of biodiversity; biological control agents used instead of pesticides and insecticides, recycling of wastes and organic matters, soil formation, pharmaceutical raw materials, bioremediation, cleaning of air from carbon dioxide to reduce global warming, genetic source to increase yields, pollination, and so on. (adom, umachandran, ziarati, sawicka, & sekyere, 2019; pimentel et al., 1997). situations that affect biodiversity also directly affect productivity (worm & duffy, 2003). therefore, biodiversity conservation and sustainable use are of great importance to humanity. biodiversity conservation has many dimensions, including social, political, and education (grace, 2009; helldén & helldén, 1970; menzel & bögeholz, 2009). education is one of the most effective ways to protect biodiversity (yen et al., 2007). because individuals who are aware of the importance of biodiversity, concerned and sensitive about biodiversity loss will ensure that biodiversity reaches future generations (montgomery, 2002). therefore, biodiversity is one of the most critical issues of education for sustainable development (esd) (grace, 2009; helldén & helldén, 1970; menzel & bögeholz, 2009; van weelie & wals, 2002). the primary way to build a sustainable future (or sustainable development) is to ensure the maintenance of biodiversity. for biodiversity education to reach its goal and have a sustainable future, it is necessary to increase the whole society's awareness about the extinction of biodiversity. for this reason, local and global projects related to biodiversity education have been carried out in all countries (gayford, 2000). when people do not have enough information or knowledge about biodiversity, they will not understand the importance of biodiversity in maintaining healthy ecosystems (hanley, spash, & walker, 1995). therefore, to achieve the goal of sustainable development, it is necessary to improve citizens' knowledge and skills on biodiversity in the formal education process (grace, 2009). esd seeks to raise individuals' analysis skills, provide active participation of individuals in protecting biodiversity, produce solutions, and involve people in projects about reducing biodiversity loss (barraza, 2001). unfortunately, the main problem in the extinction of many species in nature, the destruction of ecosystems, and the destruction of habitats of living things is the unconscious destruction of nature by anthropogenic impacts. it causes the irreversibility extinction of species in nature (díaz, fargione, chapin, & tilman, 2006; gayford, 2000). the researchers pointed out that individuals who know about biodiversity and understand the importance of living things in nature exhibit more sensitive behaviors in protecting the natural environment. researchers generally emphasize that education has an essential effect on protecting biodiversity (aivelo & huovelin, 2020; beery & jørgensen, 2018; gayford, 2000; hardy & hardy, 2018; helldén & helldén, 1970; kim, 2019; schneiderhan-opel & bogner, 2020; schultz & joordens, 2014; tuparevska, 2022; unger, rollins, tietz, & dumais, 2021; wyner & doherty, 2021; zhang, stevenson, & martin, 2022). educators aim to improve students' knowledge and skills about biodiversity by making curriculum revisions (grace, 2009). it is not only limited to primary schools, but also by integrating biodiversity in university education programs, it is ensured that individuals at all levels of education receive biodiversity education. because pre-service teachers' having sufficient knowledge and skills on biodiversity will contribute to the development of students' biodiversity values (lindemann-matthies et al., 2009). in this way, teaching strategies that increase the interaction of students with the natural environment have developed. helldén and helldén (1970) stated that individuals who interact with their natural environment from early childhood exhibit more sensitive behaviors in terms of the protection of biodiversity. some researchers have also emphasized the necessity of designing school schoolyards in accordance with nature. it was expressed that the schoolyards designed schools that are compatible with the natural environment help the students develop their environmental values (zhang et al., 2022). the mentioned literature explains the importance of biodiversity for humanity and sustainable ecosystems. however, biodiversity has decreased over the years, and many species have been extinct. the increasing human population and over-consuming natural resources cause species extinction at a higher rate. (pimentel et al., 1997). due to the rapid decrease and extinction of biodiversity, it will be challenging to replace the resources provided by biodiversity. this situation will cause significant damage to both the deterioration of ecosystems and the world economy (adom et al., 2019). there are approximately 9 million living things on earth. humans use many raw materials from biological sources. however, the vast majority of biodiversity loss on earth has been affected journal of science learning article doi: 10.17509/jsl.v6i1.50831 36 j.sci.learn.2023.6(1).34-47 directly or indirectly by human activities. it was declared at the earth summit in rio de janeiro in 1992 (cardinale et al., 2012). therefore, it brought the protection of biodiversity among the most critical problems in the world. considering the existing literature, it is seen that education has an important role in protecting biodiversity and building a sustainable future for the next generation. the growing of individuals who are aware of the protection of natural resources, active participation in preventing the loss of biodiversity, and living as a part of the ecosystem is achieved through education. for this reason, improving students' knowledge and skills about biodiversity through education is one of the essential tools for sustainable development and conservation of biodiversity. since biodiversity has many benefits for the ecosystem and humanity, much research has been carried out in this area. previous review studies have evaluated the benefits and impacts of biodiversity in many ways; biodiversity and its importance for human health (houlden, jani, & hong, 2021), relationship between biodiversity and ecosystem service (mace et al., 2012; wilson, 2013; worm & duffy, 2003), the effects of species extinction on the future of humans (adom et al., 2019; alho, 2012; cardinale et al., 2012; díaz et al., 2006), monitoring marine ecosystems (di ciaccio & troisi, 2021), pastoralism and biodiversity (bilali et al., 2022), relationship between the economic benefit of biodiversity and biodiversity loss (knapp, 2019; pimentel et al., 1997), the role of zoos in improving citizens' awareness about biodiversity (wheater, 1995; whitehead, 1995), the link between biodiversity and ecotourism (nyaupane & poudel, 2011), importance of sacred natural sites in terms of biodiversity conservation (zannini et al., 2021), endangered plants (xu et al., 2022), trends in landscape change (hernández, echeverría, & nelson, 2021), importance of biodiversity offsetting (yu, cui, xie, man, & fu, 2022), and effect of biodiversity on crop sustainability (grant, 2007). as a result of the analysis related to the review studies, there has been an increase in the subjects of bibliometric analysis, environment, environmental change, and human health. as a result, review studies have increased in recent years. furthermore, bibliometric analysis studies were primarily published concerning ecosystem service, sustainability, nature-based solutions, remote sensing, protected areas, and biotechnology. however, unlike previous studies, the educational aspect of biodiversity still needs to be evaluated. therefore, considering the importance of education in biodiversity conservation, it is crucial to reveal the relationship between biodiversity and education and the trends in biodiversity education. this aim was achieved with studies carried out between 1993-2022. finally, the place of biodiversity in the field of education was examined. 2. method to review scientific studies, the bibliometric method was used. the bibliometric method is widely used to reveal the relationship between scientific studies, the effect of the studies, and the effect of the researchers and journals in a particular field. with this method, data can be visualized by bibliometric mapping and heat mapping techniques. today, many software programs, such as citespace ii (chen, 2006), network workbench (nwb) tool (börner et al., 2010), and vosviewer (van eck & waltman, 2010), have been used for bibliometric maps. vosviewer (www.vosviewer.com) was used in this study. vosviewer is an open-access program used to make large bibliometric maps. bibliometric maps of authors (or journals) can be constructed based on citations. in addition, researchers can construct maps of keywords in their studies using cooccurrence data (van eck & waltman, 2010). the web of science (wos) is one of the most comprehensive database sources. it is used by many researchers to find information and to review special topics in their fields (xu et al., 2022; yan & liu, 2021). within the scope of the research, two types of searches have been made. first, “biodiversity education” was chosen as a keyword for the wos search. all periods and document types were included, and 84 results were reached. the results related to this section were analyzed under the heading "biodiversity education." in line with the results obtained, web of science categories associated with biodiversity education was determined. these categories were chosen for use in the second part of the research (figure 3c). second, concerning the biodiversity literature, two words were searched in the following terms; "biodiversity" and "education," and 23,597 results were found. after that, step-by-step limitations were refined: document types (article: 21,236), languages (english: 20,946), web of science index (sciexpanded, ssci, and esci: 20,700), and web of science categories (12,727) respectively. the data were downloaded in tab delimited file format. later, these files were combined in the excel file and saved as a .txt file. the data saved in excel format and pure science publications were excluded, and 1561 studies related to education and social science were included. the data were saved again in excel and .txt format. analyses were made after the data were uploaded into the vosviewer. 3. findings the oldest studies on biodiversity education were published in 1995, with two authors (wheater and whitehead) publishing studies in the same journal (biodiversity and conservation). the number of publications published between 1995 and 2022 shows a spike in three dates. after 2019, there has been an increase in studies. however, no publications were published on some dates (1996, 2002, 2007, and 2012) (figure 1a). most publications in the field of biodiversity education were http://www.vosviewer.com/ journal of science learning article doi: 10.17509/jsl.v6i1.50831 37 j.sci.learn.2023.6(1).34-47 published in the areas of education, education research, environmental sciences, environmental studies, and education scientific disciplines (figure 1c). however, in biodiversity and education (figure 1b), the oldest article in this section was published in 1993. a likert-type scale was used in the research, and it assessed opinions about kosi tappu wildlife reserve (heinen, 1993). only some studies were published between 1993 and 2007. however, as of 2008, there has been an increase in publications in this field. in other words, it is seen that there has been an increase in the publications published in recent years. 3.1 biodiversity education figure 2 shows studies from different parts of the world. it consists of 8 clusters, each shown in a different color. while at least four countries represent the other clusters, only cluster 8 (brown) includes two countries. the most significant contribution in this field is the studies carried out in germany (documents: 23, citations: 318, total link strength: 23), the usa (documents: 20, figure 1 a summary of studies on biodiversity education: (a) and (b) distribution of the number of publications in “biodiversity education” and “biodiversity” and “education”, respectively; (c) shows subject areas of biodiversity education. figure 2 visualization of co-authorship worldwide journal of science learning article doi: 10.17509/jsl.v6i1.50831 38 j.sci.learn.2023.6(1).34-47 citations: 732, total link strength: 27), and england (documents: 11, citations: 677, total link strength: 22). although germany had many publications, the usa is the country that cooperated in the highest number of publications with other countries worldwide. the total number of keywords used in publications was 302. the minimum number of occurrences of a keyword was set as 2. therefore, 34 words met this criterion. in other words, 34 words were considered in the bibliometric mapping. the most common keywords used were biodiversity education (occurrences: 26, total link strength: 35), education for sustainable development (occurrences: 10, total link strength: 20), and biodiversity (occurrences: 12, total link strength: 10). they are shown with larger circles as seen in figure 3. keywords shown in yellow indicate topics that have been studied in recent years (biodiversity learning, citizen science, sustainability education, group discussions, place-based education, higher education, and native species) (figure 3). 3.2 biodiversity and education co-occurrence and author keywords were chosen to determine the numbers and densities of the keywords used by the researchers in publications. as seen in the figure, clusters shown in different colors were obtained. each color represents a cluster. as a result of the determined criteria, seven clusters were obtained. of the 5505 keywords, 53 keywords meet the threshold. biodiversity, conservation, ecosystem service, climate change, environmental education, sustainability, protected areas, biodiversity conversation, education, and agriculture are the most used keywords. considering the concepts to which biodiversity is related, it is seen to be linked to all clusters (figure 4 and table 1). the figure indicates that biodiversity interacts with many disciplines. in other words, there are publications on biodiversity in different fields. biodiversity does not only consist of genetic, species, and ecosystem diversity. biodiversity has many dimensions: education, sustainability, climate change, species diversity, land use change, agriculture, ecotourism, conversation, ecology, and so on. studies in the field of biodiversity and climate change are among the issues of recent years. citizen science (cs) in cluster 2 has recently appeared more widely in biodiversity publications. figure 3 shows that it is prominently involved in studies published in 2019. one of the main aims of biodiversity education is to raise individuals' sensitivity to the environment. moreover, it is expected to take an active role in environmental issues. one of the most effective ways depends on individuals' direct interaction with the natural environment. the figure 3 the top keywords used in publications by years table 1 the ten most-used keywords in biodiversity education keyword* occurrences % of 5505 total link strength 1 biodiversity 223 4.05 176 2 conservation 96 1.74 80 3 ecosystem service 78 1.42 67 4 climate change 67 1.22 65 5 environmental education 57 1.04 57 6 sustainability 51 0.93 51 7 protected areas 52 0.94 49 8 biodiversity conversation 53 0.96 44 9 education 32 0.58 33 10 agriculture 23 0.42 28 *the first ten words were included. journal of science learning article doi: 10.17509/jsl.v6i1.50831 39 j.sci.learn.2023.6(1).34-47 research indicated that students who spend time in their daily life observing and collecting data about the environment show more sensitive behavior towards the environment (kim, 2019). a positive correlation was found between childhood nature activities and environmentally sensitive citizen behaviors (hoover, 2021). individuals who interact with the natural environment, even for a brief time, exhibit positive attitudes toward the environment. as a matter of fact, with camp activities, which are a part of informal education, individuals can live in the natural environment and explore nature (samperiz & herrero, 2018). supporting environmental education with informal education, which is a suitable area for field trips and outdoor learning activities, helps to achieve the objectives of environmental education (jose, patrick, & moseley, 2017). with outdoor learning activities, individuals gain more than expected in their experience and learning. individuals explore their environment and experience the methods used by scientists while collecting data in nature (jesus-leibovitz, faria, baioa, & borges, 2017). therefore, environmental education studies have begun to give more space to outdoor activities in recent years. activities can be in the form of outdoor learning, citizen science, field trip, or camping. biodiversity is related to many fields. it was stated in the studies that these concepts affect each other and have a solid link to each other (pimentel et al., 1997). each area's contribution ensures biodiversity's sustainable existence (worm & duffy, 2003). the importance of biodiversity has been emphasized in agriculture, ecosystem service, and protected area due to the effect of biodiversity on crop yield, biological control (e.g., the use of biological agents instead of chemical agents), and the development of more resistant species. furthermore, humans derive most of their food supply from plants and animals. economically, most of the world's raw material resources depend on raw materials derived from biodiversity (grant, 2007; knapp, 2019; mace et al., 2012; peter, diekötter, & kremer, 2019; pimentel et al., 1997; wilson, 2013; worm & duffy, 2003; yan & liu, 2021). another keyword is climate change. one of the most critical problems affecting ecosystems is climate change. depending on temperature changes (or global warming), the dynamic of ecosystems is negatively affected (e.g., habitat destruction, species migration, or extinction). therefore, this situation directly affects the diversity of living things in nature (lovejoy, 2008). one of the most critical problems in the world is the loss of biodiversity. for this reason, education and biodiversity conservation link to biodiversity. biodiversity protection can be achieved with a society that is aware of using nature sustainably. the formal and informal education processes try to increase society's awareness of biodiversity (krombaß & harms, 2008; figure 4 analyzing keywords in terms of clusters journal of science learning article doi: 10.17509/jsl.v6i1.50831 40 j.sci.learn.2023.6(1).34-47 lindemann-matthies et al., 2009; samperiz & herrero, 2018; tuparevska, 2022; van weelie & wals, 2002). another important keyword is sustainability. it is pointed out that society and ecosystems are seriously affected due to the loss of biodiversity. therefore, the protection of biodiversity, identification of biological resources, determination of situations that affect biodiversity, and sustainable use of natural resources were considered among the most important objectives of education for sustainable development (esd) (collins-figueroa, 2012). some of the activities that researchers have used regarding out-of-school activities in recent years are given in table 2. studies on the interaction of students with the natural environment are used. in addition, technological developments are also used. integrating outdoor learning and technology aims to increase students' awareness and sensitivity to biodiversity. educational scientists point out that the time spent in nature is decreasing more than in previous years. individuals are estranged from the natural environment. it was emphasized that more interaction with nature would cause us to become aware of the reasons that threaten nature and the ecosystem (tuparevska, 2022). positive perceptions of students develop in environments with sufficient facilities in terms of schoolyards. having natural environment opportunities in a school garden positively reflects individuals' attitudes (zhang et al., 2022). it was stated that interacting with nature has many benefits regarding environmental attitude, behavior, awareness, sensitivity, and learning. including out-of-school activities in biodiversity, education allows getting to know nature closely. therefore, the studies published in recent years have led to the intense involvement of out-of-school activities (beery & jørgensen, 2018; fančovičová & prokop, 2018; gormley, birdsall & france, 2022; guilherme, faria, & boaventura., 2016; hardy & hardy, 2018; jesus-leibovitz et al., 2017; kamudu, rollnick, & nyamupangedengu, 2022; krombaß & harms, 2008; schultz & joordens, 2014; sukhontapatipak & srikosamatara, 2012; wyner & doherty, 2021; zhang et al., 2022). although there are no experimental studies that include all of the activities given in table 2 and compare them with each other, researchers highlighted that the activities engaging the active participation of students (e.g., hands-on, outdoor learning, citizen science, field trip, and camping in nature) have a positive effect on students' success, awareness, attitude, and behavior (beery & jørgensen, 2018; bögeholz, 2006; bogner, 2018; braun & dierkes, 2017; collado, corraliza, staats, & ruíz, 2015; jansson, gunnarsson, mårtensson, & andersson, 2014; kassas, 2002; kelemen-finan, scheuch, & winter, 2018; figure 5 the visualization of co-authorship worldwide table 2 activities for biodiversity education publication activity participants 1 plant biodiversity (doup, 2018) outdoor learning students 2 biodiversity literacy (schneiderhan-opel & bogner, 2020) citizen science in class students 3 inaturalist (unger et al., 2021) using mobile app undergraduate students 4 specie variation (gormley, birdsall, france, et al., 2022) hands-on activity 5-11 years olds 5 sustainable development (rodríguez-loinaz et al., 2022) citizen science and ict tool teachers and students journal of science learning article doi: 10.17509/jsl.v6i1.50831 41 j.sci.learn.2023.6(1).34-47 kossack & bogner, 2012; ma & nickerson, 2006; navarroperez & tidball, 2012; prokop, prokop, & tunnicliffe, 2007; randler & hulde, 2007). in the present study, which included 133 countries, mapping was made by setting at least ten publications. regarding cooperation between countries, 6 clusters were obtained (figure 5 and table 3). the figure indicates that there is more cooperation between countries where they share the same geography. in addition, countries in asia, europe, and america are in cooperation with each other. many countries represent countries in these clusters (green, red, and blue). the usa, germany, and china are the most productive countries in terms of biodiversity. the usa, germany, and england are the countries most closely cooperating with other countries. more than half of the articles published in this field belong to the usa, germany, and china. in similar studies (ardoin & bowers, 2020; deng, liang, li, & wang, 2021; haghani, bliemer, goerlandt, & li, 2020), the usa and china rank first among the countries with the most publications. furthermore, depending on the development of science worldwide, there has been a significant increase in the number of publications. primarily due to interdisciplinary studies, fields of science now contribute to each other with common study subjects (haghani et al., 2020; moosa & shareefa, 2020). in recent years, biological conservation (documents: 236, citation: 9883, total link strength: 137) and sustainability (documents: 282, citation: 1984, total link strength: 118) have come first among the preferred sources of biodiversity. although the articles were published in journals journal of biological education (documents: 40, citation: 285, total link strength: 55), environmental conservation (documents: 42, citation: 888, total link strength: 35), and the australian journal of the environment in previous years, as of 2018, the trend in the sources has started to be towards education science, environmental development, and sustainability, journal of sustainable tourism, and international journal of environment. the journals with the highest publications are bc, s, and aee. jbe is the journal with the highest number of publications for educational publications (rise, ijse, and eer) (figure 6 and table 4). 4. discussion to summarize the topics where biodiversity is related, the research findings were evaluated under four main table 3 co-authorship worldwide in terms of documents, citations, and total link strength country* documents % of 1561 citations total link strength 1 usa 324 20.76 9595 402 2 germany 280 17.94 7260 349 3 england 141 9.03 3580 288 4 australia 101 6.47 2799 182 5 china 223 14.29 2269 164 6 france 55 3.52 2417 141 7 italy 64 4.10 924 134 8 spain 101 6.47 2734 133 9 canada 60 3.84 1412 132 10 netherlands 61 3.91 1676 132 *the first ten countries were included. table 4 the most used sources in terms of documents, citations, and total link strength source* documents % of 1561 citations total link strength 1 bc 236 15.12 9883 137 2 s 282 18.07 1984 118 3 ijse 21 1.35 433 83 4 rise 7 0.45 112 72 5 eer 34 2.18 418 70 6 jbe 40 2.56 285 55 7 aee 131 8.39 3287 54 8 gecco 106 6.79 809 52 9 ec 42 2.69 888 35 10 jnc 62 3.97 893 31 *bc (biological conservation), s (sustainability), ijse (international journal of science education), rise (research in science education), eer (environmental educatıon research), jbe (journal of biological education), aee (agriculture, ecosystems & environment), gecco (global ecology and conservation), ec (environmental conservation), jnc (journal for nature conservation). **the first ten sources were included. journal of science learning article doi: 10.17509/jsl.v6i1.50831 42 j.sci.learn.2023.6(1).34-47 headings. findings associated with each main keyword were evaluated considering the keywords identified. finally, the publications reached in the research were summarized using the sources associated with each heading. 4.1 education (citizen science and environmental education) citizen science (cs) is widely used in biodiversity studies. it has taken a place in this field as a popular keyword in recent years and has different definitions. the most commonly accepted definition is “the participation of society in scientific activities.” people use scientific methods to collect, analyze, and interpret data. only more recently has it been used in educational activities. students explore and collect data about the environment (socientize project, 2013). it is widely used for students' definition of biodiversity and for observing living things in the natural environment (rodríguez-loinaz, ametzaga-arregi, & palacios-agundez, 2022). practices in this field are not limited to schools but are made with the participation of individuals from different parts of society. people find the opportunity to experience scientific activities by participating in projects. it was pointed out that there are positive reflections on individuals' knowledge, attitudes, and behaviors about biodiversity through citizen science activities (peter et al., 2019). students who examine the environment scientifically reach various conclusions in line with the data they have obtained. it was stated that students with opposing views about some living things have positive changes due to citizen science activities (aivelo & huovelin, 2020). citizen science has made significant contributions to studies on the functioning of ecosystems. the data obtained by citizen scientists provide data on pollution, human impacts on the environment, endangered species, the effects of climate change, and so on. in other words, it provides a large amount of data about the ecosystem by many volunteer and amateur researchers (dickinson et al., 2012). there is a wide range of research areas from the micro to the macro scale (bonney et al., 2014). the number of individuals involved in this project is in the millions. these groups, amateur environmentalists, have an essential role in protecting the world and reducing environmental damage (merenlender, crall, drill, prysby, & ballard., 2016). in various countries, projects are carried out to observe natural biodiversity changes. these groups of volunteers (students or the public) obtain various data by photographing or observing some species in nature. data are used to determining the changes that occur in the species over time (donnelly, crowe, regan, begley, & caffarra, 2014). citizen science-based curriculums are designed for students. in such environmental education, students can collect data from nature through fieldwork (bopardikar, bernstein, & mckenney, 2021). with the industrial revolution, environmental problems have become one of the most critical issues in the world due to the damage caused by humans to the environment (or nature). in this context, raising the awareness of individuals on environmental protection was the main agenda of the stockholm conference in 1972. environmental problems include all situations that affect the environment and ecosystem. therefore, the steps to protect biodiversity and to raise individuals related to figure 6 the visualization of journals used in recent years journal of science learning article doi: 10.17509/jsl.v6i1.50831 43 j.sci.learn.2023.6(1).34-47 biodiversity are given with environmental education (kassas, 2002). in other words, environmental and biodiversity education's main objectives are to increase citizens' sensitivity to the environment (kim, 2019). however, the extreme extinction of species due to anthropogenic effects has increased interest in biodiversity protection. therefore, the concepts of biodiversity and environmental education have been included in all studies. situations that affect the environment directly or indirectly affect biodiversity (cardinale et al., 2012; kassas, 2002). for this reason, all of the behaviors that are tried to be gained by citizens with environmental education will also protect biodiversity and prevent the extinction of species in nature (kim, 2019). 4.2 sustainability (ecotourism and ecosystem service) one of the keywords in the same cluster (blue) is ecotourism (figure 4). it is a concept that emerged in the 1980s and is based on discovering nature. it is a valuable method for observing nature's diversity and natural beauty. ecotourism can be thought of as both a trip and an informal learning environment. it has an essential function in protecting ecosystems and biodiversity. individuals get more positive thoughts about preserving the nature they observe and have experienced (lengieza, hunt, & swim, 2022; stronza, hunt, & fitzgerald, 2022). as of 2007, there has been a substantial increase in the articles published in this field (stronza et al., 2022). similar findings were also obtained in our study. there has been an increase in biodiversity publications in recent years. today, ecotourism and biodiversity are intertwined because these two areas interact intensely. therefore, it has led to a parallel development with biodiversity studies because the positive and negative effects on biodiversity are addressed in studies on ecotourism (kiss, 2004; stronza et al., 2022). society is making significant efforts to protect biodiversity by increasing income from ecotourism. as long as the ecosystem, wildlife, and diversity exist, income from ecotourism will continue to rise (kiss, 2004). furthermore, governments are developing policies for protecting wildlife. thus, they hope to generate more benefits from ecotourism (wilson & tisdell, 2003). however, scientists constantly express their concerns on this issue. considering only the income aspect of ecotourism, there will be adverse effects on the ecosystem and biodiversity unless managed sustainably (goodwin & swingland, 1996). ecosystems provide many benefits to humans; ecotourism is only one of them. tourism and biodiversity are evaluated in publications on ecotourism, and many studies are currently published in this field, introducing the concept of ecotourism to biodiversity studies. the main parameters are tourism, biodiversity, and sustainable management (stronza & pêgas, 2008). one of the most used keywords in biodiversity education is ecosystem service (see figure 4 and table 1). ecosystem services are a concept that covers the benefits of the ecosystem and nature to human beings. therefore, it constitutes one of the essential concepts that individuals should be aware of in biodiversity education. citizens who are aware of the resources that biodiversity offers for humanity will take a more active role in preventing the extinction of biodiversity. therefore, the importance of ecosystem services for humanity should be taught by people through the education process. scientists assess the benefits of ecosystem services in two ways (regulation and raw material supply). for example, the regulation includes soil formation, sustained fertility, pollination and seed dispersal, climate regulation, and protection against hazards, but raw material supply involves food, medicine, and freshwater (cardinale et al., 2012; díaz et al., 2006; pimentel et al., 1997). the benefits provided in these two ways are necessary not only for ecosystem sustainability but also for the sustainability of the world economy (adom et al., 2019; mace et al., 2012). 4.3 climate change climate change has been used extensively in biodiversity publications. it shows the interaction between climate change and biodiversity. it is one of the most critical problems threatening the environment and our world. changes in the environment affect biodiversity directly or indirectly. the quality of the air deteriorates due to the gases released into the atmosphere. with the increase in the world's temperature, the waters are rising, the glaciers are melting, and living things are negatively affected. therefore, climate change and biodiversity issues are affected by each other (lovejoy, 2008; redlin & gries, 2021). scientists have researched the effects of climate change on the ecosystem in the context of the impact of this situation on species (lovejoy, 2008). various methods are being developed to prevent the extinction of endangered species. the distribution of species is observed, and efforts are being made to determine how climate change affects the distribution of species in the ecosystem (midgley, hannah, millar, rutherford, & powrie, 2002). unfortunately, observations showed that climate change's effects on living populations are increasing (root et al., 2003). 4.4 conversation at the rio earth summit in 1992, the importance of biodiversity, why it should be protected, its significance for a sustainable future, and how it should be protected were emphasized. among the objectives of the meeting, the importance of sustainable use of biodiversity resources was stated. it was stressed that biodiversity resources will be used for the people of today and future generations (cdb, 1992). biodiversity education aims to acquire the importance of natural resources and use them sustainably. with biodiversity education, individuals recognize the functioning of the ecosystem. thus, they learn about what needs to be done for a sustainable ecosystem. unfortunately, the ecosystem is declining due to humans' journal of science learning article doi: 10.17509/jsl.v6i1.50831 44 j.sci.learn.2023.6(1).34-47 overuse of natural resources (lindemann-matthies et al., 2009). the concept of biodiversity emerged in the sciences and has found its place in many disciplines today. efforts are made to raise students' awareness of this issue through environmental education (van weelie & wals, 2002). similar to the findings obtained in the study, it was stated that biodiversity is related to many concepts. it was pointed out that all dimensions of this concept should be emphasized in environmental education (gayford, 2000). because the importance of biodiversity and why it should be protected will be better understood, enabling students to understand that biodiversity education is not just about species diversity or ecosystem diversity. 5. conclusion in this study, biodiversity was evaluated in terms of education using the bibliometric analysis method. in this context, the keywords on biodiversity, the related fields, the most cited authors and journals, the most productive countries, and the trends in recent years were compared. research findings showed that germany, usa, and england are among the most productive countries in biodiversity education. it was determined that the most used keywords included biodiversity, conservation, ecosystem service, climate change, environmental education, sustainability, protected area, biodiversity conservation, education, and agriculture. moreover, recent research results showed that studies related to biodiversity tend to out-of-school (outdoor) activities. in other words, activities that engage the student in interaction with nature (e.g., field trips, citizen science, hands-on activity) were used. based on the findings and literature discussion, the following suggestions were recommended. • countries that are less productive in this area can increase their productivity by co-authorship with more productive countries. • researchers are advised to develop activities that involve students interacting more with the environment (nature). • considering the trend in recent years, studies related to climate change, ecosystem service, ecotourism, sustainability, and citizen science can be carried out. references adom, d., umachandran, k., ziarati, p., sawicka, b., & sekyere, p. (2019). the concept of biodiversity and its relevance to mankind: a short review. journal of agriculture and sustainability, 12(2), 219– 231. aivelo, t., & huovelin, s. (2020). combining formal education and citizen science: a case study on students’ perceptions of learning and interest in an urban rat project. environmental education research, 26(3), 324–340. https://doi.org/10.1080/13504622.2020.1727860 alho, c. j. r. (2012). the importance of biodiversity to human health : an ecological perspective. estudos avançados, 26(74), 151–165. ardoin, n. m., & bowers, a. w. (2020). early childhood environmental education: a systematic review of the research literature. educational research review, 31(july), 100353. https://doi.org/10.1016/j.edurev.2020.100353 barker, s., & elliott, p. (2000). planning a skills-based resource for biodiversity education. journal of biological education, 34(3), 123–127. https://doi.org/10.1080/00219266.2000.9655701 barraza, l. (2001). environmental education in mexican schools: the primary level. the journal of environmental education, 32(3), 31–36. https://doi.org/10.1080/00958960109599143 beery, t., & jørgensen, k. a. (2018). children in nature: sensory engagement and the experience of biodiversity. environmental education research, 24(1), 13–25. https://doi.org/10.1080/13504622.2016.1250149 bermudez, g. m. a., pérez-mesa, r., & ottogalli, m. e. (2022). biodiversity knowledge and conceptions in latin american : towards an integrative new perspective for education research and practice. international journal of education in mathematics, science, and technology, 10(1), 175–217. https://doi.org/https://doi.org/10.46328/ijemst.2105 bilali, h. el, dambo, l., nanema, j., bassole, i. h. n., & calabrese, g. (2022). biodiversity-pastoralism nexus in west africa. aims agriculture and food, 7(1), 73–95. https://doi.org/10.3934/agrfood.2022005 bögeholz, s. (2006). nature experience and its importance for environmental knowledge, values and action: recent german empirical contributions. environmental education research, 12(1), 65– 84. https://doi.org/10.1080/13504620500526529 bogner, f. x. (2018). environmental values (2-mev) and appreciation of nature. sustainability (switzerland), 10(350), 1–10. https://doi.org/10.3390/su10020350 bonney, r., shirk, j. l., phillips, t. b., wiggins, a., ballard, h. l., millerrushing, a. j., & parrish, j. k. (2014). next steps for citizen science. science, 343, 1436–1437. https://doi.org/10.1126/science.1251554 bopardikar, a., bernstein, d., & mckenney, s. (2021). designer considerations and processes in developing school-based citizenscience curricula for environmental education. journal of biological education, latest articles, 1–26. https://doi.org/10.1080/00219266.2021.1933134 börner, k., huang, w., linnemeier, m., duhon, r. j., phillips, p., ma, n., zoss, a. m., guo, h., & price, m. a. (2010). rete-netzwerkred: analyzing and visualizing scholarly networks using the network workbench tool. scientometrics, 83(3), 863–876. https://doi.org/10.1007/s11192-009-0149-0 braun, t., & dierkes, p. (2017). connecting students to nature–how intensity of nature experience and student age influence the success of outdoor education programs. environmental education research, 23(7), 937–949. https://doi.org/10.1080/13504622.2016.1214866 cardinale, b. j., duffy, j. e., gonzalez, a., hooper, d. u., perrings, c., venail, p., narwani, a., mace, g. m., tilman, d., wardle, d. a., kinzig, a. p., daily, g. c., loreau, m., grace, j. b., larigauderie, a., srivastava, d. s., & naeem, s. (2012). biodiversity loss and its impact on humanity. nature, 486(7401), 59–67. https://doi.org/10.1038/nature11148 cdb. (1992). convention on biological diversity. 1–28. https://www.cbd.int/doc/legal/cbd-en.pdf, 24.08.2022 chen, c. (2006). citespace ii: detecting and visualizing emerging trends and transient patterns in scientific literature. journal of the american society for information science and technology, 57(3), 359–377. https://doi.org/10.1002/asi collado, s., corraliza, j. a., staats, h., & ruíz, m. (2015). effect of frequency and mode of contact with nature on children’s selfreported ecological behaviors. journal of environmental psychology, 41, 65–73. https://doi.org/10.1016/j.jenvp.2014.11.001 collins-figueroa, m. (2012). biodiversity and education for sustainable development in teacher education programmes of four jamaican educational institutions. journal of education for sustainable development, 6(2), 253–267. https://doi.org/10.1177/0973408212475257 journal of science learning article doi: 10.17509/jsl.v6i1.50831 45 j.sci.learn.2023.6(1).34-47 deng, w., liang, q., li, j., & wang, w. (2021). science mapping: a bibliometric analysis of female entrepreneurship studies. gender in management, 36(1), 61–86. https://doi.org/10.1108/gm-12-20190240 di ciaccio, f., & troisi, s. (2021). monitoring marine environments with autonomous underwater vehicles: a bibliometric analysis. results in engineering, 9(january), 100205. https://doi.org/10.1016/j.rineng.2021.100205 díaz, s., fargione, j., chapin, f. s., & tilman, d. (2006). biodiversity loss threatens human well-being. plos biology, 4(8), 1300–1305. https://doi.org/10.1371/journal.pbio.0040277 dickinson, j. l., shirk, j., bonter, d., bonney, r., crain, r. l., martin, j., phillips, t., & purcell, k. (2012). the current state of citizen science as a tool for ecological research and public engagement. frontiers in ecology and the environment, 10(6), 291–297. https://doi.org/10.1890/110236 donnelly, a., crowe, o., regan, e., begley, s., & caffarra, a. (2014). the role of citizen science in monitoring biodiversity in ireland. international journal of biometeorology, 58(6), 1237–1249. https://doi.org/10.1007/s00484-013-0717-0 doup, m. l. (2018). using an outdoor activity on local plant biodiversity to teach conservation ecology and promote environmentally responsible behaviors. american biology teacher, 80(5), 359–364. https://doi.org/10.1525/abt.2018.80.5.359 dreyfus, a., wals, a. e. j., & van weelie, d. (1999). biodiversity as a postmodern theme for environmental education. canadian journal of environmental education, 4, 155–176. fančovičová, j., & prokop, p. (2018). effects of hands-on activities on conservation, disgust and knowledge of woodlice. eurasia journal of mathematics, science and technology education, 14(3), 721–729. https://doi.org/10.12973/ejmste/80817 gayford, c. (2000). biodiversity education: a teacher’s perspective. environmental education research, 6(4), 347–361. https://doi.org/10.1080/713664696 goodwin, h., & swingland, i. r. (1996). ecotourism, biodiversity and local development. biodiversity and conservation, 5(3), 275–276. https://doi.org/10.1007/bf00051773 gormley, k., birdsall, s., & france, b. (2022). same, same but different! exploring children’s understandings of within-species variation. journal of biological education, latest articles, 1–22. https://doi.org/10.1080/00219266.2022.2081244 grace, m. (2009). developing high quality decision-making discussions about biological conservation in a normal classroom setting. international journal of science education, 31(4), 551–570. https://doi.org/10.1080/09500690701744595 grant, s. m. (2007). the importance of biodiversity in crop sustainability: a look at monoculture. journal of hunger & environmental nutrition, 12(1), 101–109. https://doi.org/https://doi.org/10.1300/j477v01n02_07 guilherme, e., faria, c., & boaventura, d. (2016). exploring marine ecosystems with elementary school portuguese children: inquirybased project activities focused on ‘real-life’ contexts. education 313, 44(6), 715–726. https://doi.org/10.1080/03004279.2015.1007884 haghani, m., bliemer, m. c. j., goerlandt, f., & li, j. (2020). the scientific literature on coronaviruses, covid-19 and its associated safety-related research dimensions: a scientometric analysis and scoping review. safety science, 129, 104806. https://doi.org/10.1016/j.ssci.2020.104806 hanley, n., spash, c., & walker, l. (1995). problems in valuing the benefits of biodiversity protection. environmental & resource economics, 5(3), 249–272. https://doi.org/10.1007/bf00691519 hardy, c. r., & hardy, n. w. (2018). adapting traditional field activities in natural history education to an emerging paradigm in biodiversity informatics. american biology teacher, 80(7), 501–519. https://doi.org/10.1525/abt.2018.80.7.501 heinen, j. t. (1993). park-people relations in kosi tappu wildlife reserve, nepal: a socio-economic analysis. environmental conservation, 20(1), 25–34. https://doi.org/10.1017/s037689290003719x helldén, g., & helldén, s. (1970). students’ early experiences of biodiversity and education for a sustainable future. nordic studies in science education, 4(2), 123–131. https://doi.org/10.5617/nordina.286 hernández, c., echeverría, c., & nelson, c. (2021). evolution and emerging research trends in the ecological impacts of landscape change: perspectives from a chilean biodiversity hotspot. landscape ecology, 36(6), 1587–1603. https://doi.org/10.1007/s10980-02101247-1 hoover, k. s. (2021). children in nature: exploring the relationship between childhood outdoor experience and environmental stewardship. environmental education research, 27(6), 894–910. https://doi.org/10.1080/13504622.2020.1856790 houlden, v., jani, a., & hong, a. (2021). is biodiversity of greenspace important for human health and wellbeing? a bibliometric analysis and systematic literature review. urban forestry and urban greening, 66, 127385. https://doi.org/10.1016/j.ufug.2021.127385 jansson, m., gunnarsson, a., mårtensson, f., & andersson, s. (2014). children’s perspectives on vegetation establishment: implications for school ground greening. urban forestry and urban greening, 13(1), 166–174. https://doi.org/10.1016/j.ufug.2013.09.003 jesus-leibovitz, l., faria, c., baioa, a. m., & borges, r. (2017). exploring marine biodiversity through inquiry with primary school students: a successful journey? education 3-13, 45(4), 437–449. https://doi.org/10.1080/03004279.2015.1107612 jose, s., patrick, p. g., & moseley, c. (2017). experiential learning theory: the importance of outdoor classrooms in environmental education. international journal of science education, part b: communication and public engagement, 7(3), 269–284. https://doi.org/10.1080/21548455.2016.1272144 kamudu, b., rollnick, m., & nyamupangedengu, e. (2022). investigating what students learnt about biodiversity following a visit to a nature reserve using personal meaning maps. journal of biological education, latest articles, 1–18. https://doi.org/10.1080/00219266.2022.2092190 kassas, m. (2002). environmental education : biodiversity. the environmentalist, 22, 345–351. kelemen-finan, j., scheuch, m., & winter, s. (2018). contributions from citizen science to science education: an examination of a biodiversity citizen science project with schools in central europe. international journal of science education, 40(17), 2078–2098. https://doi.org/10.1080/09500693.2018.1520405 kim, m. (2019). fostering environmental sensitivity by observing everyday environments. journal of geography, 118(4), 157–168. https://doi.org/10.1080/00221341.2018.1564351 kiss, a. (2004). is community-based ecotourism a good use of biodiversity conservation funds? trends in ecology and evolution, 19(5), 232–237. https://doi.org/10.1016/j.tree.2004.03.010 knapp, s. (2019). people and plants: the unbreakable bond. plants people planet, 1(1), 20–26. https://doi.org/10.1002/ppp3.4 kossack, a., & bogner, f. x. (2012). how does a one-day environmental education programme support individual connectedness with nature? journal of biological education, 46(3), 180–187. https://doi.org/10.1080/00219266.2011.634016 krombaß, a., & harms, u. (2008). acquiring knowledge about biodiversity in a museum are worksheets effective? journal of biological education, 42(4), 157–163. https://doi.org/10.1080/00219266.2008.9656134 lengieza, m. l., hunt, c. a., & swim, j. k. (2022). ecotourism, eudaimonia, and sustainability insights. journal of ecotourism, latest articles, 1–16. https://doi.org/10.1080/14724049.2021.2024215 lindemann-matthies, p., constantinou, c., junge, x., köhler, k., mayer, j., nagel, u., raper, g., schüle, d., & kadji-beltran, c. (2009). the integration of biodiversity education in the initial education of primary school teachers: four comparative case journal of science learning article doi: 10.17509/jsl.v6i1.50831 46 j.sci.learn.2023.6(1).34-47 studies from europe. environmental education research, 15(1), 17–37. https://doi.org/10.1080/13504620802613496 lovejoy, t. e. (2008). climate change and biodiversity. review of science and technology, 27(2), 331–338. luvison araújo, l. a., & dos santos alitto, r. a. (2021). teaching native biodiversity: an exploratory study with brazilian teachers. journal of biological education, latest articles, 1–11. https://doi.org/10.1080/00219266.2021.2006271 ma, j., & nickerson, j. v. (2006). hands-on, simulated, and remote laboratories: a comparative literature review. acm computing surveys, 38(3), 1–24. https://doi.org/10.1145/1132960.1132961 mace, g. m., norris, k., & fitter, a. h. (2012). biodiversity and ecosystem services: a multilayered relationship. trends in ecology and evolution, 27(1), 19–26. https://doi.org/10.1016/j.tree.2011.08.006 menzel, s., & bögeholz, s. (2009). the loss of biodiversity as a challenge for sustainable development: how do pupils in chile and germany perceive resource dilemmas? research in science education, 39(4), 429–447. https://doi.org/10.1007/s11165-008-9087-8 merenlender, a. m., crall, a. w., drill, s., prysby, m., & ballard, h. (2016). evaluating environmental education, citizen science, and stewardship through naturalist programs. conservation biology, 30(6), 1255–1265. https://doi.org/10.1111/cobi.12737 midgley, g. f., hannah, l., millar, d., rutherford, m. c., & powrie, l. w. (2002). assessing the vulnerability of species richness to anthropogenic climate change in a biodiversity hotspot. global ecology and biogeography, 11(6), 445–451. https://doi.org/10.1046/j.1466-822x.2002.00307.x montgomery, c. a. (2002). ranking the benefits of biodiversity: an exploration of relative values. journal of environmental management, 64(3), 313–326. https://doi.org/10.1006/jema.2002.0553 moosa, v., & shareefa, m. (2020). science mapping the most-cited publications on workplace learning. journal of workplace learning, 32(4), 259–272. https://doi.org/10.1108/jwl-10-2019-0119 navarro-perez, m., & tidball, k. g. (2012). challenges of biodiversity education : a review of education strategies for biodiversity education. international electronic journal of environmental education, 2(1), 12–30. nyaupane, g. p., & poudel, s. (2011). linkages among biodiversity, livelihood, and tourism. annals of tourism research, 38(4), 1344– 1366. https://doi.org/10.1016/j.annals.2011.03.006 peter, m., diekötter, t., & kremer, k. (2019). participant outcomes of biodiversity citizen science projects: a systematic literature review. sustainability, 11(2780), 1–18. https://doi.org/10.3390/su11102780 pimentel, d., wilson, c., mccullum, c., huang, r., dwen, p., flack, j., tran, q., saltman, t., & cliff, b. (1997). economic and environmental benefits of biodiversity. bioscience, 47(11), 747– 757. https://doi.org/10.2307/1313097 prokop, p., prokop, m., & tunnicliffe, s. d. (2007). is biology boring? student attitudes toward biology. journal of biological education, 42(1), 36–39. https://doi.org/10.1080/00219266.2007.9656105 randler, c., & hulde, m. (2007). hands-on versus teacher-centred experiments in soil ecology. international journal of phytoremediation, 25(3), 329–338. https://doi.org/10.1080/02635140701535091 redlin, m., & gries, t. (2021). anthropogenic climate change: the impact of the global carbon budget. theoretical and applied climatology, 146(1–2), 713–721. https://doi.org/10.1007/s00704021-03764-0 rodríguez-loinaz, g., ametzaga-arregi, i., & palacios-agundez, i. (2022). ict tools and citizen science: a pathway to promote science learning and education for sustainable development in schools. journal of biological education, latest art, 1–17. https://doi.org/10.1080/00219266.2022.2092192 root, t. l., price, j. t., hall, k. r., schneider, s. h., rosenzweig, c., & pounds, j. a. (2003). fingerprints of global warming on wild animals and plants. nature, 421(6918), 57-60. samperiz, a., & herrero, j. (2018). evaluation of a summer camp environmental education program in spain. applied environmental education and communication, 17(1), 79–90. https://doi.org/10.1080/1533015x.2017.1366881 schneiderhan-opel, j., & bogner, f. x. (2020). futureforest: promoting biodiversity literacy by implementing citizen science in the classroom. american biology teacher, 82(4), 234–240. https://doi.org/10.1525/abt.2020.82.4.234 schultz, j. g. w., & joordens, s. (2014). the effect of visitor motivation on the success of environmental education at the toronto zoo. environmental education research, 20(6), 753–775. https://doi.org/10.1080/13504622.2013.843646 slingsby, d. (2009). editorial: charles darwin, biological education and diversity: past present and future. journal of biological education, 43(3), 99–100. https://doi.org/10.1080/00219266.2009.9656161 socientize project. (2013). green paper on citizen science. citizen science for europe: towards a society of empowered citizens and enhanced research. european commission, 1–54. https://digitalstrategy.ec.europa.eu/en/library/green-paper-citizen-scienceeurope-towards-society-empowered-citizens-and-enhancedresearch, 22.08.2022 stronza, a. l., hunt, c. a., & fitzgerald, l. a. (2022). ecotourism for conservation? routledge handbook of ecotourism, 372–397. https://doi.org/10.4324/9781003001768-28 stronza, a., & pêgas, f. (2008). ecotourism and conservation: two cases from brazil and peru. human dimensions of wildlife, 13(4), 263– 279. https://doi.org/10.1080/10871200802187097 sukhontapatipak, c., & srikosamatara, s. (2012). the role of field exercises in ecological learning and values education: action research on the use of campus wetlands. journal of biological education, 46(1), 36–44. https://doi.org/10.1080/00219266.2011.554574 tuparevska, e. (2022). learning in nature: an amplified human rightsbased framework. educational philosophy and theory, latest articles, 1– 11. https://doi.org/10.1080/00131857.2022.2035721 unger, s., rollins, m., tietz, a., & dumais, h. (2021). inaturalist as an engaging tool for identifying organisms in outdoor activities. journal of biological education, 55(5), 537–547. https://doi.org/10.1080/00219266.2020.1739114 van eck, n. j., & waltman, l. (2010). software survey: vosviewer, a computer program for bibliometric mapping. scientometrics, 84(2), 523–538. https://doi.org/10.1007/s11192-009-0146-3 van oudenhoven, a. p. e., & de groot, r. s. (2012). evidence and people’s perceptions of the importance of biodiversity and integrated land use management for ecosystem services and local livelihoods. international journal of biodiversity science, ecosystem services and management, 8(3), 187–189. https://doi.org/10.1080/21513732.2012.716961 van weelie, d., & wals, a. (2002). making biodiversity meaningful through environmental education. international journal of science education, 24(11), 1143–1156. https://doi.org/10.1080/09500690210134839 wheater, r. (1995). world zoo conservation strategy: a blueprint for zoo development. biodiversity and conservation, 4(6), 544–552. whitehead, m. (1995). saying it with genes, species and habitats: biodiversity education and the role of zoos. biodiversity and conservation, 4(6), 664–670. wilson, c., & tisdell, c. (2003). conservation and economic benefits of wildlife-based marine tourism: sea turtles and whales as case studies. human dimensions of wildlife, 8(1), 49–58. https://doi.org/10.1080/10871200390180145 wilson, j. b. (2013). biodiversity theory applied to the real world of ecological restoration. applied vegetation science, 16(1), 5–7. https://doi.org/10.1111/avsc.12008 worm, b., & duffy, j. e. (2003). biodiversity, productivity and stability in real food webs. trends in ecology and evolution, 18(12), 628–632. https://doi.org/10.1016/j.tree.2003.09.003 wyner, y., & doherty, j. h. (2021). seeing the trees: what urban middle school students notice about the street trees that surround them. https://doi.org/10.1080/00219266.2022.2092192 journal of science learning article doi: 10.17509/jsl.v6i1.50831 47 j.sci.learn.2023.6(1).34-47 journal of biological education, 55(2), 155–177. https://doi.org/10.1080/00219266.2019.1667407 xu, j., xiao, p., li, t., & wang, z. (2022). research progress on endangered plants : a bibliometric analysis. biodiversity and conservation, 31, 1125–1147. https://doi.org/https://doi.org/10.1007/s10531-022-02392-y yan, h., & liu, g. (2021). fire ’ s effects on grassland restoration and biodiversity conservation. sustainability, 13(12016), 1–15. https://doi.org/https:// doi.org/10.3390/su132112016 yen, c. f., yao, t. w., & mintzes, j. j. (2007). taiwanese students’ alternative conceptions of animal biodiversity. international journal of science education, 29(4), 535-553. https://doi.org/10.1080/09500690601073418 yu, s., cui, b., xie, c., man, y., & fu, j. (2022). bibliometric review of biodiversity offsetting during 1992–2019. chinese geographical science, 32(2), 189–203. https://doi.org/10.1007/s11769-0221265-5 zannini, p., frascaroli, f., nascimbene, j., persico, a., halley, j. m., stara, k., midolo, g., & chiarucci, a. (2021). sacred natural sites and biodiversity conservation: a systematic review. biodiversity and conservation, 30(13), 3747–3762. https://doi.org/10.1007/s10531021-02296-3 zhang, z., stevenson, k. t., & martin, k. l. (2022). use of nature-based schoolyards predicts students’ perceptions of schoolyards as places to support learning, play, and mental health. environmental education research, 28(9), 1271–1282. https://doi.org/10.1080/13504622.2022.2032612 https://doi.org/10.1080/09500690601073418 a © 2022 indonesian society for science educator 439 j.sci.learn.2022.5(3).439-451 received: 22 january 2022 revised: 29 may 2022 published: 27 november 2022 augmented reality and animation supported-stem activities in grades k12: water treatment necla dönmez usta1*, neslihan ültay1 1faculty of education, giresun university, giresun, turkey *corresponding author: necla.donmezusta@giresun.edu.tr abstract animation is used to increase the interest and engagement of students in the learning environment. animation is exciting and fun, and using animation, abstract concepts are easy to present, display, and convey to students. augmented reality, like animation, can help make it easier to understand abstract concepts. in many areas, augmented reality is used in education since it perfectly integrates virtual content with the natural world. stem education is one of the areas where augmented reality can be used effectively. in this context, this research aimed to reveal the fourth-grade students' opinions about augmented reality and animation-supported stem activities on the "water treatment" topic. these stem activities were carried out with 15 (7 females, 8 males) fourth-grade students from a primary school in turkey and lasted for 6 lesson hours. this study is a case study. as data collection tools in the study, the know-wantlearned chart, and the application general evaluation form, which consists of eight open-ended questions developed by the researchers, were used. based on the study's findings and the researchers' observations, it was determined that the augmented reality and animation-supported stem activities are appropriate for acquisition, content, application, active participation, duration, and student level. in addition, the activities were enjoyable, humorous, engaging, and exciting. it is recommended in this context to conduct similar studies on different disciplines or concepts. keywords animation, augmented reality, stem activity, primary school students, water treatment 1. introduction in the twenty-first century, having creative, innovative, and productive individuals who are curious, questioning, researching, analytical thinking, problem-solving skills, and able to transfer what they learn to real life is more important than having a labor-based workforce. these skills are called 21st-century skills, and these skills (partner 21st-century century learning (p21, 2007) are expressed in three basic dimensions: learning and innovation, digital literacy, and career and life. in the twenty-first century, with changes made in education programs in recent years to train the human resources that countries require, these p21 skills are being attempted to be gained by individuals ranging in age and education level from pre-school to higher education. interdisciplinary teaching comes to the forefront in acquiring these skills, particularly when science, technology, engineering, and mathematics disciplines are attempted to be taught by integrating. in this way, the concept of stem arising is an educational approach that integrates the teaching of science, technology, engineering, and mathematics at all educational levels, beginning with pre-school (gonzalez & kuenzi, 2012). in addition to the objectives stated above, stem education aims to provide the necessary human resources to compete in the long term, particularly on the pisa global scale, and to raise turkey's average (uştu, 2019). for this reason, the ministry of national education (mne) issued a report on stem education in 2016 that stated that steps should be taken to conduct stem education research, establish stem education centers, train stem-competent teachers, and update the curriculum (mne, 2016). as a result, the theme 'science, engineering, and entrepreneurship practices' was added to the science curriculum to respond practically to the needs of students from fourth to eighth grade and real-life problems with a product they designed. stem education aims to educate students in science, mathematics, engineering, and technology, with activities ranging from mailto:necla.donmezusta@giresun.edu.tr journal of science learning article doi: 10.17509/jsl.v5i3.43546 440 j.sci.learn.2022.5(3).439-451 pre-school to post-doctoral. however, it appears that stem education is mentioned in turkey's new curriculum under the heading of scientific and engineering applications. it takes place from fourth to eighth grade (hiğde, 2018), although stem education is appropriate for all grades. the foundations of the scientific process abilities necessary for stem activities are created at a young age (ültay et al., 2020). therefore, it is more crucial to undertake stem activities at an early age, for instance, in the 4th grade at this point. this study is significant because it focuses on fourth-grade students. there are numerous findings on the effectiveness of stem activities in turkey and abroad when reviewing the literature on science education. many of these have a positive impact on students' academic performance and scientific process skills (cotabish, dailey, robinson, & hunghes, 2013; kong & huo, 2014; ormancı, 2020; sarıca, 2020; sırakaya & alsancak-sırakaya, 2020; ültay, balaban, & ültay, 2021). furthermore, stem-focused studies show that students successfully create a positive attitude toward stem and stem disciplines (chen, huang, & wu, 2021; hackman, zhang, & he, 2021; karahan, canbazoğlu bilici, & ünal, 2015). when students' attitudes toward stem disciplines are examined separately, it is discovered that the most significant improvement is from high to low in engineering, science, technology, and mathematics (tseng, chang, lou, & chen, 2013). in stem activities, it is required that at least two stem disciplines are included in the activity (kennedy & odell, 2014). interestingly, the commonly preferred discipline is "engineering" in stem activities (moore et al., 2014; ültay et al., 2021). it is an expected result because formerly, it was mentioned that students' attitude towards engineering has mostly increased among stem disciplines. however, in today's world, especially after the covid-19 pandemic, technology has become an essential part of the educational system. during the pandemic period, most courses were based on online learning after april 2020, and most teachers had no experience designing their courses for online teaching (marek, chew, & wu, 2021). additionally, stem-based online learning activities are carried out, and remarkable results are seen. for example, sarnita, fitriani, utama, and suwarma (2021) reported that online stem courses had increased students' creative thinking skills. similarly, benli özdemir (2021) found that online stem activities improved students' 21st-century skills. from this, it is clearly understood that technology is an indispensable part of our lives. even though stem activities include technology as a discipline, using technology as a complementary pedagogy in stem activities can still be viewed as a modern-day necessity. in this research, technology is not a separate discipline but a complementary pedagogy. this study used augmented reality and animation in stem activities. augmented reality (ar) has been widely used in the last ten years, depending on smart devices such as smartphones, tablets, etc. (schiavi, havard, beddiar, & baudry, 2022). ar contributes to better structuring of concepts/subjects (turan-güntepe & dönmez-usta, 2021). with ar being used in the learning environment, abstract concepts can be transformed into visible, audible, and visual dynamic concrete concepts (cai, liu, wang, liu, & liang, 2021). many studies have concluded that ar improves student achievement and significantly contributes to stem education (hsu, lin, & yang, 2017; wu, hwang, yang, & chen, 2018). the fact that ar is a technology that today's students can easily use means that the use of ar in stem education supports the learning and teaching process (sırakaya & alsancak-sırakaya, 2020). ar in science is a simulation that could involve learners more deeply in the investigatory project activity than traditional simulations could (hwang, wu, chen, & tu, 2016). for this reason, ar can significantly impact stem education (phon, ali, & halim, 2015). considering the advantages mentioned earlier of ar in the learning environment, many applications such as "anatomy 4d, quiver 3d, animal 4d+, elements 4d, and octaland 4d" were designed to use ar in science education (buluşkırıkkaya & şentürk, 2018). one of the applications of ar, "quiver 3d," is used in this study. animations have been used as educational resources to support science teaching and learning for several decades in science education. animations of dynamic views and abstract events/subjects positively affect learning (lowe, 2003). the animations used in learning environments significantly affect students' attitudes towards the course and academic success (powell, aeby jr, & carpenter-aeby, 2003). in addition to speeding up and slowing downtime, it can be said that it provides many contributions, such as being able to be analyzed, simplifying complex systems, being valuable and inexpensive, and motivating in learning environments (daşdemir & doymuş, 2012). ceylan (2014), in her study on stem education, concluded that the methods and techniques she used for the technology discipline, such as slow transition animation techniques, collaborative groups, and the engineering design process, contributed to students' scientific creativity skills. ar and animation-based materials help students to visualize, experiment, and collect data on stem subjects by enabling them to apply what they have learned and use computers with special applications such as computer simulations and animations (lantz, 2009). in this respect, this study is also essential. this research aimed to reveal the fourth-grade students' opinions about ar and animation-supported stem activities on the "water treatment" topic in the "human and environment" unit. at this point, the following questions form the focus of the study: ⚫ what are the students’ opinions on what they know about the topic, what they want to learn, and what journal of science learning article doi: 10.17509/jsl.v5i3.43546 441 j.sci.learn.2022.5(3).439-451 they learned at the end of the ar and animationsupported stem activities? ⚫ what are the students' opinions about augmented reality and animation-supported stem activities?. ⚫ what are the students' opinions about the designed water treatment system? 2. method based on the general characteristics of case studies, yin (2003) distinguishes four patterns: single holistic case, embedded single case, holistic multiple cases, and embedded multiple cases. the holistic single case pattern can be applied to situations involving individuals or institutions (creswell, plano clark, gutmann, & hanson, 2003). researchers reveal the situation described as the heart of the study in case studies and clearly express the underlying reality and boundaries (miles & huberman, 1994). the holistic single-case design is used to study unique situations that do not meet the standards of a single analysis unit. because the unique situations mentioned in the design cannot be numerous or even multiple, they can be the subject of independent research (yıldırım & şimşek, 2018). at this point, it was determined that it was appropriate to conduct the case study, one of this study's qualitative research designs, which is about ar and animation-supported stem activities on water treatment, within the framework of a holistic single case pattern. the study employed the embedded single-case methodology to focus on a single instance where ar and animationsupported stem activities were considered vital. thus, students' perspectives on ar and animation-supported stem activities were solicited, and an attempt was made to shed light on these interlaced concerns concerning the water treatment mechanism and application process. 2.1 sample ar and animation-supported stem activities were carried out with 15 (7 females, 8 males) fourth-grade students from a primary school in the eastern black sea region in turkey, selected as a stem primary school in the second term of the 2020-2021 academic year. students are between the ages of 9-10 and have participated in stem activities on different subjects in previous years. the primary school teacher has 23 years of experience, is skilled in stem activities, and is eager to put her knowledge into practice. the school is a stem practice school. therefore, primary school teachers have a certain level of theory and practice in stem. the stem activities were explained to the primary school teacher in detail before the application and discussed. the primary school teacher also examined the applications of animation and ar in stem activities. the primary school teacher wanted the activities to be done by the researchers since they were technological applications. she said that we could also benefit from the materials in the class regarding the water purification system. thus, the researchers and the primary school teacher jointly created the materials such as cotton, coal, newspaper, pet bottles, etc. the students were divided into groups of 2-3 people by their primary school teacher, and a total of six groups were formed. the primary school teacher guided the homogeneous distribution and mert's water treatment system when mert awakens in the morning, he goes to the bathroom to wash his hands and face, and when he turns on the bathroom faucet, he notices a difference in the color of the water. as the water continues to flow, it becomes completely cloudy. as a result, mert believes he is late for school and arrives without washing his hands and face. when he walks into the classroom, his teacher notices mert is sleepy and thoughtful, and he asks. figure 1 mert's water treatment system teacher: mert, did you sleep late last night? mert: not at all, teacher. i could not wash my face in the morning because the water was very cloudy when i turned on the faucet. teacher: so, children, have you ever had this problem in the morning? a few students who lived near mert's house reported having a similar problem. when the teacher returned to the classroom, he said, "you've heard about the problems your friends are having." so, how can we help them? what solution can we offer? ????? journal of science learning article doi: 10.17509/jsl.v5i3.43546 442 j.sci.learn.2022.5(3).439-451 dynamism of the groups, was present in the classroom throughout all applications, and allowed students to carry out activities in their natural environment. on the other hand, the primary school teacher preferred that the researchers carry out the application. 2.2 designing and creating ar and animation supported stem activities this section presents the points to consider when designing the event in detail. furthermore, a worksheet has been developed so teachers and students can quickly implement ar and animation-supported stem activities. in the turkish science lesson curriculum (mne, 2016), the "human and environment" unit is in the 4th grade and is recommended for 6 lesson hours. the outcomes of the "human and environment" unit of the turkish science curriculum (meb, 2018) are listed as "f.4.6.1.1. students pay attention to behaving economically in the use of resources. a. it emphasizes the importance of saving resources such as electricity, water, and food. b. emphasis is placed on the importance of reuse. f.4.6.1.2. students realize the importance of resources and recycling necessary for life. resources such as water, food, and electricity are mentioned" as examples suitable for daily life situations were investigated in the environment where the students would apply the activity for their outcomes. within the scope of the researched examples, a context related to water treatment was chosen because of the importance of water awareness, saving water, and decreasing water resources in our daily lives. when selecting this context, consideration was given to the fact that it is a problematic situation encountered in everyday life. when determining the problem, the researchers' prepared story titled "mert's water treatment system" is intended to be given to the students as a problematic case. therefore, the relevant story is depicted in figure 1. in the preceding story, students will be asked to assist mert in setting up a water purification system to purify water. soil, gravel, coal dust, filter paper, cotton, fine cloth, newspaper, funnel, water, pet bottle, plastic container, scissors, and towel paper are required for a water treatment setup. at the stage of presenting solutions to the problem situation, consideration has been given to the criteria, limitations, and multiple solutions of the water treatment system they are expected to design and the ability to test it. however, because testing the given problem situation in real life is impossible, the criteria and limitations are given in accordance with the prototype they can apply in the classroom. similar applications can also be found in the literature (hacıoğlu, 2020). the design, which allows for the clean treatment of contaminated water, is structured as an engineering design problem. therefore, the criteria and limitations of the prototype they designed for solving the students' problems are also included. it was also noted that the given problem has multiple solutions and that the solutions can be tested. furthermore, it is hoped that they will require engineering design skills that demonstrate science and design abilities while solving the problem. for this, instructions have been created on the worksheet. students are given opportunities to use the discipline of mathematics while creating and testing the prototype they create for the problem solution. for example, students are expected to generate the amount of balanced cotton or coal dust used and increase or decrease the amounts after testing the setup and using mathematical modeling. students are also expected to test and evaluate the setup. here, they are allowed to evaluate the instructions and mechanisms in the 7th section of the worksheet. 2.3 implementation the implementation was carried out in 6 lesson hours (2 * 3 weeks) as recommended in the curriculum by the researchers. problem identification (1 lesson hour) the students were shown mert's one day animationi, prepared by the researchers, during the first stage of the lesson. this animation was created with the vyond application (“vyond”, 2022), which allows users of all skill levels to create animations. mert's mistakes in using resources such as water, electricity, and food in his environment are depicted in the animation. figure 2 contains animation sample screenshots. in addition, incorrect behaviors are depicted in the images with an arrow. following the viewing of the relevant animation, the students discussed animation. then, the students attempted to identify incorrect behaviors in the animation. unfortunately, researchers have not provided any scientific guidance in this case. following that, the researchers attempted to determine/define the problem situation by reading the story titled mert's water treatment system (figure 1). figure 2 sample screenshots of mert's one day animation-i journal of science learning article doi: 10.17509/jsl.v5i3.43546 443 j.sci.learn.2022.5(3).439-451 presenting potential solutions and determining the best solution (1 lesson hour) at this point, students were introduced to the materials needed for the water purification system and brought to the classroom. figure 3 shows a sample image from related materials. the students arrived at the material table in groups formed by the primary school teachers, examined the materials, and designed by debating the suggestions and the best solution for the proposal. the students were asked to draw the relevant suggestions for the mechanism in the third section of the worksheet. water treatment system design, testing, and revision (2 lesson hours) the students attempted to make the two-dimensional mechanisms they designed for the solution proposals in their worksheets testable in three dimensions. figure 4 contains sample images of the mechanisms created by the students water treatment system presentation (1 lesson hour) students were asked to present the water purification systems they created. during the presentation, they were questioned about how they used science, mathematics, and engineering knowledge and skills in the design of the mechanism and how they used this knowledge. process evaluation (1-course hour) mert's one day animation-ii was shown after the student presentations. this animation was also created with the vyond application, which allows users of all skill levels to create animations. mert's mistakes in using resources such as water, electricity, and food in his environment are corrected in the animation. sample screenshots of the animation are included in figure 5. following the viewing of the relevant animation, the students discussed animation. a comparison with animation-i has been made. following that, the proper and efficient use of resources such as water, food, and electricity, as well as recycling and being a conscious consumer, were discussed. next, students were instructed to draw pictures depicting the subject or concept/concepts they learned during these activities. students were given empty frames that could be used for free on quivervision 3d augmented reality coloring apps, and they were asked to draw pictures on them. this app provides high-quality augmented reality (ar) experiences by combining amazing 3d graphics and digital technologies. combining traditional coloring and cutting-edge technology helps to make learning fun and exciting for people of all ages. quiver uses augmented reality to bring drawing or coloring pages to life in 3d animation (“quivervision”, 2022). figure 6 includes sample images of students making their two-dimensional drawings three-dimensional, viewing, and coloring them. following the visualization of the students' figure 4 sample images of the prepared water treatment system figure 3 example materials required for a water treatment setup figure 5 sample screenshots of mert's one day animation-ii journal of science learning article doi: 10.17509/jsl.v5i3.43546 444 j.sci.learn.2022.5(3).439-451 drawings in 3d, the implementation was completed using data collection tools. 2.4 data collection tools as data collection tools in the study, the know-wantlearned (kwl) chart, and the application general evaluation form, which consists of eight open-ended questions developed by the researchers, were used. the application general evaluation form includes questions about what students learned, their struggles, the water purification system they designed, activities, and the learning environment they experienced during the activation process. during the development of this form, the perspectives of two science educators with experience in stem activities were solicited. the application general evaluation form was finalized after receiving feedback from the relevant experts. 2.5 data analysis the information gathered from the data collection tools was analyzed using content analysis. the basic process in content analysis is to collect similar data within the framework of specific concepts and themes, then organize and interpret it so the reader can understand (bauer, 2003). both researchers thoroughly examined the data during the data analysis process and created appropriate codes. the codes created were combined based on their similarities and differences, and subcategories/categories were determined by grouping the codes under different headings. the themes were reached through the created categories and continued until the themes were fixed. the percentage of compliance was calculated by miles and huberman (1994) with the percentage of agreement formula (% of agreement = [agreement/disagreement + agreement] * 100) as .89 was founded to determine the reliability of the study. 2.6 ethics in the study within the bounds of ethical principles, the researchers personally followed the application process. data was collected at every study stage, following the principles of follow-up and respect for people. the study did not include private conversations between researchers and students during the data collection due to privacy and confidentiality concerns (drew, hardman, & hart, 1996). furthermore, the students were informed that some demographic information would be shared with the readers, and their permission was obtained. to ensure confidentiality, students who participated in the data collection process within the research ethics framework were coded as s1, s2, s3,..., s15. 3. result and discussion the results of the first study question are shown in table 1: "what are the students' opinions on what they figure 6 sample images of 3d displays of students' drawings table 1 the results of the kwl chart know ƒ want ƒ learn ƒ conscious consumers (s1, s3, s4, s5, s9, s10, s12, -, 15) 10 recycling (s7, s8, s10, s11, s12, s15) 6 how we should clean the water (s1, s2, s3, s4, s5, s6, s7, s8, s9, s10, s11, s12, s13, s14) 14 saving and recycling (s5, s6, s7, s8, s9, s10, s11, s12, s13, s15) 10 how to be a conscious consumer (s5, s6, s7, s8) 4 how to be a conscious consumer (s1, s4, s5, s8, s11, s12, s13, s15) 8 environment concept (s2, s8, s9, s11, s12) 5 saving (s2, s3, s5, s8) 4 the importance of saving (s1, s2, s6, s11, s12) 5 the importance of the environment (s4, s6, s8, s12) 4 recycling (s4, s7, s13, s15) 4 the consequences of human impact on the environment (s4, s6, s9, s13) 4 how to get drinking water from seawater (s9) 1 what is inside the soil? (s1) 1 technology (s1) 1 journal of science learning article doi: 10.17509/jsl.v5i3.43546 445 j.sci.learn.2022.5(3).439-451 know about the topic, what they want to learn, and what they learned at the end of the ar and animationsupported stem activities?" as seen in table 1, the first column shows what students know about the topic, and the students said that they know some knowledge about "conscious consumers (10), saving and recycling (10) and environment concept (5)." the second column shows what students want to learn. furthermore, students said that they want to learn about "recycling (6), some said how to be a conscious customer (4), saving (4), the importance of the environment (4), how to get drinking water from seawater (1), what is inside soil (1) and technology (1)". the third column shows what students learned at the end of the ar and animation-supported stem activities. for example, al students said that they learned about "how we should clean the water (14), how to be a conscious consumer (8), the importance of saving (5), and recycling (4)". the results of the second and third study questions are shown in table 2: "what are the students' opinions about augmented reality and animation-supported stem activities" and "what are the students' opinions about the water treatment system they designed?". table 2 summarizes the students' answers to the application general evaluation form. as seen from table 2, the students were asked eight questions. some questions (1, 2, 3, 8) were about the students' general thoughts on ar and animation-supported stem activities, while others (4, 5, 6, 7) were about the water treatment system project. students said active participation (9) and fun (7) during ar and animation supported-stem activities. furthermore, they said that "positively affected (12), enjoyable (5) and exciting (4) your motivation towards the lesson. moreover, they indicated that these stem activities were enjoyable (8) and lovely (8). in general, positive feelings appeared in students' minds. the student said that she/he paid attention to the "filtration method (11)" while designing his/her water treatment system project. besides, they indicated that "separation of dirty water (4) and while setting up the mechanism in the filtration method (4)" had trouble learning with their your project. moreover, they said that they were "very pleased (14), clean water from dirty water made me happy (9) and enjoyable (2)," satisfied with the project they prepared. they indicated that they "improve the separation mechanism (5), obtain different dirty water (4), and obtain potable water (3)" to prepare it again. table 2 the results of the application general evaluation form 1. explain whether you feel active or not, along with the reason during this stem activity. codes ƒ codes ƒ active participation (s3, s4, s5, s7, s8, s9, s11, s12, s13) 9 intriguing (s1) 1 funny (s1, s2, s4, s6, s10, s14, s15) 7 exciting (s3) 1 experiment (s6, s7, s13) 3 creative (s8) 1 instructive (s2, s10) 2 2. would you like to use applications similar to this stem activity in your different courses? explain. codes ƒ codes ƒ yes (s1, s2, s3, s4, s5, s6, s8, s9, s10, s11, s12, s13, s14) 13 no (s15) 1 enjoyable (s1, s5, s6, s8, s11, s13, s14) 7 it takes too much time (s15) 1 i wish that all courses were taught through experiments (s4, s9) 2 in all courses (s7, s12) 2 in mathematics and turkish (s2) 1 in science courses (s10) 1 active participation (s12) 1 3. how do you think this stem activity affected your motivation toward the lesson? explain. codes ƒ codes ƒ positively affected (s1, s4, s5, s6, s7, s8, s9, s10, s11, s12, s13, s15) 12 carefully listened (s2) 1 enjoyable (s4, s5, s7, s11, s15) 5 instructive (s11) 1 exciting (s1, s2, s3, s10) 4 experiments (s12) 1 eager to learn (s6, s13) 2 happy (s8, s10) 2 learning with playing (s4) 1 4. what did you pay attention to while designing your project during the activity? codes ƒ codes ƒ filtration method (s1, s2, s4, s5, s6, s7, s8, s9, s10, s12, s15) 11 everything (s3) 1 the weight of water (s1) 1 acting with thoughts (s11) 1 experiment (s13) 1 journal of science learning article doi: 10.17509/jsl.v5i3.43546 446 j.sci.learn.2022.5(3).439-451 two figures are created from the codes to get a clearer picture. the first figure summarizes student opinions on stem activities in general (figure 7), while the second summarizes student opinions on water treatment systems (figure 8). these figures are made with word art (“wordart”, 2022). as summarized in figure 7, students have positive feelings and opinions about the ar and animationsupported stem activities. for example, an s2-coded student stated, "i did the activity feeling excited and happy." an s4-coded student stated, "i felt active myself, so it was enjoyable." finally, an s6-coded student stated, "the activities directed us to learn, and we learned while having fun." as can be seen in figure 8, the majority of them were happy to be able to get clean water from dirty water. for example, the s15 code student stated, "to obtain crystal clear water from dark water was impressive." s10, s12, and s13 coded students stated, "if we had the opportunity to change something in our projects, we would change it to obtain potable water because the water in this activity could not be consumed due to the presence of microscopic bacteria." some students added a lot of coal dust to get dirty water, so it was challenging to get clear water for them. therefore, s2, s8, s11, and s14 stated, "if we had the table 2 the results of the application general evaluation form (continued) 5. what were you having trouble learning with your project? codes ƒ codes ƒ no trouble (s3, s4, s5, s6, s7, s8, s11) 7 while setting up the mechanism in the filtration method (s1, s12, s13, s15) 4 separation of dirty water (s2, s9, s10, s14) 4 6. are you satisfied with the project you prepared? codes ƒ codes ƒ very pleased (s2, s3, s4, s5, s6, s7, s8, s9, s10, s11, s12, s13, s14, s15) 14 enjoyable (s5, s7) 2 clean water from dirty water made me happy (s1, s6, s8, s9, s10, s12, s13, s14, s15) 9 7. if you had the chance to prepare it again, what would you like to change in your new project? codes ƒ codes ƒ improve the separation mechanism (s3, s5, s6, s7, s15) 5 nothing (s1, s4) 2 different dirty water (s2, s8, s11, s14) 4 more fast system (s9) 1 obtaining potable water (s10, s12, s13) 3 8. what do you think about these stem activities in general? explain in detail. codes ƒ codes ƒ enjoyable (s1, s2, s4, s6, s8, s9, s11, s15) 8 difficult (s2) 1 lovely (s1, s3, s5, s7, s10, s12, s14, s15) 8 a useful project in daily life (s13) 1 instructive (s8, s9, s11) 3 improve social skills (s5) 1 improve motivation (s6, s9) 2 exciting (s10) 1 figure 7 opinions about the ar and animation-supported stem activities figure 8 opinions on the water treatment system journal of science learning article doi: 10.17509/jsl.v5i3.43546 447 j.sci.learn.2022.5(3).439-451 opportunity to change something in our projects, we would form a different dirty water." the findings from the kwl chart showed that the majority of the students stated that they were aware of conscious consumers (f = 10) and saving and recycling (f = 10) and that they wanted to learn more about recycling (see table 1). the students stated at the end of the application that they learned how to clean the water and to be a conscientious consumers. most students (f = 9, see table 2) stated that they actively participated in ar and animation-supported stem activities and found them entertaining (f = 7, see table 2). this situation may be related to the student's active participation in the process and their design of suitable products for potential solutions. hacıoğlu and dönmez usta (2020), in their research on digital game design-based stem activities, stated that working as engineers to solve the problem situation and designing appropriate products aided the students' more active involvement in the process. in his study, morrison (2006) stated that stem education allows learners to learn subjects or concepts more cheerfully and entertainingly. dönmez (2017) discovered that students enjoyed the activities in his study on robotic tournaments within stem education. according to ensari (2017) 's research, the lessons are more enjoyable with stem activities, and the subjects/concepts become more understandable with these activities requiring active participation. similarly, pekbay (2017) discovered that students enjoy stem activities and learn science concepts through group work. in this context, the findings of the literature support the study's findings. almost all of the study's students (f = 13) expressed a desire to use ar and animation-supported stem activities in various lessons. students can gain knowledge in various areas by combining design, experimenting, analyzing, interpreting, and natural phenomena with stem activities (wang, 2012). it is claimed that science and mathematics lessons are rich in content and that technology material such as ar and animation make them more interesting for students (schaefer, sullivan, & yowell, 2003). besides, teaching materials by integrating teaching materials with stem, it is expected that student learning motivation will increase. teaching materials with stem require text, graphics, animation, simulation, and video because the use of stem-integrated teaching materials also can improve students' thinking skills (zahara, abdurrahman, herlina, widyanti, & agustiana, 2021). moreover, it can be said that teaching materials and environments containing ar technology contribute positively to students' learning. sobrino, vallejo, morcillo, redondo, & schez (2020) found that ar-supported learning material positively affects the student's motivation in learning environments. this could have stemmed from students' intense interest in ar, which positively affects their motivation and increases their desire to learn. this study found that ar and animation technology-supported stem activities brought together more than one discipline in an interconnected way by contributing to students' knowledge in various fields and their thinking skills. moreover, it was determined that animation-based teaching materials prepared using ar technology increase students' interest and motivation in the subject and their use in lessons that include teaching problem-solving skills, such as mathematics, physics, and chemistry in literature (cevahir, özdemir, & baturay 2022). this situation may be related to the students' desire to use activities similar to those used in different lessons within the scope of the study. most students (f = 12) reported that ar and animation-supported stem activities increased their motivation for the lesson. stem education has increased students' interest and motivation in science classes (green, 2012; kang, ju, & jang, 2013; park & yoo, 2013; yamak, bulut, & dündar, 2014). it is stated that science-based stem education can be a helpful pedagogy for teaching students science and creating excitement and motivation for them to learn science (siew, amir, & chong, 2015). in many studies in which students view the use of animation in science lessons, a positive relationship was found between students' use of animation and their attitude towards technology, and their knowledge transfer and motivation towards science and technology lessons (rosen, 2009; önal & söndür, 2017). in this context, it is clear that ar and animation-supported stem activities positively impact students' motivation. furthermore, it is believed that there is a directly proportional relationship between students' motivations and their active participation in the process and creating ideas and products consistent with these ideas. when designing the water purification system in ar and animation-supported stem activities, students stated that they focus primarily on the filtration method (f = 11). this situation can be explained by the fact that students with ar and animation-supported stem education will create a goal not only as a way of learning but also with an attitude toward science (douglas & strobel, 2015). stem activities encourage students to learn, research, and ask questions about potential and existing problems (tabaru, 2017). in this regard, the problem encountered in mert's water treatment system story is a situation that anyone can face in everyday life. to solve mert's problem, students divided into groups and designed various water purification systems. in view of akgündüz and akpnar (2018), the product design stage in students' stem activities is the most challenging stage. separation of dirty water and setting up the mechanism in the filtration method as the point at which some students (f = 4) have difficulty in the activity is consistent with the literature in this study. the fact that some of the students (f = 7) also stated that they had no trouble can be explained by their familiarity and experience with stem activities. the journal of science learning article doi: 10.17509/jsl.v5i3.43546 448 j.sci.learn.2022.5(3).439-451 stem school was chosen from the school where the application was submitted. as a result, the fact that students are used to and experienced in stem activities helps to explain why some students do not struggle with design. at this point, it is possible to state that the study improves problem-solving and solution-generation skills by supplementing the students' experiences. specifically, the students stated that they would like to improve the separation mechanism, and different dirty water is allowed to design products. in this case, it indicates that students' ability to solve problems encountered in everyday life has improved. it is claimed that stem activities help students become aware of real-world problems, as well as their knowledge and skills (kanadlı, 2019), generate solutions (roberts, 2012; ültay et al., 2020), and contribute to problem-solving abilities (barak & doppelt, 2000). according to fortus, dershimer, krajcik, marx, and mamlok-naaman (2004), science education practices and engineering design activities should be used to help students develop skills such as problem-solving and solution generation. by incorporating engineering design activities into science education applications, this study added to the body of knowledge. almost all students (f = 14) expressed satisfaction with the devised water purification system. furthermore, using the purification devices they created, the students obtained clean water from dirty water and were relieved that they had found a solution to the problem. furthermore, uyar, canpolat, and şan (2021) found that students were very satisfied with the stemactivities, which made the lesson fun and allowed for active participation, in a study in which they examined the opinions of teachers and students in the stem center on stem education. these findings corroborate ours. namely, students stated that they provide active participation in ar and animation supported-stem activities and that they find it funny. furthermore, students' satisfaction with the water treatment system they designed can be attributed to the fact that stem education is an educational approach that improves students' problem-solving skills (roberts, 2012) and allows the transformation of remaining knowledge into products, applications, and discoveries (tüsi̇ad, 2014, p.16). the students responded to the question of what they would change if they had the opportunity to re-prepare the water purification systems by saying they would improve the separation mechanism (f = 5), use different dirty water (f = 4), and obtain potable water (f = 3). these answers are related to students' ability to examine the problem, find solutions to problems with unique ideas, communicate in group work (sparkes, 2017), and improve their ability to find solutions to daily life problems (roberts, 2012). it can also be explained by the fact that ar and animationsupported stem applications help students develop 21stcentury skills by encouraging scientific thinking, imagination, self-expression, and self-confidence (ültay, dönmez usta, & ültay, 2021; young, house, wang, singleton, & klopfenstein, 2011). students stated that ar and animation-supported stem activities are generally enjoyable, fun, active participation, exciting, improve motivation, and are so good. in stem education applications, there is an emphasis on technology and engineering; this allows students to gain a multidisciplinary perspective and put their knowledge into practice (akgündüz et al., 2015). due to this situation, stem is in a crucial position in today's information and technology age. given the importance of stem education in a country's ability to lead in science and technology while also becoming economically strong (lacey & wright, 2009), it is thought that ar and animation-supported stem activities should be included more in the lessons so that students can develop different solutions to the daily life problems they encounter and holistically evaluate the events. painting according to learning objectives in augmented reality applications in education, multimedia materials such as text, audio, 3d objects, 2d or 3d animation, and video can be used together (wang, kim, love, & kang, 2013). in this study, augmented reality applications were used together with animation. thus, it has contributed to students actively participating and helped to learn to be effective in the learning process. the students focus on the filtration method, obtaining clear water, obtaining potable water, a different separation system, a different dirty system, and the difficulty in setting up the mechanism. these points of view are consistent with the students' status as members of generation z. since this generation grew up with the internet, it is a generation that can quickly access information, is techsavvy, does not adhere to formalities, learns quickly, and embraces diversity (twenge, campbell, hoffman, & lance, 2010). these characteristics, in general, can be said to support their views on the water treatment system. 4. conclusion based on the study's findings and the observations of the researchers who were putting the process in place, it was determined that the ar and animation-supported stem activity is appropriate in acquisition, content, application, active participation, duration, and student level. within the scope of the study, it was determined that ar and animation-supported stem events are enjoyable, humorous, engaging, and exciting. therefore, the study designed stem activities with ar and animation support. this activity is a trailblazer in demonstrating how technology such as ar and animation in science education can be integrated with stem education. given that stem activities should be structured according to at least two disciplines (pitt, 2009), it is clear journal of science learning article doi: 10.17509/jsl.v5i3.43546 449 j.sci.learn.2022.5(3).439-451 that this ar and animation-supported activity employs three to four disciplines. at this point, it appears that three to four disciplines can be carried out in a coordinated manner in stem activities. the students' water purification system designed and tested is said to meet the learning outcomes. 5. recommendation when the results of the study were evaluated, the following suggestions were made: studies should be conducted on developing and implementing ar and animation-supported stem activities for various subjects and concepts such as traffic lights, substance, and properties, separation methods of substances, intensity, etc. studies should be conducted to design and implement studies in which various technologies such as virtual reality and simulation are integrated into stem activities. it is recommended in this context to conduct similar studies on different disciplines or concepts. in addition, studies focusing on different sample groups or research methods (such as the experimental method) should be conducted. references akgündüz, d., & akpınar b. b. (2018). okul öncesi eğitimde fen eğitimi temelinde gerçekleştirilen stem uygulamalarını öğrenci, öğretmen ve veli açısından değerlendirilmesi [evaluation of stem applications based on science education in pre-school education in terms of students, teachers, and parents]. journal of education for life, 32(1), 1-26. akgündüz, d., aydeniz, m., çakmakçı, g., çavaş, b., çorlu, m. s., öner, t., & özdemir, s. (2015). stem eğitimi türkiye raporu [stem education turkey report]. i̇stanbul: scala basım. barak, m., & doppelt, y. (2000). using portfolios to enhance creative thinking. journal of technology studies, 26(2), 16-25. bauer, m. w. (2003). classical content analysis: a review. in m. w. bauer & g. gaskell (eds.), qualitative researching with text, image and sound (pp.131). london: sage publication. benli özdemir, e. (2021). views of science teachers about online stem practices during the covid-19 period. international journal of curriculum and instruction, 13(1), 854-869. buluş-kırıkkaya, e. & şentürk, m. (2018). the impact of using augmented reality technology in the solar system and beyond unit on the academic achievement of the students. kastamonu education journal, 26(1), 181-189. cai, s., liu, c., wang, t., liu, e., & liang, j. c. (2021). effects of learning physics using augmented reality on students' self-efficacy and conceptions of learning. british journal of educational technology, 52(1), 235–251. cevahir, h., özdemir, m., & baturay, m. h. (2022). the effect of animation-based worked examples supported with augmented reality on the academic achievement, attitude and motivation of students towards learning programming. participatory educational research, 9(3), 226-247. ceylan, s. (2014). a study for preparing an instructional design based on science, technology, engineering, and mathematics (stem) approach on the topic of acids and bases at secondary school science course. (unpublished master dissertation). uludağ university, bursa. chen, y. l., huang, l. f., & wu, p. c. (2021). preservice pre-school teachers" self-efficacy in and need for stem education professional development: stem pedagogical belief as a mediator. early childhood education journal, 49(2), 137-147. creswell, j. w., plano clark, v. l., gutmann, m. l., & hanson, w. e. (2003). advanced mixed methods research designs. handbook of mixed methods in social and behavioral research, 209-240. cotabish, a. dailey, d., robinson, a., & hughes, g. (2013). the effects of a stem intervention on elementary students" science knowledge and skills. school science and mathematics, 113(5), 215-226. daşdemir, i̇., & doymuş, k. (2012). the effect of using animation on primary science and technology course students' academic achievement, retention of knowledge and scientific process skills. pegem journal of education and instruction, 2(3), 33-42. dönmez, i. (2017). the views of students and team coaches about robotic competitions on the stem education framework (case of first lego league). journal of research in education, science and technology, 2(1), 25-42. douglas, k. a., & strobel, j. (2015). hopes and goals survey for use in stem elementary education. international journal of technology and design education, 25(2), 245-259. drew, c. j., hardman, m. l., & hart, a. w. (1996). designing and conducting research: inquiry in education and social science (2nd ed.). boston: allyn and bacon. ensari, ö. (2017). pre-service teachers' views on stem education and stem activities. (unpublished master dissertation). yüzüncü yıl university, van. fortus, d., dershimer, r. c., krajcik, j., marx, r. w., & mamlok‐ naaman, r. (2004). design‐based science and student learning. journal of research in science teaching, 41(10), 1081-1110. gonzalez, h. b., & kuenzi, j. j. (2012). science, technology, engineering and mathematics (stem) education: a primer. library of congress. congressional research service. retrieved from https://fas.org/sgp/crs/misc/r42642.pdf access date: 08.04.2021. green, a., (2012). the integration of engineering design projects into the secondary science classroom. (unpublished master dissertation). michigan state university, michigan. hacıoğlu, y. (2020). ‘implementation of thematic stem education: friction force example. boğaziçi university journal of education, 37, 3-21. hacıoğlu, y., & dönmez-usta, n. (2020). digital game design-based stem activity: biodiversity example. science activities, 57(1), 1-15. hackman, s. t., zhang, d., & he, j. (2021). secondary school science teachers" attitudes towards stem education in liberia. international journal of science education, 43(2), 1-24. hwang, g. j., wu, p. h., chen, c. c., & tu, n. t. (2016). effects of an augmented reality-based educational game on students’ learning achievements and attitudes in real-world observations. interactive learning environments, 24(8), 1895–1906. https://doi.org/10.1080/10494820.2015.1057747 hiğde, e. (2018). the investigation of the effect of the stem activities prepared for 7th class students in terms of different variables. (unpublished doctoral dissertation). aydın adnan menderes university. hsu, y. s., lin, y. h., & yang, b. (2017). impact of augmented reality lessons on students’ stem interest. research and practice in technology enhanced learning, 12(1), 1-14. kanadlı, s. (2019). a meta-summary of qualitative findings about stem education. international journal of instruction, 12(1), 959-976. kang, j., ju, e. j., & jang, s., (2013). the effect of science-based steam program using a portfolio on elementary students’ formation of science concepts. elementary science education, 32(4), 593-606. karahan, e., canbazoğlu bilici, s., & ünal, a. (2015). the integration media design processes in stem education. eurasia journal of educational research, 15(60), 221-240. kennedy, t., & odell, m. (2014). engaging students in stem education. science education international, 25(3), 246-258. kong, y. t., & huo, s. c., (2014). an effect of steam activity programs on science learning interest. advanced science and technology letters, 59, 41-45. https://fas.org/sgp/crs/misc/r42642.pdf journal of science learning article doi: 10.17509/jsl.v5i3.43546 450 j.sci.learn.2022.5(3).439-451 lacey, t. a., & wright, b. (2009). employment outlook: 2008-18occupational employment projections to 2018. monthly lab. rev., 132, 82. lantz, h. b. (2009). science, technology, engineering, and mathematics (stem) education what form? what function? report, currtech integrations, baltimore. lowe, r. k. (2003). animation and learning: selective processing of information in dynamic graphics. learning and instruction, 13(2), 157176. marek, m. w., chew, c. s., & wu, w. c. v. (2021). teacher experiences in converting classes to distance learning in the covid-19 pandemic. international journal of distance education technologies (ijdet), 19(1), 40-60. miles, m. b., & huberman, a. m. (1994). qualitative data analysis: an expanded sourcebook. sage. milli eğitim bakanlığı (meb). (2018). fen bilimleri dersi öğretim programı (i̇lkokul ve ortaokul 3, 4, 5, 6, 7 ve 8. sınıflar [science cirriculum: elementary and middle schools 3,4,5,6,7 and 8th grades.]. ankara, turkey: talim terbiye kurulu başkanlığı https://mufredat.meb.gov.tr/programdetay.aspx?pid=325 ministry of national education (mne). (2016). stem education report. from http://yegitek.meb.gov.tr/stem_egitimi_raporu.pdf access date: 31.12.2017. moore, t. j., stohlmann, m. s., wang, h. h., tank, k. m., glancy, a. w., & roehrig, g. h. (2014). implementation and integration of engineering in k–12 stem education. in, s. purser, j. strobel, & m. cardella (eds.), engineering in precollege settings: research into practice (pp. 35-60). west lafayette, in: purdue university press. morrison, j. (2006). attributes of stem education: the student, the school, the classroom. ties (teaching institute for excellence in stem), 20, 2-7. ormanci, ü. (2020). thematic content analysis of doctoral theses in stem education: turkey context. journal of turkish science education, 17(1), 126-146. önal, n. t., & söndür, d. g. (2017). i like technology usage in lessons and animations in science!. the journal of academic science studies, (55), 97-118. park, s. j., & yoo, p. k. (2013). the effects of the learning motive, interest and science process skills using the "light" unit in sciencebased steam. elementary science education, 32(3), 225-238. partnership for 21st century learning (p21). (2007). framework for 21st century. pekbay, c. (2017). effects of science technology engineering and mathematics activities on middle school students. (unpublished doctoral dissertation). hacettepe university, ankara. phon, d. n. e., ali, m. b., & halim, n. d. a. (2015). learning with augmented reality: effects toward student with different spatial abilities. advanced science letters, 21(7), 2200–2204. pitt, j. (2009). blurring the boundaries stem education and education for sustainable development. design and technology education: an international journal, 14(1), 37–48. powell, j. v., aeby jr, v. g., & carpenter-aeby, t. (2003). a comparison of student outcomes with and without teacher facilitated computer-based instruction. computers & education, 40(2), 183-191. quivervision. (2022). retrieved from https://quivervision.com/ roberts, a. (2012). a justification for stem education. technology and engineering teacher, 71(8), 1-4. rosen, y. (2009). the effects of an animation-based on-line learning environment on transfer of knowledge and on motivation for science and technology learning. journal of educational computing research, 40(4), 451-467. https://doi.org/10.2190/ec.40.4.d sarica, r. (2020). analysis of postgraduate theses related to stem education in turkey: a meta-synthesis study. acta didactica napocensia, 13(2), 1-29. sırakaya, m., & alsancak-sırakaya, d. (2020). augmented reality in stem education: a systematic review. interactive learning environments, 1-14. sarnita, f., fitriani, a., utama, j. a., & suwarma, i. r. (2021, march). application of stem-based online learning to train creative skills of students in covid-19 pandemic periods. in journal of physics: conference series (vol. 1806, no. 1, p. 012039). iop publishing. schaefer, m. r., sullivan, j. f., & yowell, j. l. (2003). standard-based engineering curricula as a vehicle for k– 12 science and math integration. frontiers in education, 2, 1–5. schiavi, b., havard, v., beddiar, k., & baudry, d. (2022). bim data fow architecture with ar/vr technologies: use cases in architecture, engineering and construction. automation in construction, 134, 104054. siew, m. n., amir, n., & chong, c. l. (2015). the perceptions of preservice and in-service teachers regarding a project-based stem approach to teaching science. springer plus, 4(1), 1-20. doi:10.1186/2193-1801-4-8 sparkes, v. p. (2017). what is the steam?. ayrıntı publishing i̇stanbul. sobrino, s. s., vallejo, d., morcillo, c. g., redondo, m. a., & schez, j. c. (2020). robotic: a serious game based on augmented reality for learning programming. multimedia tools and applications, 79, 34079–34099. https://doi.org/10.1007/s11042-020-09202z tabaru, g. (2017). i̇lkokul 4. sınıf öğrencilerine fen bilimleri dersinde uygulanan stem temelli etkinliklerin çeşitli değişkenlere etkisi [elementary school 4th grade the effects of stem-based activities applied in science lessons to primary education students in terms of various variables]. (unpublished master thesis). niğde ömer halisdemir university (in turkish). tseng, k. h., chang, c. c., lou, s. j., & chen, w. p. (2013). attitudes towards science, technology, engineering and mathematics (stem) in a project-based learning (pjbl) environment. international journal of technology and design education, 23(1), 87-102. turan-güntepe, e., & dönmez-usta, n. (2021). augmented reality application based teaching material's effect on viscera learning through algorithmic thinking. journal of science learning, 4(4), 365375. tüsi̇ad. (2014). stem (science, technology, engineering and mathematics, fen, teknoloji, mühendislik, matematik) alanında eğitim almış iş gücüne yönelik talep ve beklentiler araştırması [for the stem-trained workforce demand and expectations research]. https://tusiad.org/tr/yayinlar/raporlar/item/8054-stemalaninda-egitim-almis-isgucune-yonelik-talep-vebeklentilerarastirmasi twenge, j. m., campbell, s. m., hoffman, b. j., & lance, c. e. (2010). generational differences in work values: leisure and extrinsic value increasing, social and intrinsic values decreasing. journal of management, 36(5), 1117-1142. doi: 10.1177/0149206309352246. ültay, e., balaban, s., & ültay, n. (2021). content analysis of the studies examining the teachers’ and pre-service teachers’ views on stem education. turkish journal of primary education, 6(2), 109-125. doi: 10.52797/tujped.953385 ültay, n., dönmez usta, n., & ültay, e. (2021). descriptive content analysis of studies on 21st century skills . sdu international journal of educational studies, 8(2), 85-101. doi: 10.33710/sduijes.895160 ültay, n., zıvalı, a., yılmaz, h., bak, h. k., yılmaz, k., topatan, m., & kara, p. g. (2020). stem-focused activities to support student learning in primary school science. journal of science learning, 3(3), 156-164. doi: 10.17509/jsl.v3i3.23705 uştu, h. (2019). preparing and implementing successful stem/steam activities in primary schools: a participatory action research with primary school teachers. (unpublished doctoral dissertation). necmettin erbakan university, turkey. uyar, a., canpolat, m., & şan. i̇. (2021) views of teachers and students in stem center on stem education: the sample of payastem center”. manas journal of social studies, 10(1), 151-170. vyond. (2022). retrieved from https://www.vyond.com/ wang, h. h. (2012). a new era of science education: science teachers' perceptions and classroom practices of science, technology, engineering, and mathematics (stem) integration. university of minnesota. https://mufredat.meb.gov.tr/programdetay.aspx?pid=325 http://yegitek.meb.gov.tr/stem_egitimi_raporu.pdf https://quivervision.com/ https://tusiad.org/tr/yayinlar/raporlar/item/8054-stem-alaninda-egitim-almis-isgucune-yonelik-talep-vebeklentiler-arastirmasi https://tusiad.org/tr/yayinlar/raporlar/item/8054-stem-alaninda-egitim-almis-isgucune-yonelik-talep-vebeklentiler-arastirmasi https://tusiad.org/tr/yayinlar/raporlar/item/8054-stem-alaninda-egitim-almis-isgucune-yonelik-talep-vebeklentiler-arastirmasi https://doi.org/10.52797/tujped.953385 https://www.vyond.com/ journal of science learning article doi: 10.17509/jsl.v5i3.43546 451 j.sci.learn.2022.5(3).439-451 wang, x., kim, m. j., love, p. e., & kang, s. c. (2013). augmented reality in built environment: classification and implications for future research. automation in construction, 32, 1-13. wordart. (2022). retrieved from https://wordart.com/ wu, p. h., hwang, g. j., yang, m. l., & chen, c. h. (2018). impacts of integrating the repertory grid into an augmented reality-based learning design on students’ learning achievements, cognitive load and degree of satisfaction. interactive learning environments, 26(2), 221–234. yamak, h., bulut, n., & dündar, s. (2014). the impact of stem activities on 5th grade students’ scientific process skills and their attitudes towards science. gazi university journal of gazi educational faculty, 34(2), 249-265. doi: https://doi.org/10.17152/gefd.15192 yıldırım, a., & şimşek, h. (2011). qualitative research methods in social sciences (8th ed.). ankara: seçkin publishing. yin, r. k. (2003). case study research design and methods (3rd ed.). london: sage publications. young, v. m., house, a., wang, h., singleton, c., & klopfenstein, k. (2011, may). inclusive stem schools: early promise in texas and unanswered questions. in highly successful schools or programs for k12 stem education: a workshop. washington, dc: national academies (vol. 1, p. 2014). zahara, m., abdurrahman, a., herlina, k., widyanti, r., & agustiana, l. (2021, february). teachers’ perceptions of 3d technologyintegrated student worksheet on magnetic field material: a preliminary research on augmented reality in stem learning. in journal of physics: conference series (vol. 1796, no. 1, p. 012083). iop publishing. https://wordart.com/ https://doi.org/10.17152/gefd.15192 a © 2020 indonesian society for science educator 132 j.sci.learn.2020.3(3).132-142 received: 13 december 2019 revised: 23 april 2020 published: 15 july 2020 virtual reality in science education: a descriptive review alper durukan1*, huseyin artun2, atilla temur1 1department of science education, faculty of education, van yuzuncu yil university, van, turkey 2department of elementary education, faculty of education, van yuzuncu yil university, van, turkey *corresponding author. a@alperdurukan.com abstract the term “virtual reality” currently refers to a profound sensory immersion of the user in a synthetically generated virtual environment. it is foreseen that virtual reality will gain a substantial role in the instruction of science. in this literature review, the purpose was to investigate the research on the utilization of virtual reality in the science education context, according to several criteria. the articles published in peer-reviewed journals and academic conferences/symposiums that are available in the databases of eric, wos, and google scholar have been reviewed. consequently, a total of 30 eligible articles reviewed and findings presented under every respective criterion. partially, findings revealed the dominance of journal article type publications, the usa and turkey found to be most prominent origins, experimental studies being preferred mostly, undergraduate students and pre-service teachers were the most studied groups, the contexts of the studies were prominently general, and the learning outcomes investigated mostly. keywords science education, virtual reality, instructional technology, science laboratory instruction, literature review 1. introduction instructional technologies (it) are a valuable tool in the arsenal of instruction in terms of constraints such as time, accessibility, and cost. it is particularly suitable for increasing student interest, reducing the time needed for classroom activities, and supporting educational efficiency (yumuşak & aycan, 2002). when used in the instruction of science, it may be considered as the main actor in bringing scientific findings to a solid-state and articulate the situations that unobservable with the senses (linn, 2003). in the past, virtual reality (vr) has been relatively under-emphasized in education due to its’ high cost and limited availability (checa & bustillo, 2019). in time, the advancements of technology led to a new understanding of it. moreover, the paradigmatic shift in terms of the utilization of technology in science education facilitated the emergence of innovative instructional technologies. as the environments in which information transferred are changing, the incorporation of such technologies in science education has been gaining importance. today, wearable technologies and augmented reality and vr can be listed as the most explicit examples of innovative technologies (çalışkan, 2017). in the context of education, vr may be described as an interface that presents a sufficient sensory immersion to users in an artificial environment generated by a computer and/or mobile device (lee, wong, & fung, 2010; mora, martín-gutiérrez, añorbe-díaz, & gonzálezmarrero, 2017). as a tangible demonstration, trindade, fiolhais, and almeida (2002) defined vr as an interface that embodies conceptual models. they also classified the essential components of the vr environment as immersion, interaction, and engagement, as demonstrated in figure 1. the immersion component refers to the convincing properties of vr that make users perceive a sufficient existential embodiment (bowman & mcmahan, 2007). this authentic perception results in a relatively genuine occasion by addressing multiple sensory inputs. secondly, interaction refers to the capability of simultaneous obtainment and manipulation of the intended experience by enabling users to manipulate the virtual environment (nalbant & bostan, 2006). vr makes it possible for the users to interact with that environment as in real life, which in turn facilitates learning-by-doing. finally, engagement translates into individuals’ voluntary participation to experience the vr environment (trindade, fiolhais, & journal of science learning article doi: 10.17509/jsl.v3i3.21906 133 j.sci.learn.2020.3(3).132-142 almeida, 2002). learners immersed through a realistic sensory embodiment tend to participate willingly in educational activities within the vr environment. the technological advancements also led vr interfaces to develop in terms of their functions and feasibility. capabilities of predecessor computer-based vr, which regarded as immersive in the past, nowadays fall short compared to the current state of immersive vr. immersive vr can be defined as an interface that authentically mimics real-world settings regarding visual, perceptional, and functional dimensions. when current opportunities are considered, it would be more appropriate to place obsolescent screen-based interfaces within the category of computer simulations. mora, martín-gutiérrez, añorbedíaz, and gonzález-marrero (2017) suggested three categories of interfaces in which virtual and augmented reality scenes are created and experienced: (a) smartphones mounted on headsets, (b) stand-alone head-mounted displays, (c) augmented reality devices. it could be concluded that science instruction and laboratory practices enriched with vr facilitate not only the consumption of authentic learning situations but also the creation of them. allcoat and von mühlenen (2018) emphasize the feasible promises of vr and state that vr is more than an additive to current practices: it is an interface that can expose learners to authentic learning experiences. moreover, rather than limiting the benefits for learners only, making use of this potential could provide meaningful opportunities for the pre-service and in-service training of science educators as well. as vr is gradually becoming more accessible, research on vr in the education of science and thought to be emphasized more in time. therefore, this review aims to investigate the research on the implementation of vr in science education by evaluating them according to several criteria. 2. method the study conducted as a literature review. literature reviews aim to examine scientific publications carried out on a specific field or subject area (galvan & galvan, 2017). in the present study, studies incorporate the use of vr technologies in the field of science education systematically investigated. findings obtained from this investigation have been presented respectively 2.1 screening of the databases to reach eligible studies, the databases of eric, wos (web of science), and google scholar were used. the following joint search query used to reach relevant research on vr in instruction science and sub-disciplines of physics, chemistry, and biology: virtual reality and science education or physics education or chemistry education or biology education. 2.2. criteria for the eligible studies the eligibility of the studies for investigation determined according to several inclusion conditions. the reasons for the exclusion of irrelevant papers were as follows: (a) not being relevant to the instruction of science and its sub-disciplines of physics, chemistry, and biology. (b) not being subjected to a peer-reviewing process such as in academic journals or conference publications: therefore, even being contextually relevant, publications such as books, book chapters, and reports have been excluded. consequently, a total of 30 articles focusing on the use of vr technology in science education found to be eligible and thereby investigated. 2.3. classification of the studies the eligible studies investigated according to the following criteria: (a) publication type, (b) country origin, (c) genre, (d) published year, (e) keywords used, (f) method, (g) sample or participant group, (h) context, (i) investigated variables and situations. figure 1 the components of the vr environment (trindade, fiolhais, & almeida, 2002) immersion interactionengagement journal of science learning article doi: 10.17509/jsl.v3i3.21906 134 j.sci.learn.2020.3(3).132-142 3. result and discussion the findings presented under respective sections. 3.1. types of publications figure 3 represents the distribution of the publication types. among reviewed studies, 23 papers (77%) were articles published in peer-reviewed journals, and the remaining seven papers (23%) were either abstracted or full-text manuscripts published in the proceeding booklets of academic conferences or symposiums. 3.2. country origins figure 4 presents the country origins of the investigated studies. determination of country origins conducted concerning the individual characteristics for each of the reviewed publications. for the experimental studies, either the country in which respective intervention(s) held or accommodated the participants, were considered. for nonexperimental studies and studies with multiple authors, corresponding authors’ institutions taken as the basis. as seen in figure 4, the usa ranked first in number by being the origin of nine studies, which was followed by turkey with seven studies, and taiwan with three studies coming third. australia, spain, israel, sweden, canada, malaysia, portugal, serbia, new zealand, and greece each being origin of one study on the use of vr in science education, per se. 3.3. genres of the studies figure 2 shows the distribution of the studies by their genres. among all, 15 studies (50%) classified as experimental, which utilized a particular manipulation regarding vr-assisted science instruction to influence specific variable(s). eight studies (27%) classified as descriptive introduced potential outcomes, available opportunities, and materials for the use of vr in education, either generally or science education specific. moreover, four studies (13%) categorized as material, which introduced the vr material developed and discussed the potential application areas. lastly, three (10%) papers classified as review, which utilized a systematic examination of the literature. figure 3 distribution of publication types figure 4 distribution map of the studies concerning their country origins figure 2 distribution of reviewed studies by genres journal of science learning article doi: 10.17509/jsl.v3i3.21906 135 j.sci.learn.2020.3(3).132-142 3.4. publication dates the grouped publication dates of the studies presented in figure 6. there was one study conducted between 19901994, one study between 1995-1999, three studies between 2000-2004, one study between 2005-2009, six studies between 2010-2014 and 18 studies between 2015-2019. 3.5. keyword usage figure 5 presents the frequencies of keywords of the studies that adjusted by acknowledged terminology in the literature. findings revealed a vast variation in the keywords, hence, infrequent and similar keywords grouped by the acknowledged terminology used in the literature. it is seen that 19 keywords used in the context of virtual reality, seven keywords in instructional technologies six keywords in chemistry, individual factors, learning outcomes and virtual worlds, five keywords in materials and tools and learning experience and four keywords in the contexts of augmented reality, education and virtual laboratories. also, keywords for the settings of biology, interactive learning environments, science teaching, immersive education, and methods used in three studies. lastly, the keywords used in the contexts of practices in subject areas, user interfaces, pre-service teachers, pedagogical situations, applied laboratories, and constructive methods were used only once, per se. figure 6 frequency of studies regarding their time-period of publication figure 5 keywords used in the studies journal of science learning article doi: 10.17509/jsl.v3i3.21906 136 j.sci.learn.2020.3(3).132-142 3.6. methods of the reviewed studies the distribution of the research methods utilized in reviewed studies is presented in figure 7. the research method not explicitly mentioned in 14 studies (46%) hence shown as x in the graph. apart from this, the most prominent method found to be quasi-experimental, utilized in eight studies (27%). moreover, there were two studies (7%) designed as pre-experimental, two studies (7%) conducted as a survey, and two studies (7%) conducted with mixed design, which utilizes both qualitative and quantitative procedures. lastly, there was one study (3%) conducted as meta-analysis, and one study (3%) found to be qualitative. 3.7. participants figure 8 shows the distribution of sample & participant groups or data sources of the reviewed studies. eleven studies (34%), which represented the code of x, did not explicitly include/state a participant group or a data source. it also is seen that five studies (15%) carried out with undergraduate students from various departments, and four studies (12%) conducted with pre-service teachers. although pre-service teachers are undergraduate students, they figure 7 distribution of the research methods figure 8 distribution of the participants journal of science learning article doi: 10.17509/jsl.v3i3.21906 137 j.sci.learn.2020.3(3).132-142 presented apart from the undergraduate students to reflect the context of teacher training better. also, three studies obtained the data from the literature, which carried out using descriptive or survey research to examine a relevant particular research topic. lastly, there was one study carried out with elementary school students (%3), one study with graduate students (%3), one study with faculty members (%3), and one study with teachers (%3). it was considering the parent category of educators, two studies in total conducted with teachers and faculty members (6%). 3.8. context figure 9 presents the distribution of the contexts emphasized in the reviewed studies. concerning the instructional contexts, ten studies (31%) classified as general, which focuses on the use of vr by briefly mentioning the instruction of science as well as other disciplines in a non-specific manner. moreover, six studies (18%), which were carried out within the context of chemistry, focused on the role of vr technologies in teaching chemistry-related topics. also, there were four studies (12%) carried out in the context of biology and physics to examine the use of vr in the instruction of these disciplines. it seems that the context of astronomy covered the topic of celestial bodies in two studies (6%). in two studies (6%) in the context of laboratory, science laboratory practices in a virtual environment were covered in general by not focusing on a specific topic. lastly, there was one study (3%) carried out in the context of anatomy, focusing on animal physiology, and one study (3%) in the context of teacher training, investigating the use of vr in the pedagogical practices of pre-service teachers 3.9. variables and situations figure 10 presents the frequencies for each investigated variable and situation among the reviewed studies. it was seen that 13 studies, which coded as x, did not examine any variable or situation. following this, six studies measured the participants’ attitudes towards vr-assisted instruction, and four studies measured the influence of vrassisted instruction on participants’ technology acceptance. moreover, three studies measured the outcomes in the variable of spatial skills through vr-assisted instruction, and three studies investigated outcomes in self-efficacy of participants’ vr-assisted instruction as well as obtained qualitative data on participant opinions. furthermore, two studies examined the influence of instruction supported with vr on retention to determine if the learning gains of participants have remained over time. likely, two studies figure 9 distribution of the contexts figure 10 variables and situations examined journal of science learning article doi: 10.17509/jsl.v3i3.21906 138 j.sci.learn.2020.3(3).132-142 investigated the perceived immersion/presence experiences of participants in educational vr environments. lastly, there was one study per se for following variables: science process skills, conceptual understanding, usability of vr in education, teaching experience investigated through the qualitative data on vr-assisted instruction, psychological effects and adverse effects of the usage of vr in science instruction, and lastly qualitative data on opinions towards creativity in the instruction enriched by vr. it seems that in terms of the type of publications, the primary research being journal articles, in turn, abstracts and full-texts published in proceeding books of conferences and symposiums, were scarce in comparison. in terms of publication type, mikropoulos and natsis (2011) examined the studies carried out on virtual educational environments published between the period of 1999-2009. the researchers found that the majority of studies were articles published in peer-reviewed journals, whereas studies proposed in conferences were rare in comparison. concluding the similarity in findings, it could be stated that published research on vr-assisted science instruction was more evident in peer-reviewed journals rather than manuscripts proposed to academic conferences or symposiums. this tendency can be interpreted as that the emergence of the vr-assisted science education research is still in progress, and the first remarks of this progress are evident in the peer-reviewed journals. besides, adequate utilization of vr in the educational contexts requires a relatively rigorous effort, time, and resources. hence, it is understandable that the researchers prefer to seize their extensive interventions on vr-assisted science education in the high standing publication options. concerning country origins, research on vr-assisted science instruction conducted mostly in the usa and followed closely by turkey. remaining origins found to be taiwan, australia, spain, israel, sweden, canada, malesia, norway, portugal, servia, new zealand, and greece, respectively. scimago lab (2019)’s revelations are similar concerning the country origins of the e-learning studies published between 1996-2017, which also reveals the usa’s lead in the research accommodation on instructional technologies. moreover, a plentiful number of developing countries also found to accommodate relevant research similar to the developed countries. this result bears the impression that there is an international emphasis on the research on the utilization of innovative technologies in instruction. from the perspective of vr, this situation is being prominent, especially in the field of science education. for developing countries, it is entirely reasonable to benefit from the experiences and parallel strategies of developed countries (hamidi, ghorbandordinejad, rezaee, & jafari, 2011) in the manner of the emphasis on the instructional technology. findings on study genres revealed the dominance of experimental research on vr-assisted science education. following this, descriptive studies also found to be abundant in this topic. literature reviews, on the other hand, found to be the most underrepresented genre among all. hew and cheung (2010) examined studies on the use of three-dimensional immersive virtual worlds concerning k-12 and higher-level education. similarly, they classified the accessed studies as descriptive and experimental. they reported an apparent preference and relatively balanced abundance for both, compared to the literature reviews. the preference for experimental and description-aimed research is evident in the findings of this review. the order of this preference may not remain the same for all. in another systematic review of research on the use of virtual reality environments in education, kim, lee, and thomas (2012) reported the prominence of descriptive studies over experimental research. however, their findings revealed a gradual increase in the number of experimental research over time. concerning the publication dates of the studies, the number of publications found to increase since the year 2000. moreover, most of the studies found to be published between 2015-2019. although not being exactly vr; akçayır and akçayır (2017) reviewed the studies on the use of augmented reality in education. they reported a similar trend regarding the increase in publications over time. the researchers also stated that this increase began to accelerate since 2013, and incrementally continued since then. turning back to the diffusion of vr, this process is still in progress for educators. as the instructional use of the vr becomes a more common practice, it is foreseen that this number will increase even more in the following years. regarding keyword usage, it seems that the cluster of the keyword on virtual reality used the most, followed by instructional technologies. similarly, liu et al. (2017), who used the bibliometric analysis to examine the studies on the use of vr in education that published between 1995-2016, reported that the virtual reality was the most frequently used keyword, and followed by interactive learning environment. the researchers also reported that, when viewed from the interdisciplinary perspective, the contents related to the discipline of chemistry were the most abundant in used keywords. at the same time, the occurrences of biology and science ranked second. besides, they noted that the keywords related to the discipline of physics found to be scarce among the disciplines. this conclusion supports our results regarding the abundance of discipline-specific terms, in identical order. another crucial point regarding keyword usage in the studies is an emphasis on the situations and variables observed in relevant studies— besides, terms referring to certain variables also evident such as; individual factors, learning outcomes, and learning experience. the use of the variables in the keywords section may be attributed to the intention of the authors to make their study more accessible for readers who screen the literature by investigated variables of the studies. apart journal of science learning article doi: 10.17509/jsl.v3i3.21906 139 j.sci.learn.2020.3(3).132-142 from that, four studies found to be lacking the keywords entirely. it is noteworthy that a remarkable proportion of research lacks an explicit statement of the research method. for remaining studies, the dominance of experimental methodology may also outshine as studies requiring a clear statement of research method are thought to be aimed at either description or the development of educational vr materials. moreover, the most prevalent type in experimental studies was the quasi-experimental design. concerning this, jensen and konradsen (2017), examined the research on immersive vr technologies with headmounted displays in education and training contexts the authors reported that the most common method applied in the quantitative studies was the quasi-experimental design. this preference somewhat understandable as for the educational research, random assignment of the subjects might not be feasible in most circumstances to conduct true-experimental interventions. also, researchers stressed the scarcity of mixed and qualitative methodologies in the research of vr-assisted education, which supports our present findings. the findings revealed that the majority of the studies did not explicitly present a participant group or sample like it has been in the methods. the least preferred groups included elementary school students and graduate students. in another systematic review on the use of vr in education, kavanagh, luxton-reilly, wuensche, and plimmer (2017) similarly reported that the studies on vr-assisted education have mostly been conducted with students from undergraduate and graduate levels. in another study carried out to examine the use of a three-dimensional virtual world game called second life in education, inman, wright, and hartman (2010) found that the studies towards the contexts of undergraduate and graduate-level instruction were more prominent in comparison. these studies support our findings regarding the under-emphasis in the literature on the use of vr in the education of the early age groups, which thought to reveal a potential gap in this manner. the reviewed studies found to mainly emphasize a general perspective in the use of vr in science education. this perspective manifests itself in the prominence of the studies on the use of vr in the instruction of multiple disciplines, rather than focusing on a specific discipline, subject, or concept. the fact that the prominence of the general context may also relate to the abundance of descriptive studies that tend to tackle the use of vr in education broadly. among discipline-specific studies, the fact that the instruction of the chemistry, biology, and physics being studied more than the instruction of science itself could be related to the subjects’ age groups and education levels, which scarce in terms of early grade levels. the science course taught in the levels of elementary and secondary school grades, while chemistry, biology, and physics courses taught as individual courses starting from the high school grade levels. moreover, further specialization under these disciplines is also evident in the upper levels of education, especially starting from the undergraduate level. as this situation applies to the majority of international educational policies worldwide, it reveals the holistic relationship between the findings. the preferences on the participants and studied contexts appear to related to this situation, which in turn revealed itself not only in the methods but also in the used keywords as well. as a substantial part of science education, laboratory practices have a central place in the research on instructional technology. in this review, studies on laboratory found to mainly implement a simulation of laboratory facilities in a complete vr environment rather than focusing on the applications, experiments, or visualizations under specific topics. innovative technologies that allow authentic three-dimensional representations such as augmented reality (yen, tsai, & wu, 2013) and vr (barnett et al., 2005), has a great potential in providing concrete observation of the phenomena related to the investigations in science laboratory courses. according to tuysuz (2010); the difficulty in carrying out experiments at schools that lack laboratories or laboratory equipment could be attenuated by technology. besides, the author concluded that the virtual chemistry laboratory facilitated the achievements and attitudes of students. in parallel, a vr assisted analytical chemistry instruction found to not only enhance students’ laboratory practices but also facilitate their research skills (bortnik et al., 2017). for the case of the biology laboratory practices, it seems possible not only to examine the physiology of a wide variety of living beings in a precise and detailed manner with vr but also to overcome the financial and ethical constraints in this context. however, although virtual laboratories provide accessibility and replicability, they cannot entirely create the feeling of existence experienced by students in traditional laboratories (ma & nickerson, 2006). not casting aside the traditional instruction, but seamlessly utilizing technologyenhanced instruction would facilitate the traditional instructional practices as well (young & nichols, 2017). in time, vr-assisted science education thought to lead the science laboratory practices to be more accessible and engaging thanks to their immersive interfaces. the feasibility of vr-assisted laboratory instruction may bolster the boundaries of physical laboratories, especially with a careful and adequate focus on the domain-specific concepts under the topics of science courses. it should be stressed that the feasible utilization of vr in science education demands component teachers, especially in terms of their technological-pedagogicalcontent knowledge. in this review, the context of teacher training constituted only the focus of one study, which orientated towards the pedagogical competencies of prejournal of science learning article doi: 10.17509/jsl.v3i3.21906 140 j.sci.learn.2020.3(3).132-142 service teachers in multiple branches. one of the promises of vr in the training of teachers argued to be related to its’ potential of facilitating pre-service teachers’ self-efficacy beliefs (durukan, 2018; nissim & weissblueth, 2017) and contributing to pre-service teachers’ pedagogical competencies in terms of providing authentic teaching experiences (durukan, temur, & artun, 2018). concerning these, the context of pre-service science teachers’ training needs more emphasis in the research on vr-assisted science education. the majority of the studies reviewed had not an explicit statement of the methods used; hence, they did not include any experimental structure to examine a variable or situation. one of the potential factors of this may be the prevalence of descriptive and material developmentoriented studies. moreover, this result also reveals a general lack of intervention in vr-assisted science education research. also, the tendency to cover a broad context may be related to the prominence of studies that were lacking a dependent variable. in interventional studies, the prominence of quantitative investigations could be associated with the prevalence of quasi-experimental and preexperimental designs. among experimental interventions, academic achievement was dominant in terms of examined variables. opposingly, some of the critical outcomes for science literacy, such as science process skills, conceptual understanding, as well as teaching experience of it, was found to be examined only once, being one study for each. durukan (2018) claims that laboratory practices enriched with immersive vr could contribute positively to the development of science pre-service teachers, science process skills, and, especially the observation, which is considered to be one of the fundamental skills. padilla (1990) even may develop using low-end vr in the science laboratory. next, although there is a common thought that vr may develop a conceptual understanding (lee & wong, 2008), there is still a need for a conclusion in this regard (whitelock, brna, & holland, 1996). the fact that there a limited number of studies examining the variables of ability, teaching experience, psychological effects, creativity, and adverse effects gives the impression that the affective aspect of vr may not be widespread in the research of science education. it hoped that the ongoing diffusion and growing accessibility of vr would result in an emphasis on the outcomes in the affective domain over time. 4. conclusion papers on vr-assisted science education majorly abundant in the peer-reviewed journals, compared to the conference & symposium publications. whereas this can be interpreted as a good indicator, it can also be concerning as the peer-reviewed journal publications require relatively more extensive labor and sufficient experience on the research methodology that is especially emerging researchers could initially lack. that alone can hinder the progression of the literature on vr-assisted science education. in order to ensure the fruitful interventions and practices to emerge, it is suggested that the conferences and symposiums specifically on the use of innovative technologies in science education should be established. besides, emerging researchers and graduate students should be encouraged to tackle this matter and share their insights with researchers of the field in these conferences. the growing emphasis on the vr-assisted science education is evident internationally when looked into the publication dates and country origins of the reviewed studies. concerning the promises of vr in making unreachable instructional practices more accessible in science, it is suggested for the researchers of underdeveloped and developing countries primarily to focus on the use of low-end vr interfaces such as google cardboard in the context of science education. the awareness towards that vr does not have to be utilized through costly and hard-to-reach settings in the classrooms can soothe the potential concerns of the researchers. therefore, this understanding may bolster the literature in terms of feasible outcomes and conclusions from a wide variety of demographical settings. genres and the further classification of them in the name of types reveal an abundance of empirical research and description for potential uses. similarly to the emergence phase of all innovative technologies, the diffusion of vr in the consumer market is still in the establishment phase. once this diffusion saturates, vr thought to be will be more accessible for consumers than ever. as a result of this accessibility, the growing number of empirical research may facilitate the richness of implications concerning vr-assisted science education. however, the scarcity of material development and review research on these topics demands more attention from the researchers. even this review thought to help researchers to address potential gaps in the literature. more extensive review and meta-analysis investigations that have a broader coverage may further benefit to the field in this manner, therefore encouraged. in turn, these implications may guide the emerging research and translate into meaningful contributions to the literature. the situation revealed in the diverse spectrum of terms used as keywords on the research on vr-assisted science education raises concerns regarding the accessibility and establishment of the relevant literature. even for review, it has become a necessity for the researchers to group infrequent and related keywords by the acknowledged terminology. in this respect, to ensure the accessibility of the studies for the target audience(s), emerging research should be encouraged to abide by common terminology, especially concerning the used keywords. the findings regarding used methods examined variables and studied participants revealed an irregularity regarding the methods section of the research on vrjournal of science learning article doi: 10.17509/jsl.v3i3.21906 141 j.sci.learn.2020.3(3).132-142 assisted science education. these fundamental components of the research found to be lacking an explicit and clear statement in most of the reviewed papers, although they are regarded as among the basics of the research methods. in addition to this, the qualitative and mixed methods, examinations on affective domains, and the participants from early-age groups appear to be underemphasized. therefore, it is explicitly stressed that the researchers should rigidly determine their variables and explicitly state research methodologies in their reports. lastly, it is suggested that for the emerging research to focus more on the mixed and qualitative inquiries with the participants of early-age groups and emphasize the affective properties of vr in science education more. lastly, there is no doubt that investigating the reflections of vr-assisted science education practices not only crucial for learners but also vital for science educators. therefore, especially in terms of the pre-service science teachers, the incorporation of innovative technologies in the science courses should be integrated more in the training practices, not only for the sake of the research but also for the practice. the implications of emerging research as the practitioners have been presented for the joint conclusions derived from this review, respectively. these implications ought to facilitate not only the research on the use of vr in science education but also aid science educators to gain valuable insights in terms of using vr in science classrooms. references akçayır, m., & akçayır, g. (2017). advantages and challenges associated with augmented reality for education: a systematic review of the literature. educational research review, 20, 1-11. doi:10.1016/j.edurev.2016.11.002 allcoat, d., & von mühlenen, a. (2018). learning in virtual reality: effects on performance, emotion and engagement. research in learning technology, 26(0). doi:10.25304/rlt.v26.2140 barnett, m., yamagata-lynch, l., keating, t., barab, s. a., & hay, k. e. (2005). using virtual reality computer models to support student understanding of astronomical concepts. journal of computers in mathematics and science teaching, 24(4), 333-356. bortnik, b., stozhko, n., pervukhina, i., tchernysheva, a., & belysheva, g. (2017). effect of virtual analytical chemistry laboratory on enhancing student research skills and practices. research in learning technology, 25(0). doi:10.25304/rlt.v25.1968 bowman, d. a., & mcmahan, r. p. (2007). virtual reality: how much immersion is enough? computer, 40(7), 36-43. doi:10.1109/mc.2007.257 çalışkan, e. (2017). eğitimde yenilikçi teknolojilerin kullanımı açısından türkiye’de son beş yıl içerisinde (2011-2015) tamamlanmış lisansüstü tezlerin incelenmesi. journal of human sciences, 14(1), 496505. doi:10.14687/jhs.v14i1.4169 checa, d., & bustillo, a. (2019). a review of immersive virtual reality serious games to enhance learning and training. multimedia tools and applications, 79(9-10), 5501-5527. doi:10.1007/s11042-019-083489. durukan, a. (2018). sanal gerçeklikle zenginleştirilmiş öğrenme ortamının fen bilimleri öğretmen adayları üzerindeki etkilerinin incelenmesi. (yüksek lisans yayınlanmamış yüksek lisans tezi). van yüzüncü yıl üniversitesi, van. (10214486). durukan, a., temur, a., & artun, h. (2018). fen bilimleri öğretmen adaylarının sanal gerçeklik destekli mikroöğretim uygulamaları: bir akran değerlendirme çalışması. paper presented at the 2. uluslararası türk dünyası eğitim bilimleri ve sosyal bilimler kongresi, antalya. galvan, j. l., & galvan, m. c. (2017). writing literature reviews: a guide for students of the social and behavioral sciences: taylor & francis. hamidi, f., ghorbandordinejad, f., rezaee, m., & jafari, m. (2011). a comparison of the use of educational technology in the developed/developing countries. paper presented at the procedia computer science. hew, k. f., & cheung, w. s. (2010). use of three-dimensional (3-d) immersive virtual worlds in k-12 and higher education settings: a review of the research. british journal of educational technology, 41(1), 33-55. doi:10.1111/j.1467-8535.2008.00900.x inman, c., wright, v. h., & hartman, j. a. (2010). use of second life in k-12 and higher education: a review of research. journal of interactive online learning, 9(1), 44-63. jensen, l., & konradsen, f. (2017). a review of the use of virtual reality head-mounted displays in education and training. education and information technologies, 23(4), 1515-1529. doi:10.1007/s10639-0179676-0 kavanagh, s., luxton-reilly, a., wuensche, b., & plimmer, b. (2017). a systematic review of virtual reality in education. themes in science and technology education, 10(2), 85-119. kim, s. h., lee, j. l., & thomas, m. k. (2012). between purpose and method: a review of educational research on 3d virtual worlds. journal for virtual worlds research, 5(1). doi:10.4101/jvwr.v5i1.2151 lee, e. a.-l., & wong, k. w. (2008). a review of using virtual reality for learning. in z. pan, a. d. cheok, w. müller, & a. el rhalibi (eds.), transactions on edutainment i (pp. 231-241). berlin, heidelberg: springer berlin heidelberg. lee, e. a.-l., wong, k. w., & fung, c. c. (2010). how does desktop virtual reality enhance learning outcomes? a structural equation modeling approach. computers & education, 55(4), 1424-1442. doi:10.1016/j.compedu.2010.06.006. linn, m. c. (2003). technology and science education: starting points, research programs, and trends. international journal of science education, 25(6), 727-758. doi:10.1080/0950069032000076670. liu, d., bhagat, k. k., gao, y., chang, t.-w., & huang, r. (2017). the potentials and trends of virtual reality in education. in d. liu, c. dede, r. huang, & j. richards (eds.), virtual, augmented, and mixed realities in education (pp. 105-130). singapore: springer singapore. ma, j., & nickerson, j. v. (2006). hands-on, simulated, and remote laboratories: a comparative literature review. acm comput. surv., 38(3), 7. doi:10.1145/1132960.1132961. mikropoulos, t. a., & natsis, a. (2011). educational virtual environments: a ten-year review of empirical research (1999– 2009). computers & education, 56(3), 769-780. doi:10.1016/j.compedu.2010.10.020. mora, c. e., martín-gutiérrez, j., añorbe-díaz, b., & gonzálezmarrero, a. (2017). virtual technologies trends in education. eurasia journal of mathematics, science and technology education, 13(1), 469-486. doi:10.12973/eurasia.2017.00626a nalbant, g., & bostan, b. (2006). interaction in virtual reality. paper presented at the 4th international symposium of interactive medial design (isimd). nissim, y., & weissblueth, e. (2017). virtual reality (vr) as a source for self-efficacy in teacher training. international education studies, 10(8), 52. doi:10.5539/ies.v10n8p52. padilla, m. j. (1990). the science process skills. retrieved from https://www.narst.org/publications/research/skill.cfm. scimago lab. (2019, 2019/02/22/). sjr international science ranking. country rankings. https://www.scimagojr.com/countryrank.php?area=3300&categ ory=3304. trindade, j., fiolhais, c., & almeida, l. (2002). science learning in virtual environments: a descriptive study. british journal of educational technology, 33(4), 471-488. doi:10.1111/14678535.00283 journal of science learning article doi: 10.17509/jsl.v3i3.21906 142 j.sci.learn.2020.3(3).132-142 tuysuz, c. (2010). the effect of the virtual laboratory on students’ achievement and attitude in chemistry. international online journal of educational sciences, 2(1), 37-53. whitelock, d., brna, p., & holland, s. (1996). what is the value of virtual reality for conceptual learning? towards a theoretical framework. paper presented at the proceedings of the european conference on artificial intelligence in education, ediçöes colibri, lisboa, portugal. yen, j.-c., tsai, c.-h., & wu, m. (2013). augmented reality in the higher education: students’ science concept learning and academic achievement in astronomy. procedia social and behavioral sciences, 103, 165-173. doi:10.1016/j.sbspro.2013.10.322 young, s., & nichols, h. (2017). a reflexive evaluation of technologyenhanced learning. research in learning technology, 25(0), 13. doi:10.25304/rlt.v25.1998 yumuşak, a., & aycan, ş. (2002). fen bilgisi eğitiminde bilgisayar destekli çalışmanın faydaları; demirci (manisa)'de bir örnek. m.ü. atatürk eğitim fakültesi eğitim bilimleri dergisi, 16, 197-204. a © 2020 indonesian society for science educator 143 j.sci.learn.2020.3(3).143-148 received: 2 january 2020 revised: 9 april 2020 published: 15 july 2020 using hots-based chemistry national exam questions to map the analytical abilities of senior high school students johnsen harta1*, novena tesalonika rasuh1, angela seriang1 1chemistry education study program, faculty of teacher training and education, sanata dharma university, yogyakarta, indonesia *corresponding author: johnsenharta@usd.ac.id abstract high order thinking skills (hots) are a significant part of the implementation of the 2013 curriculum to develop student competencies in the 21st century. not only in the context of learning are hots considered important. they contribute to the national examination held in indonesia, so careful preparation for this exam is needed. chemistry has basic concepts that require students to search and find facts, understand concepts, apply them procedurally, and analyze deeply. this study aims to map the analysis ability of students in yogyakarta in working on hots-based chemistry national exam questions. this research is a quantitative-qualitative descriptive study with xii year students at a senior high school in yogyakarta. the instruments used in this study were hots essay questions, interview sheets, and questionnaire sheets, analyzed with the rasch model. the results show that the students' analysis ability was still relatively low, with person reliability values of 0.56 and 0.39, and a cronbach's alpha of 0.78. the quality of the hots questions was very good, with item reliability 0.97 and 0.95. not many students in the two senior high schools have good analysis ability, so students must be better trained to optimize their potential in hots-based chemistry learning. keywords analysis ability, national exam, chemistry, higher order thinking skills (hots) 1. introduction the national exam is one of the programs organized by the indonesian ministry of education and culture to measure the academic achievements of class xii students who will complete their level of education at senior high school / equivalent level. this program has caused controversy considering the unequal education in indonesia. constraints such as lack of adequate school facilities, not many teachers in several regions, and leakage of national exam answer keys that have occurred have caused the exam to be considered inadequate as a determinant of students' graduation. schools are considered more equitable in deciding the graduation of students who have been undergoing learning and teaching activities. since 2016, the moratorium of the national exam was rejected, so the 2017 national exam up to now has been packaged with a different appearance, for example, senior at high school level in natural sciences class, only tested four subjects namely mathematics, indonesian, english, and one elective subject namely chemistry/biology/physics. in addition, the government has also begun implementing a computer-based national exam throughout indonesia. the exam held contains questions that are based on hots because the hots concept is an important part of the 2013 curriculum in indonesia. dinni (2018) explains that hots include the ability in problem-solving, creative thinking, critical thinking, the ability to reason, and the ability to make decisions. at least 10-15% of hots questions are presented in the computer-based national exam (maulipaksi, 2019), for all subjects tested. the majority of senior high schools in yogyakarta have implemented the 2013 curriculum. demands on the competence of chemistry teachers to be able to train and develop hots for class xii senior high school students in the context of national exam preparation are important because chemistry subject has potential to develop many hots questions that can train and develop students' abilities to reach the hots level which includes analyzing (c4), evaluating (c5), and creating (c6) so that chemistry teachers are expected to be able to apply it at senior high school learning. mailto:johnsenharta@usd.ac.id journal of science learning article doi: 10.17509/jsl.v3i3.22387 144 j.sci.learn.2020.3(3).143-148 the presence of hots brings new nuances and challenges in the chemistry national exam because students must be prepared to work on some hots-based chemistry questions. saputra (2016: 91-92) emphasizes that the main purpose of the emergence of hots is to improve students' thinking abilities at a higher level, especially those related to the ability to think critically in receiving various information, think creatively in solving a problem using the knowledge they have and make decisions in complex situations. the earliest hots level to do is analysis ability. anderson & krathwohl (2001) explained that indicators to measure the ability to analyze (c4) could be investigated through students' efforts in fragmenting several components and connecting these components to obtain a complete understanding of concepts. based on the results of interviews with chemistry teachers in sma a, the initial students' ability of class xii natural science 4 is relatively good, but there are still grades that are below average. in chemistry learning, the teacher has already developed hots and tested the question in the mid-term and end semester assessment, although the number of items given is not much. meanwhile, the chemistry teachers at sma b said that the ability of students of class xii natural science 6 was in the good domain, and the chemistry teacher had given hots questions during the study, but the intensity was not very frequent. the teacher still wants students to finalize their readiness, especially later national exam questions so that they are more maximized in obtaining the final results. both chemistry teachers agreed to carry out trials of the items used in this research. the chemistry teachers revealed that this research could help observe students 'ability to answer questions and investigate the students' analysis ability. politsinsky & demenkova (2015) explain that the process of practicing problem-solving skills is important, so the ability to analyze is considered as an effective part of the teaching process at school. pratama & retnawati (2018) state that students are not only applying what they have learned, but ways to analyze also help to improve the various problems that can be encountered in life. furthermore, it is emphasized that analytical skills in the 21st century are very important to be needed in solving daily problems (facione, 2015). chemistry teachers are required to design learning that can direct students' abilities towards analysis, even though the facts are quite difficult because students need a process to start the process of thinking analysis. this ability can be trained, measured through test instruments developed in the topic of chemistry high school, and conducted a study of the analysis ability of high school students in dealing with hots-based chemistry questions. 2. method this research is a quantitative-qualitative descriptive study that focuses on investigating the analysis ability of senior high school class xii students while working on hots-based chemistry national exam questions. the population in this study were all students of class xii natural sciences at sma a and b in yogyakarta. sampling was done by random sampling technique for each school so that two field trial samples were obtained, namely xii natural science 4 with 34 students from sma a and xii natural science 6 with 30 students from sma b. the instrument used in this study was a validation sheet from two experts from lecturers and two chemistry teachers in sma a and b, hots-based chemistry national exam questions, which consists of five items and a questionnaire sheet to analyze student responses. all figures and tables were attached in the supplementary information. results of the validation from experts and chemistry teachers were analyzed using the aikens' v formula to get the validity coefficient. the results of interviews with chemistry teachers were analyzed descriptively, while the results of the hots-based chemistry national exam essay results and student response questionnaire sheets were analyzed using the rasch model and descriptive. before the research was conducted, a visit to several senior high schools in yogyakarta was chosen, and two classes from two senior high schools were selected as the research sample. this data is used as a preliminary reference to further investigate students' abilities and is processed together with trial results data in each school. 3. result and discussion in mapping the analysis ability of class xii senior high school students in working on hots-based chemistry national exam questions, initial preparations were made in the form of validation of experts and chemistry teachers, trials in two classes, and analysis of test results to investigate students' ability to analyze (reaching c4 level in hots). 3.1 drafting and developing research instruments for measuring analysis ability from the end of july until august 2019 was carried out the preparation and development of research instruments such as validation instrument on a scale of four, an interview sheet with chemistry teachers, test instrument that consists of five hots items that had been selected from the chemistry national exam questions from 2014 to 2018 and modified into chemistry essay questions related to periodic table properties, acid-base titration, salt hydrolysis, and solubility products, voltaic cells, and polymers. thaneerananon, triampo, & nokkaew (2016) emphasize that tests can help develop students' abilities in reflecting and developing critical thinking skills. after that, the student response questionnaire sheet was developed as journal of science learning article doi: 10.17509/jsl.v3i3.22387 145 j.sci.learn.2020.3(3).143-148 a supporting instrument to determine student responses to hots items. 3.2 validating research instruments for measuring analysis ability validation for the research instrument was carried out by two experts from lecturers and two chemistry teachers in sma a and b. based on the results of experts validation, the average acquisition score for the validation of the interview sheet was 3.8, and this indicated the quality of the instrument was classified as good. meanwhile, the results of the validation of experts and chemistry teachers on the five items of essay questions from the aspects of content, construction, and grammar get an average score of 3.9 and the average validity coefficient of 0.96. thus, all items have been declared valid and very good, and deserve to be tested on students of class xii in selected senior high schools. the results of the validation of the student response questionnaire sheet to hots essay items tested were given an average score of 3.8 and the average validity coefficient of 0.94. this instrument was also declared valid and was appropriate to be given after students finished working on hots-based essay questions. 3.3 field trial after validating and qualifying for quality, as many as five hots-based chemistry essay items were tested in class xii natural sciences 4 and xii natural sciences 6. analysis of all components related to items and students' analysis ability on working hots-based chemistry national exam questions was conducted by referring to the rasch model. sumintono & widhiarso (2015: 84) state that the value of information conveyed through the rasch model using the winstep program is comprehensive and very good in the development of items in learning. irawati & mahmudah (2018) emphasize that the analysis ability is the basic ability of hots that must be developed. laksono, rohaeti, suyanta, & irwanto (2017) state that the analysis ability as part of hots has three distinctive indicators, namely the ability to distinguish, organize, and connect. these three indicators are used to investigate the achievement of students' analysis ability and map them. this is in line with what was stated by falvo (2008) that the learning process includes the activities of choosing (in this case, the context is to differentiate, organizing, and integrating) knowledge that occurs during the development stage of students' memory. mapping students’analysis ability of class xii natural science 4 at sma a kusuma, rosidin, abdurrahman, & suyatna (2017) relate the relationship between analytical skills students must have and the competencies that must be achieved in 2013 curriculum which are writing structure and classifying (factual), explaining and analyzing (conceptual), differentiating (procedural), and designing to find (metacognitive). all of these abilities are expected to be implemented in chemistry learning and equip students to pursue more in the fields of science, such as chemistry. the distribution of students' abilities is shown in figure s1 (supplementary information) and shows that the student's position is in the lower area, which indicates that the student's analysis ability still needs to be improved. based on the information in figure s4 (supplementary information), ab students have the highest score to solve these five hots questions, and student j is the student with the lowest score in this trial. in an effort to reach the c4 level in hots, it is necessary to map students' analysis abilities (in terms of differentiating, organizing, and connecting) specifically in each item that can be explained as follows. a. in item number 1, the students' analysis ability is very weak. a total of 12 students do not have the good analytical ability, then 20 students are at a poor level, and only two students are good enough to distinguish information about electron affinity and electronegativity, then rearrange and relate it. weak understanding of student concepts regarding the electronegativity of elements and their properties is still found. reading experiences make it easier for students to master concepts (probosari, widyastuti, sajidan, suranoto, & prayitno, 2018). futhermore, prastiwi & laksono (2018) state that the assessment developed by the teacher could have an opportunity to stimulate students' analytical thinking skills, and the ability of chemistry literacy also influenced the ability to analyze. as in this study, the hots test instrument is able to reveal the level of students' analytical skills, can students have to enrich themselves by reading, understanding, and analyzing in-depth. b. in item number 2, there were still six students who were unable to answer because they did not have competence in analyzing. in addition, the level of unfavorable is still occupied by 18 students, the level is quite good only achieved by three students, the good level is successfully done by six students, and only one student is the best in analyzing this item. the majority of students are still unable to rearrange pieces of information and relate them through calculations involving acid-base titration. c. in item number 3, there are ten students who are not competent in terms of distinguishing, organizing, and connecting two concepts in a single unit. a total of 22 students are at a poor level, and only two students are the most competent at analyzing. vong & kaewurai (2017) reveal that identifying and investigating are just a few of the keys to successful learning. these two parts are certainly part of the ability to analyze students. the majority of students are suspected of being unable to differentiate and relate the concepts of salt hydrolysis and solubility products. d. in item number 4 regarding voltaic cells, only six students were poor in terms of rearranging information and linking it, four students were at a good level, four students journal of science learning article doi: 10.17509/jsl.v3i3.22387 146 j.sci.learn.2020.3(3).143-148 were at a good level, and 20 students dominated at a very good level in analyzing. the majority of students can already work on this item well. analytical thinking involves a series of mental processes needed to find the right solution to overcome various problems so that this ability can be trained and developed during the learning process (anwar & mumthas, 2014). e. in item number 5, there are 31 students who are still incompetent in differentiating, rearranging, and connecting each concept in the polymerization reaction, then two students are at a poor level, and only one student is good enough to analyze this problem. zhang (2003) and yang & lin (2004) explained that identify is like combining informative instruction, so students have a variety of approaches and develop hots abilities, which in this case are specialized in analytical skills. based on the above information, results of the mapping, as seen in the supplementary information (figure s1), and also the ability of students (figure s4), it does seem that the students' analysis ability is still low in hots-based chemistry national exam questions. this is also supported by the value of the person reliability of 0.56 (figure s3), indicating the analysis ability of students who are still weak. anggraini, budiyono, & pratiwi (2018) state that each student has a different way of finding and solving a problem. with this in mind, the inculcation of problemsolving skills in analyzing is needed and must be built. in the chemistry test as many as five essay items for class xii natural science 4, amounting to 34 students had an item reliability value of 0.97 (figure s3), which showed the quality of the items developed was very special. according to the results of the measurement and analysis of item measures in figure s5, it was noted that item 5 had the highest difficulty level (a measure of 1.47), and question number 4 was the easiest problem (measure -1.86) that students could work on. mapping students’ analysis ability of class xii natural science 6 at sma b the distribution of students' abilities is shown in figure s2 (supplementary information) and shows that the student's position is in the lower area, which indicates that the student's analysis ability still needs to be trained and developed again. based on the information in figure s7 (supplementary information), ad students have the highest scores for completing these five hots questions, and x students are the students with the lowest scores in this trial. in an effort to reach the c4 level in hots, it is necessary to map students' analysis abilities (in terms of differentiating, organizing, and connecting) specifically in each item that can be explained as follows. a. in item number 1, the students' analysis ability is very weak. a total of 19 students did not have good analysis abilities in terms of differentiating, organizing, and connecting. meanwhile, as many as 11 students are at a bad level. the students' understanding of concepts is weak and needs to be sharpened in order to be able to distinguish electron affinity and electronegativity of elements, rearrange new concepts, and link the relationships between the two personality traits of these elements. in addition, literacy-based questions are also important to be understood and analyzed further. mahatoo (2012) states that the ability to read carefully and thoroughly and understand the context becomes an important part of analyzing. futhermore, prawita, prayitno, & sugiyarto (2019) explains that generative learning and reading interest can improve analytical thinking skills. b. in item number 2, there were still five students who were unable to analyze the questions because they did not have competency in analyzing. in addition, the level of unfavorable is still occupied by three students, the level of good enough is dominated by 13 students in arranging the information to be done, the good level is only successful in 3 students, and six students are the best in analyzing this item. the majority of students are quite capable of rearranging, but it is still not stable in connecting information and calculations involving acid-base titration. singaravelu (2017) revealed that students' analytical skills are still weak, so it is suggested that teachers can emphasize more problem-based exercises and develop students' problem-solving. c. in item number 3, there are 26 students who are not competent in terms of distinguishing, organizing, and connecting several concepts in the solution of chemistry. as many as two students are at a poor level, and only two students are the most competent at analyzing. politsinsky, demenkova, & medvedeva (2015) state that the ability to analyze is a logical basis that relates the picture, nature, and relationships of each section, which can be separately identified into a new entity. the majority of students are suspected of being unable to differentiate and relate the concepts of salt hydrolysis and solubility products. seventika, sukestiyarno, & mariani (2018) emphasize that the weak ability of students to analyze can be caused by one of them because students solve problems without proper analysis procedures, weak mathematical logic, and cannot apply the concepts needed for analytical skills. d. in item number 4 regarding voltaic cells, there are 19 students dominating at the level of incompetence in rearranging information and linking it, two students who are not good at analyzing, two students are at a good enough level, three students are at a good level, and only four students who are very good at analyzing. students still need a lot of learning to understand, rearrange, and connect them appropriately. e. in item number 5, there are 28 students who are not competent in distinguishing, rearranging, and linking some information relating to the polymerization reaction. there are only two students who are at a poor level in analyzing journal of science learning article doi: 10.17509/jsl.v3i3.22387 147 j.sci.learn.2020.3(3).143-148 this item. students still need to learn a lot about basic concepts and slowly practice analytical skills in learning. organizing information is a process needed to analyze so that a larger collection of information will be obtained and will produce a good understanding (areesophonpichet, 2013). changwong, sukkamart, & sisan (2018) state that problem-solving requires the ability to think critically, and it is stressed here that the ability to analyze and create is at the heart of critical thinking. thus, analytical skills must be mastered and sharpened by students. based on the results of the mapping (figure s2) and students' ability (figure s7), it does seem that the students' analysis ability is still low in hots-based chemistry national exam questions. this is also supported by the value of the person reliability of 0.39 (figure s6), indicating the ability of students who are still weak. yulina, permanasari, hernani, & setiawan (2018) explain that analytical thinking skills expose students to logical thinking and relate to the current situation in terms of categorizing, explaining, comparing, contrasting, and re-examining everything that is done. in the chemistry test as many as five essay items for class xii natural science 6, amounting to 30 students had an item reliability value of 0.95 (figure s6), which showed the quality of the items developed was very special. according to the results of the measurement and analysis of item measures in figure s8 (supplementary information), it was noted that item 5 had the highest difficulty level (a measure of 1.59), and question number 2 was the easiest problem (measure -1.31) that students could work on. the analysis ability of xii students in both schools is still relatively weak, even though the quality of all chemistry items tested to each class has been classified as very good. this needs to be supported by the role of chemistry teachers who must strengthen the basic abilities and develop hots abilities of students during learning at school in order to improve students' abilities in answering chemistry national exam questions. hadi, retnawati, munadi, apino, & wulandari (2018) state that difficulties in analyzing are still experienced by most students. therefore, efforts should be made so that hots can be developed slowly in learning. furthermore, harta (2019: 72) explained that the development of hots in learning is in accordance with the orientation of the 2013 curriculum in the 21st century so that students are more capable of mastering science and its applications. the learning process that is getting better, meaningful, and quality will determine the ability of students to develop competent analysis ability. through the information entered, according to winarti (2015), states that the activity of analyzing starts from structuring information into smaller parts, recognizing a thing, and formulating questions. after students finish working on the hots questions, students are asked to fill in the students' response questionnaire sheet to the item being tested. the majority of students find it quite difficult to analyze problems; students can understand concepts but are hesitant in connecting information. the relationship between data and theory made by students has not been able to provide a related picture that is being studied (saptono, rustaman, saefudin, & widodo, 2016). however, students are challenged to try other items, later on, to enrich their insights and improve their analytical ability. the final results of the analysis of this questionnaire sheet gave cronbach's alpha value of 0.78, which indicates that the consistency of this instrument is quite good. cesarandari, aswandi, & oikurema (2019) state that the ability to analyze will affect other skills such as the ability to present orally can even change a student's bad habits, making students more responsible. in this way, a holistic assessment can be carried out. irwanto, rohaeti, widjajanti, & suyanta (2017) revealed that the analytical thinking skills used in theoretical chemistry learning are very important to be developed and further integrated into chemical experiments, which also require analytical skills. analytical skills are not only important in terms of developing students' abilities in learning, but also contribute to life problems (schumacher & ifenthaler, 2018). 4. conclusion based on the research and discussion that has been done, it can be concluded that the analysis abilities in terms of distinguishing, organizing, and connecting possessed by students of class xii natural science 4 and 6 are still relatively weak. this is evident from the students' person reliability values of 0.56 and 0.39 which indicate the weakness of the analysis ability, even though the quality of hots items tested was valid and feasible to use, as well as the results of the questionnaire analysis with cronbach's alpha value of 0.78 which included both in terms of consistency. the results of the mapping analysis of students' analysis ability in each item also showed the students' weakness in distinguishing some basic chemistry concepts, the difficulty of rearranging information, and being unable to link the information available and the initial knowledge students had. this analysis ability needs to be familiarized and trained during chemistry learning at senior high school, so students are more mature and can improve their analysis competency. acknowledgment the authors would like to say many thanks to the institute for research and community service, sanata dharma university, yogyakarta, chemistry education study program of sanata dharma university, yogyakarta, and also sogang university, south korea. this research was supported by the leading university project for international cooperation through the national research journal of science learning article doi: 10.17509/jsl.v3i3.22387 148 j.sci.learn.2020.3(3).143-148 foundation of korea (nrf) funded by the ministry of education (moe) (grant number= 2015h1a7a2a02037456). references anderson, o. w. & krathwohl, d. r. (2001). a taxonomy for learning, teaching, and assessing (a revision of bloom’s taxonomy of educational objectives). addision wesley longman, inc. anggraini, n. p., budiyono & pratiwi, h. (2018). analysis of higher order thinking skills students at junior high school in surakarta. journal of physics: conference series, 1211(012077), 1-9. anwar, b. & mumthas, n. s. (2014). taking triarchic teaching to classrooms: giving everybody a fair chance. international journal of advanced research, 2(5), 455-458. areesophonpichet, s. (2013). a development of analytical thinking skills of graduate students by using concept mapping. the asian conference on education, 1, 15. cesarandari, s., aswandi & oikurema. (2019). applying the higher order thinking skills (hots) to spoken analytical exposition in oral presentation assessment. iosr journal of research & method in education, 9(2), 1-5. changwong, k., sukkamart, a., & sisan, b. (2018). critical thinking skill development: analysis of a new learning management model for thai high schools. journal of international studies, 11(2), 37-48. dinni, h. n. (2018). hots (high order thinking skills) dan kaitannya dengan kemampuan literasi matematika. prisma, prosiding seminar nasional matematika, 1, 170-176. facione, p. a. (2011). critical thinking: what it is and why it counts. insight assessment, 2007(1), 1-23. falvo, d. (2008). animations and simulations for teaching and learning molecular chemistry. international journal of technology in teaching and learning, 4(1), 68-77. hadi, s., retnawati, h., munadi, s., apino, e., &wulandari, n. f. (2018). the difficulties of high school students in solving higher-order thinking skills problems. problems of education in the 21st century, 76(4), 520-532. harta, j. (2019). kajian kurikulum kimia sma dan smk. deepublish. irawati, t. n. & mahmudah, m. (2018). pengembangan instrumen kemampuan berpikir analisis siswa smp dalam menyelesaikan soal pemecahan masalah matematika. kadikma, 9(2), 1-11. irwanto, rohaeti, e., widjajanti, e., & suyanta. (2017). students’ science process skill and analytical thinking ability in chemistry learning. aip conference proceedings, 1868(030001), 1-4. https://doi.org/10.1063/1.4995100 kusuma, m. d., rosidin, u., abdurrahman, & suyatna, a. (2017). the development of higher order thinking skill (hots) instrument assessment in physics study. journal of research & method in education, 7(1), 26-32. laksono, e. w., rohaeti, e., suyanta, & irwanto. (2017). instrumen penilaian kemampuan berpikir analitis dan keterampilan proses sains kimia. jurnal kependidikan. 1(1), 100-110. mahatoo, j. (2012). scientific literacy and nature of science as it impacts on boys' achievement in south trinidad (doctoral dissertation). maulipaksi, d. (2019, march 27). tingkat kesulitan soal un 2019 tidak berubah, ini komposisi soalnya. kementrian pendidikan dan kebudayaan. https://www.kemdikbud.go.id/main/blog/2019/03/tingkatkesulitan-soal-un-2019-tidak-berubah-ini-komposisi-soalnya politsinsky, e. & demenkova, l. (2015). the organization of the training of technical college students using practice-oriented tasks. asian social science, 11(1), 187-192. politsinsky, e., demenkova, l., & medvedevа, o. (2015). ways of students training aimed at analytical skills development while solving learning tasks. procedia social and behavioral sciences, 206 (1), 383 – 387. pratama, g. s. & retnawati, h. (2018). urgency of higher order thinking skills (hots) content analysis in mathematics textbook. journal of physics: conference series, 1097 (1), 1-8. 10.1088/17426596/1097/1/012147 prastiwi, m. n. b. & laksono, e. w. (2018). the ability of analytical thinking and chemistry literacy in high school students learning. journal of physics: conference series, 1097(1), 1-8. 10.1088/17426596/1097/1/012061 prawita, w., prayitno, b. a., & sugiyarto (2019). effectiveness of a generative learning based biology module to improve the analytical thinking skills of the students with high and low reading motivation. international journal of instruction, 12(1), 1459-1476. probosari, r. m., widyastuti, f., sajidan, s., suranoto, s., & prayitno, b. a. (2018). reading for tracing evidence : developing scientific knowledge through science text. journal of physics: conference series. 1022(1), 1-6. saptono, s., rustaman, n. y., saefudin, & widodo, a. (2016). memfasilitasi hots dalam perkuliahan biologi sel melalui model integrasi atribut asesmen formatif. unnes science education journal, 5(3), 1403-1412. saputra, h. (2016). pengembangan mutu pendidikan menuju era global: penguatan mutu pembelajaran dengan penerapan hots (high order thinking skills). smile’s publishing. schumacher, c., & ifenthaler, d. (2018). features students really expect from learning analytics. computers in human behavior, 78, 397–407. seventika, s. y., sukestiyarno, y. l., & mariani, s. (2018). critical thinking analysis based on facione (2015) – angelo (1995) logical mathematics material of vocational high school (vhs). journal of physics: conf. series, 983(1), 1-6. 10.1088/1742-6596/983/1/012067 singaravelu, s. (2017). problem solving ability of higher secondary chemistry students. iosr journal of research & method in education (iosr-jrme), 7(4), 19-22. sumintono, b. & widhiarso, w. (2015). aplikasi pemodelan rasch pada assesment pendidikan. trim komunikata. thaneerananon, t., triampo, w., & nokkaew, a. (2016). development of a test to evaluate students’ analytical thinking based on fact versus opinion differentiation. international journal of instruction, 9(2), 123-138. vong, s. a., & kaewurai, w. (2017). instructional model development to enhance critical thinking and critical thinking teaching ability of trainee students at regional teaching training center in takeo province, cambodia. kasetsart journal of social sciences, 38(1), 88-95. winarti. (2015). profil kemampuan berpikir analisis dan evaluasi mahasiswa dalam mengerjakan soal konsep kalor. jurnal inovasi dan pembelajaran fisika, 2(1), 19-24. yang, s. c., & lin, w. c. (2004). the relationship among creative, critical thinking and thinking styles in taiwan high school students. journal of instructional psychology, 3(1), 33-45. yulina, i. k., permanasari, a., hernani, h., & setiawan, w. (2018). analytical thinking skill profile and perception of pre service chemistry teachers in analytical chemistry learning. journal of physics: conference series, 1157(4), 1-7. 10.1088/1742-6596/1157/4/042046 zhang, l. f. (2003). contributions of thinking styles to critical thinking. journal of psychology, 137(6), 517-544. a © 2020 indonesian society for science educator 185 j.sci.learn.2020.3(3).185-195 received: 19 january 2020 revised: 29 july 2020 published: 31 july 2020 an analysis of the relationship between students’ scientific attitude and students’ learning style in junior high school ikmanda nugraha1*, nabilah kinanti putri1, hayat sholihin2* 1international program on science education (ipse) study program, faculty of mathematics and science education, universitas pendidikan indonesia, bandung, indonesia 2department of chemistry education, faculty of mathematics and science education, universitas pendidikan indonesia, bandung, indonesia *corresponding author ikmandanugraha@upi.edu abstract the objectives of this study were to profile the relationship of scientific attitude level and learning style preference among junior high school students in bandung. this study utilized a survey research design with a total sample size of 110 students. a scientific attitude questionnaire and a visual, auditory, and kinesthetic (vak) learning style inventory were administered in this study. the questionnaire measures five aspects of scientific attitude, specifically rationality, curiosity, open-mindedness, aversion to superstition, and objectivity. the vak learning style inventory evaluates the preferred means of receiving sensory information. the scientific attitude questionnaire provided consistent results, as indicated by its reliability coefficient (0.896). the results show that junior high school students have an average level of scientific attitude and generally prefer a kinesthetic learning style. there was a medium relationship between scientific attitude and learning style among the students (cramer’s v coefficient = 0.239). it is concluded that learning style must be considered in implementing a science lesson, especially in the indonesian context. keywords scientific attitude, vak learning style, junior high school student 1. introduction science has always been an essential aspect of everyday life. its existences have helped human being to cease their activities, including processing information, especially in the educational process. three primary goals of science education, are including development in science knowledge (cognitive domain), science process skills (psych motive domain), and scientific attitude (affective domain) (ali khan, shah, makhdoom, mahmood, & zareen, 2012). the first two domains have been studied proportionally. however, the assessment of scientific attitude as the affective domain is not as easy as the two others. it’s the number of researches less grow proportionally (punia & bala, 2009). due to the hesitation to use affective measures for grading purposes and the result that develop slowly compared to assessment in the cognitive aspect (krathwol, bloom, & masia, 1964). this lack of assessment in scientific attitude has been considered as the factor that is causing poor scientific orientations among science students, which causes a decrease in several aspects of students’ daily activities in the society, such as productivity, development, and values (zain, samsudin, rohandi, & jusoh, 2010). an affective domain that being discussed in some science education-related literature is concerning attitudes related to science (laforgia, 1988). gardner (1975) suggested two main categories of attitudes related to science; attitudes towards science (e.g., interests in science, attitudes towards scientists, and attitudes towards social responsibility in science) and scientific attitudes (e.g., openmindedness, honesty, skepticism). ozden & yenice (2014) brought out the importance of scientific attitude towards the cognitive process as a must-have skill by science people to reach new knowledge in science. teaching and learning science supposed to be perpendicular to its function and purposes, which is developing a scientific attitude (istikomah, hendrato, & bambang, 2010). oloruntegbe & omoifo (2005) stated that one of the factors that might cause students to have poor scientific attitude orientation is the lack of assessment in students’ journal of science learning article doi: 10.17509/jsl.v3i3.22873 186 j.sci.learn.2020.3(3).185-195 scientific attitudes. teachers should give the students chances to develop their scientific attitude (istikomah, hendratto, & bambang, 2010). teachers could put scientific attitudes through experiments or exploration to give students a chance to develop their scientific attitude because problem-solving skills and scientific attitudes are important things in 21st-century demand. but this importance has again been getting less attention as the learning purpose due to some difficulties found by teachers in designing strategy and document to measure scientific attitude (widowati, nurohman, & anjarsari, 2017). curry & adams (1991) explained that learning style is needed habitually to acquire knowledge, skills, or attitudes. federico (2000) said that by understanding students’ learning styles, students could improve their planning, producing, and implementing educational experiences; thus, the analysis of student attitudes and learning styles will help in designing, developing, and delivering more effective and efficient educational environments. the concept of learning style is a diversity of individual’s preferences towards learning approaches (joy & kolb 2009). students in the classroom may have their preferred learning style. still, those learning styles will generally cover: (1) an attempt to learn maximum knowledge, solely from the lecturer (authority) for later regurgitation or (2) an ongoing commitment to learn and reorganize knowledge, particularly in collaboration with peers and, e.g., the lecturer (el-farargy, 2010). learning style as a concept has been raising interest among professional educators in all education levels. the excitement gets increased regarding its capability of being acceptable to a broader community, not only among the educators but also among parents and the public in general. pashler, mcdaniel, rohrer, & bjork (2009) said that this acceptance could happen because learning style has many offerings such as tests, assessment devices, and technologies to help educators in identifying students learning styles, which in the future can be useful to create adaptive and active learning processes. one of the strategies to facilitate the different learning styles is the use of multimedia and innovative approach (nugraha & eliyawati, 2019; suryawati & osman, 2018). the popularity of learning styles has been proved by a significant number of researchers in the literature trying to describe learning styles from various points of view. coffield, moseley, hall, & ecclestone (2004) stated that there had been enormous reviews related to learning styles over the past years, and the number is still growing. in the united states, a very popular learning styles inventory was developed by kolb in 1984. kolb’s learning styles inventory has two main dimensions: preferred mode of perception (concrete to abstract) and preferred mode of processing (active experimentation to reflective observations), with four classifications: divergers (concrete, reflective), assimilators (abstract, reflective), convergers (abstract, active), and accommodators (concrete, active) (pashler, mcdaniel, rohrer, & bjork, 2009). another learning style questionnaire (lsq) was developed by honey and mumford in 1986 by modifying kolb’s. the lsq classified learners into activists (competitive activities and respond well to challenges), reflectors (need time to prepare in advance), theorists (required to understand complex problems), and pragmatists (seek distinct advantages to learning a given task) (shaw & marlow, 1999). vaishnav (2013) stated that some students might learn best by visual, auditory, and kinesthetic. these three learning styles said as the most popular one and a favorite one in the learning community due to its benefits in providing a broader perspective of students’ dominant thinking, learning style, and strengths. yet, this visual, auditory, and kinesthetic (vak) learning style doesn’t overlay gardner’s theory of multiple intelligence or kolb’s theory (gholami & bagheri, 2013; vaishnav, 2013). visual, auditory, and kinesthetic (vak) learning style models were developed by dunn & dunn (sanni & emeke, 2017). they developed the vak learning style model, which focuses on three main sensory receivers to determine dominant learning. students with visual learning styles will tend to write what they learned, remembering the shape and color of what they observed and remembered faces easily. students with auditory learning styles will choose to speak what they learned aloud or to themselves, love to listen to music but easily get distracted by noise, and remember names better than faces. while students with kinesthetic learning styles will show the tendency to learned better from what they have done, love to do physical activities, and found it hard to make them sit still. based on the theory, one or two learning styles will appear dominantly on learners. the domination shows the best way of learners in filtering information, and it might be changed due to the different tasks given. one’s may prefer a learning style on one task and combine two learning styles on another (penger & tekavcic, 2009; magulod, 2019; weng, ho, yang, & weng, 2019). the relationship between students’ scientific attitudes and students’ learning styles has been studied by some researches. kant & singh (2015) argued that science students have different learning styles, and achievement in science subjects was not significantly different in some groups of learning styles, but it was significant in the others. while students were having more and less scientific attitudes of various categories of learning style, the learning styles were not substantial overall, but, in some cases, they were significantly different. sanni & emeke (2017) showed that age, extroversion, sensing, thinking, and kinesthetic had direct effects on biology achievement, and gender, age, and thinking had indirect effects on biology achievement. pitafi & farooq (2012) showed that secondary school students’ scientific attitude was moderately scientific about journal of science learning article doi: 10.17509/jsl.v3i3.22873 187 j.sci.learn.2020.3(3).185-195 the element “curiosity” and secondary school students’ scientific attitude was slightly scientific about the element rationality, willingness to suspend judgment, openmindedness, critical mindedness, objectivity, honesty, and humility. students have a good attitude when the science lesson is interesting (kurniawan, astalini, & sari, 2019). ataha & ogumogu (2013) found that the level of students’ scientific attitude was average, and there is no significant difference between the students’ scientific attitude acquired by male and female students. and vaishnav (2013) revealed that the kinesthetic learning style was found to be more prevalent than visual and auditory learning styles among secondary school students. there exists a high positive correlation between kinesthetic learning style and academic achievement. moreover, there was a significant association between academic performance and the reading/writing learning style preference (akhlaghi, mirkazemi, jafarzade, & akhlaghi, 2018). thus, to reduce the lack of assessment in affective domain and to help students in acquiring scientific attitude, this research brings the urge to develop a suitable measurement of students’ scientific attitude, analyzing students’ learning style, and finding the possible relationship between them into an analysis of students’ scientific attitude and students learning style survey. elaborating on the research problem, the research attempts to explore the following questions: 1) how is the profile of students’ scientific attitudes in junior high school? 2) how is the profile of students’ learning styles in junior high school? and 3) how is the relationship between students’ scientific attitudes and students’ learning styles? 2. method since this research included as non-intervention, with the purpose is to describe the tendency of a population, a survey research design was employed. this research addressed two questionnaires to profile the level of students’ scientific attitude and to analyze students’ learning styles. the data were collected through an online platform. students’ scientific attitude questionnaire has a reliability coefficient of 0.896, which means that the scientific attitude questionnaire has gained its consistency. the questionnaire consists of 45 statements, with a four-scale likert scale. table 1 shows the total statements being included in the final form of the scientific attitude questionnaire. curiosity aspect has the largest number of statements because the aspect consists of more indicators than another aspect. also, the aversion to superstition aspect has the smallest number of statements because the aspect consists of only two indicators. each statement is paired with a four-scale likert scale. the vak learning style inventory employed in this research was the vak learning style inventory developed by victoria chislett and alan chapman in 2005. this vak learning style inventory was constructed by 30 statements with multiple choices. the vak learning style inventory has a reliability coefficient of 0.767, which means that the scientific attitude questionnaire has gained its consistency. each statement had three different choices that each choice was representing a sensory receiver. table 2 explains that choice ‘a. read the instruction first’ is representing visual sensory receiver by it statements that ‘reading’ as the chosen word in the statement. while choice ‘b.’ represents auditory sensory receiver as the word ‘listen’ applied in the statement, and choice ‘c.’ represents visual sensory receiver as the word ‘try’ applied in the statement. those choices are always representing one sensory receiver; ‘a.’ for the visual sensory receiver, ‘b.’ for the auditory sensory receiver, and ‘c.’ for the kinesthetic sensory receiver. the population was students of public junior high school in bandung in the 2018/2019 academic year; the targeted population was the 9th-grade students of junior high school in bandung in the 2018/2019 academic year. there were eleven classes with a total number of 9th-grade table 1 items of scientific attitude questionnaire no dimension number of statements 1 rationality 11 2 curiosity 15 3 open-mindedness 8 4 aversion to superstition 5 5 objectivity 6 total 45 table 2 example of choices in vak learning style inventory that represent each sensory receiver no. statement choice sensory receiver 1. when i operate new equipment i generally: a. read the instruction first visual b. listen to an explanation from someone who has used it before auditory c. go ahead and have a go, i can figure it out as i use it kinesthetic table 3 range of students’ scientific attitude point and its level range level 150 – 200 high 100 – 149 average 50 – 99 low journal of science learning article doi: 10.17509/jsl.v3i3.22873 188 j.sci.learn.2020.3(3).185-195 students are 418 students. a simple random sampling was used to determine the sample; the sampling allows every member of the population has an equal and independent chance of being selected (fraenkel, wallen, & hyun, 2013). thus, the samples are randomly chosen to choose ten students from every class that made the total sample size of 110 students. to obtain the result from the scientific attitude questionnaire and analyzed the profile of students’ scientific attitude, points gained by the students first need to accumulate. students who have answered all 45 statements in the scientific attitude questionnaire will gain the minimum point of 45 and the maximum point of 180— the range of the point presented in table 3. the range written in table 3 is the range of points gained by students when they finished answering all the 45 statements in the scientific attitude questionnaire. for example, if a student gained 132 points from all the statements, he/she answered from the questionnaire, he/she will be profiled as having an average scientific attitude level. to be able to analyze the profile of students’ learning style, all of the choices made by students need to be summed up. after answering all the 30 statements of vak learning style inventory, students who have mostly choose a. b. or c. the choice will be analyzed if a student's choice is mostly a. means that the student has visual learning style if a student choice is mostly b. means that the student has auditory learning style, and is a student choice are mostly c. means that the student has a kinesthetic learning style. the interpretation displayed in table 4. in order to analyze the relationship between the two variables, microsoft excel and ibm spss software were applied to analyze, calculate, and find the possible relationship between the two variables. the microsoft excel run to convert raw data gained through an online questionnaire, analyze the profile of students’ scientific attitude and students’ learning style, and also to visualize the data. at the same time, ibm spss software was run to find the possible relationship between two variables using the crosstabs chi-square technique and cramer’s v value. the chi-square technique applied to analyze whether a relationship or association existed between two variables because the data collected through the questionnaire were not nominal data (king, rosopa, & minium, 2011). scientific attitude questionnaire resulted in ordinal data (low, average, high), and the vak learning style inventory resulted in categorical data (visual, auditory, kinesthetic). cramer’s v value was applied to determine the power or strength that occurs in the relationship between variables (gravetter & wallnau, 2009). table 4 analysis and interpretation of students’ answers in vak learning style inventory students’ answers interpretation mostly ‘a’ visual learner mostly ‘b’ auditory learner mostly ‘c, kinesthetic learner table 5 mean score of students’ scientific attitude n minimum maximum mean s.d decision scientific attitude 110 101 180 146.77 16.008 average scientific attitude figure 1 analysis of student’s responses within each aspect of scientific attitude questionnaire 0% 20% 40% 60% 80% 100% rationality curiosity open-mindedness aversion to supertition objectivity student's response s c ie n ti fi c a tt it u d e strongly disagree disagree agree strongly agree journal of science learning article doi: 10.17509/jsl.v3i3.22873 189 j.sci.learn.2020.3(3).185-195 3. result and discussion 3.1. profile of students’ scientific attitude 3.1.1. level of students scientific attitude the data collected were analyzed by using spss software to obtain a mean score of students’ scientific attitude level in junior high school in bandung. to answer the research question, “how is the profile of students’ scientific attitude in junior high school?” a summary of the mean score of students’ scientific attitude is presented in table 5. table 5. shows that out of 110 samples, the lowest point of scientific attitude gained by students is 101, and the highest is 180. the value of mean shows that most of the students in junior high school gained 146.77 scientific attitude points; thus, it can be decided that students in junior high school obtained the average scientific attitude level. this finding is supported by the previous research, which reported that the scientific attitudes of students in pakistan secondary schools were moderate (pitafi & farooq, 2012). a similar finding also indicated that secondary school students have a scientific attitude at the level average (ozden & yenice, 2014). ataha & ogumogu (2013) argued that the average level obtained by students is not enough to fulfill the needed scientists in order to scientifically and technologically developing the society to be ready to compete with another country. ozden & yenice (2014) stated that the basic science and technology education provided was not very influential to student’s scientific attitudes and values. thus, to develop programs that will increase students’ scientific attitude of students in secondary school is needed (ataha & ogumogu, 2013). figure 2 students’ response towards indicators in rationality aspect 0% 20% 40% 60% 80% 100% commitment to rationality in problem solving belief in science as means of influencing environment awareness of falability of human effort challenge of authority seeking for natural causes of events and identification of cause and effect students' response r a ti o n a li ty a s p e c t strongly disagree disagree agree strongly agree figure 3 students’ response towards indicators in curiosity aspect 0% 20% 40% 60% 80% 100% desire for new knowledge or ideas desire for additional information seeking for evidence to support conclusions made from scientific materials expression of interest in scientific discoveries students' response c u ri o s it y a s p e c t strongly disagree disagree agree strongly agree journal of science learning article doi: 10.17509/jsl.v3i3.22873 190 j.sci.learn.2020.3(3).185-195 3.1.2. aspects analysis of students scientific attitude the scientific attitude was constructed by 5 aspects; rationality, curiosity, open-mindedness, aversion to superstition, and objectivity. thus, the analysis of students’ responses within each aspect of the scientific attitude questionnaire was necessary to be analyzed. the analysis is summed up in figure 1. figure 1. shows students’ responses in each statement of the scientific attitude questionnaire. it shows that students mostly strongly agree with all statements. even in the objectivity aspect, 60% of students state strongly agree toward the statements within the aspect dominating all three other degrees of agreement. while in the curiosity aspect, only 40% of students state strongly agree towards the statements within the aspect. thus, it can be concluded that students in junior high school in bandung are having more objectivity and curiosity in their profile of scientific attitude. pitafi & farooq (2012) also found that students were highly scientific in the objectivity aspect and moderately scientific in the curiosity aspect. 3.1.3. indicators analysis of students’ scientific attitude the scientific attitude questionnaire consists of five aspects; rationality, curiosity, open-mindedness, aversion to superstition, and objectivity. to be able to profile the students’ attitudes within each aspect, there were indicators constructed in each aspect. figure 4 students’ response towards indicators in open-mindedness aspect 0% 20% 40% 60% 80% 100% willingness to subject data and opinion to criticism and evaluation to others willingness to consider new evidence rejection of singular rigid approach to people, things, and ideas students' response o p e n -m in d e d n e s s a s p e c t strongly disagree disagree agree strongly agree figure 5 students’ response towards indicators in aversion to superstition aspect 0% 20% 40% 60% 80% 100% rejection of supertitious beliefs preference for scientific explanations students' response a v e rs io n t o s u p e rt it io n a s p e c t strongly disagree disagree agree strongly agree journal of science learning article doi: 10.17509/jsl.v3i3.22873 191 j.sci.learn.2020.3(3).185-195 3.1.3.1. rationality rationality aspect consists of five aspects; there are; commitment to rationality in problem-solving, belief in science as means of influencing the environment, awareness of the fallibility of human effort, the challenge of authority, and seeking natural causes of events and identification of cause and effect. in figure 2, it shows that there are 64% and 63% of students are strongly agree toward the statements representing ‘awareness of the fallibility of a human effort’ and ‘seeking for natural causes of events and identification of cause and effect’ indicators, respectively. while in ‘belief in science as a means of influencing environment’ indicator, only 40% of the students are strongly agree toward the statements representing the indicator. as it has been explained, it can be stated that the ‘awareness of the fallibility of a human effort’ and ‘seeking for natural causes of events and identification of cause and effect’ indicators are more able to profile students’ scientific attitude in rationality aspect compare to other indicators within the aspect. in their finding, pitafi & farooq (2012) found that students have a high scientific attitude in rationality aspect while lacap (2015) found that students were having a moderate scientific attitude, which means that students were less committed in their identification of cause and effect relationship done by nature or human being. 3.1.3.2 curiosity curiosity aspect consists of five aspects; there are; desire for new knowledge or ideas, desire for additional information, seeking evidence to support conclusions made from scientific materials, expression of interest in scientific discoveries, and desire for explanations. in figure 3, it shows that there is 54% of students that are strongly agree toward the statements representing ‘desire for knowledge or ideas’ indicator. while in ‘expression of interest in scientific discoveries’ indicator, there is the almost equally same number of students in each degree of agreement. as has been explained, it safe to state that the ‘desire for knowledge or ideas’ indicator is more able to profile students’ scientific attitude in curiosity aspect compare to other indicators within the aspect. lacap figure 6 students’ response towards indicators in objectivity aspect 0% 20% 40% 60% 80% 100% preference for statements supported by evidence over unsupported ones sensitivity to accuracy data preference for scientific generalization that have withstood the rest of critical review students' response o b je c ti v it y a s p e c t strongly disagree disagree agree strongly agree table 6 crosstabs calculation between students’ scientific attitude and students’ learning style learning style visual auditory kinesthetic scientific attitude average number 18 10 28 percentage 32.1% 17.9% 50% high number 11 21 22 percentage 20.4% 38.9% 40.7% table 7 chi-square test of the relationship between students’ scientific attitude and students’ learning style value df asymptotic significance (2-sided) pearson chi-square 6.279a 2 .043 likelihood ratio 6.381 2 .041 table 8 symmetric measures value approximate signficance nominal by nominal phi .239 .043 cramer’s v .239 .043 journal of science learning article doi: 10.17509/jsl.v3i3.22873 192 j.sci.learn.2020.3(3).185-195 (2015) also found that students. curiosity was high, and students were showing good traits as science students. while pitafi & farooq (2012) found that students’ curiosity is moderately scientific compare to another aspect of scientific attitude. 3.1.3.3. open-mindedness open-mindedness aspect consists of three aspects; there are; rejection of a singular rigid approach to people, things, and ideas, willingness to consider new evidence, and willingness to subject data and opinion to criticism and evaluation to others. in figure 4, it shows that there is 56% of students that are strongly agree toward the statements representing ‘willingness to subject data and opinion and evaluation to others’ indicator. while there is only 3% of the students that are strongly disagree toward statements representing ‘willingness to consider new evidence’ indicator. as it has been explained, concluded that the ‘willingness to subject data and opinion and evaluation to others’ indicator is more able to profile students’ scientific attitude in openmindedness aspect compare to other indicators within the aspect. lacap (2015) found that students with high openmindedness aspects of scientific attitude determined their good attitude to respect and listened to others' ideas and criticism to accept reliable evidence. students also had a high willingness to learn new things. 3.1.3.4. aversion to superstition aversion to the superstition aspect consists of two aspects, and there are; rejection of superstitious beliefs and preference for scientific explanations. figure 5 shows that there is 20% of students that are strongly disagree with the statements representing the ‘rejection of superstitious beliefs’ indicator. while there are only 9% of the students that are strongly disagree toward statements representing ‘preference for scientific explanations’ indicator. lacap (2015) also found that students were highly scientific towards this aspect, but this aspect was not significantly related to students’ performance. as it has been explained, it is safe to state that the ‘preference for scientific explanations’ indicator is able to profile students’ scientific attitude in aversion to superstition aspect compare to other indicators within the aspect. 3.1.3.5 objectivity the objectivity aspect consists of three aspects, and there are; preference for statements supported by evidence over unsupported ones, sensitivity to accuracy data, and preference for scientific generalization that have withstood the test of critical review. figure 6 shows that the indicators almost have a similar amount in strongly agree as the degree of agreement. while lacap (2015) found that students were scientifically high in this aspect, pitafi & farooq (2012) found that students are only moderately scientific in this aspect. as has been explained, it safe to state that the statements made for objectivity indicator are able to profile students’ scientific attitude. 3.2. profile of students’ learning style to answer the third research question: “how is the profile of students’ learning style in junior high school?” the processes were needed to be broken down as followed. students’ learning style preference in this research was measured using the vak learning style inventory by victoria chislett & alan chapman. the inventory consists of 30 statements with multiple choices in each statement. the choices are written in a, b, and c options. those alphabets represent each learning style. students with more “a” options preference tend to be visual learners, those who choose more “b” options preference tend to be auditory learners, and those with more “c” options preference tend to be kinesthetic learners. to answer the research question, “how is the profile of students’ learning style in junior high school.” the data of the vak learning style inventory were analyzed and shown in figure 7. figure 7 shows that students who prefer to learn in visual learning style are only 26% of the total sample, it means that students mostly prefer to choose ‘a’ option as their answer in the vak learning style inventory. the following are students who prefer to learn in auditory learning style by 28% of the total sample, by having auditory learning style preference, it means that students mostly prefer to choose ‘b’ option as their answer in the vak learning style inventory. while students’ with kinesthetic learning style dominated overall students with 46% means that they mostly prefer to choose the ‘c’ option as their answer in the vak learning style inventory. previous research by vaishnav (2013) also resulted that kinesthetic learners are analyzed more than visual and auditory learners in the research. 3.3. relationship between students’ scientific attitude and students’ learning style to answer the last research question: “how is the relationship between students’ scientific attitude and figure 7 profile of students’ learning style percentage 26% 28% 46% visual auditory kinesthetic journal of science learning article doi: 10.17509/jsl.v3i3.22873 193 j.sci.learn.2020.3(3).185-195 students’ learning style in junior high school?” the processes were needed to be breaking down in the form of descriptive statistics. the relationship was analyzed through the relationship from each level of students’ scientific attitude towards students’ learning style, and the relationship from each aspect of scientific attitude towards learning style. 3.3.1 relationship between level of students scientific attitude and students’ learning style first, table 6 shows the relationship between the level of students’ scientific attitude (low, average, high) and students’ learning style (visual, auditory, kinesthetic). as shown in table 6, students with kinesthetic learning styles are always dominating both levels of scientific attitude. half of the students in the average level of scientific attitude are kinesthetic learners, and 40.7% of students in high level of scientific attitude are also kinesthetic learners. it might be caused by indicators of scientific attitude that asked students to do more than just observe, but also doing or moving toward something that makes the kinesthetic learners have more dominant performance in scientific attitude. to be able to analyze whether there is a significant statistical relationship between two variables, a chi-square test needs to be conducted. these crosstabs with the chisquare technique measure the relationship between students’ learning style preference with their level of scientific attitude. the result shows in table 7. the result of the chi-square test between the two variables shows in table 7 shows that the value of pearson chi-square between the two variables is 6.279. to be able to describe whether the two variables are associated, the counted value of pearson chi-square needs to be bigger than the table value of pearson chi-square. the table value of pearson chi-square shows the value of 5.995 with df=2. thus, it can be concluded that the counted value of pearson chi-square is bigger than the table value of pearson chi-square, 6.279 > 5.995, or in other words, it can be concluded that there is a statistical association between the two variables. to be able to identify the strength of association, asymmetric measure to determine the cramer’s v coefficient is needed. the result is shown in table 8. table 8 shows that the value of cramer’s v coefficient is .239. the strength of an association determined by cramer’s v association has a range between 0 to 1. if the coefficient is close to zero, then there is no association between the measured variables, and if the coefficient is close to one, then there is an association occurs between the measured variables. by knowing df = 2 from the chi-square test result, it can be seen that the strength of association between students’ scientific attitude and students learning style is a medium association, because the calculated cramer’s v coefficient is .239 or bigger than .21. thus, it can be concluded that there is a medium association between student's scientific attitude and students’ learning style of junior high school in bandung. the medium association means that student's scientific attitude and students’ learning style must be considered during the science lesson to facilitate the unique and difference of student’s learning style and attitude. this difference between learning style and attitude can be explained by the fact that learning styles and attitudes of students are an inner structure that develops from life experiences (kolb, 1984). 3.3.2. relationship between aspects of scientific attitude and learning style another relationship that needs to be found in this research is the tendency of students’ learning style in each table 9 tendency of students learning style in each aspect of scientific attitude learning style visual auditory kinesthetic scientific attitude rationality number 29 31 50 percentage 26.36% 28.18% 45.45% curiosity number 29 32 49 percentage 26.36% 29.09% 44.54% openmindedness number 29 30 51 percentage 26.36% 27.27% 46.36% aversion to superstition number 29 31 50 percentage 26.36% 28.18% 45.45% objectivity number 29 31 50 percentage 26.36% 28.18% 45.45% figure 8 distribution of students’ prefered learning style in each aspect of scientific attitude 29 29 29 29 29 31 32 30 31 31 50 49 51 50 50 25 30 35 40 45 50 55 0 1 2 3 4 5 l e a rn in g s ty le f re q u e n c y aspects of scientific attitude v a k journal of science learning article doi: 10.17509/jsl.v3i3.22873 194 j.sci.learn.2020.3(3).185-195 aspect of scientific attitude. data on the relationship summed up in table 9. table 9 shows the tendency of students’ learning style in each aspect of scientific attitude. students who prefer to learn in visual learning style in every aspect of scientific attitude share the same frequency, which is 29 persons and 26.36% from the total sample. while for students who prefer to learn in auditory learning style are tend to be varied in each aspect. open-mindedness aspect tends to have the lowest frequency of students who prefer to learn in auditory learning style, the frequency is 30 persons or 30% of the total sample, while in curiosity aspect students who prefer to learn in auditory learning style tend to be more than others, which the frequency is 32 persons or 29.09% of the total sample. in rationality, aversion to superstition, and objectivity aspects, they share the same frequency, which is 31 persons or 28.18% of the total sample prefer to learn in auditory learning style. same as auditory learner, students who prefer to learn in kinesthetic learning style ten to be variously distributed in each aspect of scientific attitude. the frequency is largely distributed in the openmindedness aspect with 51 persons or 46.36% of the total sample, and few are distributed in the curiosity aspect with 49 persons or 44.54% of the total sample. for another three aspects, which are; rationality, aversion to superstition, and objectivity aspects, they share the same frequency, which is 50 persons or 45.45% of the total sample. the distribution is shown in figure 8. it can also be said that students who prefer to learn with kinesthetic learning styles are mostly distributed in every aspect of scientific attitude. while students who prefer to learn in auditory learning style are moderately distributed in every aspect of scientific attitude, and students who prefer to learn in visual learning style are having the same amount of frequency in every aspect of scientific attitude and tend to be least distributed in every aspect of scientific attitude. since teaching intervention given in accordance with the learning styles affect the attitude levels towards the lesson (white, 1999), the lessons that will be given during the learning activities, stages should consider the learning styles of students in order to increase the attitude levels towards the lessons and hence the success of the students. 4. conclusion it is concluded from the study that students in junior high school in bandung were profiled as having average and high level of scientific attitude. in the relationship between the level of students’ scientific attitude and students’ learning style, it was found that 50% of students with average level of scientific attitude were preferred to learn with kinesthetic learning style, 32.1% were preferred to learn with visual learning style, and 17.9% were preferred to learn with auditory learning style. meanwhile, in the relationship between aspects of scientific attitude and students’ learning style, it was found that in all aspects of scientific attitude, 26.36% of students were prefer to learn with visual learning style. in rationality, aversion to superstition, and objectivity aspects, 28.18% of students preferred to learn with auditory learning style, and 45.45% preferred to learn with kinesthetic learning style. while for curiosity and open-mindedness aspect, the distribution was found to be different. in curiosity aspect students who prefer to learn with auditory learning style was 29.09% which is higher than other aspect and those who prefer to learn with kinesthetic learning style was 44.54% which is lower than another aspect. finding in open-mindedness aspect shows the opposite from the curiosity aspect where 27.27% of students preferred to learn with auditory learning style, which is lower than another aspect. in comparison, 46.36% of students preferred to learn with kinesthetic learning style, which is higher than another aspect of scientific attitude. references akhlaghi, n., mirkazemi, h., jafarzade, m., & akhlaghi, n. (2018). does learning style preferences influence academic performance among dental students in isfahan, iran? journal of educational evaluation for health professions, 15(8), 1–6. ali khan, s., shah, a., makhdoom, s., mahmood, z., & zareen, r. (2012). scientific attitude development at secondary school level: a comparison between methods of teaching. language in india, 12(9), 439–454. ataha, u. c., & ogumogu, a. e. (2013). an investigation of the scientific attitude among science students in senior secondary schools in edo south senatorial district, edo state. journal of education and practice, 4(11), 12–17. coffield, f., moseley, d., hall, e., ecclestone, k., & ecclestone, k. (2004). learning styles and pedagogy in post-16 learning: a systematic and critical review. curry, l., & adams, c. (1991). patterns of learning style across selected medical specialties. educational psychology, 11(3), 37–41. el-farargy, n. (2010). the views, attitudes and learning style preferences of higher national chemical engineering students. education for chemical engineers, 5(4), 55–71. federico, p. (2000). learning styles and student attitudes toward various aspects of network-based instruction. computers in human behavior, 16, 359–379. fraenkel, j. r., wallen, n. e., & hyun, h. h. (2013). how to design and evaluate research in education. new york: mcgraw-hill. gardner, p. l. (1975). attitudes to science: a review. studies in science education, 2, 37–41. gholami, s., & bagheri, m. s. (2013). relationship between vak learning styles and problem-solving styles regarding gender and students’ fields of study. journal of language teaching and research, 4(4), 700–706. gravetter, f. j., & wallnau, l. b. (2009). statistics for the behavioral sciences. california: wadsworth cengage learning. istikomah, h., hendratto, s., & bambang, s. (2010). penggunaan model pembelajaran group investigation untuk menumbuhkan sikap ilmiah siswa. jurnal pendidikan fisika indonesia, 6(1). joy, s., & kolb, d. a. (2009). are there cultural differences in learning style?. international journal of intercultural relations, 33, 69–85. kant, r., & singh, m. d. (2015). relationship between learning styles and scientific attitude of secondary school students and their achievement in science subject. journal of educational sciences & psychology, 5(1), 01–10. journal of science learning article doi: 10.17509/jsl.v3i3.22873 195 j.sci.learn.2020.3(3).185-195 king, b. m., rosopa, p. j., & minium, e. w. (2011). statistical reasoning in the behavioral sciences. new jersey: john wiley & sons, inc. kolb, a. d. (1984). experiential learning experience as the source of learning and development. new jersey: prentice hall. krathwol, d. r., bloom, b. s., & masia, b. b. (1964). taxonomy of educational objectives book 2 affective domain. new york: longman inc. kurniawan, d. a., astalini, & sari, d. k. (2019). an evaluation analysis of students’ attitude towards physics learning at senior high school. jurnal penelitian dan evaluasi pendidikan, 23(1), 26–35. lacap, m. p. (2015). the scientific attitudes of students major in science in the new teacher education curriculum. asia pacific journal of multidisciplinary research, 3(5), 7–15. laforgia, j. (1988). the affective domain related to science education and its evaluation. science education, 72(4), 407–421. magulod, g. c. (2019). learning styles, study habits and academic performance of filipino university students in applied science courses: implications for instruction. journal of technology and science education, 9(2), 184–198. nugraha, i., & eliyawati. (2019). the use of video laboratory report to develop presentation skills in science teacher education students. journal of physics: conference series. 1157(2), 1-6. oloruntegbe, k., & omoifo, c. (2005). development and validation of an instrument of scientific attitude that is curriculum based. african journal of curriculum studies, 2(2), 147–153. ozden, b., yenice, n., & an, n. (2014). an analysis of the secondary education students ’ scientific attitudes. international journal of contemporary educational research, 1(2), 86–97. pashler, h., mcdaniel, m., rohrer, d., & bjork, r. (2009). learning styles; concepts and evidence. a journal of the association for psychological science, 9(3), 105–119. penger, s., & tekavcic, m. (2009). testing dunn & dunn’s and honey & mumford’s learnins style theories: the case of the slovenian higher education system. management, 14(2), 1–20. pitafi, a. i., & farooq, m. (2012). measurement of scientific attitude of secondary school students in pakistan. academic research international, 2(2), 379–392. punia, v., & bala, r. (2009). scientific attitude amongst the science and non-science pupil teachers: a comparative analysis. srde’s higher education journal, 1, 9-16. sanni, k. t., & emeke, e. a. (2017). direct and indirect effects of personality type and learning style preferences on students’ achievement in senior secondary school biology in osun state, nigeria. european journal of education studies, 3(11), 825–839. shaw, g., & marlow, n. (1999). the role of student learning styles, gender, attitudes and perceptions on information and communication technology assisted learning. computer & education, 33, 223–234. suryawati, e., & osman, k. (2018). contextual learning: innovative approach towards the development of students’ scientific attitude and natural science performance. eurasia journal of mathematics, science and technology education, 14(1), 61–76. vaishnav, r. (2013). learning style and academic achievement of secondary school students. voice of research, 1(4), 1–4. weng, f., ho, h. j., yang, r. j., & weng, c. h. (2019). the influence of learning style on learning attitude with multimedia teaching materials. eurasia journal of mathematics, science and technology education, 15(1), 1–9. white, j. (1994). individual characteristics and social knowledge in ethical reasoning. psychological reports, (75), 627-649. widowati, a., nurohman, s., & anjarsari, p. (2017). developing science learning material with authentic inquiry learning approach to improve problem solving and scientific attitude. jurnal pendidikan ipa indonesia, 6(1), 32–40. zain, a. n. m., samsudin, m. a., rohandi, r., & jusoh, a. (2010). using the rasch model to measure students' attitudes toward science in" low performing" secondary schools in malaysia. international education studies, 3(2), 56-63. a © 2020 indonesian society for science educator 106 j.sci.learn.2020.3(2).106-116 received: 27 desember 2019 revised: 9 march 2020 published: 18 march 2020 the correlation between social media usage in academic context and self-efficacy towards tpack of prospective science teachers in indonesia heru setiawan1,2*, shane phillipson2 1sma global mandiri jakarta, jakarta, indonesia 2faculty of education, monash university, victoria, australia *corresponding author. hset0001@gmail.com abstract the purpose of this study is to investigate the relationship between the frequency of social media usage (smu) in an academic setting and self-efficacy beliefs towards tk, tck, tpk, and tpack of indonesian prospective science teachers. this research is quantitative based-research design using a self-administered survey. the research was conducted during the second semester of the academic year 2018/2019 from october to november 2018 in the faculty of mathematics and natural science of a state university located in semarang city, indonesia. the sample consists of 217 indonesian prospective science teachers from the science and biology education department. the result shows that the average social media usage frequency has a statistically high correlation with tk self-efficacy, tpk, and tpack. however, in general, it does not correlate with tck. second, social media usage for download media (dm), searching information (si), and entertainment and motivation (em) generally have a statistically medium correlation with tk, tpk, and tpack for both male and female participants. third, social media usage for professional development (pd) has a medium correlation with tk and high correlation with tpk and tpack. this study implies that training focuses on the application of social media in teaching, and learning should be integrated to improve indonesian prospective science teachers' tpack. keywords social media usage, self-efficacy, tpack, prospective science teacher, indonesia 1. introduction professional science teachers are crucial to improve the quality of education in indonesia. they must have four competencies, including pedagogy, personality, professional, social, where they should not only master in their subject matter but also technology and pedagogy. the framework that can show a teacher's knowledge related to pedagogy, technology, and content is tpack. tpack's theoretical framework, which was developed by mishra & koehler (2006), stands for technological pedagogical content knowledge (tpack) (figure 1). teachers should have good tpack to effectively integrate technology into their teaching. however, some research has shown that most of indonesian science teachers did not efficiently use technology in their teaching (kazu & erten, 2014; semiz & ince, 2012; smeets, 2005). for example, kamulung (2017) investigated the level of integration of information technology by indonesian biology, chemistry, and physics teachers in grade xi high school using the technology integration matrix (tim) method. he found that the ability of indonesian science teachers to integrate technology in learning science is only 55% in a medium category between the interval 41%-60%. furthermore, another researcher found that although most of the indonesian teachers had the technology to support students learning, their ability to use technology was still relatively low, where most teachers still used conventional methods based on the results of the questionnaire, observations, and interviews (styaningrum, 2016). there are some complex factors affecting teachers' use of technology in k-12 classrooms such as computer proficiency, computer availability, and technical support such as electricity and hardware facilities (eteokleous, 2008; hew & brush, 2007; inan & lowther, 2010). in indonesia, some researchers show that the main factors are science teachers’ lack of skills, knowledge for effective technology use, and management issues because they are mailto:hset0001@gmail.com journal of science learning article doi: 10.17509/jsl.v3i2.22242 107 j.sci.learn.2020.3(2).106-116 not adequately prepared to integrate technology in the classroom (styaningrum, 2016; sumintono, wibowo, mislan, & tiawa, 2012). most importantly, evidence shows that low self-efficacy among indonesian science teachers becomes an obstacle to integrating technology because it affects their motivation to try and to learn technology (styaningrum, 2016). self-efficacy is an individual's confidence in his ability or competence to carry out tasks or actions and overcoming obstacles needed to achieve certain results or outcomes (bandura, 2001). many educational experts supported that teachers’ self-efficacy has a significant contribution affecting the intention of educational technology integration (abbitt & klett, 2007; ertmer, 2005; lee & lee, 2014; uzun, ekici, & sağlam, 2010). several studies showed that teachers’ use of technology in their classrooms also correlates with their frequency usage of educational internet use (sahin, celik, akturk, & aydin, 2013). one of the platforms of technology that is commonly used by students and teachers in an academic context in recent years is social media. social media is "the internet-based tool and platform that facilitates sharing of information including the transfer of text, photos, audio, video, and information in general" (bassell, 2010). it has been an integral part of learning, communication activities, content sharing, and social networking for several decades (kaplan & haenlein, 2010; shirky, 2011). in an academic context, social media is used in university, teacher education, and school settings because it plays a decisive role in meeting students' needs and accessible on the variety of devices (ahern, feller, & nagle, 2016). for example, lecturers often use blogs, skype, wikis, facebook; and even mobile apps, such as whatsapp in distance learning in university (mnkandla & minnaar, 2017; owusu-ansah, gontshi, mutibwa, & ukwuoma, 2015; callaghan & fribbance, 2016; hussain, cakir, & candeger, 2018). the most common use of social media includes searching and studying in collaborative online activities, download media, sharing information, and entertainment (wheeler, yeomans, & wheeler, 2008; al-daihani, jumanah, & sara, 2018). in teacher education, teacher educator used social media for motivating teacher candidate to collaborate with a cooperative teacher or mentor; providing a friendly atmosphere to encourage reflection to identify and address challenges of teaching practice; giving feedback and continue following up on their progress in teaching practice (pitiporntapin & lankford, 2015). in the actual setting of the teaching and learning process, in-service teachers are also reported integrating social media such as facebook (blonder & rap, 2017) and youtube (blonder, jonatan, bar-dov, benny, rap, & sakhnini, 2013) in their science classrooms. although flourishing research evidence shows the potential of social media, there is little research connect between social media use affect tpack self-efficacy of prospective teachers. this study has some potential significance, both theoretically and practically. by investigating the correlation between social media and tpack, future researchers might consider some factors of social media usage that need attention to design the program or intervention that focuses on developing equitable tpack for all psts. this is because prospective teachers who have low self-efficacy towards technology integration tend not to use it in their future teaching practice. meanwhile, integrating technology into the sciences classroom is the requirement of national science standard not only in indonesia but also in many countries. practically, the result of this study would be significant for policy and practice. teacher educators can also consider the finding of designing the best practice of a professional development program (pdp) through ict course evaluation. for example, social media can be integrated to improve their ktt self-efficacy. there are some operational definitions of the key terms of this research. tpack framework stands for technological pedagogical content knowledge. it has three essential elements, including ck (content knowledge), pk (pedagogical knowledge), tk (technological knowledge). the intersection between them is pck (pedagogical content knowledge), tck (technological content knowledge), and tpk (technological pedagogical knowledge) and tpack. since this study would focus on the element related to technology, ck, pk, and pck were excluded from the analysis. social media usage (smu) in this project would only focus on five constructs of smu in the academic life of university students including searching and studying (ss), download media (dm), sharing media (sm), professionalism development (pd), and entertainment figure 1 tpack and its components (koehler & mishra, 2009), p. 63 journal of science learning article doi: 10.17509/jsl.v3i2.22242 108 j.sci.learn.2020.3(2).106-116 and motivation (em). smu in an informal setting or leisure activity was excluded. 1.1 tpack framework technological knowledge (tk) generally includes knowledge on a wide variety of different technologies from simple technology such as a pen to advance technology (archambault & crippen, 2009; ozden, mouza, & shinas, 2016; schmidt et al., 2009). however, tk in the tpack framework only specifically includes technology associated with digital platforms and computers (mishra & koehler, 2006). it also consists of the knowledge of using computer software (basic office programs, operating system, and software related to instruction), using hardware (projection device, interactive whiteboard, etc.), web or the internet research devices (chuang, 2013; handal, campbell, cavanagh, petocz, & kelly, 2013) and using communication platform (social networks, email, forums, chat tools) (chai, koh, & tsai, 2013). teachers with a high level of technical knowledge tend to use technology more efficiently, which also has an implication in their classroom teaching and learning (abbitt, 2011; harris, mishra, & koehler, 2007). knowledge of teaching with technology (ktt) in tpack framework is the simplification of three elements in tpack frameworks, including technological content knowledge (tck), technological pedagogical knowledge (tpk), and technological pedagogical content knowledge (tpack). the three components are simplified into ktt because psts generally interpreted the items related to technology as being conceptually similar (koh, chai, & tsai, 2010). firstly, in the science education context, tck refers to the effectiveness of teaching science with technology integration (schmidt et al., 2009). it implies that science teachers should master not only their knowledge of science concepts but also an understanding of how it can change with technology that more appropriate to be integrated (mishra & koehler, 2006; schmidt et al., 2009). secondly, tpk refers to an understanding of how technology can affect the teaching and learning process. for example, teachers should be able to consider the challenges, constraints, and affordance of technology by considering their teaching environment and curriculum (harris, mishra, & koehler, 2007). this also includes what kind of technology more suitable for a particular teaching strategy, classroom management, and students’ differences. thirdly, tpack is the presentation of science by being integrated with a pedagogical approach by using various technologies (chai, koh, & tsai, 2013). current research on tpack mainly focuses on three major themes, including measuring (students) teachers tpack, the relationship between tpack and other components of technology integration, and strategies for developing prospective teachers tpack (setiawan, phillipson, & isnaeni, 2019). firstly, two main methods to measure participants' tpack, including via self-report and via performance (e.g., kafyulilo, fisser, pieters, & voogt, 2015; maeng, mulvey, smetana, & bell, 2013). the distinction between them is that tpack can reflect in one end as knowledge (via self-report) (e.g., abbitt, 2011; alqurashi, gokbel, & carbonara, 2017), and in the other end as competence or performance that can be observed (via performance) (e.g., kafyulilo, fisser, pieters, & voogt, 2015; maeng, mulvey, smetana, & bell, 2013). 1.2 social media usage (smu) and tpack some researchers argue that social media has a beneficial impact on the academic life of university students (ahmed, ahmad, ahmad, & zakaria, 2018; alrahmi & zeki, 2017; ali, yaacob, endut, & langove, 2016; arquero & esteban, 2013). for example, in social media, prospective teachers can contribute to express their idea or perspectives to the group debates to respond to the education issues (mudaly, morgan, van laren, singh, & mitchell, 2015). online chatting in a whatsapp group can be used as a tool to support prospective teachers’ professional development through sharing their experience in a group chatting and support emotionally each other (cansoy, 2017). kivunja (2013) supported that integrating social media is to improve their digital pedagogy as the preparation for them teaching using digital technologies. “digital pedagogy in pre-service higher education is crucial to better prepare teachers in the new classroom because children in their classrooms will be digital natives, with skills for digital fluency” (kivunja, 2013). students have grown up in a world surrounded by and using computers. in the actual setting, science teachers also integrating social media such as youtube (blonder, jonatan, bardov, benny, rap, & sakhnini, 2013), facebook forum (blonder & rap, 2017; köseoglu & mercan, 2016), google docs, skype, social networks and wikis (ahern, feller & nagle, 2016), to support students learning. social media is also integrated into moodle to support distance learning (buus, 2012). lau (2018) investigates the correlation between smu in informal settings (outside of classroom context), including for media sharing, internet searching, and video gaming with tpack level of pre-service teachers with path analysis. he found that media sharing is a useful predictor of technological knowledge (tk) and knowledge of teaching with technology (ktt). in contrast, internet searching and video gaming do not correlate with any components of tpack. however, the research is conducted in a broad context and only focuses on smu in an informal setting. meanwhile, some research shows that university students in teacher education using not only social media to media sharing, internet searching, and video gaming but also other activities such as to download media, entertainment, improve motivation, to improve their professionalism as prospective science journal of science learning article doi: 10.17509/jsl.v3i2.22242 109 j.sci.learn.2020.3(2).106-116 teachers. therefore, further research that investigates the relationship between smu in an academic context and their tpack self-efficacy in science education needs to be conducted in academic settings. 1.3 gaps in the literature although previous research shows, tpack correlates with the use of technology level (kazu & erten 2014), educational internet use (sahin, celik, akturk, & aydin, 2013), attitude towards computer (baturay, gökçearslan, & sahin, 2017), attitude towards web-based instruction (kavanoz, yüksel, & ozcan, 2015) and smu in informal setting or leisure activities (lau, 2018), no research connect it with smu in an academic setting or school activities. lau (2018) suggested that extended research of smu in an academic setting and specific domain should be identified as an area for future research, especially in the context of science education. previous studies (kaya, kaya, & emre, 2013; lin, tsai, chai, & lee, 2013; pamuk, ergun, cakir, yilmaz, & ayas, 2015; schmidt et al. 2009) supported that tpack could be domainspecific. therefore, in this study, this research would analyze the association between prospective science teachers' demographic characteristics, smu in their school activities, and their tpack self-efficacy in the indonesian context. 1.4 research question (rq) considering the importance of tpack self-efficacy, and the wide use of social media among prospective science teachers supported by given gaps of literature, this study proposes to examine issues related to tpack surveys with a study of indonesian pre-service teachers. the research question addressed in this study is: what is the relationship between the frequency of social media usage (smu) in an academic setting and self-efficacy beliefs towards tk, tck, tpk, and tpack of prospective science teachers in indonesia? 2. method 2.1 research design this research is quantitative based-research design. the research was conducted using a self-administered survey that provides a quantitative or numeric description of the opinions of a population (creswell, 2008; robson & mccartan, 2016). 2.2 time, location, and participants the research was conducted during the second semester of the academic year 2018/2019 from october to november 2018 in the faculty of mathematics and natural science of a state university located in semarang city, central java province, indonesia. the sample of this research study consists of 217 indonesian prospective science teachers from approximately 250 population invited to participate in this study. the population includes psts from the science education department and biology education department. the response rate of the survey is 86.8 %. the high response rate might because of the reward offered before the study. psts have been studying science to become secondary school teachers majoring in science education and biology education at various levels. the second year of 112 bachelor degree students (51.61%) was not experience with ict courses, which includes computer and technology integration in the science classroom because it is offered in the fifth semester. in contrast, 105 master's degree students (48.38%) were experienced with ict courses that are expected to be better informed with technology application in their teaching. the majority of participants were female, with 124 students (58%), and a smaller proportion of male students responded to the survey with 93 students (42%). the unbalance ratio between females and males might because the research was conducted in the faculty where strong female-bias in indonesia. 2.3 instrument tpack self-efficacy scale (tpack-ses) the first research instrument to evaluate tpack selfefficacy used in this research was explicitly developed for prospective science teachers (psts) by bilici, yamak, kavak, & guzey (2013). the item has been validated with 808 psts from teacher preparation institutions at 17 different universities in turkey. the items are valid and reliable with the result of item-total correlation coefficients of exploratory factor analysis (efa), and confirmatory factor analysis (cfa) showed a good fit. the item also reliable since cronbach's alpha coefficient was found to be high. the format for measurement of this instrument was modified from a 100-point rating scale to a 5-point likert scale (1=very low, 2=low, 3=medium, 4=high, 5=very high). content knowledge (ck) construct, as well as pedagogical knowledge (pk) construct and the intersection, was deleted from the survey because the primary focus is the technology constructs of tpack. technological knowledge (tk) self-efficacy has six constructs, and knowledge of teaching with technology (ktt) self-efficacy has 14 constructs. ktt is the composite of technological pedagogical knowledge (tpk), technological content knowledge (tck), and technological pedagogical content knowledge (tpack). the three components are simplified into ktt because prospective teachers generally interpreted the items related to technology as being conceptually similar (koh, chai, & tsai, 2010). the questionnaire was translated by the author into bahasa indonesia (the national language of indonesia). social media usage in academic context the second research instrument, smu in an academic context, is developed by looking at the common use of social media both theoretically and practically. the former, extensive literature review is conducted by reviewing related peer-reviewed articles to discuss media journal of science learning article doi: 10.17509/jsl.v3i2.22242 110 j.sci.learn.2020.3(2).106-116 and technology usage. we found that the instrument to identify the frequency of social media use has been initially developed by lau (2018). however, the instrument focus on developing an instrument to measure smu in an informal setting which not fully reflect smu in academic life. therefore, an extended literature review is conducted to explore students' motivation and purpose to use social media in higher education (e.g., ali, yaacob, endut, & langove, 2016; bal & bicen, 2017; balakrishnan, teoh, pourshafie, & liew, 2017; arquero & esteban, 2013; gupta, 2014; rosen, whaling, carrier, cheever, & rokkum, 2013) and in teacher preparation institution (carpenter & krutka, 2015). the latter, the common use of social media, is conducted not only theoretically but also practically. some graduate students were asked what kind of practical activity in social media that they do during their study. this study was conducted by interviewing three pre-service science teachers enrolled in a teacher preparation program in indonesia and two graduate students at monash university australia. they were asked the common use of social media in their academic life. the result was rewritten to create the questions. based on the result of the literature review and elicitation study, the author them modify smu in an informal setting, which is developed by lau (2018). the instrument has five constructs includes five including searching and studying (ss), download media (dm), sharing media (sm), professionalism development (pd), and entertainment and motivation (em). media sharing, which is developed by lau (2018), initially has four subitems, including the use of social media to watch tv shows, to watch video clips on a computer, to download media files, and to share media files on a computer. in the modified instrument, we remove smu to watch tv because it is not related to the academic setting and teaching practice. professional development (pd) is added as an additional item because prospective teachers also use social media to enhance their professionalism. the scale of smu frequency uses a five-point scale (never, rarely, sometimes, often, and always). the format of measurement slightly different from lau (2018) that uses ten-point frequency scales of smu in an informal setting. this newly-modified instrument was presented to a group of graduate students studying at the faculty of education, monash university (n=3), and doctoral students (n=1) to be reviewed for content validity. the results were reviewed, and slight alterations were made in language, and questions were added or dropped to strengthen the sub-items of the construct. minor alterations were made, and the final instrument was used to collect data. 2.4 data collection ethical approval was obtained from the monash university human research ethics committee (muhrec) with approved number 2018-17330-24853 to ensure that the research follows research ethics principles such as informed consent, privacy, and confidentiality. before the survey was conducted, the institution where the research was conducted was given a consent form to obtain a letter of permission. the prospective participants were also given information implicitly related to the purpose of research, what they need to do, how long they need to complete the survey, the potential benefits, and risk. the survey was anonymous, so they will not explicitly identify themselves to make sure the data remains confidential. however, they need to input their email address before completing the survey to avoid double response. data were collected through a web-based survey using google form because it is free and efficient to collect information (djenno, insua, & pho, 2015). the researcher shares the link of a web-based survey to all of the selected students with the help of lecturers in the faculty of mathematics and natural sciences who teach the participants. after the link was shared using social networks (whatsapp group), the participants completed three parts of the survey sequentially. first, they completed their demographic characteristics (gender, level of education, major, and age) in a multiple-choice format. some alternative answers are provided for each question. participants can only choose one answer for gender, level of education, major, and age. the second and third part of the survey is tpack self-efficacy and smu in the academic context, respectively. they can only respond to one answer for each item. fifteen minutes were granted for the students to complete the survey. participants had a two-week timeframe in which to complete the survey. after the survey was conducted, the data was downloaded from google form and transferred into microsoft excel 2016. all of the students' responses were entered as numerical values so that the computer can read and analyze the data. therefore, the first step was coding, which aims to transform non-numerical information into numerical information. for example, the self-efficacy scale response, very high, was coded as 5, and 1 for very low. the data were organized into a cross-tabulation. journal of science learning article doi: 10.17509/jsl.v3i2.22242 111 j.sci.learn.2020.3(2).106-116 2.5 data analysis the data analysis was inferential statistics. the examination of pearson’s correlation was used to analyze whether the psts’ differences in perception of tpack were related to their smu. the correlational analysis was presented in a scatterplot. the multiple regression analysis was used to test for the significance of the relationships. the statistics of the multiple regressions of importance in determining the relationship will be the pearson r, pvalue, and effect size (gall, gall, & borg, 2003). spearman's rank correlation coefficient (r) was counted to see the linear trend using microsoft excel. the direction of correlation can be positive, negative, and no correlation (mertens, 2016). a positive correlation means that the two variables move together in the same direction. a negative correlation means that the two variables differ inversely in the opposite direction. if the correlation coefficient is near zero, no relationship exists. the interpretation to assess the strength of the correlation coefficient and the direction (cohen, 1992) are presented in table 1. figure 2 the correlation between the frequency of social media in an academic setting and tk self-efficacy figure 3 the correlation between the frequency of social media in an academic setting and tck self-efficacy table 1 interpretation of spearman's rank correlation coefficient (r) r-value interpretation r =.10 to .29 or r = −.10 to −.29 low r = .30 to .49 or r = −.30 to −.49 medium r = .50 to 1.0 or r = −.50 to −1.0 high table 2 correlation between social media usage (smu) and tpack factor (pearson's r) according to gender smu gender tk tck tpk tpack r r2 r r2 r r2 r r2 searching and studying male (n=93) .51 ** .26 .07 .01 .53 ** .28 .50 ** .25 female (n=124) .37 ** .14 .03 .00 .42 ** .18 .42 ** .18 download media male (n=93) .49 ** .24 .10 .01 .47 ** .22 .42 ** .18 female (n=124) .45 ** .21 .02 .00 .53 ** .28 .48 * .23 searching information male (n=93) .33 ** .11 .08 .01 .42 ** .17 .41 ** .16 female (n=124) .37 ** .14 .03 .00 .31 ** .10 .33 ** .11 professional development male (n=93) .47 * .22** .14 .02 .47 ** .22 .53 ** .28 female (n=124) .36 ** .13** .15 .02 .55 ** .30 .58 ** .34 entertainment and motivation male (n=93) .52 ** .27 .04 .00 .48 * .23 .39 ** .15 female (n=124) .41** .17 .08 .01 .36 ** .13 .35 ** .12 social media usage all (n=217) 0.53** .27 .07 .00 0.58** .34 0.57** .31 * p<0.05; **p<0.01 journal of science learning article doi: 10.17509/jsl.v3i2.22242 112 j.sci.learn.2020.3(2).106-116 the p-value indicates how confident each independent variable has some correlation with the dependent variable. in other words, the p-value indicates whether the relationship found in the r is statistically significant. when p<.05, the relationship is statistically significant. 3. result and data analysis we found three important findings when the correlation between all construct of smu and tpack are analyzed based on gender (table 2). first, smu for searching and studying has a high statistically significant correlation with tk self-efficacy, tpk, and tpack (r>0.5) among male participants but has a medium correlation among female participants (0.3<r<0.5). second, smu for download media (dm), searching information (si), and entertainment and motivation (em) generally have a statistically medium correlation with tk, tpk, and tpack for both male and female participants. third, smu for professional development (pd) has a medium correlation with tk and high correlation with tpk and tpack. interestingly, this research found that tck only has a low correlation with smu for pd for both male (r=.22, p<.05) and female (r=.13, p<.05), but it does not correlate with another construct of smu including ss, dm, si, and em. figure 2 shows that the average of social media usage frequency has a statistically high correlation with technological knowledge self-efficacy of prospective teachers (r=-.53, p<.01). in contrast to tk, figure 3 shows that the average of social media usage frequency has no correlation with the technological knowledge self-efficacy of prospective teachers (r<.1). similar to the correlation between tk and smu, social media usage has a statistically high correlation with technological pedagogical knowledge (r=-.58, p<.01) (figure 4) and high correlation with tpack (r=-.57, p<.01) (figure 5). 4. discussion the purpose of this paper is to investigate the relationship between smu in an academic setting and tpack. this research found that the frequency of smu, in general, has a statistically high correlation with tk (r=.53, p<.01), tpk (r=-.58, p<.01), and tpack (r=-.57, p<.01). however, the frequency of smu does not correlate with tck (r<.1). the result of this study aligns with the research of lau (2018) investigating the relationship between smu in an informal setting and tpack. he found that although the relationship is not strong, tk correlates with media sharing (ms) (r=.237, p<.05), internet searching (is) (r=.124, p>.05), and video gaming (vg) (r=.204, p<.01). however, the result of this study is slightly different from his findings. smu in an informal setting is moderately correlated with all construct of knowledge of teaching with technology (ktt), which is the composite of tck, tpk, and tpack. it might have the effect of the context of formal and informal setting usage of smu. some similar research also supports the result of this study. who found positive relationships exist between the knowledge dimensions in the tpack and the self-efficacy beliefs in educational internet use (sahin, celik, akturk, & aydin, 2013; kazu & erten, 2014; kavanoz, yüksel, & ozcan, 2015; bingimlas, 2018). a study conducted by kavanoz, yüksel, & ozcan (2015) also revealed that the level of general self-efficacy among participants regarding tpack is associated with their attitudes towards web-based instructions. some research reported that science teachers who integrate social media such as youtube (blonder, jonatan, bardov, benny, rap, & sakhnini, 2013), facebook forum (blonder & rap, 2017; köseoglu & mercan, 2016), google docs, skype, social networks and wikis (ahern, feller & nagle, 2016), to support students learning is figure 4 the correlation between the frequency of social media in academic setting and tpk self-efficacy figure 5. the correlation between the frequency of social media in an academic setting and self-efficacy towards technological pedagogical and content knowledge journal of science learning article doi: 10.17509/jsl.v3i2.22242 113 j.sci.learn.2020.3(2).106-116 found an increased efficacy towards tpack. therefore, integrating social media in teacher education institutions would be beneficial for psts. when further analyzed, the sub-items of smu in academic settings according to gender, searching, download, and sharing media have a positive correlation with tk, tpk, and tpack for both males and females with the size of correlation among female psts is weaker than males. smu for professional development also has a high correlation with tpack self-efficacy for both males and females. professional development in this context refers to social media to learn new technological tools for teaching, to create learning media, to update information related to current new technology for teaching, to join distance learning workshop or online short course and to get information from professional teachers about how they apply ict in their science classroom. then, smu for entertainment and motivation (em) also has a positive correlation with their tk, tpk, and tpack for both males and females. em in this research refers to the use of social media to read online news, listen to music, play educational games, watch online videos for entertainment and chat in social media to improve learning motivation from others. the existence of gender difference as the moderator of the relationship between age and tpack self-efficacy might be the effect of the ict-user profile. ict user profiles are defined as "a framework which is used to identify and categorize students based on how frequently they use ict" (tomte & hatlevik, 2011). an increased level of self-efficacy in ict is related to both an increased level of educational use and leisure use of ict. 5. conclusion the average smu frequency has a statistically high correlation with tk (r=-.53, p<.01), tpk (r=-.58, p<.01), and tpack (r=-.57, p<.01). however, the frequency of smu does not correlate with tck (r<.1). when the correlation was analyzed based on gender, we found three important findings. first, among male participants, smu for searching and studying has a high statistically significant correlation with tk self-efficacy (r=.51), tpk (r=53), and tpack (r=.50), but has a medium correlation among female participants with tk selfefficacy (r=.37), tpk (r=.42), and tpack (r=.42). second, smu for download media (dm), searching information (si), and entertainment and motivation (em) generally have a statistically medium correlation with tk, tpk, and tpack for both male and female participants (r = .30 to .49). third, smu for professional development (pd) has a medium correlation with tk for both males (r=.47) and female (r=.36). however, pd has high correlation with tpk and tpack (r = .50 to 1.0). interestingly, this research found that tck only has a low correlation with smu for pd for both males and females (r=.14), but it has no correlation with another construct of smu(r<.1). in this research, generally, social media usage for all purposes has a high correlation with technological knowledge (r=.53), technological pedagogical knowledge (r=.58), and tpack (r=.57). 6. limitation we exclude some construct of tpack that is not related to technology, including content knowledge (ck), pedagogical knowledge (pk), and the intersection, pedagogical content knowledge (pck). thirdly, the items in the instrument to measure smu in an academic context were the result of modification from smu in an informal setting from previous researchers, but the modified items were not tested for validation due to time constraints to do the project. 7. recommendation 1. future research should investigate tpack among in-service science teachers in indonesia who enroll in pendidikan profesi guru/ppg or graduate certificate for teacher profession. some topics that can be explored are: (1) the effectiveness of professional development of technology and social media (facebook, twitter, youtube) among in-service teachers in indonesia to improve their tpack competence (2) the effectiveness of graduate certificate for teacher profession in indonesia to improve in-service teachers’ competencies in technology integration. 2. the correlational research between smu and tpack in this research is only conducted in the context of science education, and the participants of this research are psts from the department of science education and biology education in one teacher education institution in indonesia. future investigation of tpack should focus on other science domains such as physics and chemistry in more than one institution. 3. future research can compare tpack across crosscultural contexts between indonesia and other countries in south east asia, such as indonesia and singapore, malaysia, or even with countries in the different continents to identify the possibility of cultural differences. this is because most of the research is conducted in one country and a lack of research comparing psts’ tpack across different cultural backgrounds (alqurashi, gokbel, & carbonara, 2017). 8. implication 8.1 implication for practice since social media usage has a positive effect on psts' tk self-efficacy and ktt, training focuses on the application of social media in teaching and learning should be integrated to improve their tpack. in this research, we especially found that smu for professional development correlates with all construct of tpack journal of science learning article doi: 10.17509/jsl.v3i2.22242 114 j.sci.learn.2020.3(2).106-116 related to technology. therefore, the lecturer can integrate professional development that is designed specifically for psts. for example, the introduction could be designed as an advanced program for guiding teachers to adopt adequate social media-based learning materials to learn new technological tools for teaching, to create learning media, to join distance learning workshops or online short courses. for example, advanced training programs could introduce and guide psts to utilize social media-related apps such as facebook, youtube, twitter in their classroom. students' motivation for learning science might be further enhanced. recently, science teachers who integrate social media such as youtube (blonder, jonatan, bar-dov, benny, rap, & sakhnini, 2013), facebook forum (blonder & rap, 2017; köseoglu & mercan, 2016), google docs, skype, social networks and wikis (ahern, feller & nagle, 2016), to support students learning is found an increased efficacy towards tpack. therefore, integrating social media in teacher education institutions would be beneficial for psts. implication for policy the teacher education also needs to reconsider its design of ict course through ict course evaluations. previous research suggested the implementation of a tpack-based course or professional development program improves tpack and self-efficacy (blonder, jonatan, bar-dov, benny, rap, & sakhnini, 2013; kafyulilo, fisser, pieters, & voogt, 2015; canbazoglu, guzey, & yamak, 2016). acknowledgment the research in this paper is funded by the financial support of the ministry of finance republic of indonesia (lembaga pengelola dana pendidikan) with allocation for thesis or research project of master's degree program (master of education) according to the scholarship contract no. prj6893/lpdp.3/2016. we also acknowledge the lecturers of unit edf5614 (research project in education) in the faculty of education at monash university that provide guidance and support for the research project. references abbitt, j. t. (2011). an investigation of the relationship between selfefficacy beliefs about technology integration and technological pedagogical content knowledge (tpack) among preservice teachers. journal of digital learning in teacher education, 27(4), 134143. doi: https://doi.org/10.1080/21532974.2011.10784670 abbitt, j. t., & klett, m. d. (2007). identifying influences on attitudes and self-efficacy beliefs towards technology integration among pre-service educators. electronic journal for the integration of technology in education, 6(1), 28-42. retrieved from http://ejite.isu.edu/volume6/abbitt.pdf ahern, l., feller, j., & nagle, t. (2016). social media as a support for learning in universities: an empirical study of facebook groups. journal of decision systems, 25(sup1), 35-49. doi: https://doi.org/10.1080/12460125.2016.1187421 ahmed, y. a., ahmad, m. n., ahmad, n., & zakaria, n. h. (2018). social media for knowledge-sharing: a systematic literature review. telematics and informatics. doi: https://doi.org/10.1016/j.tele.2018.01.015 al-daihani, s., jumanah, s. a., & sara, a. a. (2018). use of social media by social science academics for scholarly communication. global knowledge, memory, and communication, 67(6), 412-424. doi: https://doi.org/10.1108/gkmc-11-2017-0091 al-rahmi, w. m., & zeki, a. m. (2017). a model of using social media for collaborative learning to enhance learners’ performance on learning. journal of king saud university-computer and information sciences, 29(4), 526-535. doi: http://dx.doi.org/10.1016/j.jksuci.2016.09.002 ali, m., yaacob, r. a. i. b. r., endut, m. n. a. a. b., & langove, n. u. (2016). strengthening the academic usage of social media: an exploratory study. journal of king saud university-computer and information sciences. doi: http://dx.doi.org/10.1016/j.jksuci.2016.10.002 alqurashi, e., gokbel, e. n., & carbonara, d. (2017). teachers' knowledge of content, pedagogy, and technology integration: a comparative analysis between teachers in saudi arabia and the united states. british journal of educational technology, 48(6), 14141426. doi: http://dx.doi.org.10.1111/bjet.12514 archambault, l., & crippen, k. (2009). examining tpack among k12 online distance educators in the united states. contemporary issues in technology and teacher education, 9(1), 71-88. retrieved from https://www.learntechlib.org/p/29332/ arquero, j. l., & esteban, r., (2013). using social network sites in higher education: an experience in business studies. journal of innovations in education and teaching international. doi: 10.1080/14703297.2012.760772 bal, e., & bicen, h. (2017). the purpose of students’ social media use and determining their perspectives on education. procedia computer science, 120, 177-181. doi: https://doi.org/10.1016/j.procs.2017.11.226 balakrishnan, v., teoh, k. k., pourshafie, t., & liew, t. k. (2017). social media and their use in learning: a comparative analysis between australia and malaysia from the learners' perspectives. australasian journal of educational technology, 33(1). bandura, a. (2001). social cognitive theory: an agentic perspective. annual review of psychology, 52(1), 1-26. bassell, k. (2010). social media and the implications for nursing faculty mentoring: a review of the literature. teaching and learning in nursing, 5(4), 143-148. doi: https://doi.org/10.1016/j.teln.2010.07.007 baturay, m. h., gökçearslan, s., & sahin, s. (2017). associations among teachers' attitudes towards computer-assisted education and tpack competencies. informatics in education, 16(1), 1-23. bilici, s. c., yamak, h., kavak, n., & guzey, s. s. (2013). technological pedagogical content knowledge self-efficacy scale (tpack-ses) for pre-service science teachers: construction, validation, and reliability. eurasian journal of educational research, (52), 37-60. retrieved from https://eric.ed.gov/?id=ej1060363 bingimlas, k. (2018). investigating the level of teachers' knowledge in technology, pedagogy, and content (tpack) in saudiarabia. south african journal of education, 38(3), 1-12. blonder, r., & rap, s. (2017). i like facebook: exploring israeli high school chemistry teachers' tpack and self-efficacy beliefs. education and information technologies, 22(2), 697-724. doi: 10.1007/s10639-015-9384-6 blonder, r., jonatan, m., bar-dov, z., benny, n., rap, s., & sakhnini, s. (2013). can youtube it? providing chemistry teachers with technological tools and enhancing their self-efficacy beliefs. chemistry education research and practice, 14(3), 269-285. doi: 10.1039/c3rp00001j buus, l. (2012). scaffolding teachers integrate social media into a problem-based learning approach? electronic journal of ejournal of science learning article doi: 10.17509/jsl.v3i2.22242 115 j.sci.learn.2020.3(2).106-116 learning, 10(1), 13-22. retrieved from https://files.eric.ed.gov/fulltext/ej969432.pdf callaghan, g., & fribbance, i. (2016). the use of facebook to build a community for distance learning students: a case study from the open university. open learning, 31(3), 260-272. doi: https://doi.org/10.1080/02680513.2016.1229176 canbazoglu-bilici, s., guzey, s. s., & yamak, h. (2016). assessing preservice science teachers’ technological pedagogical content knowledge (tpack) through observations and lesson plans. research in science & technological education, 34(2), 237-251. doi: https://doi.org/10.1080/02635143.2016.1144050 cansoy, r. (2017). teachers' professional development: the case of whatsapp. journal of education and learning, 6(4), 285-293. doi: 10.5539/jel.v6n4p285 carpenter, j. p., & krutka, d. g. (2015). engagement through microblogging: educator professional development via twitter. professional development in education, 41(4), 707-728. chai, c.s., koh, j. h.l., & tsai, c.c. (2013). a review of technological pedagogical content knowledge. educational technology & society, 16(2), 31–51. retrieved from https://www.jstor.org/stable/jeductechsoci.16.2.31 chuang, h. (2013). a case study of e-tutors’ teaching practice: does technology drive pedagogy? international journal of education in mathematics, science and technology, 1(2), 7582. retrieved from https://files.eric.ed.gov/fulltext/ej1057552.pdf cohen, j. (1992). quantitative methods in psychology: a power primer. psychological bulletin, 112(1), 155-159. creswell, j. (2008). educational research: planning, conducting and evaluating quantitative and qualitative research (3rd ed.). upper saddle river, nj: pearson prentice hall. djenno, m., insua, g. m., & pho, a. (2015). from paper to pixels: using google forms for collaboration and assessment. library hi tech news, 32(4), 9-13. ertmer, p. a. (2005). teacher pedagogical beliefs: the final frontier in our quest for technology integration?. educational technology research and development,53(4), 25-39. doi: 10.1007/bf02504683 eteokleous, n. (2008). evaluating computer technology integration in a centralized school system. computers & education, 51(2), 669686. doi: https://doi.org/10.1016/j.compedu.2007.07.004 gall, m. d., gall, j. p., & borg, w. (2003). educational research: an introduction. boston: person education. gupta, d. s. (2014). social media for teachers of english: a hub for professional development. research journal of english language and literature, 2(2), 34-38. handal, b., campbell, c., cavanagh, m., petocz, p., & kelly, n. (2013). technological pedagogical content knowledge of secondary mathematics teachers. contemporary issues in technology and teacher education, 13(1), 22-40. harris j.b., mishra p., & koehler m.j. (2007). teachers’ technological pedagogical content knowledge: curriculum-based technology integration reframed. journal of research on technology in education, 41, 393–416. doi: https://doi.org/10.1080/15391523.2009.10782536 hew, k. f., & brush, t. (2007). integrating technology into k-12 teaching and learning: current knowledge gaps and recommendations for future research. educational technology research and development, 55(3), 223-252. doi: 10.1007/s11423-006-9022-5 hussain, i., cakir, o., & candeger, ü. (2018). social media as a learning technology for university students. international journal of instruction, 11(2), 281-296. doi: https://doi.org/10.12973/iji.2018.11219a inan, f. a., & lowther, d. l. (2010). factors affecting technology integration in k-12 classrooms: a path model. educational technology research and development, 58(2), 137-154. kafyulilo, a., fisser, p., pieters, j., & voogt, j. (2015). ict use in science and mathematics teacher education in tanzania: developing technological pedagogical content knowledge. australasian journal of educational technology, 31(4), 381–394. retrieved from https://ajet.org.au/index.php/ajet/article/viewfile/1240/1288 kamulung, p. (2017). evaluasi pengintegrasian teknologi dalam pembelajaran sains di sma kristen satya wacana [evaluation of technology integration in science learning at satya wacana christian high school] (doctoral dissertation, program studi pendidikan teknologi informasi dan komputer fti-uksw). retrieved from http://repository.uksw.edu/handle/123456789/14001 kaplan, a. m., & haenlein, m. (2010). users of the world, unite! the challenges and opportunities of social media. business horizons, 53(1), 59-68. kavanoz, s., yüksel, h. g., & özcan, e. (2015). pre-service teachers' self-efficacy perceptions on web pedagogical content knowledge. computers & education, 85, 94-101. doi: https://doi.org/10.1016/j.compedu.2015.02.005 kaya, z., kaya, o. n., & emre, i. (2013). adaptation of technological pedagogical content knowledge scale to turkish. educational sciences: theory and practice, 13(4), 2367–2377. retrieved from https://files.eric.ed.gov/fulltext/ej1016741.pdf kazu, i. y., & erten, p. (2014). teachers' technological pedagogical content knowledge self-efficacies. journal of education and training studies, 2(2), 126-144. doi: http://dx.doi.org/10.11114/jets.v2i2.261 kivunja, c. (2013). embedding digital pedagogy in pre-service higher education to better prepare teachers for the digital generation. international journal of higher education, 2(4), 131-142. doi: http://dx.doi.org/10.5430/ijhe.v2n4p131 koh, j. h. l., chai, c. s., & tsai, c. c. (2010). examining the technological pedagogical content knowledge of singapore pre‐service teachers with a large‐scale survey. journal of computer assisted learning, 26(6), 563-573. köseoglu, p., & mercan, g. (2016). the educational use of facebook as a social networking site in animal physiology classes. world journal on educational technology: current issues, 8(3), 258-266. retrieved from https://files.eric.ed.gov/fulltext/ej1142257.pdf lau, w. w. (2018). relationships between pre-service teachers’ social media usage in informal settings and technological pedagogical content knowledge. eurasia journal of mathematics, science and technology education, 14, 12. lee, y., & lee, j. (2014). enhancing pre-service teachers' self-efficacy beliefs for technology integration through lesson planning practice. computers & education, 73, 121-128. doi: https://doi.org/10.1016/j.compedu.2014.01.001 lin, t.-c., tsai, c.-c., chai, c. s., & lee, m.-h. (2013). identifying science teachers’ perceptions of technological, pedagogical, and content knowledge (tpack). journal of science education and technology, 22, 325–336. doi: 10.1007/s10956-012-9396-6 maeng, j. l., mulvey, b. k., smetana, l. k., & bell, r. l. (2013). preservice teachers’ tpack: using technology to support inquiry instruction. journal of science education and technology, 22(6), 838857. doi: 10.1007/s40299-016-0305-2 mertens, d. m. (2016) research and evaluation in education and psychology (4thed.). london: sage. mishra, p., & koehler, m.j. (2006). technological pedagogical content knowledge: a framework for teacher knowledge. teachers college record, 108(6), 1017-1054. doi: 10.1.1.523.3855 mnkandla, e., & minnaar, a. (2017). the use of social media in elearning: a metasynthesis. international review of research in open and distributed learning, 18(5), 227-248. doi: http://dx.doi.org/10.19173/irrodl.v18i5.3014 mudaly, r., morgan, k. p., van laren, l., singh, s., & mitchell, c. (2015). connecting with pre-service teachers' perspectives on the use of digital technologies and social media to teach socially relevant science. perspectives in education, 33(4), 23-41. owusu-ansah, c. m., gontshi, v., mutibwa, l., & ukwuoma, s. (2015). applications of social media and web 2.0 for research support in selected african academic institutions. journal of balkan libraries union, 3(1), 30-39. journal of science learning article doi: 10.17509/jsl.v3i2.22242 116 j.sci.learn.2020.3(2).106-116 ozden, s. y., mouza, c., & shinas, v. h. (2016). teaching knowledge with curriculum-based technology: development of a survey instrument for pre-service teachers. journal of technology and teacher education, 24(4), 471-499. retrieved from https://www.learntechlib.org/p/172178/ pamuk, s., ergun, m., cakir, r., yilmaz, h. b., & ayas, c. (2015). exploring relationships among tpack components and development of the tpack instrument. education and information technologies, 20(2), 241–263. doi: 10.1007/s10639-013-9278-4 pitiporntapin, s., & lankford, d. m. (2015). using social media to promote pre-service science teachers' practices of socio-scientific issue (ssi) based teaching.asia pacific forum on science learning and teaching, 16(2), 1-28. robson, c., & mccartan, k. (2016). real world research. new york: john wiley & sons. rosen, l. d., whaling, k., carrier, l. m., cheever, n. a., & rokkum, j. (2013). the media and technology usage and attitudes scale: an empirical investigation. computers in human behavior, 29(6), 25012511. sahin, i., celik, i., akturk, a. o., & aydin, m. (2013). analysis of relationships between technological pedagogical content knowledge and educational internet use. journal of digital learning in teacher education, 29(4), 110-117. retrieved from https://files.eric.ed.gov/fulltext/ej1010761.pdf schmidt, d. a., baran, e., thompson, a. d., mishra, p., koehler, m. j., & shin, t. s. (2009). technological pedagogical content knowledge (tpack): the development and validation of an assessment instrument for preservice teachers. journal of research on technology in education, 42(2), 123-149. doi: https://doi.org/10.1080/15391523.2009.10782544 semiz, k., & ince, m. l. (2012). pre-service physical education teachers' technological pedagogical content knowledge, technology integration self-efficacy and instructional technology outcome expectations. australasian journal of educational technology, 28(7), 1248-1265. setiawan, h., phillipson, s., & isnaeni, w. (2019). current trends in tpack research in science education: a systematic review of literature from 2011 to 2017. journal of physics: conference series, 1317 (1), p. 012213. doi: 10.1088/1742-6596/1317/1/012213 shirky, c. (2011). the political power of social media: technology, the public sphere, and political change. foreign affairs, 28-41. retrieved from http://www.jstor.org/stable/25800379 smeets, e. (2005). does ict contribute to powerful learning environments in primary education?. computers & education, 44(3), 343-355. doi: 10.1016/j.compedu.2004.04.003 styaningrum, a. (2016). analisis hambatan guru dalam pengintegrasian teknologi di smpn 1 grabag [analysis of teacher obstacles in technology integration at middle school grabag 1] (doctoral dissertation, program studi pendidikan teknologi informasi dan komunikasi fti-uksw). retrieved from http://repository.uksw.edu/handle/123456789/10781 sumintono, b., wibowo, s. a., mislan, n., & tiawa, d. h. (2012). penggunaan teknologi informasi dan komunikasi dalam pengajaran: survei pada guru-guru sains smp di indonesia. jurnal pengajaran mipa, 17(1), 122-131. doi: http://dx.doi.org/10.18269/jpmipa.v17i1.251 tomte, c., & hatlevik, o. e. (2011). gender-differences in selfefficacy ict related to various ict-user profiles in finland and norway. how do self-efficacy, gender and ict-user profiles relate to findings from pisa 2006. computers & education, 57(1), 1416-1424. doi: http://dx.doi.org.ezproxy.lib.monash.edu.au/10.1016/j.compedu. 2010.12.011 uzun n., ekici, g., & saglam, n. (2010). a study on the primary school secondary level students’ self-efficacy perception of their computer competence. kastamonu education journal, 18(3), 775– 788. wheeler, s., yeomans, p., & wheeler, d. (2008). the good, the bad and the wiki: evaluating student‐generated content for collaborative learning. british journal of educational technology, 39(6), 987-995. doi: https://doi.org/10.1111/j.1467-8535.2007.00799.x a © 2022 indonesian society for science educator 250 j.sci.learn.2022.5(2).250-265 received: 28 october 2021 revised: 16 march 2022 published: 27 july 2022 stem club evaluation scale: validity and reliability study hasan gokce*1, seyide eroglu1, melek karaca2, oktay bektas3 1ministry of national education, kayseri, turkey 2institute of educational sciences, erciyes university, kayseri, turkey 3faculty of education, erciyes university, kayseri, turkey *corresponding author: hasangokce3838@gmail.com abstract in stemnet's report, 76% of 500 teachers interviewed stated that joining the stem club increased students' ability to solve real-world problems. this study aims to develop a valid and reliable measurement tool for evaluating stem clubs. the research sample consisting of 149 teachers who carry out stem club activities in schools in turkey was determined using the purposive sampling method. content and construct validity and reliability analyses have been performed for this purpose. to ensure content validity, (1) a pool of questions based on the literature was created, (2) draft scale items were determined, (3) an expert was allowed to check them, and (4)item difficulty and discrimination index were calculated. to ensure construct validity, (1) exploratory factor analyses (efa) and (2) confirmatory factor analyses (cfa) were performed on both the same and different samples. as a result of the analyses, having the same data set be analyzed with different software was sufficient for verifying the factor structure. a three-factor structure consisting of 29 items was obtained, which explains 52% of the variance. cronbach’s alpha of reliability for the overall scale was calculated as .92. as a result, a valid and reliable scale was determined to have been developed for researchers and program practitioners to evaluate stem clubs. suggestions have been made that the scale can be used on stem clubs at the provincial, district, and school levels to determine their efficiency and productivity. keywords science education, stem, scale development, validity, reliability 1. introduction stem education emerged as an educational reform. turkey adopted it in 2014 and entered it onto the national education policy agenda. therefore, the number of studies on stem education and its importance has also increased (corlu, capraro & capraro, 2014; ministry of national education [mone], 2016; turkish industry & business association [tusiad], 2014). this issue became official and started to be discussed in education circles for the first time with the stem education report published by the mone-affiliated general directorate of innovation and educational technologies (2016). this report emphasized that stem education urgently needs to be included in turkey’s current education system for sustainability in the economy; an action plan was created, and solution suggestions were listed. some of the solution suggestions included in the action plan involve: establishing and disseminating stem centers in provinces, encouraging educational researchers to conduct research in this field, supporting teachers with pre-service and in-service training, updating curricula, and providing an environment for students to perform stem education activities regardless of time or place (mone, 2016). considering the stem education report, studies have been initiated to integrate stem education into lessons. first of all, changes were made to the curricula. accordingly, the objectives and achievements of the science course curriculum were revised in 2018 and harmonized with stem education. thus, the aim is also to create a suitable framework for integrating stem into lessons. according to the 2018 curriculum, students are given opportunities to propose solutions to daily life problems, experience engineering applications, and use different disciplines in their courses (mone, 2018). however, when examining the studies on teachers’ or teacher candidates’ opinions on how to conduct stem activities in classes, they are seen to state the content and duration of the course to be a major obstacle in making the applications (siew, amir, chong, 2015). when examining journal of science learning article doi: 10.17509/jsl.v5i2.39826 251 j.sci.learn.2022.5(2).250-265 the studies on the problems experienced in stem education in turkey (akgunduz et al., 2015; altunel, 2018; eroglu & bektas, 2016), teachers and teacher candidates are seen to stated being unable to carry out studies on stem education due to the course load and curriculum density. this problem can be solved by changing the curriculum, course content, and class hours. however, because this solution implies long-term radical change, the most practical way to apply stem is to conduct activities outside the classroom. for this reason, stem studies mainly use out-of-school learning environments (baran, bilici, mesutoglu & ocak, 2016; kalkan & eroglu, 2017; national research council, 2015). out-of-school learning environments have significant potential for increasing student learning and providing them with a rich learning environment (robelen, 2011). through the activities carried out in after-school programs, students acquire various skills, produce many solutions to daily life problems, and learn to cooperate and communicate (mahoney, parente & lord 2007). out-of-school learning environments cover a wide area, such as social, cultural, and technical trips around the school, field studies, project studies, sports activities, nature training, or club activities (karadogan, 2016). by applying stem activities in out-of-school learning environments, students are supported in terms of career choices, meaningful learning, and interest in science lessons (dabney et al., 2012). the related literature states that stem activities in out-of-school learning environments direct students' career plans to stem fields (dabney et al., 2012). in addition, it is underlined that stem activities carried out in out-of-school learning environments are essential in providing deep learning for students (bybee, 2001). one of the out-of-school learning environments where stem activities are carried out is social clubs (afterschool alliance, 2015; bell, lewenstein, shouse & feder, 2009). stem clubs are expressed as flexible working environments created without regard to time or place where out-of-school stem studies are carried out (blanchard, hoyle & gutierrez, 2017). while stem clubs develop students’ emotional skills such as belonging and peer-to-peer communication, they also enhance students’ 21st-century skills and help them learn current content. in addition, the activities carried out in stem clubs support the formation of career awareness in students and their orientation toward stem professions (blanchard, hoyle & gutierrez, 2017). in the related literature, many studies reveal the positive effects of stem studies carried out under social clubs on students (ayers, wade-jaimes, wang, pennella & pounds, 2020; baldridge, nutt, vaughn, hartley-lewis & amos, 2019; lipuma, bukiet & leon, 2021). as one of these studies, the stem club study conducted by ayers, wadejaimes, wang, pennella & pounds (2020) with different partner schools enabled students to face real-life problems. they called st. jude stem club (sjsc), and students conducted pediatric cancer research with accurate data for a 10-week. as a result of the study determined that students' attitudes toward science changed positively, their interest in the stem field, and their participation in club activities increased (ayers, wade-jaimes, wang, pennella & pounds, 2020). in another study, activities in cooperation with a university and a high school within the step (the student and teacher enhancement partnership) program conducted by baldridge, nutt, vaughn, hartley-lewis & amos (2019). as a result of the activities carried out with mentors from the university, they determined that many skills, including scientific literacy, writing, and continuing science skills, developed. based on those above, it can be stated that while stem activities carried out under social club activities eliminate the time problem, it also supports formal education and contributes to the development of students. for this reason, stem clubs are seen as an appropriate way to carry out stem studies effectively (straw, branson, neumann & dickinson, 2011). based on this, mone yegitek sent an official letter dated may 2018 to all schools in turkey. accordingly, the official article stated that stem clubs might be formed to carry out stem studies more effectively in schools. in this direction, schools have started to form stem clubs as of the 2018-2019 academic year (coskun, alakurt, & yilmaz, 2020). stem clubs have been formed based on mone’s (2017) educational institutions social activities regulation. when examining this regulation, it is seen to involve general principles. although these are the same regarding the basic principles of stem club activities in schools, no legislation exists that sets out stem clubs’ working structures or activity frameworks. that suggests that differences may exist in how content is applied. determining the standards for the studies carried out in stem clubs and establishing stem clubs within the framework of scientific and other criteria are extremely important for carrying out school applications without interruption (vural, 2018). when examining the relevant studies in the literature, out-of-school studies are found to have been conducted with stem clubs in the title (ferrara et al., 2017; gottfried & williams, 2013; sahin, 2013). however, no studies were found to have examined, evaluated, or determined the deficiencies of stem club studies. for this reason, the current situation must be revealed in all its aspects to organize stem club activities and eliminate their problems. obtaining the opinions of the teachers who carry out stem club work on the subject is essential for determining the current situation. while the effects of stem club activities on students are frequently investigated in the related literature (ferrara et al., 2017; gabrielson, strachan, warner & lafleche, 2009), no study journal of science learning article doi: 10.17509/jsl.v5i2.39826 252 j.sci.learn.2022.5(2).250-265 is found to have obtained the opinions of teachers who carry out stem club activities. however, we think it is more appropriate to start researching stem clubs from teachers since they are the practitioners of the programs who work in the field and can best identify the positive and negative aspects of the studies. in addition, we believe that it would be more appropriate to start the application with the students, starting with the teachers, since they are the teachers who know and observe their students best. for this reason, primarily teachers were studied; in addition to the teacher element, detailed analyses were made in the study in terms of planning and implementation and student dimensions, which are other important elements. in the related literature, there are many scale development studies in the field of stem (cevik, 2017; faber et al., 2013; guzey, harwell & moore, 2014; milner, horan & tracey, 2014). when these studies are examined in detail; we have seen that they focus mostly on cognitive and affective areas such as awareness of stem fields (cevik, 2017), attitude towards stem (buyruk & korkmaz, 2016, faber et al., 2013), self-efficacy (milner, horan & tracey, 2014). however, to increase the capacity and impact of stem studies, the studies should be evaluated according to specific criteria. the current situation should be revealed in all aspects to identify the good or bad aspects of stem studies in or out of school and make improvements if necessary. in this context, evaluation tools for stem studies are needed, but instead of special scales, alternative assessment and evaluation tools are preferred (zengin, kaya & pektaş, 2020). however, it is important to determine the effectiveness of different stem practices, and situation-specific scales need to be developed. still, no measurement tool can evaluate all aspects of the stem club activities that teachers carry out in their classrooms. therefore, the decision has been made to conduct such a study for the following reasons: to be able to carry out investigations on stem education through out-of-school clubs, stem clubs are a way to adapt learning environments to stem education, a limited number of studies are found in the literature on stem clubs, legislation that sets out the operational structures and activity frameworks of stem clubs is lacking and creates non-standard situations in practice, no studies are found on measuring the effectiveness of stem clubs, and no measurement tool exists that can be used to evaluate stem club activities. this research aims to develop a measurement tool that will reveal the level to which the studies carried out in the stem clubs established in their schools have been implemented, the problems experienced in the implementation process, and the advantages of the applications. validity and reliability studies of the developed measurement tool have been carried out for this purpose. in other words, this research aims to bring a practical and easily applicable measurement tool to the table 1 draft scale in blueprint format dimension no items 1 2 3 4 5 teacher (factor 1) 1. club activities are carried out regularly in my school. 2. teacher selection for the stem club in my school is done voluntarily . 3. i have sufficient knowledge about stem education. 4. student selection for the stem club at my school is not voluntary. 5. i do not have any difficulties while carrying out stem club activities at my school. 6. i act according to the club plan while performing stem club activities. 7. the stem club plans i use are associated with science achievements. 9. the stem club plans i use are related to information technology gains. 11. stem club activities in my school contribute to the success of the students. 17. it contributes to the use of stem club activities in my school and different teaching strategies in my lessons. 18. stem club activities in my school contribute to the students' ability to solve their daily life problems. 19. it gives students an interdisciplinary perspective on stem club work at my school. 21. students take an active role in stem club activities at my school. 22. stem club activities in my school contribute to the development of positive attitudes of students towards school. 24. stem club activities in my school contribute to the career choice of students. 27. i would like to take an active part in the stem club every year. 30. when organizing stem club work, i create the club plan myself. 32. the stem club plans i use are associated with technology design achievements. journal of science learning article doi: 10.17509/jsl.v5i2.39826 253 j.sci.learn.2022.5(2).250-265 literature to enlighten authorities on how to create a special framework plan for stem clubs. furthermore, to determine the functioning of social clubs in schools, enable administrators to determine the effectiveness of club activities, and reveal practitioners positive and negative experiences. furthermore, the study will help bring new scales to the field, determine the strengths and weaknesses of some special applications, make the necessary revisions, carry out studies without interruption, and increase the maximum benefit of the applications for students. finally, the study will also facilitate the work of experts in this field. the research questions determined in line with the aims of the study are as follows; 1. is the scale developed for determining the level of teachers’ implementation of stem club activities valid? 2. is the scale developed to determine the level of teachers’ implementation of stem club activities reliable? 2. method this section explains the research design, universe, sample, data collection, and analysis. 2.1. research design this study has chosen the survey design, a quantitative research method. survey designs are generally defined as the numerical expression of attitudes, tendencies, and opinions about the community, using the answer options determined by the researcher from a community (creswell, 2017; fraenkel, wallen & hyun, 2012). the survey design has been preferred within the scope of the current research to numerically express the stem club evaluation scale (sces) scores the teachers who constitute the study sample received and to perform analyses with these scores. the survey design meets these needs. 2.2. population and sample this research has identified the accessible population as the teachers who carry out stem club activities in schools in turkey. to make generalizations in validity and reliability studies, it is necessary to reach five times the number of items (tabachnick & fidell, 2007). for the results to be statistically significant for sces, a sample of five times the 34 items in the scale was tried to be reached, and this number was determined as 149 (hair, black, tatham & anderson, 2019). official registration of the number of teachers in the accessible population is not permitted. for this reason, the authors could not study with at least 10% of the population. the study has preferred purposive sampling. purposeful sampling is determining a sample of people and situations suitable for the research (johnson & christensen, 2014). purposive sampling is preferred because the investigation is conducted with table 1 draft scale in blueprint format (continued) dimension no items 1 2 3 4 5 planning and implementation (factor 2) 10. my school's facilities are not sufficient to run stem club activities. 12. i can easily obtain materials used in stem club studies. 14. different types of stem club activities are not carried out in my school. 23. the time allocated for stem club activities carried out in my school is not enough. 29. the fact that stem club activities are seen as a lesson activity and carried out in one lesson prevents the effectiveness of the activities. 31. during stem club activities, most of the time, there are no activities and the studies remain on paper. 33. during stem club activities, there is no cooperation with official and voluntary organizations in the district. student (factor 3) 8. stem club activities in my school do not contribute to the students' ability to use technology effectively. 13. stem club activities at my school have no contribution to students' discovery of their talents. 15. few students participate in stem club activities. 16. the stem club plans i use are not suitable for the student level. 20. stem club activities in my school do not contribute to the development of different materials in my lessons. 25. i do not think that stem club activities at my school improved my communication skills with my students. 26. the stem club activities at my school do not contribute to the students' ability to think like scientists. 28. students who carry out stem club activities at my school are expected to meet certain criteria. 34. the stem club plans i use are not associated with math achievements. journal of science learning article doi: 10.17509/jsl.v5i2.39826 254 j.sci.learn.2022.5(2).250-265 teachers who carry out stem club activities in the schools where they teach. 2.3. data collection tool the scope of the current research aims to develop a data collection tool. the authors have conducted validity and reliability studies of the sces as a measurement tool. the draft scale created by the authors based on science education and stem literature is given in table 1 in blueprint format. the authors wanted to develop a measurement tool to evaluate the effectiveness of out-ofschool stem practices. therefore, it was paid attention to creating items for the teachers who are the implementers of the program and the student who is the addressee of the program, which is suitable research. since the efficiency of stem clubs requires good planning and implementation, articles for planning and implementation are also written. in short, it was predicted that the draft scale consisted of teacher, student, planning, and implementation dimensions. 2.4. data collection at the level that teachers implement stem club activities, the following steps have been followed within the scope of the sces development studies: 1. a data collection tool has been prepared as a result of the relevant literature review. 2. the sample over which the study will be conducted has been determined. 3. the authors transferred the sces to the google questionnaire application and allowed it to be filled out online. 4. after the teachers filled out the sces online, the authors transferred the data obtained from the google survey application to the package program spss 25. 2.5. data analysis the obtained data have been analyzed using the package programs spss.25 and lisrel 8.80. the study data have been evaluated at a significance level of p < .05. it is explained in detail in the findings section. general headings are given in the data analysis to avoid repetition. to provide evidence for the validity and reliability of the scale, devellis’ (2014) eight steps for scale development were adhered to, and the following analyses have been made in order: • literature review, question pool creation, expert opinion, and item index analyses to ensure content validity; • explanatory and confirmatory factor analyses over the same and different samples to construct validity; • cronbach alpha reliability analysis was performed to ensure reliability. it is common in the literature that confirmatory factor analysis should be done with data obtained from a different sample than the sample in which exploratory factor analysis was performed. this study discusses the accuracy of this common idea by using the same and different data sets for exploratory and confirmatory factor analysis. the expression "same sample" means that the data set used in exploratory factor analysis is also used in confirmatory factor analysis. the expression "different sample" means that the data set used in exploratory factor analysis is different from that used in confirmatory factor analysis. the draft sces is a 5-point likert-type scale consisting of 34 items. the scale has 21 positive items (items 1, 2, 3, 5, 6, 7, 9, 11, 12, 15, 17, 18, 19, 21, 22, 24, 25, 27, 28, 30, 31, and 32) and 13 negative items (items 4, 8, 10, 13, 14, 16, 20, 23, 25, 26, 29, 33, and 34). in scale development studies, one way to check that participants read and answer the scale items is to use positive and negative items that measure the same dimension. scales containing positive and negative statements are widely used to lessen the acquiescent response bias (qasem & gul, 2014). before the analysis, negative items were reverse coded using the recode command. the five points of “strongly disagree” (1 pts.), “disagree” (2 pts.), “undecided” (3 pts), “agree” (4 pts.), and “strongly agree” (5 pts.) have been used to determine the level to which each item in the data collection tool has been realized. the kolmogorov-smirnov and shapiro-wilk tests have been used to determine the normal distribution assumption of the sces scores from the research data. histograms and skewness-kurtosis coefficients, mean, mode, median, and standard deviation values have been examined. for teachers’ sces scores to meet the assumption of normal distribution, the mean and median values should have similar values, and the skewness-kurtosis coefficients should be between -2 and +2 (george & mallery, 2016). based on the sample size, the decision was made to use either the kolmogorov-smirnov or shapiro-wilk test. this study used the kolmogorov-smirnov test because the sample size was greater than 50 (buyukuzturk, kiliccakmak, akgun, karadeniz & demirel, 2016). in data analysis, the data are assumed to have normal distribution when p > 0.05 (pallant, 2016) 3. findings 3.1. normality analysis findings the mean, median, and mode values for each item in the developed scale were close, and the kurtosis and skewness values are between -2 and +2. therefore, because the items on the draft scale have a normal distribution, no items were removed (george & mallery, 2016). the kolmogorov-smirnov test results were also in the desired range (pallant, 2016). in addition, the entire sample within the scope of construct validity was determined to have a normal distribution for the efa and cfa. to perform cfa on a different sample, the sample was split in two to ensure normal distribution. the average distribution of the samples was determined by examining the histograms to show the mean, median, and mode values to be close to journal of science learning article doi: 10.17509/jsl.v5i2.39826 255 j.sci.learn.2022.5(2).250-265 one another and the kurtosis-skewness values to vary between -2 and +2 (fraenkel & wallen, 1996). 3.2. validity analysis findings validity is the degree to which the measurement tool serves its purpose (turgut & baykul, 2015). content and construct validity analyses were made to ensure the validity of the developed scale. the obtained results are presented in order. 3.2.1. content validity findings the items on the sces scale were created based on constructivist theory, the theoretical framework of which is the philosophical approach on which stem education is based. in addition, devellis’ (2014) 8-step scale development method has been followed (see figure 1). content validity can be defined as the extent to which the items making up the test represent the behavioral universe to be measured. therefore, the content and framework must be consistent for a study to have content validity (fraenkel & wallen, 1996). in this context, how the sces items were created is explained in detail. in other words, while preparing the scale, the structure to be measured was defined, an item pool was created, which type of scale it would be was decided, expert opinions were sought, and the items were revised and finalized (devellis, 2014). in creating the scale, the literature on the subject was first reviewed (ferrara et al., 2017; gonsalves, rahm, & carvalho, 2013; gottfried & williams, 2013; sahin, ayar, & adigüzel 2014). based on the literature, an interview form consisting of 22 open-ended questions and various probes was created at the beginning of the study. next, the form was revised, reduced to 17 items, and presented to an expert for their opinions. finally, the interview form was examined by a science education specialist and an assessment and evaluation specialist. as a feedback result received from the expert and the change of opinion to apply the study to a wider audience, the decision was made to convert the interview form to a likert-type scale. in the beginning, 28 statements had been written based on the literature. before receiving its final form, opinions from three experts (an academician in science education, assessment and evaluation, and a science teacher) were obtained regarding the scale. line data obtained from the experts’ opinions, the scale was re-examined by the researchers in terms of clarity, appropriateness, and adequacy of the questions. same items were changed, others removed, and still a few others added in line with the experts’ opinions to arrive at a 34-item scale. the item “i create the club plan myself when organizing stem club activities” was found to be excessive and removed from the scale. apart from this, other expressions were decided to be reverse coded. for example, “stem club activities in figure 1 scale development steps table 2 item difficulty and discrimination index values for the scale item number item discrimination index item difficulty index item number item discrimination index item difficulty index 1 .76 .54 18 .50 .75 2 .58 .68 19 .34 .83 3 .50 .72 20 .32 .84 4 .37 .45 21 .63 .68 5 .76 .57 22 .53 .74 6 .58 .68 23 .61 .49 7 .50 .64 24 .55 .67 8 .21 .84 25 .32 .71 9 .53 .66 26 .34 .83 10 .66 .41 27 .45 .78 11 .66 .67 28 .07 .33 12 .79 .53 29 .50 .38 13 .42 .79 30 .66 .64 14 .42 .45 31 .02 .14 15 -.05 .23 32 .39 .75 16 .32 .84 33 .45 .51 17 .34 .77 34 .39 .77 journal of science learning article doi: 10.17509/jsl.v5i2.39826 256 j.sci.learn.2022.5(2).250-265 my school contribute to students’ ability to think like scientists” was edited to say “do not contribute.” lastly, items 8, 10, 15, 29, 31, and 33 were added to the scale. some of the statements were formed by considering the literature. for example, based on the statement “only willing students should be recruited for club work,” the reverse-coded statement “student selection for the stem club in my school is not done voluntarily” was created (polat, 2017). another example statement, “you work in cooperation with public and private non-governmental organizations as well as parents” (birturk, 2015), which is used as the new scale item “there is no collaboration with official or volunteer organizations in the district during stem club activities.” the constructivist theory was considered while creating the scale items as the basic philosophical approach on which stem education is based. accordingly, the items include interdisciplinary expressions by the nature of stem education where the student is placed in the center to determine whether students take an active role in the process or not and their status regarding being able to use different skills and competencies. this scale is scored as a 5-point likert-type scale ranging from strongly disagree (1) to agree (5) strongly. 3.2.2. item index analysis findings item difficulty and discrimination indexes have been calculated to contribute to the content validity of the developed scale (see table 2). the criteria determined by ebel & frisbie (1991) were taken into consideration when evaluating the item discrimination index (see table 3) when examining table 3, the discrimination index for items 8, 28, and 31 have been determined to be low (r8 = 0.21, r28 = 0.07, r31 = 0.02) and the discrimination index for item 15 to be low and negative (-0.05). after examining the other analysis results, these items were decided to be removed from the scale. the discrimination indices for other items apart from these four items vary between 0.32 and 0.79, and the item difficulty indexes vary between 0.38 and 0.84. 3.3. construct validity findings after completing the content validity, efa and cfa were performed on the same and different samples to ensure the scale's construct validity. a scale development study uses factor analyses to determine the measurement tool's factor structure and verify a specific factor structure (secer, 2017). before performing the factor analysis, cronbach’s alphas of reliability were examined for each item on the draft scale and their impact on the reliability of the entire scale if the item were to be deleted. as a result of the analyses, the reliability coefficients for items 15, 28, and 31 on the draft scale were negative. therefore, the decision was made not to include item 4 (α = .069) in the factor analysis as the value is less than .20. after removing the four items from the item index and reliability analyses, the factor analyses were performed on the remaining 30 items from the scale. 3.3.1. factor analysis findings for the same sample within the scope of the current study, factor analyses have been performed on the same and different samples. this section explains the efa and cfa performed on the same sample using the entire research sample (n = 139). kmo and bartlett’s test results have been calculated using spss 25 to check the suitability of the research data for factor analysis (see table 4). when examining table 4, the kmo value has been determined to be greater than the minimum value of .60 required for analysis and to be statistically significant (tabachnick & fidell, 2007). therefore, factor analysis was performed without factor limitation, and a 6-factor structure was obtained. however, the factor analysis was repeated by limiting the number of factors to three, as suggested by the scree plot, due to a large number of overlapping items and the presence of only one item in the last factor (figure 2). when examining table 5, the developed scale has a structure consisting of three factors and 29 items that explain 52.02% of the total variance. when examining the table 3 item discrimination index criterion item discrimination index value evaluation 0.19 or smaller it should never be scaled or completely corrected. between 0.20 and 0.29 it can be corrected and taken to the test. between 0.30 0.39 can be tested without correction. 0.40 or higher very good conditions can be taken as the test. table 4 kmo and bartlett's test results for the same sample kaiser-meyer-olkin measure of sampling adequacy .890 bartlett's test of sphericity approx. chisquare 2228.706 df 406 sig. .000 figure 2 scree plot for the same sample journal of science learning article doi: 10.17509/jsl.v5i2.39826 257 j.sci.learn.2022.5(2).250-265 items contained in the factors (table 6), factor 1 became “teacher,” factor 2 became “planning and implementation,” and factor 3 became “student.” to verify the obtained structure, cfa was performed on the same sample using lisrel 8.80. the number of samples required to perform cfa was determined to be 75.75, and the analysis was performed on the people in the current study (n = 139). the raw data and scale items were grouped according to the factors specified in the efa results, and syntax commands were written and made suitable for cfa. figure 3 provides the t values obtained as a result of the cfa. jörskog and sörbom (1993) drew attention to looking for the presence of a red arrow while examining these t values. when a red arrow indicates no item, the model interpretation may proceed. as can be seen in figure 3, the analysis continued with the same items as the t values had no issues. factor 1 here represents the teacher, factor 2 represents planning and implementation, and factor 3 represents the students. the next step makes sure that each item has a factor loading value of at least .30. when examining figure 4, the factor loading values for all items are noted to be .30 or greater. factor loadings, t values, the desired criteria, and the model fit indices have been examined. first, the χ2 value compatibility index and the ratio of this value to the degrees of freedom (df) are examined. when examining the path diagram, the χ2 value is understood to be 790.16 (df = 377, p = 0.00, χ2 / df = 2.09). the χ2 value is understood to be low but significant in terms of the model fit indices; χ2 / df is less than three, and the current scale has a perfect fit (jöroskog & sörbom, 1993). although the initial data on the model fit was good, other fit indices have also been examined as χ2 has a significant value (see table 7). the sces was determined during its development phase to consist of 29 questions and three factors resulting from the efa; the cfa has confirmed this structure. table 5 total explained variance for the sces for the same sample component initial eigenvalues extraction sums of squared loadings total % of variance cumulative % total % of variance cumulative % 1 10.463 36.079 36.079 10.463 36.079 36.079 2 2.597 8.955 45.034 2.597 8.955 45.034 3 2.027 6.988 52.022 2.027 6.988 52.022 4 1.389 4.791 56.813 5 1.159 3.995 60.808 6 1.056 3.643 64.450 7 .865 2.984 67.434 8 .854 2.945 70.379 9 .811 2.797 73.175 10 .739 2.548 75.723 11 .685 2.362 78.086 12 .624 2.150 80.236 13 .584 2.014 82.249 14 .575 1.982 84.231 15 .536 1.849 86.081 16 .492 1.695 87.776 17 .434 1.497 89.273 18 .414 1.427 90.700 19 .388 1.339 92.040 20 .350 1.206 93.246 21 .328 1.131 94.377 22 .291 1.005 95.382 23 .280 .964 96.346 24 .245 .843 97.190 25 .224 .772 97.961 26 .173 .595 98.556 27 .161 .555 99.112 28 .140 .483 99.595 29 .117 .405 100.000 journal of science learning article doi: 10.17509/jsl.v5i2.39826 258 j.sci.learn.2022.5(2).250-265 3.3.2. factor analysis findings for different samples to ensure normality, the study divided the sample in two (n = 70 for the efa and n = 69 for the cfa). the kmo and bartlett’s test results have been examined to check the suitability of the research data for factor analysis figure 3 path diagram containing the cfa t values for the sces (same sample) figure 4 path diagram containing the standardized factor loadings for the sces (same sample) journal of science learning article doi: 10.17509/jsl.v5i2.39826 259 j.sci.learn.2022.5(2).250-265 (see table 8). when examining table 8, the kmo value was determined to be greater than the minimum value of .60 required for analysis; this value was statistically significant (tabachnick & fidell, 2007). therefore, because the 3-factor structure was confirmed over the same sample, a structure has been obtained explaining 46.79% of the variance when performing the efa and limiting the number of factors to three (see table 9). when examining table 10, the developed scale consists of four factors and 24 items, explaining 53.48% of the variance. to verify the obtained structure, cfa has been performed on different samples. the number of samples required to perform cfa has been specified as 37.40, and the study performed the analysis over a sample of n = 69 individuals. figure 6 provides the t values obtained as a result of the cfa. when examining the t values, items 12, 14, 23, 29, and 33 were determined to have been indicated with red arrows, thus showing these items to be problematic. in addition, when examining the path diagram showing the standardized factor loading values, items are found with values less than .30 (see figure 6). the t values and standardized factor loading values obtained from the cfa analysis using different samples are not in the desired range, so the data on the other model fit index was examined (table 11). when examining the path diagram, although the χ2 value is understood to be 481.37 (df = 252, p = 0.00, χ2 / df = 1.91) and to have the perfect fit, the other goodness-of-fit indices do not have desired values (see table 11). during the development phase of the sces, the scale has been determined to consist of 24 items table 6 pattern matrix values for sces (same sample) item number pattern matrix values factor 1 factor 2 factor 3 22 .758 18 .749 19 .726 7 .725 21 .715 6 .708 11 .703 24 .703 27 .696 17 .685 5 .665 .437 9 .616 32 .598 30 .569 1 .558 .429 2 .520 3 .512 23 .799 10 .703 33 .663 29 .510 14 .487 26 .730 8 .716 20 .674 13 .641 16 .327 .607 25 .510 34 .319 .508 table 7 cfa fit indices and results for the sces (same sample) fit indexes acceptable limit perfect fit limit value of scale the scale's fit decision nfi .90 and above .95 and above 0.90 acceptable nnfi .90 and above .95 and above 0.93 acceptable ifi .90 and above .95 and above 0.93 acceptable rfi .90 and above .95 and above 0.89 rejected cfi .95 and above .97 and above 0.94 rejected gfi .85 and above .90 and above 0.72 rejected agfi .85 and above .90 and above 0.67 rejected rmsea between =.050 and =.080 between = .000 and <.050 .080 acceptable tablo 8 kmo and bartlett's test results for different samples kaiser-meyer-olkin measure of sampling adequacy. .717 bartlett's test of sphericity approx. chisquare 794.327 df 276 sig. .000 figure 5 scree plot for the different samples journal of science learning article doi: 10.17509/jsl.v5i2.39826 260 j.sci.learn.2022.5(2).250-265 and four factors as a result of the efa (figure 5); however, this structure could not be confirmed in the cfa using different samples. 3.4. reliability analysis findings the reliability coefficients scale items have been examined for the draft sces consisting of 34 items within the scope of reliability studies, after which the validity studies began. in addition, cronbach’s alpha of reliability has been calculated for the 29 items, and the 3-factor structure was obtained and verified as a result of the validity studies (see table 12). the sces was determined to explain 52.02% of the variance, consist of 29 items and three factors, and have a reliability value of .914 due to the reliability analysis. table 13 provides the reliability coefficients for the sces’s 29 items and their level of impact on reliability upon being removed from the scale. when examining the table, all items can be said to have acceptable values (cronbach, 1951). 4. discussion this study has aimed to develop a valid and reliable scale for evaluating stem clubs. when examining the literature in terms of scale studies, scale adaptation studies are noteworthily predominant (derin, aydin & kirkic, 2017; gelen, akcay, tiryaki & benek, 2019; hacıomeroglu & bulut, 2016) as rearranging an existing scale concerning table 9 total explained variance for the sces (different samples) component initial eigenvalues extraction sums of squared loadings total % of variance cumulative % total % of variance cumulative % 1 7.045 29.353 29.353 7.045 29.353 29.353 2 2.389 9.952 39.306 2.389 9.952 39.306 3 1.781 7.421 46.727 1.781 7.421 46.727 4 1.623 6.761 53.487 1.623 6.761 53.487 5 1.400 5.833 59.321 6 1.093 4.555 63.875 7 1.031 4.295 68.170 8 .978 4.076 72.246 9 .903 3.761 76.007 10 .761 3.170 79.177 11 .725 3.022 82.199 12 .587 2.448 84.646 13 .566 2.360 87.007 14 .550 2.290 89.297 15 .496 2.066 91.363 16 .417 1.738 93.101 17 .386 1.606 94.707 18 .320 1.332 96.039 19 .253 1.054 97.093 20 .184 .768 97.861 21 .176 .733 98.595 22 .147 .613 99.207 23 .111 .465 99.672 24 .079 .328 100.000 table 10 pattern matrix values for sces (different samples) item number pattern matrix values factor 1 factor 2 factor 3 11 .784 27 .776 18 .743 19 .742 24 .696 22 .684 16 .660 17 .636 8 .622 .415 21 .584 30 .578 3 .543 23 .771 4 .681 29 .633 33 .526 12 .464 25 .712 14 .631 13 .359 .603 2 .346 9 7 6 .402 journal of science learning article doi: 10.17509/jsl.v5i2.39826 261 j.sci.learn.2022.5(2).250-265 a different culture and language rather than developing a new scale is thought to save both time and money (oner, 2008). this situation leads to the absence of scales for specific areas (acar-güvendir & özer-özkan, 2015). the current study has developed a scale to evaluate the work of teachers who carry out stem club activities by choosing a subject area that has not been previously studied. in addition, the increase in scale development and adaptation studies in the literature related to stem and science teachers draws attention (corlu et al., 2014; derin, aydin & kirkic, 2017; hacıomeroglu & bulut, 2016; unlu, dokme, & veli, 2016). when examining these studies, they are seen to mostly focus on affective factors such as attitudes (derin, aydin & kirkic, 2017), motivation (donmez, 2020), and awareness (buyruk & korkmaz, 2016; cevik, 2017). no scale study was found regarding obtaining teachers' opinions toward stem clubs or the work carried out by these clubs. the scale developed in this respect is thought to fill a gap in the field and help organize stem club activities. the current research has developed the stem club evaluation scale for teachers who conduct stem club activities. this study followed devellis’ (2014) 8-step scale development method. in addition, the scale items were created based on the constructivist theory, the philosophical approach on which stem education is based, and upon which the scale focuses. although scales having a theoretical basis is important, a limited number of studies are seen to have been shaped within the framework of a theoretical structure in the literature (kizilay, yamak & kavak, 2019). similarly, kizilay, yamak & kavak (2019) study carried out the scale development process according to devellis’ (2014) scale development steps and took into account the motivation-based arcs model. a theoretical framework was formed to ensure the scale’s content validity, a literature review was conducted, expert opinions were taken, and item difficulty and table 11 cfa fit indices and results for the sces (different samples) fit indexes acceptable limit perfect fit limit value of scale the scale's fit decision nfi .90 and above .95 and above .80 rejected nnfi .90 and above .95 and above .87 rejected ifi .90 and above .95 and above .88 rejected rfi .90 and above .95 and above .78 rejected cfi .95 and above .97 and above .88 rejected gfi .85 and above .90 and above .63 rejected agfi .85 and above .90 and above .56 rejected rmsea between =.050 and =.080 between = .000 and <.050 .12 rejected table 12 reliability of the factors and sces dimensions cronbach's alpha cronbach's alpha based on standardized items n of items sces .914 .928 29 teacher .922 17 planning and execution .698 5 student .830 7 table 13 reliability analysis results of sces items item number corrected item total correlation cronbach's alpha if item deleted 1 .569 .910 2 .452 .912 3 .436 .912 5 .578 .910 6 .709 .908 7 .551 .911 8 .505 .911 9 .559 .911 10 .360 .914 11 .719 .908 13 .503 .911 14 .290 .917 16 .683 .910 17 .542 .911 18 .690 .909 19 .652 .910 20 .596 .910 21 .650 .909 22 .731 .908 23 .247 .916 24 .561 .911 25 .217 .918 26 .640 .910 27 .616 .910 29 .260 .917 30 .543 .911 32 .496 .912 33 .369 .914 34 .612 .910 journal of science learning article doi: 10.17509/jsl.v5i2.39826 262 j.sci.learn.2022.5(2).250-265 discrimination indexes were calculated. as a result of these calculations, items were seen to have low item discrimination indexes. however, these items were not immediately removed from the scale until the other analysis results were examined. after enabling the scale's content validity, its construct validity was checked by performing efa and cfa. when examining scale development studies in the field of science education, studies are found where only efas have been performed and no cfas (cermik & kara, 2020; firdaus, subchan & narulita, 2020), where efa and cfa were conducted over the same sample (kizilay, yamak & kavak, 2019; pedaste, baucal & reisenbuk, 2021), and where efa and cfa were performed over different samples (akkus, 2019; burak & gultekin, 2021; fidan & tuncel, 2021; yildirim & sahin-topalcengiz, 2018). when looking at scale development studies in this context, the structure efa reveals is seen should be verified using cfa. on the other hand, a disagreement exists in the literature regarding whether cfa should be performed over the same sample as in the efa or over a different one. therefore, the current study has conducted its efa and cfa using both the same sample and different samples to clarify the confusion in the literature. as a result of the efa performed over the same sample, a structure was obtained consisting of three factors and 29 items explaining 52.02% of the variance, and the cfa confirmed this structure. an analysis that explains 50-75% of the total variance is considered a valid analysis (beavers et al., 2013). the developed scale can be said to be valid in this respect. as a result of the efa performed on different samples, a structure was obtained consisting of four factors and 24 items explaining 53.48% of the variance; however, cfa did not confirm this structure. although the literature argues that a different sample should be used for cfa and efa, in light of the results the current study obtained, the same sample should at least be split in half and analyses made over the two groups. analyzing the same data set with different software has also been said to be sufficient for confirming the factor structure (yaslioglu, 2017). the cfa did not confirm the structure obtained in the current study as a result of the efa made by creating two groups and providing normality for both groups as suggested by the literature. this result can be considered a reference for future scale development studies on the point of factor analysis validation. the factor loads for the items from the three factors have values between .487 and .799. for an item to be included under any factor, it must have a value of at least .30. we calculated item discrimination and difficulty indices before factor analysis. the discrimination index of item 15 was low and negative. as a result of item reliability analysis, we found the reliability of items 15, 28, and 31 negative. karakaplan & yildiz (2010) also removed four items that harmed reliability while calculating the alpha value in their scale development studies. therefore, we did not include these three items in the factor analysis. since the reliability of item 4 was below .20, we did not include it in the factor analysis. the item's correlation coefficient does not affect the item reliability coefficient (cronbach's alpha) value, so the item-total score correlation value should be at least 0.20 (tavsancil, 2002). in the exploratory factor analysis, item 12 was excluded from the scale because it did not fit under any factor. similarly, yolagiden & bektas (2021), in their study of "developing an entrepreneurship scale for science course", removed item 21 from the scale because it did not fall under any factor. the factor loading values for the items on the sces have been considered good in this respect. the factors that were identified by receiving expert opinions are the teacher (factor 1), planning and implementation (factor 2), and student (factor 3). noteworthily, a similar scale development study in the literature found the following factor names: stem’s effect on the student, stem’s effect on the lesson, and stem’s effect on the teacher (cevik, 2017). for example, the "i have sufficient knowledge about stem education." item was asked to determine teacher proficiency regarding stem club practices. for this reason, it is foreseen that this item will be included in the teacher dimension. although the item"the stem club plans i use are not suitable for the student level." mentions stem club plans, it has been evaluated in the student dimension since it emphasizes the suitability of the plans for the student level. although the item "the time allocated for stem club activities carried out in my school is not enough." it seems to be for the teacher and has been included in the planning and implementation dimension because it stems from the structure of the social club rules. within the scope of reliability studies, the reliability coefficients from the draft scale first consisted of 34 items examined each item individually. the validity studies began after examining the reliability coefficients. cronbach’s alpha of reliability was calculated for the 29-item 3-factor scale and was determined and verified as a result of the validity studies. as a result of the reliability analysis, the spss was concluded to explain 52.02% of the variance, consisted of 29 items and three factors, and have a reliability value of .914. yılmaz & cavas (2007) study calculated the reliability of each item by subjecting them to separate reliability analyses in the program spss; the reliabilities calculated for each factor of the scale were seen to range from .54 to .85. when examining the reliabilities for each item in this study, the values were determined to vary between .217 and .719. in addition, the reliability was calculated for each factor. for example, the reliability for the factor of the teacher is .922. the reliability factor for planning and implementation was calculated as .698, and student factor was calculated as .830. thus, the sces is concluded to have high overall reliability for each factor and item (cronbach, 1951). in this context, knowing journal of science learning article doi: 10.17509/jsl.v5i2.39826 263 j.sci.learn.2022.5(2).250-265 whether the obtained results are reliable using the whole scale and for each factor was desired. 5. conclusion as a result of the validity checks, we determined that 29 items and a three-factor structure explained 52% of the variance. as a result of the reliability checks, we calculated the cronbach's alpha reliability as .92. as a result, a valid and reliable measurement tool has been developed by which program practitioners and researchers can measure the effectiveness and efficiency of stem clubs. suggestions • the sces can determine the effectiveness and efficiency of stem clubs at the provincial, district, and school levels. • the developed scale can be used as a data collection tool to create framework plans for stem club studies. • the sces can be applied to different levels of education by making it suitable in terms of language and intelligibility. • the sces can be used as a data collection tool in studies conducted in different socioeconomic and geographical regions regarding academic achievement, gender, and family. references acar-güvendir, m., & özer-özkan, y. (2015). the examination of scale development and scale adaptation articles published in turkish academic journals on education. electronic journal of social science, 14(52), 23-33. https://doi.org/10.17755/esosder.54872 afterschool alliance. (2015). full stem ahead: afterschool programs step up as key partners in stem education. washington, d.c. retrieved from http://www.afterschoolalliance.org/aa3pm/. akgunduz, d., aydeniz, m., cakmakcı, g., cavas, b., corlu, m. s., oner, t., & ozdemir, s. (2015). stem education turkey report. istanbul: scala press. akkus, a. (2019). developing a scale to measure students’ attitudes toward science. international journal of assessment tools in education, 6(4), 706-720. https://doi.org/10.21449/ijate.548516 altunel, m. (2018). stem education and turkey: opportunities and risks. seta perspective, 207, 1-7. ayers, k. a., wade-jaimes, k., wang, l., pennella, r. a., & pounds, s. b. (2020). the st. jude stem clubs: an after-school stem club for upper elementary school students in memphis, tn. journal of stem outreach, 3(1), 1-26. https://doi.org/10.15695/jstem/v3i1.13 baldridge, a., nutt, a., vaughn, m., hartley-lewis, c., & amos, a. (2009). the stem club at marietta high school. asee southeast section conference. baran, e., bilici, s. c., mesutoglu, c., & ocak, c. (2016). moving stem beyond schools: students’ perceptions about an out-of-school stem education program. international journal of education in mathematics, science and technology, 4(1), 9-19.. http://dx.doi.org/10.18404/ijemst.71338 beavers, a. s., lounsbury, j. w., richards, j. k., huck, s. w., skolits, g. j., & esquivel, s. l. (2013). practical considerations for using exploratory factor analysis in educational research. practical assessment, research, and evaluation, 18(1), 6. https://doi.org/10.7275/qv2q-rk76 bell, p., lewenstein, b., shouse, a.w., & feder, m.a. (2009). learning science in informal environments: people, places, and pursuits. washington, dc: the national academies press. birturk, h. (2015). the functionality of social clubs in schools. unpublished master's thesis. yeditepe university institute of educational sciences, istanbul. blanchard, m. r., hoyle, k. s., & gutierrez, k. s. (2017). how to start a stem club. science scope, 41(3), 88-94. burak, d., & gultekin, m. (2021). verbal-visual learning styles scale: developing a scale for primary school students. international journal on social and education sciences, 3(2), 287-303. https://doi.org/10.46328/ijonses.171 buyruk, b., & korkmaz, ö. (2016). stem awareness scale (sas): validity and reliability study. journal of turkish science education, 11(1), 3-23. https://doi.org/10.12973/tused.10179a buyukuzturk, s., kilic-cakmak, e., akgun, o. e., karadeniz, s. & demirel, f. (2016). scientific research methods (22nd edition). ankara: pegem academy. bybee, r.w. (2001). achieving scientific literacy: strategies for ensuring that free choice science education complements national formal science education efforts. in j.h. falk (ed.), free choice education: how we learn science outside of school (pp. 44–63). cermik, h., & kara, i. (2020). physics course attitudes scale for high school students: a validity and reliability study. international journal of assessment tools in education, 7(1), 62-72. https://doi.org/10.21449/ijate.693211 cevik, m. (2017). a study of stem awareness scale development for high school teachers. journal of human sciences, 14(3), 2436-2452. doi:10.14687/jhs.v14i3.4673 corlu, m. s., capraro, r. m., & capraro, m. m. (2014). introducing stem education: implications for educating our teachers in the age of innovation. education and science, 39(171), 74-85. http://hdl.handle.net/11693/13203 coskun, t. k., alakurt, t., & yilmaz, b. (2020). stem education from the perspective of information technologies teachers. abant izzet baysal university journal of the faculty of education, 20(2), 820-836. https://doi.org/10.17240/aibuefd.2020..-536856 creswell, j. w. (2017). research design qualitative, quantitative, and mixedmethod studies (3rd edition) (s. b. demir, trans. ed.). ankara: educating book. cronbach, l. j. (1951). coefficient alpha and the internal structure of tests. psychometrika, 16(3), 297-334. dabney, k. p., tai, r. h., almarode, j. t., miller-friedmann, j. l., sonnert, g., sadler, p. m., & hazari, z. (2012). out-of-school time science activities and their association with a career interest in stem. international journal of science education, part b, 2(1), 63-79. https://doi.org/10.1080/21548455.2011.629455 derin, g., aydin, e., & kirkic, k. a. (2017). a scale on the attitudes towards stem education. el-cezeri journal of science and engineering, 4(3), 547-559. https://doi.org/10.31202/ecjse.336550 devellis, r. f. (2014). scale development: theory and applications. https://doi.org/10.14689/ejer.2016.63.2 donmez, i. (2020). adaptation of stem motivation scale into turkish: validity and reliability study. yyu journal of education faculty, 17(1), 486-510. https://doi.org/10.33711/yyuefd.693825 ebel, r. l. & frisbie, d. a. (1991). essentials of educational measurement. englewood cliffs, nj: prentice-hall. eroglu, s., & bektas, o. (2016). ideas of science teachers took stem education about stem-based activities. journal of qualitative research in education, 4(3), 43-67. https://doi.org/10.14689/issn.2148-2624.1.4c3s3m faber, m., unfried, a., wiebe, e. n., corn, j., townsend, l. w., & collins, t. l. (2013, june). student attitudes toward stem: the development of the upper elementary school and middle/high school student surveys. in 2013 asee annual conference & exposition (pp. 23-1094). https://doi.org/10.17755/esosder.54872 http://www.afterschoolalliance.org/aa3pm/ https://doi.org/10.21449/ijate.548516 doi:10.15695/jstem/v3i1.13. https://doi.org/10.15695/jstem/v3i1.13 http://dx.doi.org/10.18404/ijemst.71338 https://doi.org/10.7275/qv2q-rk76 https://doi.org/10.46328/ijonses.171 https://doi.org/10.21449/ijate.693211 http://hdl.handle.net/11693/13203 https://doi.org/10.17240/aibuefd.2020..-536856 https://doi.org/10.1080/21548455.2011.629455 https://doi.org/10.31202/ecjse.336550 journal of science learning article doi: 10.17509/jsl.v5i2.39826 264 j.sci.learn.2022.5(2).250-265 ferrara, m., mason, h., wee, b., rorrer, r., jacobson, m., & gallagher, d. (2017). enriching undergraduate experiences with outreach in school stem clubs. fidan, m., & tuncel, m. (2021). developing a self-efficacy scale toward physics subjects for lower-secondary school students. journal of baltic science education, 20(1), 38. https://doi.org/10.33225/jbse/21.20.38 firdaus, f., subchan, w., & narulita, e. (2020). developing stembased tgt learning model to improve students' process skills. jpbi (jurnal pendidikan biologi indonesia), 6(3), 413-422. https://doi.org/10.22219/jpbi.v6i3.12249 fraenkel, j. r., wallen, n. e., & hyun, h. h. (2012). how to design and evaluate research in education. fraenkel, j., & wallen, n. (1996). validity and reliability. how to design and research in education. new york: mcgraw-hill, inc, 3, 153-171. gabrielson, e. a., strachan, j. l., warner, m. t., & lafleche, s. h. (2009). evaluating the london science museum’s activity boxes at uk stem clubs. gelen, b., akcay, b., tiryaki, a., & benek, i. (2019). pre-service science teachers’ self-efficacy toward science, technology, engineering, mathematics (stem) survey: an adaptation to turkish, validity and reliability study. journal of theory and practice in education, 15(1), 88-107. https://doi.org/10.17244/eku.395204 george, d., & mallery, p. (2016). ibm spss statistics 26 step by step: a simple guide and reference. (14th ed.). routledge. gonsalves, a., rahm, j., & carvalho, a. (2013). “we could think of things that could be science”: girls' re‐figuring of science in an out‐ of‐school‐time club. journal of research in science teaching, 50(9), 1068-1097. https://doi.org/10.1002/tea.21105 gottfried, m. a., & williams, d. (2013). stem club participation and stem schooling outcomes. education policy analysis archives, 21, 79. doi:10.14507/epaa.v21n79.2013 guzey, s. s., harwell, m., & moore, t. (2014). development of an instrument to assess attitudes toward science, technology, engineering, and mathematics (stem). school science and mathematics, 114(6), 271-279. https://doi.org/10.1111/ssm.12077 hacıomeroglu, g., & bulut, a. s. (2016). integrative stem teaching intention questionnaire: a validity and reliability study of the turkish form. journal of theory and practice in education, 12(3), 654669. http://eku.comu.edu.tr/article/view/5000176286/5000164803 hair, j. f., black, w. c., tatham, r. l. & anderson, r. e. (2019). multivariate data analysis (eighth edition). cengage learning emea. johnson, b., & christensen, l. (2014). educational research: quantitative, qualitative and mixed approaches (trans. ed. demir, s. b.). ankara: egiten book. jöroskog, k., & sörbom, d. (1993). lisrel 8: structural equation modeling with the simplis command language. lincolnwood: scientific software international, inc. kalkan, c., & eroglu, s. (2017). designing sample activities based on stem materials for gifted/talented students in support education rooms. journal of gifted education and creativity, 4(2), 36-46. https://dergipark.org.tr/tr/pub/jgedc/issue/38702/449432 karadogan, s. (2016). in education-school learning practices and daily classroom problems. in academic evaluations and solution suggestions for educational problems in turkey-1, (ed. r aksu), publiser maya academy, 47-84. karakaplan, s., & yildiz, h. (2010). a study on developing a postpartum comfort questionnaire. maltepe university journal of nursing science and art, 3(1), 55-65. kizilay, e., yamak, h., & kavak, n. (2019). motivation scale for stem fields. journal of computer and education research, 7(14), 540-557. https://doi.org/10.18009/jcer.617514 lipuma, j., bukiet, b. g., & leon, c. (2021). hands-on developmental playbook for stem clubs in elementary schools. stem for success resources. 3. https://digitalcommons.njit.edu/stemresources/3 mahoney, j. l., parente, m. e., & lord, h. (2007). after-school program engagement: links to child competence and program quality and content. the elementary school journal, 107(4), 385-404. milner, d. i., horan, j. j., & tracey, t. j. (2014). development and evaluation of stem interest and self-efficacy tests. journal of career assessment, 22(4), 642-653. https://doi.org/10.1177%2f1069072713515427 ministry of national education. (2016). stem education report. http://yegitek.meb.gov.tr/stem_egitimi_raporu.pdf, (access date: 10 september 2021). ministry of national education. (2018). curriculum monitoring and evaluation system-curriculums. http://mufredat.meb.gov.tr/programlar.aspx, (access date: 10 september 2021). mone, (2017). mone educational institutions social activities regulation. national research council (2015). identifying and supporting productive stem programs in out-of-school settings. national academies press. oner, n. (2008). examples of psychological tests used in turkey: a reference (extended 2nd edition). istanbul: bogazici university publishing. pallant, j. (2016). spss user guide step-by-step data analysis with spss. (s. balci & b. ahi, trans.). ankara: ani publishing. pedaste, m., baucal, a., & reisenbuk, e. (2021). towards a science inquiry test in primary education: development of items and scales. international journal of stem education, 8(1), 1-19. polat, b. s. (2017). investigation of opinions of administrators, teachers and students about the effectiveness of social club activities in secondary schools. unpublished master's thesis. ataturk university institute of education sciences, erzurum. qasem, m. a. n., & gul, s. b. a. (2014). effect of items direction (positive or negative) on the factorial construction and criterion-related validity in likert scale. khazar journal of humanities and social sciences, 17(3), 7784. http://hdl.handle.net/20.500.12323/3240 robelen, e. (2011). new stem schools target underrepresented groups. education week, 31(1), 18-19. sahin, a. (2013). stem clubs and science fair competitions: effects on post-secondary matriculation. journal of stem education: innovations and research, 14(1), 5-11. sahin, a., ayar, m. c. & adigüzel, t. (2014). stem related afterschool program activities and associated outcomes on student learning. educational sciences: theory and practice, 14(1), 309-322. http://dx.doi.org/10.12738/estp.2014.1.1876 secer, i̇. (2017). practical data analysis with spss and lisrel: analysis and reporting (3rd ed). ankara: ani publishing. siew, n. m., amir, n., & chong, c. l. (2015). the perceptions of preservice and in-service teachers regarding a project-based stem approach to teaching science. springerplus, 4(1), 8. http://dx.doi.org/10.1186/2193-1801-4-8 straw, a. d., branson, k., neumann, t. r., & dickinson, m. h. (2011). multi-camera real-time three-dimensional tracking of multiple flying animals. journal of the royal society interface, 8(56), 395-409. tabachnick, b. g. & fidell, l. s. (2007). using multivariate statistics (5th ed.). boston, ma: allyn & bacon/pearson education. tavsancil, e. (2002). measuring attitudes and data analysis with spss. ankara: nobel publishing.turgut, m. f., & baykul, y. (2015). measurement and evaluation in education (7th edition). ankara: pegema publishing. tusiad (2014). a research on the demands and expectations of the workforce trained in stem (science, technology, engineering and mathematics, science, technology, engineering, mathematics). tusiad. unlu, z. k., dokme, i., & veli, u. n. l. u. (2016). adaptation of the science, technology, engineering, and mathematics career interest survey (stem-cis) into turkish. eurasian journal of educational research, 16(63), 21-36 http://dx.doi.org/ 10.14689/ejer.2016.63.2 vural, c. (2018). the determination of school pririncipals’ and teachers’ attitude towards educational school clubs. unpublished master's https://doi.org/10.33225/jbse/21.20.38 https://doi.org/10.22219/jpbi.v6i3.12249 https://doi.org/10.17244/eku.395204 https://doi.org/10.1002/tea.21105 https://doi.org/10.1111/ssm.12077 http://eku.comu.edu.tr/article/view/5000176286/5000164803 https://dergipark.org.tr/tr/pub/jgedc/issue/38702/449432 https://doi.org/10.18009/jcer.617514 https://digitalcommons.njit.edu/stemresources/3 https://doi.org/10.1177%2f1069072713515427 http://hdl.handle.net/20.500.12323/3240 http://dx.doi.org/10.1186/2193-1801-4-8 journal of science learning article doi: 10.17509/jsl.v5i2.39826 265 j.sci.learn.2022.5(2).250-265 thesis. i̇stanbul sabahattin zaim university, social sciences institute, i̇stanbul. yaslioglu, m. m. (2017). factor analysis and validity in social sciences: application of exploratory and confirmatory factor analyses. istanbul university journal of the school of business, 46, 74-85. yildirim, b., & sahin-topalcengiz, e. (2018). stem pedagogical content knowledge scale (stempck): a validity and reliability study. journal of baltic science education, 20(1), 38-49. yılmaz, h., & çavaş, p. h. (2007). reliability and validity study of the students’ motivation toward science learning (smtsl) questionnaire. elementary education online, 6(3). yolagiden, c., & bektas, o. (2021). development of entrepreneurship scale for science lesson: a validity and reliability study. afyon kocatepe university journal of social sciences, 23(4), 1349-1365. https://doi.org/10.32709/akusosbil.903893 zengin, n., kaya, g., & pektaş, m. (2020). stem temelli araştırmalarda kullanılan ölçme ve değerlendirme yöntemlerinin incelenmesi. gazi university journal of gazi educational faculty (gujgef), 40(2), 329355. https://doi.org/10.32709/akusosbil.903893 a © 2022 indonesian society for science educator 452 j.sci.learn.2022.5(3).452-468 received: 25 january 2022 revised: 03 august 2022 published: 27 november 2022 the effectiveness of stem-supported inquiry-based learning approach on conceptual understanding of 7th graders: force and energy unit hasan bakirci1, merve gül kirici1, yilmaz kara2* 1department of primary education, faculty of education, van yuzuncuyil university, van, turkey 2department of science education, faculty of education, bartin university, bartin, turkey *corresponding author: yilmazkaankara@gmail.com abstract this research examines the effectiveness of the stem supported inquiry-based learning approach on the conceptual understanding of 7th graders. in the study, a mixed-method design was adopted. the research was carried out with 64 students studying in a secondary school. the study used a conceptual understanding test (cut) and an interview form as data collection tools. quantitative data obtained in the research were analyzed using ancova and t-test. qualitative data were proceeded by subjecting them to content and descriptive analysis. examining the study results, stem supported inquiry-based learning approach increased students’ conceptual understanding in the experimental group and the inquiry-based learning approach in the control group. it was determined that the science teaching in the experimental group was more effective in the conceptual understanding of 7th graders. the students stated that the science teaching in the experimental group was fun, created excitement, made them feel happy, instilled cooperation and team spirit, and thought by doing and living. depending on the results obtained from the research, it was implicated that stem-supported education should be carried out by determining the engineering design process steps suitable for the middle school level. keywords inquiry learning, stem education, conceptual understanding 1. introduction stem activities are learning processes that include students’ knowledge in stem fields, in-school or out-ofschool, and include today’s skills. with such activities, it is seen that students learn science and mathematics concepts more effectively and take the information out of the abstract and make it concrete (göloğlu-demir, tanıkönal, & önal, 2021). at the same time, it is seen that it is effective in permanent learning as it allows information transfer among fields. while stem activities are being prepared, they should not be limited to integrating science and mathematics. there is a need to settle a balance among the fields by ensuring equally strong integrations for engineering and technology. stem activities can be applied to students outside of school as well as in school. it has been observed that the activities carried out in the school are generally implemented within the scope of science or science applications courses, while the activities carried out outside the school are performed under the name of after-school activities, projects, and summer camps. thus, carrying out the processes that depend on engineering design can be the healthiest method to prepare for stem activities (felix, bandstra, & strosnider, 2010). the engineering design cycle, which expresses the engineering design processes described in the science and engineering applications unit in the 5th grade science coursebook published by the ministry of national education (mone) in 2018, consists of the stages: determining the problem, imagining, planning, designing, and testing-development. these design cycles are distributed in all textbook units under "engineering and entrepreneurship practices" (mone, 2018). thus, the engineering design in the 5th grade science textbook consists of processes: asking questions, imagining, planning, creating, and developing steps. in this study, the lesson plans developed as a guide for the teacher were prepared following the current mone curriculum, taking into account the inquiry learning model. following this mailto:yilmazkaankara@gmail.com journal of science learning article doi: 10.17509/jsl.v5i3.43647 453 j.sci.learn.2022.5(3).452-468 model, the engineering design process of hynes et al. (2011) was taken into account and applied indirectly to the steps in the curriculum. furthermore, the engineering design process was considered in developing and implementing the activities used in the learning force and energy unit. force and energy is a unit that is considered difficult by the students and includes more than one discipline at the middle school level (yürümezoğlu, ayaz, & çökelez, 2009). also, force and energy are among the critical concepts that students have difficulty structuring (stylianidou, ormerod, & ogborn, 2002). it has been determined in many studies that even after well-operated instructions, most learners are unable to construct kinetic and potential energy, which are types of energy (coştu, ayas, & ünal, 2007; taşdemir & demirbaş, 2010). furthermore, a study found that middle schoolers had misconceptions about mass and weight (demir & çökelez, 2012). it was reported that the students do not adequately understand the concepts in the force and energy unit, have misconceptions about these concepts, and they contain abstract concepts. it was also mentioned that classical methods were insufficient to remove the misconceptions in this unit (demir & çökelez, 2012). in light of this information in the literature, stem has the potential to enable more understanding in learning the force and energy unit. thus, it is essential to investigate the effectiveness of the stem-supported inquiry-based learning approach in eliminating the misconceptions of 7th graders and learning the concepts. studies on the stem approach differ according to the level of education. in this context, it was found that the most studied groups were middle school students (dumanoğlu, 2018; irkıçatal, 2016; savran-gencer, 2015), teachers, and teacher candidates (wood, knezek, & christensen, 2010). in addition, it was identified that stem studies conducted with preschool and primary school students were limited (faber, unfried, wiebe, corn, & collins, 2013). therefore, this study was performed because there are few studies on stem activities (gülhan & şahin, 2016; savran-gencer, 2015). furthermore, students focus on conceptual understanding in central exams in turkey, and the success rate of students is low in this exam. furthermore, due to the abstractness of the force and energy unit concepts, students have difficulties understanding this unit due to many misconceptions. so, this study has importance in eliminating these limitations and problems in the literature. therefore, the main problem for the research can be phrased as “does stemsupported inquiry-based learning approach has an effect on the conceptual understanding of 7th graders?”. answers to the following sub-problems were sought within the scope of this fundamental problem. 1. is there a significant difference between the experimental and control groups' total scores of conceptual understanding in the pretest and post-test? 2. is there a significant difference between the ancova results of the post-test scores of the experimental and control group students? 3. what are the views of 7th graders about stemsupported inquiry-based science teaching? 2. method 2.1 research design this study, which examines the effect of the inquirybased stem approach on students’ science concepts, was carried out by adopting the mixed method. the mixed method allows the use of both qualitative and quantitative research techniques in a way that eliminates the weaknesses arising from the method or supports each other (yıldırım & şimşek, 2013). the quantitative research dimension consists of a quasi-experimental design to reveal students’ misconceptions at the beginning and end of the study. through the unbiased assignment, one of the classes was assigned as the experimental group while the other was the control group. the quasi-experimental research design allows for the examination of the effect of the application made after the learning activities on the predetermined variables of the two groups that were more or less equal to each other before the applications. in other words, it is in question to examine the reflection of the variables that are assumed to change depending on the application of the post-test scores in two similar groups whose pretest scores are close to each other (christensen, 2004). in addition, this method is one of the methods that provide the most effective results among scientific methods. in the qualitative aspect of the study, interviews were conducted by adopting the case study method to support the findings reached by the quasi-experimental method and obtain more insights into the teaching process. for this reason, the interviewees were assigned only to the experimental group. the adopted research design in the study is presented in table 1. table 1 cat implementation process group pretest implementation process post-test experimental cut hand-on papers and lesson processing + stem activities prepared according to research inquiry-based learning approach cut semi-constructed interviews control hand-on papers and lesson processing prepared according to research inquiry-based learning approach cut journal of science learning article doi: 10.17509/jsl.v5i3.43647 454 j.sci.learn.2022.5(3).452-468 2.2 participants the study was realized with the participation of 64 students. all participants are between 12-13, and 54.68% are female students. participants are studying in a public school where families with low and middle socio-economic income levels are densely populated. while 29 participants were assigned to the experimental group, 35 were to the control group. all students were students at a middle school in the 2018-2019 academic year. the participants were determined according to the easily accessible sampling method. the advantages of this sampling method, such as being economical, enabling the comparison among comparison groups, and providing speed and practicality to the research, enabled the use of this sampling type in this study (çepni, 2011). in the experimental group, ten selected students were interviewed after the implementation. the post-test scores of the conceptual understanding test and voluntariness were considered to select the interviewees. cut test scores are divided into four quartiles according to the low, medium, and high-class average. interviews were conducted with three students from the low and high groups and four from the middle group. due to the ethics of the research, the students were coded as s1, s2, s3… s10. 2.3 data collection tool the conceptual understanding test (cut), consisting of 14 open-ended questions and a semi-structured interview form, was used in the research. these data collection tools are described in detail below. the conceptual understanding test (cut) the researchers developed the cut. the questions were prepared in line with the "force and energy" unit objectives included in the 2018-2019 science curriculum to ensure the scope validity of the questions in the cut. it was submitted to the opinion of three science educators having positions at the university as assistant professors and two science teachers working in the state middle school for 5-10 years to ensure the validity of the cut, which initially consisted of 22 items. the corrections made due to opinions and suggestions separated some questions into a and b sections, and some were removed from the test. the final version of the test consisted of 14 questions. thus, cut was ready for pilot implementation and applied to 70 students in 8th graders. the t-test results for cut are given in table 2. a significant difference was found between the mean scores of the upper and lower groups of 27% in the remaining questions, except for question 1a, question 1b, question 7, question 9, question 13a, and question 13b considering the average of the lower and upper groups. therefore, questions 1a and 1b, question 7, question 9, questions 13a, and 13b were excluded from the test, looking at the data in table 2. the final version of the cut consisted of 10 questions and was implemented as it is. a question in the cut is given in figure 1. table 2 t-test results for item means of upper-lower groups of test items question # groups n m sd t p 1a upper group 19 3.3158 0.74927 1.061 0.296 lower group 19 3.0526 0.77986 1.061 1b upper group 19 3.3158 0.82007 0.404 0.689 lower group 19 3.2105 0.78733 0.404 2 upper group 19 3.6316 0.49559 19.298 0.000 lower group 19 0.4737 0.51299 19.298 3 upper group 19 3.3684 0.49559 17.690 0.000 lower group 19 0.4737 0.51299 17.690 4 upper group 19 3.3684 0.49559 20.171 0.000 lower group 19 0.2632 0.45241 20.171 5 upper group 19 3.3684 0.49559 20.171 0.000 lower group 19 0.2632 0.45241 20.171 6 upper group 19 3.3684 0.49559 17.400 0.000 lower group 19 0.3158 0.58239 17.400 7 upper group 19 3.3158 0.74927 1.061 0.296 lower group 19 3.0526 0.77986 1.061 8 upper group 19 3.6842 0.47757 19.640 0.000 lower group 19 0.5263 0.51299 19.640 9 upper group 19 3.3684 0.49559 17.690 0.689 lower group 19 0.4737 0.51299 17.690 10 upper group 19 3.3158 0.82007 0.404 0.000 lower group 19 3.2105 0.78733 0.404 journal of science learning article doi: 10.17509/jsl.v5i3.43647 455 j.sci.learn.2022.5(3).452-468 interview form this study applied a semi-structured interview form to identify the students’ opinions on the stem supported inquiry-based learning approach. with the semistructured interview, the researchers determined the questions to be researched or asked before interviewing the relevant person or people (yıldırım & şimşek, 2013). the researchers developed the interview form. the opinions of three assistant professors in science education and two science teachers were taken, and the pilot application was made by making the necessary corrections to ensure the intelligibility of the form. at the end of the pilot application, final arrangements were made regarding the questions' clarity, and the form was given its final form. the semi-structured interview form included five questions. the questions in the interview form are given below. 1. what are your thoughts on the applications made in the force and energy unit? 2. what would you like to say about implementing these applications in the “force and energy unit” into other units? 3. do you think that stem activities implemented in terms of the “force and energy unit” contribute to your problem-solving skills? 4. what are the points where you have the most fun and difficulty among the stem activities held in the force and energy unit? 5. do you have any suggestions to make stem activities more fun and educational? 2.4 implementation this study lasted six weeks in the experimental and control groups (24 hours, four hours a week). courses in the control group were performed regarding the inquirybased learning approach. the experimental group carried out activities according to the stem supported inquirybased learning approach. the researchers prepared the lesson plan and worksheets in both groups regarding inquiry-based learning. in addition, stem activities prepared by the researcher were applied to the students in the experimental group. the researcher prepared a teacher table 3 t-test results for item means of upper-lower groups of test items (continued) question # groups n m sd t p 11 upper group 19 3.3684 0.49559 17.400 0.000 lower group 19 0.3158 0.58239 17.400 12 upper group 19 3.6842 0.47757 19.640 0.000 lower group 19 0.5263 0.51299 19.640 13a upper group 19 3.3684 0.49559 17.400 0.701 lower group 19 0.3158 0.58239 17.400 13b upper group 19 1.8947 0.45883 0.387 0.664 lower group 19 1.8421 0.37463 0.387 14a upper group 19 1.9474 0.52427 -0.438 0.000 lower group 19 2.0000 0.00000 -0.438 14b upper group 19 1.4211 0.50726 27.450 0.000 lower group 19 0.4737 0.51299 27.450 figure 1 cut example of a question in the final version journal of science learning article doi: 10.17509/jsl.v5i3.43647 456 j.sci.learn.2022.5(3).452-468 stem activity plan and a student worksheet depending on the stem activities. a total of five stem activities were developed for the force and energy unit. in addition, the experimental group applied a stem activity every week. one of the stem activities developed by the researchers in the study is given in appendix 1, with the evaluation rubric in appendix 2. one sample worksheet is provided as an example of the activities in the control group in appendix 3. the implementations made in the experimental and control groups are explained in detail below. implementation in the experimental group the courses were performed according to the stem supported inquiry-based learning approach. the researchers prepared five different stem activities. these activities were applied in the experimental group, respectively; “let’s build a bridge,” “bow and arrow construction,” “water slide construction,” “catapult construction competition,” and “boat construction.” in these activities, the students were in groups of four, each time coinciding with their different friends. it was table 4 the implementation process of the “force and energy unit” for the experimental group implementations p re t e s t before starting the application, conceptual understanding test (cut) for force and energy unit was applied as a pretest. students were given one lesson hour (40 minutes) for the cut. a c ti v it y 1 : l e t’ s b u il d b ri d g e s introduction: in each of the activities, the main concepts related to the work they will do were expressed to the students. for the “let’s build bridge activity,” information about force weight and bridge force transmission was given to the students. it is explained what should be considered in the construction of the bridge. inquiry: the problem situation for the activity is presented. “people in village a cannot cross directly to the opposite side because of the stream, and they can go up to 10 km from the village and cross from there. the villager reported this problem to the municipality. upon the proposal of the construction companies interested in bridge construction, the municipality says that it will give the construction job to the company that makes the most durable bridge construction. you are one of the construction companies dealing with this bridge business. you aim to make a bridge model that is the strongest, as cheap as possible, and aesthetically appropriate. marbles will be used to measure the durability of the model. “who will be the company that makes the bridge model carries the most marbles as cheap as possible and aesthetically beautiful?” according to the problem, the student bought the materials on the teacher’s desk for an imaginary fee. by sharing tasks between students under the name of “group discussion,” a draft of the bridge model they will make was drawn under the title of “drawing,” and then model making was started under the title of “construction.” evaluation: at this stage, the bridge model was scored by calculating its suitability for purpose, usage status, and durability and, accordingly, how much fictitious money it was made, that is, its cost. in the “changedevelopment” section, they were asked to write down where they would pay attention if given a chance to make the desired model again. finally, students were required to criticize themselves under the questions asked in the “analysis” section. a c ti v it y 2 : b o w a n d a rr o w m a k in g introduction: in each of the activities, the main concepts related to the work they will do were expressed to the students. it explained what the students should pay attention to for "bow and arrow making.” inquiry: the problem given for the activity is as follows: "mustapha lives in the village of pay. his uncle, who came from city during the summer vacation, bought him a toy bow and arrow set as a gift. unfortunately, mustapha broke it while playing with his bow and arrow team friends. seeing that mustapha was very upset, his friends wanted to design a bow and arrow with the means at hand. in this case, consider yourself mustapha's friend. how would you design a sturdy bow and arrow set, as inexpensive as possible and aesthetically pleasing? your goal is to produce a durable, inexpensive, and aesthetic bow and arrow. according to the problem, the student bought the materials on the teacher’s desk for an imaginary fee. by sharing the task between the students under the name of “group discussion,” the bow and arrow they will make were drafted under the title of “drawing,” and then model making was started under the title of “construction.” evaluation: in this section, scoring was made by calculating the suitability of the bow model (by shooting), its use, its durability, and, accordingly, how much fictitious money it was made, that is, its cost. in the “change-development” section, students were asked to write down where they would pay attention if given a chance to make the desired model again. finally, students were required to criticize themselves under the questions asked in the "analysis" section. journal of science learning article doi: 10.17509/jsl.v5i3.43647 457 j.sci.learn.2022.5(3).452-468 evaluated and scored according to the rubric the groups gave during and at the end of the activity. some visuals are included in appendix 4 from the products of students. during the study, the applications made in the experimental group are summarized in table 3. implementation in the control group the courses were conducted regarding the inquirybased learning approach. the 7th-grade science textbook of the mone was used as a source. the inquiry-based learning approach has been handled in three steps. table 5 the implementation process of the “force and energy unit” for the experimental group (continued) implementations a c ti v it y 3 : w a te r s li d e c o n st ru c ti o n introduction: in each of the activities, the main concepts related to the work they will do were expressed to the students. it explained what the students should pay attention to for the “water slide.” inquiry: the problem given for the activity; “ali went to the water park with his family on a summer weekend. as soon as he entered the park, he tried all the slides and wanted to find the one that provided the fastest transition to the water. however, while passing from the slides to the water, he noticed none of them was too fast. in the meantime, he thought about what a waterslide should be like, providing quick water access. if you were the owner of the manufacturer company that designed these slides, how would you offer a solution to ali’s problem? your purpose is to make a model of the slide that is robust, aesthetically beautiful, and as cheap as possible. it provides a quick transition to the water." according to the problem, the student bought the materials from the teacher’s desk for an imaginary fee. by sharing tasks between the students under the name of “group discussion,” the draft of the water slide they will build was drawn under the title of “drawing." then model making was started under the title of "construction.” evaluation: in this section, the water slide model’s suitability for purpose, usage, durability, and, accordingly, how much fictitious money it was made, that is, its cost, were calculated and scored. in the “change-development” section, they were asked to write down where they would pay attention if given a chance to make the desired model again. finally, students were required to criticize themselves under the questions asked in the "analysis" section. a c ti v it y 4 : b u il d in g a c a ta p u lt introduction: in each of the activities, the main concepts related to the work they will do were expressed to the students. it explained what the students should pay attention to for "catapult making.” inquiry: the problem given for the activity: “you are a catapult master living in the 1350s. you learned that the army would also go to conquer a castle. in preparations for the conquest, you and several masters were asked to make catapults to use in battle. how would you make a catapult? your purpose; is to make a solid, inexpensive, and visually aesthetic catapult”. according to the problem situation, the student bought the materials on the teacher’s desk for an imaginary fee. by sharing tasks under the name of “group discussion,” the draft of the catapult they will make was drawn under the title of “drawing.” then model making was started under the title of “construction.” evaluation: in this part, the suitability of the catapult model for its purpose, its use, its durability, and, accordingly, how much fictitious money it was made, that is, its cost, were calculated and scored. in the “changedevelopment” section, students were asked to write down where they would pay attention if given a chance to make the desired model again. finally, students were required to criticize themselves under the questions asked in the "analysis" section. a c ti v it y 5 : b o a t b u il d in g introduction: in each of the activities, the main concepts related to the work they will do were expressed to the students. it explained what the students should pay attention to for "boat building.” inquiry: the problem given for the activity: “boats will race to cross a river. the group that will be the first in this competition takes the boat across the fastest compared to the others’ boats. the engine power of the boats will be the same as the competition. your goal is to get the boat across in the shortest time (fastest). which team will achieve this?” according to the problem situation, the students bought the materials on the teacher’s desk for an imaginary fee. by sharing tasks between the students under the name of “group discussion,” the draft of the boat they will build was drawn under the title of “drawing." then model making was started under the title of "construction.” evaluation: at this stage, scoring was made by calculating the suitability of the boat model for its purpose, its use, its durability, and, accordingly, how much fictitious money it was made, that is, its cost. in the “changedevelopment” section, they were asked to write down where they would pay attention if given a chance to make the desired model again. finally, students were required to criticize themselves under the questions asked in the "analysis" section. p o s t t e s t after the application, the conceptual understanding test (cut) for the force and energy unit was administered as a post-test. students were given one lesson hour (40 minutes) for the cut. at the same time, semi-structured interviews were conducted with the students voluntarily. journal of science learning article doi: 10.17509/jsl.v5i3.43647 458 j.sci.learn.2022.5(3).452-468 during the study, the applications made in the control group are summarized in table 4. 2.5 analysis of data a study by abraham, grzybowski, renner, and marek (1992) benefited from analyzing the data obtained from the answers to the force and energy unit conceptual understanding test (cut) of 7th-grade secondary school students. the cut, prepared by the researcher, was calculated over 72 points (18 x 4 = 72). therefore, the total score for each question is 4. while creating the answer key table 6 the implementation process of the force and energy unit for the control group implementations p re t e s t before starting the implementation, conceptual understanding test (cut) for force and energy unit was applied as a pretest. students were given one lesson hour (40 minutes) for the cut. a c ti v it y 1 : m e a su re m e n t w it h a d y n a m o m e te r introduction: does anyone know the difference between weight and mass?”, “is there a relationship between kinetic energy and gravitational potential energy?” “do you think the gravitational forces of celestial bodies are the same?” the students asked. it was explained to the students that the movement of a person who goes up the mountain is more comfortable, but the movement of a person who lives at sea level is not more comfortable. the reason for this was the focus. the students were asked why an astronaut who went to the moon jumped from the ground higher than on earth, and the reasons were emphasized. both cases were based on the same scientific fact, and students were asked to form their hypotheses about the current situation. inquiry: the class was divided into groups of four, and the necessary materials were distributed to each group. they were first asked to fill in the estimation section on the worksheet. the groups then recorded the necessary measurements for the study. evaluation: the values obtained through the activity were compared. it was concluded that as the number of books increased, the numerical values in the measurement increased. it was stated that if this activity is carried out in high regions, it will decrease. the reason for this was the decrease in the gravitational force exerted by the earth as you go up. the questions in the worksheets were solved. a c ti v it y 2 : c o m p a ri s o n o f k in e ti c e n e rg ie s introduction: “how does increasing the mass of a moving object affect the kinetic energy? “how does increasing the speed of a moving object affect kinetic energy?” questions were asked to the students. after the student predictions, case studies related to the activity were explained. then the students were asked to form their hypotheses about the current situation. inquiry: the class was divided into groups of four, and the necessary materials were distributed to each group. they were first asked to fill in the estimation section on the worksheet. then it was put into practice. first, one end of the wooden board was placed on the floor, and the other was placed on the side where the three books were placed on top to create an inclined plane. next, a wooden block was placed at a distance of 5 cm from the ground-contacting the end of the inclined plane. finally, it was calculated how much the plastic ball and miniature basketball ball sent from the inclined plane moved the wooden block. the same work was repeated by placing four books and the fifth book on the higher side of the wooden block. data saved. evaluation: at the end of the activity, when the plastic ball and basketball ball were released from the same height, it was observed that the basketball dragged the wooden block more. it was commented that this was due to the large mass of the basketball ball. furthermore, as the number of books increased, both balls dragged the wooden block more. it was interpreted that the reason for this situation was the increase in height. a c ti v it y 3 : c o m p a ri n g p o te n ti a l e n e rg ie s introduction: “what does the potential energy of an object depend on?” “does mass affect potential energy?” “does height affect potential energy?" were asked to the students. finally, students were asked to form their hypotheses about the current situation. inquiry: the class was divided into groups of four, and the necessary materials were distributed to each group. they were first asked to fill in the estimation section on the worksheet. next, the basketball ball and the plastic ball were dropped from 50 cm and 100 cm, respectively. finally, the amount of pitting formed in the sand was measured in each process. evaluation: at the end of the activity, the basketball ball was observed to create more depth in the sand when the plastic ball and basketball ball were dropped from the same height. this is because the basketball ball has more mass than the plastic ball. it was observed that the sand depth increased when the height was 100 cm. it was interpreted that the increase in height caused this situation. a c ti v it y 4 : e n e rg y t ra n s fo rm a ti o n s introduction: questions were asked such as “do you think it is possible to switch between energies?” and “how do we explain the motion of an object released from above when it comes to the ground?”. students were asked to form their hypotheses about the current situation. inquiry: the class was divided into groups of four, and the necessary materials were distributed to each group. they were first asked to fill in the estimation section in the worksheet. then, the students released the plastic ball in their hands from a height of 100 cm without using force. when he got to the ground, they saw that he had speed. evaluation: the students stated that the object’s potential energy when it was 100 cm high turned into kinetic energy when it was on the ground. p o s t t e s t after the application, the conceptual understanding test (cut) for the force and energy unit was implemented as a posttest. students were given one lesson hour (40 minutes) for the cut. journal of science learning article doi: 10.17509/jsl.v5i3.43647 459 j.sci.learn.2022.5(3).452-468 for the cut questions, five criteria were determined for the answer part. these criteria are characterized as “complete understanding (4)”, “partial understanding (3)”, “partial understanding with a certain misunderstanding (2)”, “misunderstanding (1)”, and “no understanding (0)". the scoring method is done as in table 5. after scoring, the kurtosis-skewness coefficients and the kolmogorov-smirnov (k-s) test were calculated to examine whether the data obtained with cut in the groups showed a normal distribution. the statistical values of the cut are given in table 6. when table 4 was examined, it was determined that the skewness and kurtosis values of the cut pretest and posttest score distributions of the experimental and control groups remained within the normal distribution limits (+2, -2). therefore, the normality test results are given in table 7 to provide more information on the normality of the data distribution. when kolmogorov-smirnov values given in table 7 were examined, it was seen that the cut pretest and posttest scores of the experimental group were normally distributed (p < 0.05). in this context, the dependent group’s t-test was used for intragroup comparisons, and ancova was used for intergroup comparisons. the analysis results were presented in the findings section in the tables. the data obtained from the force and energy unit semi-structured interview were recorded with a voice recorder and transferred to the electronic environment. later, the data were transcribed and converted into written documents. in this context, the data were simplified by rereading the written documents and removing the subjects outside the research scope. the data were categorized in the findings section, considering the common and divergent points. in this context, interview data were subjected to content analysis. while performing content analysis, three experts read and coded it. the obtained table 7 cut scoring table levels of understanding scoring criteria score complete understanding • answers that include all aspects of a valid answer 4 partial understanding • answers that include one aspect of the valid answer but not all aspects. 3 partial understanding with a certain misunderstanding • responses that show a partial understanding of the information but also contain a misconception 2 misunderstanding • scientifically incorrect answers 1 no understanding • answers containing expressions such as leaving blank, "i do not know,” "i do not understand,” • repeating the question exactly, • irrelevant or unclear answers 0 table 8 experimental and control group cut pre-test and post-test statistics values groups tests n min m sd variance skewness kurtosis experimental pretest 29 3 8.59 3.09 9.54 0.356 -0.679 post-test 29 15 35.24 8.40 70.62 -0.515 -0.317 control pretest 35 2 8.20 3.38 11.40 0.439 0.515 post-test 35 9 24.20 7.76 60.46 0.051 -1.037 table 9 experimental and control group cut pre and post-test normality analysis results groups tests kolmogorovsmirnov shapirowilk statistic sd p statistic sd p experimental pretest 0.161 29 0.051 0.946 29 0.146 post test 0.134 29 0.194 0.961 29 0.351 control pretest 0.133 35 0.124 0.965 35 0.312 post test 0.117 35 0.200 0.959 35 0.217 (p < 0.05) journal of science learning article doi: 10.17509/jsl.v5i3.43647 460 j.sci.learn.2022.5(3).452-468 codes and frequencies are presented in the form of a table. in addition, remarkable views and statements about the codes emphasized by the participants are given under the relevant tables in italics and quotation marks. 3. findings the conceptual understanding test (cut) was administered to the experimental and control groups as a pretest and post-test. in addition, the t-test was used for dependent groups to determine whether there was a significant difference between the pretest and post-test scores. the t-test results are given in table 8. when table 8 is examined, it is seen that there is a statistically significant difference between the cut pretestposttest mean scores of the experimental group in favor of the post-test (t(28) = -19.117; p < 0.05). as a result of this finding, the research shows that applied stem activities effectively develop students' conceptual understanding. at the same time, it is seen that there is a statistically significant difference between the mean cut pretest-posttest scores of the control group in favor of the post-test (t(34) = 13,387; p < 0.05). since it was determined that the cut data of the experimental and control groups were normally distributed, ancova was used to determine whether there was a significant difference between the cut post-test scores of these groups. the pretest scores of the groups from the cut were determined as covariance and included in the analysis to eliminate group differences and the situation of students being affected by the pretest. the statistical values of the cut post-test averages of the experimental and control groups obtained at the end of the analysis are given in table 9. when table 9 is examined, it is seen that the post-test mean score corrected for the pretest was x = 35.01 for the experimental group and x = 24.39 for the control group. in this case, it is understood that the conceptual understanding improved more in the experimental group in which the stem activities were applied than in the control group. however, ancova was conducted to see whether these values differed statistically. the ancova results are given in table 10. when table 10 is examined, it is seen that there is a statistically significant difference between the post-test mean scores according to the cut pretests of the experimental and control groups (f1; 61 = 33.235, p < 0.05). in this situation, it is seen that stem activities create a significant difference between the developmental levels of the conceptual understanding levels of the experimental group and control group students. at the same time, the impact power of the study was found to be 0.353 (in etasquare). from this point of view, cut has a moderate effect and is significant. the views of 7th graders about stem supported inquiry-based science teaching are given in table 11. the analysis of the first question on the interview form, in which students' opinions about stem activities were collected, is listed in table 11 under the "feelings.” participants answered the first question with codes such as “feeling of competing,” “feeling of happiness,” “feeling of responsibility,” “team spirit,” “workload,” “fear,” “fun,” “excitement,” and “inventive sense.” for example, the table 10 dependent groups’ t-test results of experimental and control groups cut pretest and post-test average scores groups tests n m sd df t p experimental pretest 29 8.59 3.09 28 -19.117 0.000* post-test 29 35.24 8.40 control pretest 35 8.20 3.38 34 -13.387 0.000* post-test 35 24.20 7.76 (p < 0.05) table 11. statistics according to cut post-test scores groups n mean s.d. corrected mean experimental 29 35.24 8.40 35.01 control 35 24.20 7.78 24.39 table 12 ancova results of post-test scores adjusted for cut pretests by groups source of variance sum of squares sd mean of squares f p eta-square model 2691.606 2 1345.803 25.069 .000 .451 pretest 758.157 1 758.157 14.122 .000 .188 group 1784.182 1 1784.182 33.235 .000 .353 error 3274.754 61 53.684 total 60547.000 64 (p < 0.05) journal of science learning article doi: 10.17509/jsl.v5i3.43647 461 j.sci.learn.2022.5(3).452-468 participant with the pseudonym s4 stated that he felt happy and responsible and expressed his opinions with fun codes. for example, in the interview with s4, "i am happy when i do the activities. i’m having fun. because we have been given a task, i take responsibility for it to be beautiful”. likewise, the participant with the pseudonym s1 said, “it’s fun and exciting… once we score at the end of the event, it’s like we compete. you have to accomplish the part you are responsible for. this is how you feel…". in addition, almost all participants stated that the activities instilled a sense of fun and responsibility and that they were happy while making the prototype. the analyzes of the second question in the interview form are given in table 9 under the “function of activities” category. participants expressed their opinions on the current question with codes such as “making joint decisions,” “active participation,” “being fun,” “achieving your mission,” “memorability,” and “lack of information/efforts to access information.” table 13. themes and codes obtained from students’ answers about stem-supported inquiry-based science teaching codes s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 f f e e li n g s the feeling of being in the competition + + + + + 5 feeling of happiness + + + + + + + 7 feeling of responsibility + + + + + + + 7 collaborative/team spirit + + + 3 workload + + + + 4 fear + + + 3 fun + + + + + + + + 8 excitement + + + 3 inventive sense + + 2 f u n c ti o n o f a c ti v it y making joint decision + + + + + + 6 active participation + + + + 4 being fun + + + + 4 achieving your mission + + 2 memorability + + + + + 5 lack of information/efforts to access information + 1 im p a c t o n p ro b le m s o lv in g creative thinking + + + + + + 6 learning by doing + + + + 4 detailed thinking + + 2 collaborative work + + + + 4 confidence in the face of troubles + + + 3 group work + + 2 f u n a n d d if fi c u lt p o in ts ignoring the visuality of the prototype + + + + + + + 7 testing the working state of the prototype + + + + + + 6 scoring at the end of the event + + + + + 5 being in a struggle + + + 3 making the prototype sturdy + + + + + + 6 time keeping + + + + + 5 making the prototype the cheapest + + + 3 journal of science learning article doi: 10.17509/jsl.v5i3.43647 462 j.sci.learn.2022.5(3).452-468 for example, the participant with the code s10 said, “of course, yes. normally, i would learn with a pencil, notebook, or stationary material if it was in a book. if i didn’t repeat, i would have forgotten. but here, we are the group decision maker. thus, through the activities, i do not forget my information, and because i did it, it stays in my mind, i never forget it”. likewise, if the participant with the code s6, “we act as a group in events. so, everyone is trying to do their best. by adding something from ourselves, we learn by ourselves”. the third question of the interview form was given under the “impact on problem solving” category. the participant with code s2 said, “i started to trust myself in problem-solving. now, i’m not saying i can’t do it when i encounter a problem. in addition, through the activities, i set the rules in my mind by creating and living them”. at the same time, the participant with the code s8 stated his opinion “if i had done this activity by myself, not as a group, i probably not be able to get it to the end. because while i approached a problem from one side, my other friends approached it from different directions. through discussions, i learned how they thoughtfully solved the problem. i noticed that my creativity increased. i can say without hesitation the ideas that seem ridiculous at first and that i am ashamed to say”. the fourth question of the interview form was analyzed under the category of “fun and difficult points.” participants answered this question as “ignoring the visuality of the prototype,” “testing the working state of the prototype,” “scoring at the end of the event,” “being in a struggle,” “making the prototype sturdy,” “timekeeping,” and “making the prototype the cheapest.” for example, the participant with code s3 said, “the things i like in the activities; i was always in a struggle. i was surprised that time passed quickly. we tried to make the model look beautiful by making it colorful and symmetrical. the most difficult thing was that the prototype worked as desired. if it was not as desired, we could not get points and were upset. we were also afraid that time would not be enough". in addition to this situation, participant s5 stated, "i had much fun scoring at the end of the activities. it was challenging to pay attention to the robustness, cost, and aesthetic categories, but i had much fun". the last question of the interview form was handled under the category of “suggestions.” through the answers, it was revealed that the suggestions reflected in participant statements could be categorized under categories such as “more event creation,” “variety of materials,” and “ignoring costs for materials.” for example, the participant with the code s7 replied, "the color is crucial. there would be a price difference if i chose the colored one. i think these should not be”. the participant with the code s9 said, “during the activities, most of the groups were immediately sharing tasks and moving on to the design of model making. however, our group was always slow. i think more time should be given to this. afterward, we got used to this pace, so we accelerated, but still, much time should be given”. 4. discussion first, the pretest and post-test scores of the students in the experimental group were compared in the study. a statistically significant difference was found in favor of post-test scores regarding conceptual understanding (see table 8). this finding shows that the stem supported inquiry-based learning approach applied in the experimental group increased students' conceptual understanding. the engineering design process effectively prepares stem activity (felix et al., 2010). in addition, the activities in the science textbook since 2018 by the mone have been prepared according to the engineering design cycle steps. using the stages of determining the problem, imagining, planning, designing, and testing-developing the problem is practical during the activities performed in the experimental group. when the control group students’ cut pretest and post-test scores were compared, a statistically significant difference was found in favor of the post-test scores (see table 10). these analyses can be interpreted as the fact that the inquiry-based learning approach applied in the control group effectively conceptualizes 7th graders. this increase in conceptual understanding in the control group is due to the worksheets developed by the researchers. these worksheets were student-centered and focused on concepts. the items in open-ended format were placed in the evaluation stage of the worksheet, and these questions measure high-level thinking skills (bakırcı & ensari, 2018). regarding similar studies, it was concluded that studentcentered activities are effective in the conceptual understanding of 8th graders (bakırcı & çalık, 2013). when the experimental and control groups' post-test scores of the cut were examined, it was determined that there was a statistically significant difference in favor of the experimental group. this situation shows that the stem supported research inquiry-based learning approach applied in the experimental group and the inquiry-based learning approach applied in the control group were more effective in conceptualizing 7th graders (see table 9). in table 14. themes and codes obtained from students’ answers about stem-supported inquiry-based science teaching (continued) codes s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 f s u g g e s ti o n s more event creation + + + + + + 6 variety of materials + + + 3 ignoring costs for materials + + + + + + + 7 giving longer time to the making of events + + + + + 5 journal of science learning article doi: 10.17509/jsl.v5i3.43647 463 j.sci.learn.2022.5(3).452-468 other words, it is understood that the conceptual understanding of the stem activities applied experimental group improved more than the control group. when the cut post-test increases of the experimental and control groups were examined, it was determined that the difference between the experimental and control groups was 10.20 in favor of the experimental group. in the ancova analysis of cut, a significant difference was found in favor of the experimental group between the control and experimental groups' post-test scores. this difference is probably caused by the implementation of the work papers prepared in parallel with the mone textbook implemented in the experimental group and the implementation of stem activities. it is thought that applying more than one way of thinking to the problem solution in the worksheets given to the students and testing them by trying helps to increase their conceptual understanding. it was mentioned that stem activities improve students’ conceptual understanding in many studies (irkıçatal, 2016). stem activities have a crucial role in improving students’ interest, success, attitude, and motivation and in concluding the problems encountered in daily life (honey, pearson, & schweingruber, 2014). each activity was prepared by the researcher and started with a problem situation to solve the problem encountered in daily life. by adhering to the engineering design process steps, students tried to solve the concept arising from the encountered problem (hynes et al., 2011). at the same time, attention was paid to the criteria that the prototype they created should appeal to the eye, be made at a low cost, and be robust. it was emphasized that it is essential to implement and conduct the prepared activities this way (savran-gencer, 2015). in the first question of the semi-structured interview form, the students stated that they competed, felt responsible, and worked in a team spirit. in addition, they stated that they had fun, were excited, and felt like inventors. so, it was the indicator for the activities that are a new practice for the students, they take an active role in the activities, and the idea of being successful in each group contributes to the positive thinking of the students. in addition, it has been determined that stem-supported science teaching increases students' motivation, develops a positive attitude toward the lesson, and provides effective learning (yıldırım, 2016). however, some students also underlined that they feared failing to succeed in the task they took during the activity (see table 11). this situation can be attributed to the belief that some students, who are generally unsuccessful in the lessons, are hesitant to attend and express themselves in a group of friends. furthermore, yıldırım and selvi (2016), in their study with pre-service teachers, identified that the pre-service teachers held both positive and negative opinions on stem. in this context, although the study group was different in the literature, it can be said that the obtained results were parallel with the results of this study. in the second question of the interview form, students were required to state opinions on the application of activities in other units. the students who participated in the interview thought positively about the performed stem applications and agreed on implementing similar activities in other units. the positive thoughts of the students can be interpreted as fruitful that they reach the information themselves during the activity, take an active part in the application, and express their opinions in group discussions. in the interview with the students, it is understood that it is easy for them to learn the subject, the information is permanent, and they learn by having fun. in addition, the worksheet related to stem activity has some features. these features include giving the problem situation to the students, determining the materials for the prototype themselves, making the prototype with their creativity, and providing the product formation with their efforts. karahan, canbazoglu-bilici, and ünal (2015) stated that students had an enjoyable learning experience in studies on the stem education approach. the third interview question examined the applications’ contributions to the students’ conceptual understanding level. students stated that they could think creatively, learn by doing and experience, think versatile and detailed, work collaboratively, and try to solve the problem by relying on themselves in case of a problem through stem activities. it is effective for students to combine the prototype for the problem given during the activity with their imagination and creativity. in addition, activities such as drawing the prototype of the activity that the students will do and creating a model are effective. in addition, the activities are effective in thinking quickly in the face of the problem, making joint decisions, and trusting themself and their group mate during model formation. therefore, it has been determined that studies on the stem approach improve students’ creativity and imagination and help them think quickly and produce solutions (tiryaki & adıgüzel, 2021). in the applications made with the fourth interview question, the points that the students had fun with and had difficulties with were questioned. for example, during the stem activities, the students stated that they had fun decorating the prototype, trying to see whether it worked, and scoring at the end of the activities. on the other hand, they stated that they had difficulties during the prototype construction due to the timing and wanted to avoid cost calculations due to the pricing of materials. it is thought that students' negative thinking is because they want to extend the required time by thinking quickly and creatively to get higher scores during the evaluation, which is evaluated based on the appropriateness of the cost calculation. the last interview question included the students' suggestions about the practices. the students stated that journal of science learning article doi: 10.17509/jsl.v5i3.43647 464 j.sci.learn.2022.5(3).452-468 more activities should be done, the duration of the activities should be extended, and the materials used for the activities should include more variety. it can be said that having fun, being happy, and spending time with their group friends for a purpose is effective the basis of the thoughts of increasing the number of activities of the students. however, insufficient application time can explain the student’s requests to extend the activity period. this situation can be associated with students' learning speed and individual differences. in addition, when the reasons underlying the subject of increasing the variety of materials are examined, there is no shortage of tools. for example, they want to buy a colored product to compensate for the time it takes to color material to ensure its visuality. 5. conclusion the findings obtained through the student interviews supported the quantitative data gathered through the cut. the students stated that stem-supported science teaching enables learning by doing and experiencing, that they take responsibility while learning the subject, and that this situation enables learning by having fun. in addition, the students stated that their problem-solving skills improved because they used the engineering design cycle in their activities. it has been understood that the designs made for force and energy unit contribute to the effective learning of subjects. in this study, it was concluded that the stem supported inquiry-based learning approach was effective in conceptualizing 7th graders in the force and energy unit. during the formation of the groups, it is recommended that the students should be allowed to form the friends they want in the first activities. then it is better to form the groups randomly for the subsequent activities. thus, the heterogeneity within the group will be reflected in the groups, and the responsibility taken by each student and the communication with his groupmates will increase. in this way, it can be ensured that students respect each other. it is recommended not to show the materials when the activity worksheets are distributed to the groups. it can be instilled that students need to decide on their own which materials they should use for a suitable solution to the problem situation. at the same time, students can be expected to write the materials they want by leaving the materials section blank. during future activities, it is recommended that the students make the desired prototype by providing more than one feature (resilience, cost, and aesthetics) by their level. at the same time, students can be informed by including the desired features in the evaluation form in the scoring. acknowledgments this study was produced from the dissertation of merve gül kirici for the master’s degree titled “the effect of stem supported research questioning based learning approach on the conceptual understanding and scientific creativity of 7th grade students”. references abraham, m. r., grzybowski, e. b., renner, j. w., & marek, e. a. (1992). understandings and misunderstandings of eighth graders of five chemistry concepts found in textbooks. journal of research in science teaching, 29, 105-120. bakırcı, h., & çalık, m. (2013). effect of guide materials developed in adaptation and natural selection subject on remedying grade 8 students’ alternative conceptions. education and science, 38(168), 215-229. bakırcı, h., & ensari, ö. (2018). the effect of common knowledge construction model on academic achievement and conceptual understandings of high school students on heat and temperature. education & science, 43(196), 171-188. christensen, l. b. (2004). experimental methodology. united states of america: pearson education. coştu, b., ayas, a., & suat, ü. (2007). misconceptions about boiling and their possible reasons. kastamonu education journal, 15(1), 123-136. çepni, s. (2011). introduction to research and project studies. trabzon: celepler printing. demir, & çökelez (2012). investigation of the conceptual learning of 6th grade students on the concepts of mass, weight and gravity. x. national science and mathematics education congress, 27-30 june nigde, turkey. dumanoğlu, f. (2018). effect of science, technology, engineering and mathematics activities on seventh grade students’ academic achievement and attitudes (master’s thesis). istanbul university, turkey. faber, m., unfried, a., wiebe, e. n., corn., j., & collins, t. l. (2013, june). student attitudes toward stem: the development of upper elementary school and middle/high school student surveys. 120th asee annual conference & exposition, 23-26 june, atlanta, georgia. felix, a. l., bandstra, j. z., & strosnider, w. h. j. (2010h). design-based science for stem student recruitment and teacher professional development. mid-atlantic asee conference, 16-17 april, easton, pennsylvania. göloğlu-demir, c., tanık-önal, n., & önal, n. (2021). investigation of middle school students’ attitudes towards science, technology, engineering and mathematics (stem) education and determination of the predictors. journal of science learning, 4(2), 101112. gülhan, f., & şahin, f. (2016). the effects of science-technologyengineering-math (stem) integration on 5th grade students’ perceptions and attitudes towards these areas. international journal of human science, 13(1), 602-620. honey, m., pearson, g,. & schweingruber, h. (2014). stem integration in k-12 education: status, prospects, and an agenda for research. washington: the national academies press. hynes, m., portsmore, m., dare, e., milto, e., rogers, c., hammer, d., & carberry, a. (2011). infusing engineering design into high school stem courses. http://ncete.org/flash/pdfs/infusing%20engineering%20hynes. irkıçatal, z. (2016). stem related after-school program actıvıtıes and assocıated outcomes on students success and on theır stem perceptıon and ınterest (master’s thesis). akdeniz university, turkey. karahan, e., canbazoglu-bilici, s., & ünal, a. (2015). integration of media design processes in science, technology, engineering and mathematics education. eurasian journal of educational research, 60, 221-240. ministry of national education [mone]. (2018). science curriculum. ankara: state books printing house. journal of science learning article doi: 10.17509/jsl.v5i3.43647 465 j.sci.learn.2022.5(3).452-468 savran-gencer, a. (2015). scientific and engineering practices in science education: twirly activity journal of inquiry based activities, 5(1), 119. stylianidou, f., ormerod, f., & ogborn, j. (2002). analysis of science textbook pictures about energy and pupils readings of them. international journal of science education, 24(3), 257-283. taşdemir, a., & demirbaş, m. (2010). the level of correlation of concepts that primary students seen topics in science and technology class with daily life. international journal of human sciences, 7(1), 124-148. tiryaki, a., & adıgüzel, s. (2021). the effect of stem-based robotic applications on the creativity and attitude of students. journal of science learning, 4(3), 288-297. wood, t. t., knezek, g., & christensen, r. (2010). instruments for assessing interest in stem content and careers. journal of technology and teacher education, 18(2), 341-363. yıldırım, b. (2016). an examination of the effects of science, technology, engineering, mathematics (stem) applications and mastery learning integrated into the 7th grade science course (phd thesis). gazi university, turkey. yıldırım, a., & şimşek, h. (2013). qualitative research methods in the social sciences. ankara: seçkin publishing. yıldırım, b., & selvi, m. (2016). examination of the effects of stem education integrated as a part of science, technology, society and environment courses. journal of human sciences, 13(3), 3684-3695. yürümezoğlu, k., ayaz, s., & çökelez, a. (2009). grade 7-9 students’ perceptions of energy and related concepts. necatibey faculty of education, electronic journal of science and mathematics education, 3(2), 52-73. appendix 1: let’s make bridge activity grade: 7th grade unit: force and energy unit duration: 40+40+ 40+40 minutes materials used the following materials will be left on the teacher’s desk as much as the number of groups. students will be free to take the required material from the table they believe (you can use the same material more than once. the calculation of the total fee will be made accordingly). the price of each material is listed under the student worksheet. at the end of such an activity, the expenditure of the students for the bridge model will be calculated. • background paper • adhesive (90ml-60ml) • paper cup • cardboard • rope • pen • scissors • silicone wick • ruler • silicone gun • marble balls • scissors • utility knife • tape • highlighter pricing (tl: turkish liras) cardboard: 5 tl, silicone wick: 1 tl, silicone gun+1 silicone wick: 30 tl, background paper: 3 tl, ruler: 2 tl, scissors: 1 tl, utility knife: 2 tl, adhesive 90 ml: 3 tl, adhesive 60 ml: 2 tl, pencil ballpoint pen: 2 tl, highlighter thick tip pen: 3 tl rope: (1 m) 2 tl(2 m) 3 tl, tape: 2 tl. preparation • students are divided into groups of 6 people. • groups find a group name. problem statement people in village a cannot cross directly to the opposite side because of the stream. they should go up to 10 km from the village and cross from there. the villager reported this problem to the municipality. upon the proposal of the construction companies interested in bridge construction, the municipality says that it will give the construction job to the company that makes the most durab le bridge construction. you are one of the construction companies dealing with this bridge business. you aim to make a bridge model that is the strongest, as cheap as possible, and aesthetically appropriate. marbles will be used to measure the durability of the model. who will be the company that makes the bridge model that carries the most marble, is as cheap as possible, and is aesthetically beautiful? research: ruined or solid? • the groups draw the bridges that will examine their durability by designing them on the worksheets. • they hypothesize how many marbles will be carried by the bridges they produce. • each group tests its designed bridges and records the obtained data. • they discuss the reasons for the collapse of the collapsed bridges. introduction to the main terms to be learned: force: the effect that stops a moving object, makes a stationary object move and changes the shape, direction, and direction of the object is called force. journal of science learning article doi: 10.17509/jsl.v5i3.43647 466 j.sci.learn.2022.5(3).452-468 weight: the gravitational force exerted by gravity on the mass of an object is called weight. weight is a force. thus, the weight applied by the marble is mentioned. transmission of force in bridges: forces were transmitted and distributed from the upper part to the lower legs, that is, to the feet. it transfers the load collected in the center towards the shores and the bridge piers. imagination: students will be allowed to think about the bridge design. they will conduct peer discussions and brainstorm on constructing a bridge model for durability, aesthetic appearance, and cheapness. planning: each group is going to make a decision on the design procedure for the imagined bridge. they are required to construct a bridge. materials were provided by the number of groups (for example, if there are five groups, there are at least five of a material). the students take the materials they need according to their designs. each group member will be responsible for an engineering task (the task can take the form of designing, building, and testing durable bridges). it should be noted that the estimates are established before the students start the designs. according to the group’s decision, the bridge model will be 45 cm long. “activity evaluation rubric,” which indicates the criteria for evaluating students at the planning stage, will be presented to the students with a smart board. creating: students begin to construct the bridge they designed. testing: the bridges are tested when the groups have completed their designs. marbles will be used in testing the durability of the bridge design. development: after the groups test their bridge’s resistance to weight, the students are asked, “what changes would you make to strengthen the bridge?”. extra time is given if the bridge wants to be improved in design. communication: groups, in turn, explain their bridges to other groups. meanwhile, groups can ask each other questions about their designs. finally, after the completion of the build and testing process, the results are discussed. • which bridge design was the most durable? • are the marbles used in the evaluation considered by mass or weight? • which kind of bridge does your model belong to? • how many marbles could the design bridges withstand? appendix 2: evaluation rubric for bridge models the bridges the groups made will be evaluated according to the rubric below. the group with the highest score will be the winner of the competition. categories excellent😉 good😊 weak☹ 1-the group understands the problem situation. understands the problem clearly. this point is given if the teacher is not asked a confirmation question about the bridge. students ask one or two confirmation questions such as “teacher, can we do this and that?” this point is given. students ask more than two confirmation questions to the teacher, such as “teacher, can we do this and that?” this score is given. 2-group work is done very well. this point is given if he can get along with his groupmates without problems. (students are monitored and scored by the teacher throughout the study.) this point is given if there is a disagreement with the group mates once or twice. (students are monitored and scored by the teacher throughout the study.) if the group has a long-term problem in the distribution of tasks among themselves and exhibits behavior that prevents the creation of the product, he gets this score. (students are monitored and scored by the teacher throughout the study.) 3-the design was an aesthetically pleasing bridge. this point is given if attention is paid to the absence of adhesive or silicone traces, the symmetry of the cuts, and color harmony in the work on the bridge. this point is given if the rules of no glue or silicone traces, symmetrical cuts, and color harmony are followed only once in work on the bridge. this point is given if the rules of no glue or silicone traces, symmetrical cuts, and color harmony are followed only once in work on the bridge or if the product still needs to be finished. 4the group made the bridge model as cheaply as possible. the group that makes the cost of the materials the cheapest among the groups gets this score. when we list the materials cost from the lowest to highest among the groups, the 2nd, 3rd, and 4th groups get this score. the group that makes the cost of the materials the most expensive among the groups gets this score. journal of science learning article doi: 10.17509/jsl.v5i3.43647 467 j.sci.learn.2022.5(3).452-468 5the bridge model made by the groups was solid. if it carries 700 marbles or more, it gets this score. if it has between 200-700 marbles, it receives this score. if it takes less than 200 marbles, it reaches this point. 6-the sketch of the group is clear and understandable. this point is given if the group paid attention to the fine details during the drawing and explained by making explanations. if the group did not pay attention to the fine details or did not explain during the drawing. if the group did not pay attention to the fine details and did not explain during the picture. 7-the analyzing part of the group is clear and understandable. the group could notice its positive or negative aspects and express them clearly. if only the positive or only negative aspects of the group are stated in the analysis section. in the analysis section, if the group has written a single word, left it, or did not make the necessary explanation. 8-the students’ estimations are compatible with the estimation results. it gets this point if it carries 25 marbles, more or less from the students’ guesses. it gets this point if it takes 50 marbles, more or less from the students’ guesses. from the students’ guesses, it reaches this point if it has more than 50 marbles or less than 50. * in the competition, scoring was based on this formulation: excellent = 3 points, good = 2 points, weak = 1 point. appendix 3: lesson plan example for inquiry-based learning approach course name: science class: 7th grade subject: friction force objective: 7.3.1.1. students call the gravitational force acting on the mass weight. duration: 4 lesson hours tools: 4 dynamometers, four packages, four textbooks implementation steps introduction: “does anyone know the difference between weight and mass?” question is asked to the students. then, it is explained to the students that the movement of a person who goes up the mountain is more comfortable, but the direction of a person who lives at sea level is not yet. the reason for this is discussed. next, students are asked why an astronaut who goes to the moon jumps from the ground higher than on earth. it focuses on why this is the case. it is emphasized that both cases are based on the same scientific fact, and students are asked to form their hypotheses about the current situation. inquiry: the “measuring with a dynamometer” activity is adapted from the meb book and distributed to students as worksheets. the class is divided into four groups, and each group is given a dynamometer, bag, and textbook. they are instructed to fill in the forecast section of the worksheet. first, the groups are asked to place a book in the bag, measure it on the dynamometer, and record the data. afterward, the number of books set in the pack increases, and the measurements are repeated in the dynamometer. evaluation: the values obtained as a result of the activity are compared. as the number of books increases, it is concluded that the numerical values in the measurement increase. it is stated that this activity will decrease if it is carried out in high regions. the reason for this is that the gravitational force exerted by the earth decreases as you go up. focus on question 1 in the worksheet. worksheet “measuring with a dynamometer” 1) prediction: in the table below, write your measurement value estimates in the dynamometer as the number of books increases. number of books estimated numerical value in dynamometer 1 2 3 4 journal of science learning article doi: 10.17509/jsl.v5i3.43647 468 j.sci.learn.2022.5(3).452-468 2) observation: in the table below, write the measurement values read in the dynamometer as the number of books increases. 3) explanation: a) as a result of your estimates and measurements, explain the relationship between the number of books and the reading on the dynamometer. b) what are your activity's dependent, independent, and control variables? the dependent variable: independent variable: control variable: question: is it possible for the weight of an object weighing 10 kg to be the same everywhere on earth? why? appendix 4: visuals of student products in experimental group products of “let’s make bridge activity” products of “bow and arrow activity” products of “water slide activity” number of books numerical value read on dynamometer 1 2 3 4 a © 2022 indonesian society for science educator 301 j.sci.learn.2022.5(2).301-320 received: 5 october 2021 revised: 10 february 2022 published: 27 july 2022 determination of science students' awareness on waste management gonca harman1, nisa yenikalayci2* 1department of science education, faculty of education, karamanoğlu mehmetbey university, karaman, turkey 2department of science education, graduate school of educational sciences, gazi university, ankara, turkey *corresponding author: nyenikalayc@gmail.com abstract in this research, it was aimed to determine science students’ awareness on waste management. eleven students studying in the first year of the science education department participated in the research. the screening model was used in the research. the data were collected through scientific newspapers prepared in line with the scientific newspaper preparation instruction within the scope of waste management. in the five sections of the instruction, students were asked to prepare scientific newspapers that they will support with written explanations and drawings on recovery, reuse, recycling, plastic bag usage, and zero waste within waste management. the data were analyzed by using content analysis. as a result of the research, it was understood that a significant part of the students was aware of the effects of education, research, and project activities on recovery, reuse, recycling, plastic bag usage, and zero waste practices within the scope of waste management, and the place and importance of the individual in waste management. on the other hand, it was understood that most of the students were not aware of the basis of waste management practices as the waste types, the separation of wastes at the source in accordance to their types, and throwing the wastes into the appropriate waste bin for their types. keywords sustainability, waste management, awareness, scientific newspaper, science student 1. introduction waste is defined as any substance or material thrown, left, or obligatory discarded to the environment by the producer or the person in possession (url). if it is appropriately managed, waste has an economic value (muljaningsih & galuh, 2018). so much so that many countries consider waste as material and energy (cucchiella, d'adamo & gastaldi, 2017). once the waste composition is known, strategies for separation, collection, and treatment alternatives such as recycling and composting can be determined. this situation allows that waste is being removed from landfills (nolasco et al., 2021). otherwise, the negative effects of the waste will occur. in this respect, waste causes various problems such as health problems, bad smells, fire hazards, atmosphere, soil, water, and visual pollution, together with social and economic losses (dharmasiri, 2019). waste is a serious problem for air, water, and soil. as a result of the burning of trash stored in open areas, toxic gases are released, such as carbon dioxide and carbon monoxide, which cause health problems. waste pollutes water by falling into rivers and puddles; at the same time, it causes an increase in the fly and mouse population. the physical properties of the soil change, and the growth of plants are adversely affected. in addition, trashes cause a decrease in the aesthetic value of the environment and the disappearance of the landscape (dharmasiri, 2019). considering all these factors, waste should be seen as a cultural problem affecting life. to minimize the negative effects of this problem on the environment, waste should be appropriately managed (naria, nasution & santi, 2018). waste management covers the activities of preventing the formation of waste, reduction at the source, reuse, separating according to its characteristic and type, accumulating, collecting, temporary storage, transportation, intermediate storage, recycling, recovery including energy, disposal, monitoring after disposal processes, control and inspection (url). waste management is a long process that is creative and full of opportunities. in this process, waste should not be considered a problem but a resource that society will benefit from (kumar & kumar, 2020). a five-step process is followed in waste management (figure 1). the first preferred step in this process is preventing waste. then, journal of science learning article doi: 10.17509/jsl.v5i2.39376 302 j.sci.learn.2022.5(2).301-320 preventing waste is followed by reuse, recycling, recovery including energy, and safe disposal is applied as the last step. benefits obtained from the regular implementation of the waste management process are preventing greenhouse gas emissions, reducing pollutants, energy-saving, conservation of resources, new employment opportunities, and developing green technologies (cucchiella, d'adamo & gastaldi, 2014). the 3r (reduce, reuse, and recycle) rule is one of the most effective methods implemented in waste management (naria, nasution & santi, 2018). this rule comes to the fore in reducing waste, which has become a significant environmental problem (aksan & çelikler, 2019). in the integrated waste management system, following the approaches covered by the 3r rule, reducing wastes at the source, reusing, and recycling; then, incineration, and final disposal processes are carried out close to the waste source (asteria & haryanto, 2021). however, 3r activities face a significant obstacle if society's awareness of separating or classifying waste is low (muljaningsih & galuh, 2018). reusing waste for another purpose or in a different way is called reuse. people can reuse many materials such as plastic bags, furniture, toys etc. and repair some broken materials they use. they may also sell to others or give to charity. additionally, recycling is when waste is reshaped and converted into raw materials that can create a new product (dharmasiri, 2019). the first step in implementing waste material recycling plans is the proper implementation. therefore, a household waste recycling program should be followed when considering the environmental, economic, and social effects of recycling (almasi et al., 2019; asteria & haryanto, 2021). so much so that harman & çelikler (2016) determined that most of the pre-service science teachers had sufficient awareness of the meaning, necessity, and purpose of recycling. in contrast, their understanding of recyclable waste was not enough. waste is a result of human activities. every human activity produces waste. the type of waste depends on the types of materials that people consume, and waste management cannot be separated from people's lifestyles (muljaningsih & galuh, 2018). lack of waste management, which is a direct result of human activity, is one of the main problems of modern civilization that has emerged from the contemporary lifestyle (kumar & kumar, 2020). the 3r rule is widely accepted in waste management to solve the waste problem. in addition to this rule, responsibility is also important in promoting waste management behaviors (minelgaitė & liobikienė, 2019). in an ideal world, everyone should be conscious of managing waste appropriately, as everyone is responsible for environmental issues, including waste management (ampofo, 2020; dharmasiri, 2019). in support of this, aksan, çelikler & yenikalaycı (2019) pointed out that science students have a high awareness of recycling, and individuals with a high level of recycling awareness will be more likely to show recycling behavior. environmental knowledge is the key to a clean and sustainable environment. lack or absence of funding, lack of educated teachers, insufficient knowledge, awareness, practice, and interest are obstacles to environmental sustainability. however, these obstacles can be overcome in schools. so much so that schools create appropriate curricula to promote sustainability through practice, awareness, and participation. this situation leads to sustainable development and cleaner and greener production (debrah, vidal & dinis, 2021). on the other hand, a lack of individual awareness and environmental knowledge weakens waste management practices, causing permanent problems for future generations (ampofo, 2020). wrong waste management pollutes the air, water, and soil, and these pollutants create severe risks to biodiversity and human health. therefore, everyone needs to participate and provide support (kumar & kumar, 2020). consequently, sustainable waste management should concern everyone (gusti, 2016). sustainability is a concept that includes the appropriate use of natural resources and processing wastes with proper methods (triguero, alvarez-aledo & cuerva, 2016). waste reduction and correct waste management are significant for sustainable environmental management. this situation reduces landfill use (cucchiella, d'adamo & gastaldi, 2014) and protects environment-human health and environmental quality (cucchiella, d'adamo & gastaldi, 2014; fagnani & guimaraes, 2017; gusti, 2016). waste management is beneficial in terms of economically and socially. with waste management, natural resources are protected, and employment is created (fagnani & guimaraes, 2017). in addition, improvement of environmental aesthetics, prevention of environmental pollution, and reduction of disease transmission routes can be provided (naria, nasution & santi, 2018). in this context, waste management practices are also at the center of innovative sustainable campus programs and activities (ifegbesan, ogunyemi & rampedi, 2017). these activities should be related to cognitive, emotional, and psychomotor areas. students should have knowledge and attitude on waste management and display this knowledge figure 1 waste management hierarchy (cucchiella, d’adamo, & gastaldi, 2014) journal of science learning article doi: 10.17509/jsl.v5i2.39376 303 j.sci.learn.2022.5(2).301-320 and perspectives with their behaviors (muljaningsih & galuh, 2018). separation of waste is essential as the waste produced causes many problems. waste separation should be done within the scope of waste management at the sources such as homes, markets, or industries to ensure recycling (dharmasiri, 2019). the biggest obstacle to recycling is the lack of a system for separate collection of wastes in many countries (tulebayeva, yergobek, pestunova, mottaeva & sapakova, 2020). waste management at the source is the basis of waste management at the household level. the education to be provided can increase knowledge, environmental awareness, practical knowledge and improve the understanding of household waste management (asteria & haryanto, 2021). it is necessary to raise the awareness of individuals so that separate collection of waste becomes a part of daily life at school, at home, and in the community. teachers are a key factor in providing this awareness (rada et al., 2016). considering that societies that have poor environmental awareness are the leading cause of ecological problems, it is a fact that environmental education and teachers, one of the most important elements of the education system, have a vital role in raising generations that act with sustainability awareness (aksan & çelikler, 2019). practical education to transform waste into valuable products improves students' environmental understanding. waste processing should be integrated with school and extracurricular activities and frequently applied to correct students' ecological awareness (nizaar, sukirno, djukri, muhardini & mas'ad, 2020). waste management practices can reduce environmental pollution and waste-related diseases. it can also improve the reuse, recycling, and recovery of waste materials (fredrick, oonyu & sentongo, 2018). education is essential in raising people's awareness about environmental protection to ensure waste management (muljaningsih & galuh, 2018; tulebayeva et al., 2020). education is used to gain knowledge, change attitudes and develop skills in the direction of resources, including waste management (fredrick, oonyu & sentongo, 2018). education, a significant and simple way to solve the waste problem, leads to increasing environmental awareness and planning appropriate solutions for sustainable development (przydatek, 2019). fredrick, oonyu & sentongo (2018) revealed that public education improves waste management practices in the city. early education ensures future generations contribute to sustainable waste management practices (zainu & songip, 2017). therefore, it is essential to disseminate waste management education starting from kindergarten. in this context, each individual's waste management roles and responsibilities should be appropriately communicated and given a directive. in this way, responsible citizens who manage waste as a resource and create zero or less waste in the future can be raised while applying the 3r rule (dharmasiri, 2019). the concept of zero waste and recycling activities are important universal gains to shape the future of countries and leave a cleaner environment for future generations (bulut, 2020). however, the amount of waste produced continues to increase without consumers’ conscious decisions about reducing, reusing, and recycling (zainu & songip, 2017). ampofo (2020) determined that high school students, teachers, and administration were worried about waste disposal. when the studies about the effectiveness of environmental education and practices on waste management are examined, liao & li (2019) revealed that environmental education is essential to ensure that high school students have the necessary knowledge and positive attitudes towards separating solid wastes. similarly, as a result of their studies, nizaar et al. (2020) revealed that the waste processing activity carried out by using the 3r rule was also effective in students' environmental awareness. when the studies about the knowledge, attitude, and behavior on waste management are examined, gusti (2016), in his study with primary school students, showed that knowledge of sustainable waste management with attitudes towards sustainable waste management; moreover, knowledge and attitudes towards sustainable waste management with the intention of sustainable waste management behavior were also related. similarly, molina & catan (2021) determined that senior high school students had sufficient knowledge about solid waste management and did the correct practices. however, as a result of the study conducted by desa, kadir & yusooff (2012), it was determined that first-year university students showed correct behaviors regarding solid waste management. on the other hand, almasi et al. (2019) found that participants' knowledge and attitudes about reduction at the source, separation, and recycling were high, but the correct practice was low. in addition, nearly half of the participants stated that they were not satisfied with the waste collection services. in support of this, baba, güneşşen & aydın (2018) examined the public’s opinions on solid waste management, and they found that most of the participants were not satisfied with the waste management in the country. teachers are expected to educate students to manage waste according to the 3r rule to increase respect for the environment (nizaar et al., 2020). recycling education in schools within qualified environmental education will be effective throughout individuals' lives. in this context, curriculum, teaching environments, and textbooks significantly affect the teaching of recycling within the scope of environmental education by well-appointed teachers (aksan & çelikler, 2019). education and awareness are the critical factors in reducing waste (zainu & songip, 2017). education and awareness on waste and waste management are becoming increasingly important from a global resource management perspective (kumar & journal of science learning article doi: 10.17509/jsl.v5i2.39376 304 j.sci.learn.2022.5(2).301-320 kumar, 2020). the knowledge and awareness obtained from education are highly effective in environmental activities and human behaviors. waste management education starts at school and affects individuals at home (dharmasiri, 2019). primary school students' sustainable waste management behavior will provide role models for their families and society (gusti, 2016). formal education for sustainable development is essential at all levels of education. teachers who have the correct knowledge, attitudes, skills, and innovation are required for better environmental sustainability or waste management sustainability education. however, most teachers do not have applied solid waste management or ecological knowledge negatively affects the students' environmental knowledge and attitudes toward solid waste management. in addition, teachers' attitudes in environmental education related to waste management are very important in the education they will give to students (debrah, vidal & dinis, 2021). considering that teachers affect societies, are role models for the growing generations, and have active roles in shaping today's world and its future, pre-service teachers should have knowledge and awareness on waste and recycling. for this reason, preservice teachers in all programs should be informed through various educational activities and courses during their university education. in addition, scientific activities such as conferences, symposiums, and panels on waste and recycling should be organized in universities. in this way, the increase in the knowledge and awareness levels of the pre-service teachers will positively affect their attitudes and behaviors (aksan & çelikler, 2019). protecting the environment can be emphasized by encouraging society to reduce waste generation through awareness programs. awareness programs can improve attitudes towards waste management and provide new business opportunities. higher education institutions play an important role in giving environmental education to students. thus, postgraduate students can explore career opportunities in solid waste management (reduction, reuse, and recycling as an alternative to disposal) (al-khatib, kontogianni, al-sari & al rajabi, 2018). when the studies on the importance of waste management awareness are examined, bulut (2020) revealed most teachers thought that children in preschool education gained insufficient understanding about zero waste and recycling. teachers stated that they wanted to gain awareness about zero waste and recycling in preschool education undergraduate programs for reasons such as contributing to the country's economy, raising children’s awareness at an early age, protecting nature and the environment, saving the country's natural resources and protecting all living things in the background. at the same time, aksan & çelikler (2019) found that awareness education increased the preservice science teachers’ knowledge level about recycling and positively changed their behavior towards recycling. the research aimed to determine the science students’ awareness on waste management. in this context, answers to the following sub-questions of research were sought. 1. what is the science students’ awareness on recovery? 2. what is the science students’ awareness on reuse? 3. what is the science students’ awareness on recycling? 4. what is the science students’ awareness on plastic bag usage? 5. what is the science students’ awareness on zero waste? 2. method 2.1 research model the screening model was used in the research. screening is a research model that aims to examine a situation such as an event, individual, or object in its original form without changing and affecting it (karasar, 2020, p. 109). 2.2 research group eleven students (nine female, two male) studying in the science education department of a state university in turkey in the spring semester of 2019-2020 voluntarily participated in the research. 2.3 data collection the data were collected through scientific newspapers prepared in line with the scientific newspaper preparation instruction within the scope of waste management. researchers designed instruction. first, the current science course curriculum (ministry of national education [mone], 2018) and the national and international literature were examined to prepare the instruction. then, a control list was designed to evaluate the statements' suitability to be included in the instruction, the adequacy of the power to represent the subject, and the content validity. concepts involved within the scope of waste management in the control list are given in figure 2. in the five sections of the instruction, students were asked to prepare scientific newspapers that they will support with written explanations and drawings on recovery, reuse, recycling, plastic bag usage, and zero waste figure 2 concepts involved within the scope of waste management journal of science learning article doi: 10.17509/jsl.v5i2.39376 305 j.sci.learn.2022.5(2).301-320 within waste management. there are three stages in each section of the instruction. after completing each step, the students were asked to put an x in the parenthesis ( ) near the relevant stage. these stages are: stage-1: provide information about recovery by supporting written explanations and drawings. ( ) stage-2: write an example from the news you have heard/read from the media about recovery. ( ) stage-3: write your thoughts about recovery. ( ) similarly, three-stage instruction was followed for reuse, recycling, plastic bag usage, and zero waste. the ability of the statements in the instruction to determine the students' awareness on waste management, including the concepts of recovery, reuse, recycling, plastic bag usage, and zero waste. whether any expressions needed correction that were not understood or that were not necessary and whether they were appropriate for the level of the students were examined by two faculty members whose fields were science education and chemistry education. 2.4 data collection process during the data collection process, separate tables were prepared for eleven students. the scientific newspaper preparation instruction within the scope of waste management, five stapled white a4 papers, and 12 colors of dry pencils was put on the tables for individually used materials by students. one student sat at each table. students were given 60 minutes to follow the instruction and prepare scientific newspapers. 2.5 data analysis the data sheets were numbered for each student in the data analysis, and the written explanations and drawings on the data sheets were transferred to the computer in their raw form. within the scope of the research, the written explanations and drawings of the students were analyzed using content analysis. the data of written explanations and drawings in scientific newspapers prepared by students were analyzed separately by two science education researchers to ensure validity and reliability. first, codes and categories were created by using content analysis by two researchers. then they were compared and arranged. finally, comparisons were made between coders to ensure the reliability of coding. the reliability between the two independent coders was calculated by using the formula agreement=[number of agreements/(number of disagreements+number of agreements)x100] (miles & huberman, 1994) and found as 81.4%. however, expert opinion was received on the suitability of the categories and codes. to ensure external validity, two researchers presented the raw data in categories by coding to make a detailed description. in addition, direct quotations from the students' written explanations and examples of their drawings were provided with the number given to each student's datasheet. ethics committee approval was obtained for this research (turkey, ondokuz mayıs university, social sciences and humanities research ethics committee, 28.02.2020, 2020/115). 3. findings the slogans in the prepared scientific newspapers about recovery, reuse, recycling, plastic bag usage, and zero waste within the scope of waste management by students are given in table 1. when table 1 is examined, it is seen that students write more slogans about recycling, plastic bag usage, and zero waste than recovery and reuse. 3.1 findings of science students’ awareness on recovery students' written explanations and drawings in the prepared scientific newspapers about recovery are analyzed and given in table 2. when table 2 is examined, it is seen that explanations and drawings in the prepared scientific newspapers about recovery mainly were concentrated on the resource type, sustainable consumption behavior, educational activities and effects, project activities and effects, cooperation, and solidarity. one student (s5) drew attention to negative situations in terms of sustainable consumption behavior with expressions such as excessive water use, leaving taps open, and not being able to prevent excessive water use. in addition, a student (s4) drew without writing an explanation, and two students (s1, s6) did not prepare a scientific newspaper page about recovery. some examples of students' written explanations are given below. “both wear and sell… in this article, we will present the idea of an association to you. i am sure that it will be a great idea for life. the members of the recovery association put forward this idea at first, then they developed and presented this idea and tried to reach an agreement with some companies. in this idea, we put a machine in certain places. we throw our clothes and shoes that we do not wear, worn out or shrinking, into this machine, and discounts are applied for you in some branded stores. this idea is first being thought of in big cities. it is being considered in istanbul, ankara, and izmir. if it holds, it will probably spread everywhere, and i think it is a good idea and that such ideas will develop and be beneficial for our country. if it is agreed and resolved, i will announce it to you in this newspaper. with love ...” (s2) “every day, we run a lot of water for everything. even while standing by the sinks, the taps are on. this situation is terrible. we need to, but we cannot prevent it. therefore, it would be a very logical approach to clean the running water and make it reusable.” (s5) journal of science learning article doi: 10.17509/jsl.v5i2.39376 306 j.sci.learn.2022.5(2).301-320 “the old collected materials are recovered by the municipality and distributed to the needy again. thanks to the recovery, support was provided to many families. thanks to the recovery of old materials by the people, they found many owners again. i think that recovery supports many families in need. this is a very nice and healthy project. table 1 slogans in the prepared scientific newspapers by students slogans student number recovery both wear and sell… s2 no losing, keep winning. we continue to win. s8 reuse why are we cutting down trees? s2 reuse the water. s4 the municipality cares about reuse! s6 don’t worry about single-use! now, you can use it again! do you know these? don't be sad that it's over! s8 recycling let's recycle and not go back! let's recycle and get back to life! s1 give your signature so that everything transforms. think ahead, recycle!!! s2 recycling is everywhere. s4 recycle if you want to live! all children are experiencing report card excitement. i recycle for more books, for my future. s5 throw into the recycle, don’t be trash. s6 we renew nature! s8 don't throw which on your hand to the ground; recycle it… s10 support recycling by throwing your books into the waste bins or through schools. s11 plastic bag usage plastic bag case! plastic bags are killing! no to the false glare. s1 should the plastic bag necessarily enter the house? s2 plastic bags do not disappear. s4 now, the plastic bags are 25 krş. is the string shopping bag coming again? s5 let's recycle; let's not pollute nature. s6 was there a plastic bag in the past? no to the plastic bag! yes to the string shopping bag! long live the string shopping bags! s8 we don't want a plastic bag. s10 “plastic bags do not disappear in nature.” we should be aware of plastic bag usage! s11 zero waste let's not drown in the water which we drink! s1 zero waste for my country... “make zero waste, get.” s2 clean nature, clean future. s3 bread is labor, don't throw it into the trash. s4 bread in africa. please, don't be indifferent, don't throw it away! s5 our world is getting better with zero waste. s7 we couldn't bear to throw. zero waste, double energy! s8 “an important step for zero waste in the dormitory.” s9 i want a world without waste. s10 s2: collecting different materials in the separate waste bin, giving a discount voucher to be used on clothes or shoes in return for the materials placed in the waste bin s5: the water flowing from the faucet to the sink is cleaned and purified with a cleaner and purifier and reaches the faucet again with the pipe s10: repairing and renewing a scrap vehicle with a flat tire figure 3 some examples of drawings in the prepared scientific newspapers about recovery by students journal of science learning article doi: 10.17509/jsl.v5i2.39376 307 j.sci.learn.2022.5(2).301-320 we can take a step in this regard, and we should raise awareness of everyone.” (s11) some examples of drawings in the prepared scientific newspapers about recovery by students are given in figure 3. 3.2 findings of science students’ awareness on reuse students' written explanations and drawings in the prepared scientific newspapers about reuse are analyzed and given in table 3. in table 3, it is seen that explanations and drawings in the prepared scientific newspapers about reuse were mostly concentrated on resource type and sustainable consumption behavior. a student (s2) drew attention to a situation that is not appropriate for the category of individual responsibility with the statement that using paper without thinking of felled trees. two students highlighted the effect of reuse on life with the statements that the cutting down of trees poses a problem for our country and the world (s2), the depletion of resources is bringing the end of humanity day by day (s5); one student emphasized the effect of reuse on natural resources with the statements that millions of trees are cut down, and trees are in danger (s2). however, a student’s (s8) statement contradicts the nature of reuse by expressing the scientists' studies on disposable materials. also, a student (s1) did not prepare a scientific newspaper page about reuse. table 2 written explanations and drawings in the prepared scientific newspapers about recovery by students category codes f resource type trees 1 wastewater 1 used clothes and shoes 1 used materials 1 scrap car 1 waste bin clothes 1 unspecified waste type 1 separation at the source recovery machines 1 recycling bins 1 sustainable consumption behavior wasting water* 3 walking to close distances 1 conscious use of oxygen, time, and money 1 educational activities and effects news about generating solutions for wastage 1 news about recovery projects 1 using the newspaper for the announcement 1 organizing book reading days 1 raising awareness of people 1 project activities and effects implementation in big cities first 1 reusing wastewater by cleaning 1 reusing the car with recovery 1 nature conservation 1 beautiful and healthy 1 beneficial for the country 1 authorized institution/organization activities and effects recovery association generating and developing an idea 1 municipality recovery 1 cooperation and solidarity giving recovered materials to the needy 2 giving their old materials by people 1 agreement between the recovery association and other companies 1 people working together 1 strengthening natural resources tree planting days 1 increasing the number of trees recovered to life 1 encourage and widespread effect shop discount for used clothes and shoes given to machines 1 acceptance and widespread of the recovery idea 1 effect on emotional state peaceful 1 surprising 1 *negative journal of science learning article doi: 10.17509/jsl.v5i2.39376 308 j.sci.learn.2022.5(2).301-320 some examples of students' written explanations are given below. “instead of throwing away the finished towel papers in the house, the woman designed something from them. my friend made an owl for homework. she had some leftover materials, and instead of throwing them away, she made cloudy pillows for her nephew. i think that we can do something with them and add color to our lives instead of discarding the leftover materials.” (s3) “the municipality cares about reuse! the municipality offers the materials for the second user with the clothes collection boxes in each district. besides clothes, toys, bags, and shoes are also collected in these boxes. it was good work to prevent new materials from being thrown away and help those in need. i invite everyone to be sensitive about this issue.” (s6) “primary school students did a study called reuse. a student made a wall mirror using a broken cabinet mirror and said that he would gift the mirror to his mother. we need generations who reuse, not destroy.” (s10) some examples of drawings in the prepared scientific newspapers about reuse by students are given in figure 4. 3.3 findings of science students’ awareness on recycling students' written explanations and drawings in the prepared scientific newspapers about recycling are analyzed and given in table 4. table 3 written explanations and drawings in the prepared scientific newspapers about reuse by students category codes f resource type used and leftover materials 3 paper, book, notebook 1 clothes, shoes, toys, bag 1 wastewater 1 broken mirror 1 general 1 waste bin clothes 1 sustainable consumption behaviour making a wall mirror from the broken mirror of the cabinet 1 designing new products from used and leftover materials 1 reusing materials over and over 1 preventing the used materials from being thrown away 1 exchange a reading book with a new one 1 preventing wastage 1 preventing water wastage 1 generations who reuse 1 individual responsibility sensitive people 1 paper usage without thinking of felled trees* 1 educational activities and effects learning that some materials are reusable 1 seeing a reuse example on the website 1 seeing a reuse example with peer observation 1 primary school students doing a study 1 research activities and effects scientists' method suggestions to prevent tree cutting 1 scientists' studies on disposable materials* 1 the invention of the machine that reproduces worn clothes 1 the invention of the machine that reproduces used notebooks and pens 1 authorized institution/organization activities municipality collecting used materials and offering to reuse 1 municipality putting a clothes collection box in each district 1 solidarity helping those in need 1 effect on life and natural resources a national and global problem felling of trees* 2 trees are in danger* 1 day-to-day depletion of resources and the end of humanity* 1 functionality beneficial 1 nice 1 encourage and widespread effect supporting studies about reuse 1 reuse is a common method today 1 effect on the emotional state feeling happy due to the news about the done studies 1 adding color to life by newly designed products 1 *negative journal of science learning article doi: 10.17509/jsl.v5i2.39376 309 j.sci.learn.2022.5(2).301-320 table 4 written explanations and drawings in the prepared scientific newspapers about recycling by students category codes f resource type paper 3 paper, book 1 paper, plastic, glass, metal 1 paper, plastic, glass, battery 1 book 1 waste oil 1 general 1 general (recyclable) 1 waste bin unspecified waste type 2 paper, plastic, glass 1 paper, plastic, glass, battery 1 separation at the source separating household wastes (paper, plastic, metal, glass) 1 campaign placing waste oil collection boxes in cities 1 symbols highlighting waste management recycling symbol 4 sustainable consumption behavior throwing wastes into the recycling bin 1 throwing recyclable wastes into the recycling bins 1 recycling used books to prevent wasting book 1 recycling takes the country away from reuse* 1 individual responsibility sensitivity 3 sensitivity towards the waste oil collection campaign 1 making an effort to beautify the environment 1 educational activities and effects seeing the consequences of recycling and being mindful 1 gaining awareness on wastes 1 gaining awareness on recycling 1 obtaining information about the paper recycling from the website 1 raising public awareness 1 project activities and effects doing a project about reducing the used materials’ effect on nature 1 high school students doing a study 1 changing the life of a disabled person with a gained wheelchair within the scope of the project 1 cooperation bringing household wastes by everybody on weekends 1 donating books and notebooks by school for recycling 1 effect on life and natural resources preventing tree cutting by paper recycling 4 healthy creatures 2 increasing life expectancy 1 the most important thing in our life 1 preventing recyclable wastes from becoming trash 1 a clean world 1 resource abundance 1 the end of the world is coming* 2 increase in tree cutting due to non-recyclable papers* 2 having trouble finding a resource in the future* 1 the effect of waste on nature* 1 the importance of natural resources trees produce oxygen 1 trees contribute to a happy nature 1 encourage and widespread effect contributing to recycling 2 increasing support for paper wastes recycling 1 implementation of paper recycling in many metropolitan cities 1 increasing recycling in the world 1 effect on emotional state adding colour to life 1 joy 1 *negative journal of science learning article doi: 10.17509/jsl.v5i2.39376 310 j.sci.learn.2022.5(2).301-320 in table 4, it is seen that explanations and drawings in the prepared scientific newspapers about recycling mainly were concentrated on resource type and effect on life and natural resources. a student (s6) drew attention to a situation that would not be appropriate for sustainable consumption behavior with the statement that recycling takes the country away from reuse. although some students expressed the effect of not recycling on natural resources as having difficulties in finding resources to use in the future (s5), tree cutting for used and discarded papers (s1), and not recycling paper will increase tree cutting (s1). regarding the used materials’ effects on nature (s8), two students stated the impact of not recycling on life as the world's end is coming (s1, s5). some examples of students' written explanations are given below. “let's throw our trash into the recycling bin and add color to life. i read on a web page that the paper wastes we throw away are transformed instead of tree cutting again. this practice has been implemented in many metropolitan cities. more trees, cleaner air, and, moreover, a happier nature. throw your wastes into the recycling bins, and add color to life.” (s3) “you can save recyclable everything from being trash by throwing them into the recycling bins! everything thrown into the recycling suspends our country away from reuse. for example, the papers we do not throw into the trash and recycle prevent the trees from dying. throw it into the recycling; it isn’t trash. with a campaign launched, oil collection boxes for recycling were made and placed in certain parts of the cities. people are already extremely sensitive.” (s6) “we can contribute to recycling by separating the trashes in our home as paper-plastic-metal-glass. contribution to recycling plays an important role in changing people's lives.” (s9) some examples of drawings in the prepared scientific newspapers about recycling by students are given in figure 5. 3.4 findings of science students’ awareness on plastic bag usage students' written explanations and drawings in the prepared scientific newspapers about plastic bag usage are analyzed and given in table 5. in table 5, it is seen that explanations and drawings in the prepared scientific newspapers about plastic bag usage mainly were concentrated on the effect on life and natural resources. some students drew attention to situations that were not appropriate for the category of individual responsibility with the expressions such as the insensitivity of people (s4), people buying insistently (s2), reaction to the increase in the plastic bag price (s8), unnecessary grouching of lazy people about environmental problems (s8). on the other hand, some students emphasized the widespread of plastic bags usage with their negative expressions, such as its use is increasing day by day (s8, s11), rising in usage with the s3: instead of throwing the leftover materials into the trash, collect them in the box and use them to make a pillow s5: reuse wax paper used in food making instead of throwing it into the trash s10: making a framed small wall mirror from the mirror of the broken cabinet door figure 4 some examples of drawings in the prepared scientific newspapers about reuse by students s1: the relationship between paper usage and cutting down trees s3: the separated waste bins according to their types (plastic, glass, paper) s7: instead of separating the wastes according to their types and throwing them into the appropriate recycling bin, throwing the waste into the same trash bucket without separating them (inappropriate drawing) figure 5 some examples of drawings in the prepared scientific newspapers about recycling by students journal of science learning article doi: 10.17509/jsl.v5i2.39376 311 j.sci.learn.2022.5(2).301-320 increase in shopping (s5), being everywhere (s8), quickly covering the world (s1). one student (s2) established a relationship between the unconsciousness of public and educational activities. seven students (s1, s2, s3, s4, s5, s10, s11) stayed long, three students (s3, s4, s8) harming to the environment, and some students caused environmental pollution (s8), causing visual pollution (s1), decreasing in natural habitats (s1), increasing in water turning into a swamp (s1) and causing global warming (s8). in addition, some students mentioned the negative effects of plastic bag usage on life. for example, three students (s1, s2, s11) harm the living things, some of the students cause the extinction of caretta caretta (s9), increasing in the fish that hit the shore (s1), and humanity will see the harm to nature (s5). in addition, it was observed that a student (s7) did not prepare a scientific newspaper page about plastic bag usage. some examples of students' written explanations are given below. table 5 written explanations and drawings in the prepared scientific newspapers about plastic bag usage by students category codes f waste bin unspecified waste type 1 symbols highlighting waste management recycling symbol 1 sustainable consumption behavior throwing wastes into the recycling bins 1 protecting the living space 1 individual responsibility insensitivity of people* 1 buying plastic bags by people insistently* 1 reaction to the price hike on the plastic bag* 1 unnecessary grouching despite irresponsibility* 1 educational activities getting information from the news 1 unconsciousness of public* 1 research activities doing research of university students 1 effect on life and natural resources staying long in nature* 7 harming to the environment* 4 harming to the living things* 3 causing visual pollution* 1 reduction of natural habitat* 1 increasing in waters turning into swamp* 1 causing global warming* 1 causing the extinction of the caretta caretta* 1 an increasing number of fish striking the shore* 1 being affected humanity by the harm to nature* 1 widespread effect increasing in usage day by day* 2 increasing in usage with the increase in shopping* 1 being everywhere* 1 quickly covering the world* 1 preferred reason easy to use 1 economic 1 needed in daily life 1 deterrent practices and effects decreasing in plastic bag usage with pricing 4 pricing 3 raising awareness of people with pricing 1 cloth bag or string shopping bag usage with pricing 1 tendency to cloth bag usage with pricing 1 solution offers preferring cardboard/cloth ones 3 disuse 2 plastic bag usage can be destroyed in nature 1 making and using our own string shopping bag 1 the usefulness of string shopping bag usage 1 *negative journal of science learning article doi: 10.17509/jsl.v5i2.39376 312 j.sci.learn.2022.5(2).301-320 “plastic bag case! another blow to the decreasing natural habitat comes from plastic bags daily. plastic bags are killing! according to the information obtained as a result of long research, plastic bags, which can be destroyed in nature in close to a thousand years, cause visual pollution and threaten the lives of many living things. the fish that hit the shore and the waters that turned into swamps increased to a level frightening. most of us turn to plastic bags saying that they are easy to use and economical. but this false glare should not mislead us. if every person in the world uses one plastic bag a day and leaves it to nature, we will have difficulty finding a space to breathe daily. the behind of false glare is dark. let's protect our world, our living space. no to false glare.” (s1) “plastic bags or every plastic item are materials that take a very long time to disappear. it has much harm to the environment. plastic bags do not disappear. it is awful that people are insensitive to this situation while it takes more than centuries for a plastic bag to disappear. the pricing of plastic bags raises people's awareness of this situation.” (s4) “one of the reasons for the extinction of caretta caretta is plastic bag usage. we can prevent such problems by increasing cloth bag usage instead of plastic bag.” (s9) some examples of drawings in the prepared scientific newspapers about plastic bag usage by students are given in figure 6. 3.5 findings of science students’ awareness on zero waste students' written explanations and drawings in the prepared scientific newspapers about zero waste are analyzed and given in table 6. in table 6, it is seen that explanations and drawings in the prepared scientific newspapers on zero waste are concentrated educational activities and the effect on life and natural resources. one student (s5) drew attention to a situation that would not be appropriate for sustainable consumption behavior. the expressions were taking more bread than needed and throwing the bread into the trash. the same student (s5) described a negative approach to individual responsibility with the word as people who have good living conditions do not know the value of what they have. some students also gave negative expressions such as harming nature by pouring waste oil into the sink (s3), the formation of a new continent from the accumulated trash in the world (s5), and the starvation to death of people who cannot find bread (s5). one student (s11) stated that zero waste practices are not common today. also, a student (s6) did not prepare a scientific newspaper page about zero waste. some examples of students' written explanations are given below. “she read that some cities had started a zero waste project on the internet. instead of pouring oils into sinks and harming nature, they collected and recycled them with a zero waste project. thanks to this project, water pollution, and many pollutions did not occur. this means that nature is cleaner. nice thing. pure nature means a clean future.” (s3) s1: giving harm to the world through plastic bag usage s3: comparison of the future that awaits individuals using cloth bag and plastic bag s9: the danger of caretta caretta's extinction with the threw plastic bags into the water figure 6 some examples of drawings in the prepared scientific newspapers about plastic bag usage by students s2: separating and disposing of wastes according to their type and obtaining food and drink in return s3: pouring waste oil into the waste oil box instead of pouring it into the sink s4: preventing bread wastage figure 7 some examples of drawings in the prepared scientific newspapers about zero waste by students journal of science learning article doi: 10.17509/jsl.v5i2.39376 313 j.sci.learn.2022.5(2).301-320 “a new continent has emerged from the trash in the world. lots of people are starving. s/he cannot find food or bread. although our situations and living conditions are so good, we do not know the value of what we have. we buy more bread than we can eat. we throw out the rest. do you think it's a beautiful thing? according to the research, 100.000 new schools can be built with the bread thrown into the trash in our country... bread in africa. again, bread was the last word of a child before s/he died. we lost an angel of hunger again. how are we going to account for this? please don't be indifferent, don't throw!” (s5) “no waste will pollute our world anymore. big steps are planned to be taken about zero waste in the coming days. a neighborhood headman in turkey informed the whole neighborhood about zero waste. in my opinion, zero waste is one of the best things we can do. i also think that everyone should be conscious about zero waste, and zero waste should be explained everywhere in the streets and buses.” (s11) some examples of drawings in the prepared scientific newspapers about zero waste by students are given in figure 7. table 6 written explanations and drawings in the prepared scientific newspapers about zero waste by students category codes f resource type water bottles 1 waste oil 1 paper, plastic, glass 1 household wastes, plastic, glass 1 waste bin waste oil 1 household wastes, plastic, glass 1 paper, plastic, glass 1 paper, plastic, glass, metal 1 sustainable consumption behavior respect the bread 1 don’t waste bread 1 prevention of wasting in the country 1 buying more bread than you need* 1 throwing the bread into the trash* 1 individual responsibility coming up with a solution by people 1 good individual contribution 1 people with good living conditions do not know the value* 1 educational activities raising people's awareness on waste reduction 1 demonstrate waste reduction in practice 1 planning major activities 1 informing people about zero waste 1 raising awareness of people about zero waste 1 talking about the zero waste issue everywhere 1 seeing a slogan about zero waste on social media 1 get information about the cities where the zero waste project was initiated from the website 1 inspirational news for people 1 project activities collecting household wastes in waste pipes in homes 1 recycling bins in dormitories (paper, glass, plastic, metal) 1 recycling of waste oil 1 effect on life and natural resources preventing water and environmental pollution 4 approaching the possibility of a world without waste 1 harming to the environment by pouring waste oil into the sink* 1 the formation of a new continent from the accumulated trash in the world * 1 the starvation to death of people who cannot find bread* 1 economic development building a school the economic value of trashed bread 1 building a school income from recovery water bottles 1 energy-saving recycling of household wastes 1 separating wastes and encourage machine separating wastes according to their types (paper, plastic, glass) 1 machine giving food and drink in exchange for waste 1 widespread effect not being common zero waste practices today* 1 *negative journal of science learning article doi: 10.17509/jsl.v5i2.39376 314 j.sci.learn.2022.5(2).301-320 unscientific explanations, drawings, and reasons for not being scientifically appropriate in the prepared scientific newspapers by students are given in table 7. 4. discussion in the research, eight students prepared scientific newspapers about waste management (recovery, reuse, recycling, plastic bag usage, and zero waste). however, s1 did not prepare a scientific newspaper page about recovery and reuse; s6 did not prepare a scientific page about recovery and zero waste; s7 did not prepare a scientific newspaper page about plastic bag usage. as a result of the research, it was understood that a significant part of the students was aware of the effects of education, research, and project activities on recovery, reuse, recycling, plastic bag usage, and zero waste practices within the scope of waste management, and the place and importance of the individual in waste management. on the other hand, it was understood that most of the students were not aware of waste types that are the basis of waste management practices, the separation of wastes at the source in accordance to their types, and throwing the wastes into the appropriate waste bin for their types. similarly, it was determined that university students had recycling awareness in specific areas. still, they did not have recycling awareness in many places, and they performed recycling at a minimum level in practice (ak & genç, 2018). furthermore, harman, aksan & çelikler (2015) determined that first-year science students' mental recycling models were limited. on the other hand, aksan, çelikler & yenikalaycı (2019) determined that first-year science students’ awareness on recycling was high. in this table 7 unscientific explanations, drawings, and reasons for not being scientifically appropriate in scientific newspapers unscientific explanations and drawings reasons for not being scientifically appropriate recovery recovering the person who quit smoking back to life (s3) a happy life awaits people who quit smoking (s3) establishing a relationship between recovery from smoking and human life reinforcing the learned information not to forget (s7) establishing a relationship between recovery and ensuring the permanence of the learned information reuse obtaining new and clean pages from used school books in paper mills (s7) being a practice within the scope of recycling obtaining fertilizers from organic wastes (s9) being a practice within the scope of composting throwing the caps and labels of pet bottles into separate containers (s11) being a practice within the scope of recycling recycling throwing wastes into the same trash bucket (s7) it is necessary to separate the wastes according to their types and throw them into the appropriate recycling bin putting trash buckets in every neighborhood (s8) it is necessary not to throw waste into the trash buckets for recycling collecting household wastes in trash buckets (s8) since it is possible to recycle household wastes, it is necessary not to be thrown them into the trash buckets opening shops producing from wastes in the neighborhoods (s8) being a practice within the scope of reuse plastic bag usage being too lazy to throw it into the trash, the plastic bags next to the trash bin (s8) it is necessary to throw the plastic bags into the recycling bin zero waste putting the waste collected separately in household, plastic, and glass waste bins into the same machine for recycling (s8) it is necessary that the waste collected separately in household, plastic, and glass waste bins are put into separate recycling machines in accordance with their types. throwing trashes on the grounds instead of trash buckets (s8) it was expressed as trash, whereas it should be expressed as waste. it was expressed as a trash bucket, whereas it should be expressed as a recycle bin. finding a solution to the disposal of trash on the ground (s8) it was expressed as trash, whereas it should be expressed as waste. trash types: food leftovers, glass bottles, underused clothes, etc. (s8) expressing the recyclable and reusable waste types as trash types journal of science learning article doi: 10.17509/jsl.v5i2.39376 315 j.sci.learn.2022.5(2).301-320 direction, they stated that individuals with heightened awareness of recycling would be more likely to show recycling behavior. similarly, pamuk & kahriman-pamuk (2019) determined that pre-service teachers' recycling intentions and behaviors differ according to the recycling opportunities on the campus where they study and, the place where they lived as children, and they currently live. the basis of waste management practices is waste types, separation of wastes at the source according to their types, and throwing the wastes into the appropriate waste bin for their classes. on the other hand, in this research, while a small number of the students gave limited examples for resource types that can be evaluated in recovery, reuse, recycling, and zero waste practices, a significant part of the students did not place waste types and waste bins according to waste types. when different studies that determined the limitation on waste types were examined, it was expressed that the recyclable waste types by limited as food waste, packaging, bottle, paper, cardboard, glass, plastic, organic waste, battery, and metal by pre-service science teachers (harman & yenikalaycı, 2020); paper, plastic, glass, solid waste, battery, pet, cardboard and pet bottles by preservice science teachers (harman & çelikler, 2016), paper, plastic and glass by pre-service physics, chemistry and biology teachers (demircioğlu, demircioğlu & yadigaroğlu, 2015), paper, glass, metal and organic matter by biology department students (soran, morgil, yücel, atav & işık, 2000), plastic, returnable cans and coke bottles by biology department students (soran et al., 2000) and secondary and university students (yılmaz, morgil, aktuğ & göbekli, 2002). science students stated waste types by limiting them to paper, plastic, glass, and battery in waste separation at the source, recyclable wastes, recycling bins, and recycling facilities (harman, aksan & çelikler, 2015). pre-service preschool teachers gave examples of recyclable products such as plastic, paper, battery, glass, and metal (dinler, simsar & doğan, 2020). pre-service science teachers stated that the wastes should be thrown into separate recycling containers for each litter and that the materials used in packaging production are recyclable (harman & çelikler, 2018). gürer & sakız (2018) determined that most adults who attended various courses in public education centers knew about the recycling of packaging wastes. before the education based on the scamper technique, science students stated that they would throw plastic, cardboard boxes, empty cans, glass bottles, and scrap metal into the trash bins and throw them into the recycling bin after the education (çelikler & harman, 2015). before recycling awareness education, pre-service science teachers wrote examples for recyclable waste types like paper/cardboard and plastic (all); glass (almost all); battery, waste oil, metal, composite, vehicle tire (majority); electronic waste, medical waste, organic waste (small number) (aksan & çelikler, 2019). it has been revealed that pre-service science teachers drew recycling bins for paper, plastic, glass, metal, and waste batteries in the garden and classroom for the school in their dreams; a small number of pre-service teachers drew a general recycling bin for all waste in the park and the classroom, and some of them drew trash buckets for waste (harman, 2017). in addition, science students emphasized the recovery of solid wastes in the models that they prepared for the teaching of waste recovery; they did not use mixed wastes, although they used solid wastes such as paper, cardboard, plastic, textile, metal, and glass in almost all of the models. it was determined that the message of “reuse of solid materials/wastes for different purposes” was given in all models on recycling (harman & çelikler, 2015). similarly, pre-service science teachers emphasized recovery in all of the models they prepared for teaching waste recovery besides recycling and reuse in half. all of the pre-service science teachers prepared models by using solid wastes. the models determined that the most commonly used plastic and paper/cardboard wastes, and they did not use metal and composite waste types (aksan & çelikler, 2018). in younger age groups, primary school first-grade students most commonly drew trash, including paper, plastic, glass, etc. about the concept of zero waste; some students drew recycling bins; a student drew battery collection boxes; a student drew separate recycling bins for paper, glass and plastic wastes, and a small number of students drew trash bin (sönmez, 2020). in the research, few students (s2, s6, s9) drew attention to separating the sources' wastes. in this context, it was understood that the majority of the students were not aware of the waste types, the separation of the wastes at the source in accordance to their types, and throwing the wastes into the appropriate waste bin for their types. however, to ensure recycling, it is a vital requirement that waste management includes waste separation at the sources such as the home, market, or industry. wastes should be separated as dry/wet or biologically degradable/nondegradable (dharmasiri, 2019). individual effort is required for waste separation behavior. providing supportive conditions for waste separation in homes is crucial for individuals to participate in separation at the source. therefore, local authorities, municipalities, and recycling organizations should facilitate the separate collection of household wastes (arı & yılmaz, 2019). ampofo (2020) determined that there were no appropriate waste dumping areas, and the scraps were disposed of without separation in the high schools within the municipality. contrary to this result, it was determined that most university students separated household wastes (gültekin, 2017), and most people tend to recycle at home (almasi et al., 2019). sayın & yerli (2020) determined that gender, age, employment status, marital status, and education were influential in the household’s environmental awareness of household waste. liao & li (2019) drew attention that journal of science learning article doi: 10.17509/jsl.v5i2.39376 316 j.sci.learn.2022.5(2).301-320 environmental education is necessary to ensure that high school students have the essential knowledge and positive attitudes toward separating solid wastes. an important part of pre-service preschool teachers stated that the obstacle to recycling is unconsciousness/ignorance, and the sensitivity of individuals can be increased through education (dinler, simsar & doğan, 2020). similarly, most pre-service science teachers stated that individuals do not know about recycling as the most important reason why recycling is not widespread. from this point, it has been said that as individuals understand the importance of recycling, their sensitivity to this issue increases. as a result, they use recycling bins by separating wastes and taking into account the colors of the recycling bins (aksan & çelikler, 2019). on the other hand, ak & genç (2018) determined that half of the university students felt lazy to throw the trash into the recycling bins. in addition, more than half of the students stated that they do not separate and recycle waste because there is no legal sanction. they do not recycle because other people do it. they do not use recycling bins because there are people who separately collect wastes from the trash, and they see recycling as a waste of time. more than half of the students pointed out that they did not know about the markings on the recycling bins, and nearly half of them did not know what happened to the recyclable wastes. adequate waste management leads to health, environmental, economic, and social benefits. activities related to waste management protect natural resources and create employment by providing positive social effects. waste management includes establishing an appropriate treatment and rethinking production processes, minimizing waste generation, reuse, and recycling. moreover, when wastes are appropriately managed, natural resources and energy consumption also decreases (fagnani & guimaraes, 2017). global environmental risks show themselves in the scarcity of natural resources, climate change, water, air, soil pollution, and endangered biodiversity and wastes. ecological problems and limited natural resources make a significant contribution to be reaffirmed the importance and role of waste management in developing countries. information on waste and its effect on health and the environment requires systematic and diverse educational approaches (kumar & kumar, 2020). there should be a constant urge to encourage and support awareness of effective waste management and sustainable environmental practices through education. generally, a direct correlation has been found between poor waste management practices and a lack of environmental knowledge and awareness (ampofo, 2020). in this context, within the scope of the research, a significant part of the students (s2, s3, s5, s7, s8, s9, s10, s11) revealed the effects of education, research and project activities and the consequences of these activities on recovery, reuse, recycling, plastic bag usage and zero waste practices with written explanations and drawings. in this direction, it was understood that the students were aware of education, research, and project activities in waste management and their effects. however, few students (s2, s6, s11) included the activities of authorized institutions/organizations. so much, so that recycling awareness education increased the knowledge levels of preservice teachers about recycling and positively affected their behaviors toward recycling (aksan & çelikler, 2019), and it has been determined that the education given to preservice science teachers was effective in increasing their awareness (aksan & çelikler, 2020). furthermore, it was determined that education based on the scamper technique effectively raised science students' awareness about the collection and utilization of solid wastes (çelikler & harman, 2015). similarly, akanyeti & kazımoğlu (2019) revealed that environmental engineering students who took environmental courses had more conceptual knowledge about solid waste pollution and management. still, the course did not positively affect students' attitudes and behaviors on issues such as throwing trash into the environment or using recycling bins. er-nas & şenel-çoruhlu (2017) emphasized that projects about the concept of sustainable development that pre-service science teachers make can increase their awareness. ersoy-quadır & temiz (2017) determined that kindergarten teachers implemented environmentallyfriendly activities with their students, and parents also offered environmentally friendly experiences to their children. it is possible to develop knowledge and improve environmental practices by creating participatory and encouraging programs between families and the municipality, emphasizing the education aspect through cities (almasi et al., 2019). likewise, karadağ & acar (2020) determined that pre-service social studies teachers attending seminars, meetings, or congresses had higher awareness of environmental problems. tosun & demir (2018) used many activities together: creative drama techniques such as dramatization, improvisation, roleplaying, multiple intelligence activities, travel-observation, game, show, music, dance, etc. in the learning-teaching process to increase the preschool children’s awareness on recycling. they determined that creative drama positively affected students' features such as learning and understanding of recycling, cooperation, self-confidence, creativity, and respect for the environment and nature. reducing waste as much as possible by processing waste in areas close to waste sources and supporting this in terms of legal, regulatory, institutional, functional, technical, financing, and community involvement consist of integrated waste management with the 3r principle (asteria & haryanto, 2021). by reducing the amount of stored solid waste and waste disposal costs, saving energy and natural resources, taking into account the crucial journal of science learning article doi: 10.17509/jsl.v5i2.39376 317 j.sci.learn.2022.5(2).301-320 benefits of recycling, the knowledge and attitude of the public towards the recycling of solid waste, and the cooperation of people, and recycling programs are the first steps in the correct use of resources (almasi et al., 2019). in the research, a significant part of the students drew attention to sustainable consumption behavior (s1, s2, s3, s4, s5, s6, s7, s8, s10), individual responsibility (s2, s4, s5, s6, s7, s8, s11), cooperation and solidarity (s2, s6, s8, s10, s11) in recovery, reuse, recycling, plastic bag usage and zero waste practices within the scope of waste management. also, an important part emphasized the effects of waste management on life (s1, s2, s5, s7, s9, s11) and natural resources (s1, s2, s3, s5, s6, s8, s10, s11) and its encourage and widespread effect (s2, s3, s10, s11). in this context, it was understood that the students were aware of the place and importance of the individual in waste management. dinler, simsar & doğan (2020) determined that pre-service preschool teachers were sensitive and willing to recycle. however, they decided that while pre-service teachers pay more attention to recycling in the school, they pay less attention at home. furthermore, pre-service teachers claimed that the primary purpose of recycling is to keep the environment clean, and the biggest obstacle to recycling is unconsciousness. the primary condition for creating environmental awareness in sustainability is the effective and efficient usage of resources. consumers are among the groups that take an active role in the consumption of these resources (kızgın, karaosmanoğlu, örmeci & taş, 2017). ateş (2018) determined that pre-service science and social studies teachers demonstrated a sufficient level of sustainable behavior and had enough knowledge. in addition, a low level of positive correlation was determined between the sustainable consumption behaviors of individuals and their knowledge levels on sustainability. on the other hand, şentürk (2019) found that the public was aware of the danger of e-waste, but they did not know the appropriate method to get rid of this waste. for this reason, she emphasized that informed consumers will support the recycling of electrical and electronic equipment wastes. in the research, few students presented written explanations and drawings for the functionality of reuse (s5, s6) and the strengthening of natural resources (s8, s9). sönmez (2020) determined that in first-grade primary school students, in the drawings about zero waste, female students mainly included the concepts of waste awareness, while male students mainly included the ideas of environmental protection. harman & yenikalaycı (2020) determined that pre-service science teachers' awareness of the zero waste approach was insufficient. they also found that the pre-service teachers’ statements about the zero waste approach were structured based on waste management and recovery, environmental effects, savings, economy, and public awareness. dal & cengiz-gökçe (2019) drew attention to the importance of integrating practices that protect raw materials and energy into life activities within the scope of the sustainability of environmental resources and the inclusion of these activities in daily life will contribute to the continuity of livable, healthy environments and landscapes. education is related not only to the cognitive field but also to the affective and psychomotor fields. because, for waste management, students should pay attention to the cognitive field and behavior and skill practices (muljaningsih & galuh, 2018). likewise, attitudes are related to feelings towards sustainable waste management and preconceived ideas that may lead to it (gusti, 2016). in this context, few students presented written explanations and drawings on the effects of recovery, reuse, recycling, plastic bag usage, and zero waste practices on the emotional state of the research. in the study conducted by bulut (2020), teachers stated that efforts to raise children’s awareness of zero waste and recycling in the learningteaching process would positively affect their social and emotional development and problem-solving skills. karakaş & dönel-akgül (2020) stated that pre-service elementary school teachers with a high sense of responsibility towards the environment and respect for nature had a higher sensitivity to environmental problems. a small number of students in their scientific newspapers wrote explanations and made drawings reflecting the evaluation of the used materials such as clothes, shoes, toys, and bags (s2, s6, s8, s11); prevention of water (s4, s5), paper/book (s1, s2, s3, s4, s5, s6, s7, s11) and bread wastage (s4, s5). similarly, a student studying in the first grade of primary school drew a clothes collection box, and a student drew a toy collection box in social responsibility and sharing. in addition, in terms of preventing wastage, a student drew stale bread, and also, a student also emphasized giving stale bread to an animal by soaking it (sönmez, 2020). in the prepared scientific newspapers about the plastic bag usage by students, it was determined that most of the students focused on the effects of the plastic bag usage on life (s1, s2, s5, s9, s11) and natural resources (s1, s2, s3, s4, s5, s8, s10, s11), deterrent practices for preventing the plastic bag usage and its effects (s4, s5, s6, s10, s11). it was seen that few students were aware of this issue. gürbüz & yılmaz (2018) also found that awareness of the harm of plastic bags, positive attitudes toward banning plastic bags, and the malleable bag usage reduction factors was effective in the statistics department students' behavior in reducing the plastic bag usage. besides, they stated that environmental bans could affect the intention and behavior of individuals, and individuals who favor banning plastic bags have the choice and behavior to reduce plastic bag usage. in addition, a student (s8) offered a solution for plastic bag usage, with the statement that everyone should make and use their string shopping bag. similarly, in the study journal of science learning article doi: 10.17509/jsl.v5i2.39376 318 j.sci.learn.2022.5(2).301-320 conducted by naria, nasution & santi (2018), students produced shopping bags from unused t-shirts. conclusion to sum up, in the research, it became clear that a significant proportion of students were aware of the impact of education, research, and project activities on recovery, reuse, recycling, plastic bag usage, and zero waste practices within waste management, and the place and importance of humans in waste management. unlike that, it was understood that most of the students were unaware of the basics of waste management practices such as waste types, segregating waste at the source according to type, and throwing trash into an appropriate waste bin for their type. in addition, in the research, it was concluded that the use of scientific newspapers could be an effective tool for measuring student awareness. recommendations in line with the results obtained within the scope of the research, the following recommendations were made. • instead of providing superficial information on waste management in environmental education courses at different education levels, courses that provide detailed information on waste management in preschool, primary, secondary and high school could be added to the curriculum. furthermore, scientific newspapers could be disseminated to raise awareness on waste management in addition to the curriculum. • in order to ensure students with the opportunity to learn by doing waste management, the number of pilot schools in preschool, primary school, secondary school, and high school could be increased, and students could be supported to take part in projects more frequently. references ak, ö. & genç, a. t. (2018). üniversite öğrencilerinin geri dönüşüm bilinci üzerine bir araştırma: sakarya üniversitesi örneği. uluslararası ekonomik araştırmalar dergisi, 4(2), 19-39. akanyeti, i̇. & kazımoğlu, ç. (2019). üniversite öğrencilerinin katı atık kirliliği ve yönetimi konusundaki farkındalıkları ve davranışlarına eğitimin etkileri. folklor/edebiyat, 25(97), 128-145. doi: 10.22559/folklor.932 aksan, z. & çelikler, d. (2018). models by science teacher candidates for the teaching the reutilization of wastes. kastamonu eğitim dergisi, 26(5), 1461-1467. aksan, z. & çelikler, d. (2019). recycling awareness education: its impact on knowledge levels of science teacher candidates. international electronic journal of environmental education, 9(2), 81-105. aksan, z., çelikler, d., & yenikalaycı, n. (2019). the determination on the science teaching candidates' awareness about the recycling. anadolu journal of educational sciences international (ajesi), 9(2), 881901. doi: 10.18039/ajesi.583817 aksan, z. & çelikler, d. (2020). creating awareness of pre-service science teachers for sustainable development about waste recycling. international electronic journal of environmental education, 10(2), 147-166. al-khatib, i. a., kontogianni, s., al-sari, m. i., & al rajabi, h. (2018). current trends in solid waste management in higher education institutions: the case of west bank region, palestine. environmental engineering and management journal, 17(8), 1887-1896. almasi, a., mohammadi, m., azizi, a., berizi, z., shamsi, k., shahbazi, a., & mosavi, s. a. (2019). assessing the knowledge, attitude and practice of the kermanshahi women towards reducing, recycling and reusing of municipal solid waste. resources, conservation & recycling, 141, 329-338. ampofo, j. a. (2020). waste disposal management practices in selected senior high schools within the wa municipality of ghana. international journal of management & entrepreneurship research, 2(4), 273-290. arı, e. & yılmaz, v. (2019). üniversite öğrencilerinin evsel atık ayırma davranışlarının planlı davranış teorisi yardımıyla araştırılması. batman üniversitesi yaşam bilimleri dergisi, 9(1), 53-68. asteria, d. & haryanto, j. t. (2021). empowerment key factors in shaping women's awareness of household waste management. global journal of environmental science and management (gjesm), 7(3), 1-14. ateş, h. (2018). fen bilgisi ve sosyal bilgiler öğretmen adaylarının sürdürülebilir tüketim davranışlarının ve bilgi düzeylerinin araştırılması. uludağ üniversitesi eğitim fakültesi dergisi, 31(2), 507531. baba, f. a. m., güneş-şen, s., & aydın, m. (2018). solid waste management and public awareness on solid waste management in libya–benghazi. bulut, a. (2020). teacher opinions about children's awareness of zerowaste and recycling in the preschool education years. review of international geographical education (rigeo), 10(3), 351-372. cucchiella, f., d’adamo, i., & gastaldi, m. (2014). strategic municipal solid waste management: a quantitative model for italian regions. energy conversion and management, 77, 709-720. cucchiella, f., d’adamo, i., & gastaldi, m. (2017). sustainable waste management: waste to energy plant as an alternative to landfill. energy conversion and management, 131, 18-31. çelikler, d. & harman, g. (2015). the effect of the scamper technique in raising awareness regarding the collection and utilization of solid waste. journal of education and practice, 6(10), 149159. dal, i̇. & cengiz-gökçe, g. (2019). sürdürülebilirlik yolunda “ileri dönüşüm”: bir atölye çalışması. i̇nönü üniversitesi sanat ve tasarım dergisi, 9(20), 30-38. http://dx.doi.org/10.16950/iujad.654300 debrah, j. k., vidal, d. g., & dinis, m. a. p. (2021). raising awareness on solid waste management through formal education for sustainability: a developing countries evidence review. recycling, 6(6), 1-21. https://doi.org/10.3390/recycling6010006 demircioğlu, g., demircioğlu, h., & yadigaroğlu, m. (2015). fizik, kimya ve biyoloji öğretmen adaylarının çevre bilinç düzeylerinin değerlendirilmesi. adıyaman üniversitesi sosyal bilimler enstitüsü dergisi, 8(19), 167-193. http://dx.doi.org/10.14520/adyusbd.41708 desa, a., kadir, n. b. a., & yusooff, f. (2012). waste education and awareness strategy: towards solid waste management (swm) program at ukm. in ukm teaching and learning congress 2011, procedia social and behavioral sciences, 59, 47-50. dharmasiri, l. m. (2019). waste management in sri lanka: challenges and opportunities. sri lanka journal of advanced social studies, 9(1), 72-85. dinler, h., simsar, a., & doğan, y. (2020). okul öncesi öğretmen adaylarının geri dönüşüme yönelik düşüncelerinin incelenmesi. çocuk ve gelişim dergisi, 3(5), 1-11. er-nas, s. & şenel-çoruhlu, t. (2017). fen bilgisi öğretmen adaylarının perspektifinden sürdürülebilir kalkınma kavramı. yüzüncü yıl üniversitesi eğitim fakültesi dergisi, 14(1), 562-580. http://dx.doi.org/10.23891/efdyyu.2017.22 http://dx.doi.org/10.16950/iujad.654300 https://doi.org/10.3390/recycling6010006 http://dx.doi.org/10.14520/adyusbd.41708 http://dx.doi.org/10.23891/efdyyu.2017.22 journal of science learning article doi: 10.17509/jsl.v5i2.39376 319 j.sci.learn.2022.5(2).301-320 ersoy-quadır, s. & temiz, g. (2017). öğretmenlerin ve ebeveynlerin okul öncesi çağı çocuklarla çevre dostu uygulamalarının incelenmesi. üçüncü sektör sosyal ekonomi, 52(2), 71-89. fagnani, e. & guimaraes, j. r. (2017). waste management plan for higher education institutions in developing countries: the continuous ımprovement cycle model. journal of cleaner production, 147, 108-118. fredrick, m., oonyu, j. c., & sentongo, j. (2018). influence of education on the solid waste management practices of communities in kampala city. journal of environment and waste management, 5(1), 261-274. gusti, a. (2016). the relationship of knowledge, attitudes, and behavioral intentions of sustainable waste management on primary school students in city of padang, indonesia. international journal of applied environmental sciences, 11(5), 1323-1332. gültekin, m. c. (2017). üniversite öğrencilerinin çevresel duyarlılıklarına dair bir inceleme: mardin artuklu üniversitesi örneği. selçuk üniversitesi sosyal ve teknik araştırmalar dergisi, (14), 172-185. gürbüz, h. & yılmaz, v. (2018). üniversite öğrencilerinin naylon poşet kullanımına ilişkin tutum ve davranışlarının yapısal eşitlik modellemesiyle araştırılmas [investigation of university students' attitudes and behaviors towards the use of plastic bags by structural equation modeling]ı. sosyoekonomi, 26(38), 135-149. gürer, a. & sakız, g. (2018). yetişkinlerin küresel ısınma ile ilgili bilgi düzeyleri ve geri dönüşüm farkındalıkları. itobiad: journal of the human & social science researches, 7(2), 1364-1391. harman, g. (2017). awareness of science teacher candidates on sustainability: my school. necatibey faculty of education electronic journal of science & mathematics education, 11(2), 243-262. harman, g., aksan, z., & çelikler, d. (2015). mental models which influence the attitudes of science students towards recycling. international journal of sustainable and green energy, 4(1-2), 6-11. doi: 10.11648/j.ijrse.s.2015040102.12 harman, g. & çelikler, d. (2015). the development of conceptual models for teaching waste material recovery to middle school students. journal of materials education, 37(5-6), 227-238. harman, g. & çelikler, d. (2016). fen bilgisi öğretmen adaylarının geri dönüşüm kavramı hakkındaki farkındalıkları. abant i̇zzet baysal üniversitesi sosyal bilimler enstitüsü dergisi, 16(1), 331-353. harman, g. & çelikler, d. (2018). the opinions of elementary science teacher candidates regarding the collection, separation and recycling of solid wastes. kastamonu education journal, 26(3), 813822. doi: 10.24106/kefdergi.413329 harman, g. & yenikalaycı, n. (2020). fen bilgisi öğretmen adaylarının sıfır atık yaklaşımına yönelik farkındalıkları. pamukkale üniversitesi eğitim fakültesi dergisi, (50), 138-161. https://doi.org/10.9779/pauefd.589781 ifegbesan, a. p., ogunyemi, b., & rampedi, i. t. (2017). students' attitudes to solid waste management in a nigerian university: implications for campus-based sustainability education. international journal of sustainability in higher education, 18(7), 12441262. karadağ, y. & acar, f. (2020). sosyal bilgiler öğretmen adaylarının çevre sorunlarına yönelik farkındalıklarının incelenmesi. uluslararası sosyal bilgilerde yeni yaklaşımlar dergisi (ijonass), 4(1), 62-78. doi: 10.38015/sbyy.693956 karakaş, e. & dönel-akgül, g. (2020). sınıf öğretmen adaylarının çevresel duyarlılık ve ahlak ilişkisi üzerine görüşleri. journal of social, humanities and administrative sciences, 6(26), 572-582. karasar, n. (2020). bilimsel araştırma yöntemi: kavramlar ilkeler teknikler. ankara: nobel yayın. kızgın, y., karaosmanoğlu, k., örmeci, g., & taş, ö. (2017). tüketicilerin sürdürülebilir tüketim bağlamında yeşil tüketim eğilimleri ve demografik özelliklere göre farklılıkları. 1st international sustainable tourism congress, november 23-25, kastamonu-turkey, s. 762-777. kumar, p. & kumar, a. (2020). role of education in waste management. iconic research and engineering journals, 4(2), 68-72. liao, c. & li, h. (2019). environmental education, knowledge, and high school students' intention toward separation of solid waste on campus. international journal of environmental research and public health, 16, 1659. doi:10.3390/ijerph16091659 miles, m. b. & huberman, a. m. (1994). qualitative data analysis. an expanded sourcebook (second edition). thousand oaks, ca: sagepublication. minelgaitė, a. & liobikienė, g. (2019). waste problem in european union and its influence on waste management behaviours. science of the total environment, 667, 86-93. https://doi.org/10.1016/j.scitotenv.2019.02.313 ministry of national education [mone] (2018). fen bilimleri dersi öğretim programı (i̇lkokul ve ortaokul 3, 4, 5, 6, 7 ve 8. sınıflar). ankara. molina, r. a. & catan, i. (2021). solid waste management awareness and practices among senior high school students in a state college in zamboanga city, philippines. aquademia, 5(1), 1-8. https://doi.org/10.21601/aquademia/9579 muljaningsih, s. & galuh, a. k. (2018). intention model of waste management education concept based on green campus in brawijaya university. indonesian journal of environment and sustainable development, 9(2), 129-139. doi: 10.21776/ub.jpal.2018.009.02.10 naria, e., nasution, e., & santi, d. n. (2018). improving environmental health through reduce reuse recycle (the 3rs) and trash cans with attractive pictures at high school in percut sei tuan sub-district. in proceedings of the international conference of science, technology, engineering, environmental and ramification researches (icosteerr 2018) research in industry 4.0, 917-921. doi: 10.5220/0010101409170921 nizaar, m., sukirno, djukri, muhardini, s., & mas'ad (2020). improving students' environmental awareness using 3r principles. universal journal of educational research, 8(11b), 6146-6151. doi: 10.13189/ujer.2020.082251 nolasco, e., vieira-duraes, p. h., gonçalves, j. p., cristina de oliveira, m., monteiro de abreu, l., & nascimento de almeida, a. (2021). characterization of solid wastes as a tool to implement waste management strategies in a university campus. international journal of sustainability in higher education, 22(2), 217-236. pamuk, s. & kahriman-pamuk, d. (2019). pre-service teachers’ intention to recycle and recycling behavior: the role of recycling opportunities. international electronic journal of environmental education, 9(1), 33-45. przydatek, g. (2019). waste management in selected national parks a review. journal of ecological engineering, 20(4), 14-22. https://doi.org/10.12911/22998993/102609 rada, e. c., bresciani, c., girelli, e., ragazzi, m., schiavon, m., & torretta, v. (2016). analysis and measures to improve waste management in schools. sustainability, 8, 840, 1-12. doi:10.3390/su8090840 sayın, a. a. & yerli, a. (2020). evsel atıklarda geri dönüşüm farkındalığı ve ekonomiye katkısının incelenmesi. opus uluslararası toplum araştırmaları dergisi, 16(29), 1849-1874. soran, h., morgil, f. i̇., yücel, s., atav, e., & işık, s. (2000). biyoloji öğrencilerinin çevre konularına olan ilgilerinin araştırılması ve kimya öğrencileri ile karşılaştırılması. hacettepe üniversitesi eğitim fakültesi dergisi, 18, 128-139. sönmez, d. (2020). i̇lkokul birinci sınıf öğrencilerinin “sıfır atık” kavramı ile ilgili çizimlerinin incelenmesi. anemon muş alparslan üniversitesi sosyal bilimler dergisi, 8(2), 593-601. şentürk, i̇. (2019). elektrikli ve elektronik eşya atıklarının geri dönüşümü konusunda halkın bilinç düzeyinin ölçülmesi: sivas ili örneği. opus uluslararası toplum araştırmaları dergisi, 11(18), 956-978. tosun, n. & demir, k. (2018). minik ayaklar geri dönüyor. yaratıcı drama dergisi, 13(1), 115-128. triguero, a., alvarez-aledo, c., & cuerva, m. c. (2016). factors influencing willingness to accept different waste management policies: empirical evidence from the european union. journal of cleaner production, 138, 38-46. http://dx.doi.org/10.1016/j.jclepro.2016.05.119 https://doi.org/10.9779/pauefd.589781 https://doi.org/10.1016/j.scitotenv.2019.02.313 https://doi.org/10.21601/aquademia/9579 https://doi.org/10.12911/22998993/102609 http://dx.doi.org/10.1016/j.jclepro.2016.05.119 journal of science learning article doi: 10.17509/jsl.v5i2.39376 320 j.sci.learn.2022.5(2).301-320 tulebayeva, n., yergobek, d., pestunova, g., mottaeva, a., & sapakova, z. (2020). green economy: waste management and recycling methods. e3s web of conferences, 159, 1-9. https://doi.org/10.1051/e3sconf/202015901012 url: atık yönetimi yönetmeliği. [available online at: https://www.mevzuat.gov.tr/mevzuat?mevzuatno=20644&mev zuattur=7&mevzuattertip=5], retrieved on may 11, 2021. yılmaz, a., morgil, i̇., aktuğ, p., & göbekli, i̇. (2002). ortaöğretim ve üniversite öğrencilerinin çevre, çevre kavramları ve sorunları konusundaki bilgileri ve öneriler. hacettepe üniversitesi eğitim fakültesi dergisi, 22, 156-162. zainu, z. a. & songip, a. r. (2017). policies, challenges and strategies for municipal waste management in malaysia. journal of science, technology and innovation policy, 3(1), 18-22. https://doi.org/10.1051/e3sconf/202015901012 https://www.mevzuat.gov.tr/mevzuat?mevzuatno=20644&mevzuattur=7&mevzuattertip=5 https://www.mevzuat.gov.tr/mevzuat?mevzuatno=20644&mevzuattur=7&mevzuattertip=5 microsoft word layout 14348-30200-1-sm.docx a © 2018 indonesian society for science educator 21 j.sci.learn.2018.2(1).21-25 received: 9 september 2018 revised: 13 december 2018 published: 16 december 2018 interactive e-module development through chemistry magazine on kvisoft flipbook maker application for chemistry learning in second semester at second grade senior high school roza linda1*, herdini1, ika sulistya s1, teja pratama putra1 1program studi pendidikan kimia, jurusan pendidikan matematika dan ipa, fakultas keguruan dan ilmu pendidikan, universitas riau, pekanbaru, indonesia *corresponding author. roza.linda@lecturer.unri.ac.id abstract this research is an innovative development on teaching materials in order to intensify the movement of "satu guru satu buku (sagusabu)" and to prepare skillful media users on mastering information and communication technologies (ict). the resulting prototype is an interactive e-module named chemistry magazine which have two editions: ionic equilibrium and ph of buffer solution and solubility equilibrium. applying research and development study design with plomp model, assessment and suggestion by validator team on material substance aspect, instructional design, display (visual communication) and software utilization using validation sheet are in valid category. percentage of legality score by media validator is 91.70% for ionic equilibrium and ph of buffer solution edition; and 94.18% for solubility equilibrium edition, whereas by material validator is 93.75% and 94.45% on each. users response score rate through questionnaire for teachers and students on ionic equilibrium edition and ph of buffer solution edition are 87.08% and 88.45%; meanwhile 94,25% and 91,43% are obtained from solubility equilibrium edition. keywords interactive e-module, chemistry magazine, kvisoft flipbook maker, buffer, solubility, plomp model 1. introduction having a good teaching material is an essential point on acquiring chemistry, it can facilitate both teachers and students while in the learning process (direktorat pembinaan sma, 2008). teaching materials are the resources a teacher uses to deliver instruction. each teacher requires a range of tools to draw upon in order to assist and support student learning. these materials play a large role in making knowledge accessible to a learner and can encourage a student to engage with knowledge in different ways. since 2016, ministry of education and culture has been promoted the motion called “satu guru satu buku (sagusabu) in order to improve teachers capability in producing paperwork. one of the examples is a module. andi zulkarnain, et al (2015) stated that a module is a learning tool that contains the materials, methods, limitations, and steps used systematically and appealing to achieve the expected competence according to the complexity level (direktorat pembinaan sma, 2008). printed modules teaching material can be modified into a filled glossary magazines form, as has been done by wulandari, azrita, & hendri (2016). this breakthrough appear due to learners are accustomed to technological advances and it can be observed that learners have a better understanding on android system technology rather than the teacher itself (syahrowardi & permana, 2016). this is in line with the aim of our national education which stated that chemistry learning is expected to deliver participants who are capable of using information and communication technologies (ict). chemistry magazine is a teaching material in the form of interactive e-module. it can be accessed easily by students using computers and various types of gadgets anywhere and anytime. this module can be created using flipbook applications namely kvisoft flipbook maker (syamsurizal & chairani, 2015). this kind of practicality can enrich digital products such as texts, images, audio, videos, animations, flash and links (www.kvisoft.com, 2015). journal of science learning article doi: 10.17509/jsl.v2i1.12933 22 j.sci.learn.2018.2(1).21-25 two of the hardest discussion, based on student’s difficulty; one is about ionic equilibrium and ph of buffer solution, and the other is solubility equilibrium; are considered as the proper material for e-module content. those subjects require a thorough understanding of theory and analysis to solve the problems, as well as proficient in calculations because the basic concepts related to solubility are described in terms of mathematical equations. based on the explanation above, the research was conducted on the development of interactive e-module chemistry magazine by utilizing kvisoft flipbook maker application for chemistry learning on a second-grade senior high school in the course of the second semester. 2. method the research is held in chemistry education study program, faculty of teachers training and education, riau university with trials in sma negeri 8 pekanbaru, sma negeri 12 pekanbaru and sma negeri 2 bangkinang kota. interactive e-module chemistry magazine by utilizing kvisoft flipbook maker application for chemistry learning on a second-grade senior high school in the course of the second semester is adopted from plomp development model. consisting of initial investigation phase, design phase, realization/construction phase, validation, trials, and revision phase, and implementation phase (rochmad, 2012). the implementation phase is not done because the purpose of the research is limited to obtain output in the form of valid product based on material substance aspect, instructional design, display (visual communication) and software utilization. data gathering related to user's validity and responses to interactive e-module chemistry magazine was obtained in validation phase, experiment and improvement using research instrument in the form of validation sheet prepared by one media validator and two material validators along with rubrics and user response questionnaires by teachers and students. the data collected is then analyzed by specific analytical techniques. analysis of validity using the formulation by rohmad, suhandini, & sriyanto (2013). the data collected is then analyzed by specific analytical techniques. analysis of validity using the formulation by rohmad, suhandini, & sriyanto (2013). p = (n/n)×100% explanation: p = percentage score (%) n = number of scores obtained n = maximum score the percentages obtained are then converted to qualitative values with the validity criteria in table 1. the user response analysis is measured by the likert fourchoice scale with scoring guidelines in table 2. the formula used in calculating the percentage of user response scores using the formulation by yamasari (2010): r = (f/n)×100% explanation: r = percentage of respondents score (%) f = number of scores obtained n = maximum score the percentages obtained are then converted to qualitative values with the validity criteria in table 3. 3. result and discussion this research development produces interactive emodule chemistry magazine products for ionic equilibrium and ph of buffer solution and solubility equilibrium materials. this product can be accessed via electronic media such as a computer, laptop, android, iphone, ipad, and other technology both online and offline. teachers and students can use the interactive e-module chemistry magazine during the learning process in the classroom and as self-teaching materials even not in lesson time. here is the exposure of results and discussion of each development phase that has been done. research stages using the plomp model include the initial investigative phase, design phase, realization/construction phase, validation, trial, and revision phase, and the implementation phase. the explanation of the results of each stage of development, as follows: 3.1 the initial investigation phase front end analysis, information and data obtained from the analysis of the front end is a little source of learning that also can generate interest and motivation of the learners, and it can help the learners to connect with the material that has been learned in daily life. the teaching materials commonly used in the learning process is printed table 1 validity criteria percentage score (%) validity criteria 75.00-100 valid 50.00-74.99 valid enough 25.00-49.99 less valid 0.00-24.99 not valid table 2 user response scoring guidelines statement of attitudes score agree (s) 4 quite agree (cs) 3 less agree (ks) 2 disagree (ts) 1 table 3 criteria for user response percentage (%) criteria for user response 75.00-100 very good 50.00-74.99 good 25.00-49.99 less good 0.00-24.99 not good journal of science learning article doi: 10.17509/jsl.v2i1.12933 23 j.sci.learn.2018.2(1).21-25 text materials in the form of textbooks, student worksheet, and copied task and digital resources in the form of powerpoint text (ppt). analysis of the front end to some relevant literature, obtained the information that the chemistry subject is expected can bring the students to fill the 21st century ability, one of that is, skilled in using media, technology, information and communications (ict) (kementerian pendidikan dan kebudayaan, 2016) and in the framework of teacher competence development in producing a paper which is intensified through movement of “satu guru satu buku (sagusabu)” by directorate of teacher and education personnel of ministry of education and culture since 2016. student’s analysis, characteristics of learners adjust with the design of the development product that has made. the results of the analysis of students showed that students at the second class of senior high school are in the range age of 16-17 years. based on piaget's cognitive development theory, on this age, the learners approach the maximum intellectual efficiency, but due to lack of experience thereby limiting their knowledge and skills to exploit what is known. many things that can be learned through experience, but learners sometimes have difficulty in understanding of the abstract concepts (piaget, 2001). the result of the questionnaire is 100%, the learner stated that the existence of the learning resource is considered important. as many as 85.54% of students find difficulties in learning chemistry subject, in this case, it’s because less of the learning resources that can help the learners to an understanding about the subject, therefore the existence of learning resources in this interactive emodule chemistry magazine is expected can help the learners in the learning process. it is also supported by the facilities for interactive e-module chemistry magazine that is 81.92% of students have pc or gadget to operate of interactive e-module chemistry magazine. material's analysis, the results of interviews with chemistry teachers found that the material of ionic equilibrium and ph of buffer solution there are three main points; the nature of the buffer solution, the ph of the buffer solution and the function of the buffer solution in the body of living beings and in daily life. sub-material properties of buffer solutions are sub-material theories that require an understanding of the concepts that the learners must really understand because these sub-materials are the first sub-matter of the ion equilibrium material and the ph of the solution, as well as for the functions of sub-material buffer solution in the body of living beings and in daily life. the calculation of ph sub-material of the buffer solution is a calculation material involving some of the precursor material such as stoichiometry and acid-base. while in the material of solubility equilibrium, learners learn about the definition of solubility, the yield of the solubility product (ksp), forecasting the formation of precipitate from mixing the ions with opposite charge, the influence of the same ion on the solubility of ionic solids in water, the factors that figure 1 examples of products in ionic equilibrium and ph of buffer solutions editions on material design part figure 2 examples of products in ionic equilibrium and ph of buffer solutions editions on magazine part figure 3 examples of products in solubility equilibrium editions on material design part figure 4 examples of products in solubility equilibrium editions on magazine part journal of science learning article doi: 10.17509/jsl.v2i1.12933 24 j.sci.learn.2018.2(1).21-25 affect the solubility, and selective precipitation (watoni, 2014). competency analysis, the ionic equilibrium, and ph of the buffer solution is one of the chemical learning materials found in the basic competence of 3.12 and 4.12 in the syllabus of chemical subjects of senior high school (kementerian pendidikan dan kebudayaan, 2016). while the solubility equilibrium material is found in the basic competencies of 3.14 and 4.14. the results of the syllabus analysis will be obtained indicators of learning achievement and learning objectives to be achieved by learners. 3.2 design phase the design phase is designed to design the initial product that will be developed in outline. the results of the design stage are; (1) the blueprint of interactive emodule chemistry magazine, (2) grid sheet validation of material experts and media experts and (3) design of user response questionnaires by teachers and students. the blueprint of interactive e-module chemistry magazine load e-module content according to direktorat tenaga kependidikan (2018), that is; 1) cover page, 2) preface, 3) table of contents, 4) position map, 5) description of content, 6) benefits, 7) instructions for use, 8) concept maps, 9) learning; which includes a) learning objectives, b) material description; composed of apperception, learning material, sample questions, formative tests; in the form of crossword and multiple choice puzzles, summaries, assignments, follow-up, magazine content; in the form of scientific articles, tips, galleries, and profile figures, 10) evaluation, 11) answer keys, 12) glossary and 13) bibliography. 3.3 realization or construction phase the results obtained in this phase are; (1) the prototype as the realization of the design result of interactive emodule chemistry magazine which is in accordance with the results of the analysis that has been carried out, the characteristics and structure of an interactive e-module, magazine content, and components of ict-based teaching materials assessment, (2) the validation sheet of material experts and media experts refers to the guidance of developing ict-based teaching materials by the direktorat pembinaan sma (2010) and (3) user response questionnaires constructed in such a way as to the needs of the study. 3.4 validation, trial and revision phase validation aims to derive valuations and suggestions from a validator team consisting of a media validator and two material validators. validation is done until the product developed in category valid. some examples of products in ionic equilibrium and ph of buffer solutions editions can be seen in figure 1 and figure 2. as for the solubility, equilibrium can be seen in figure 3 and figure 4. for the validation of the ionic equilibrium and the ph of the buffer, solution edition can be seen in figure 5 and figure 6. as for the result of validation of the equilibrium solubility edition can be seen in figure 7 and figure 8. overall, the validation results show very good criteria, which means that the e-module chemistry magazine that has been developed has met the requirements set by the direktorat pembinaan sma (2010) in the guide to the development of ict-based teaching materials that covering substance aspect, instructional design, display (communication visual) and software utilization. the trial aims to derive user-side assessments and suggestions of teachers and students of interactive emodule chemistry magazine. scores of respondents of ionic equilibrium and ph of buffer solution edition were obtained from teachers questionnaire sheet of 87.08% and 88.45% of students questionnaires. as for the solubility equilibrium edition, a percentage of 94.25% was obtained from the teachers and 91.43% from students. based on table 4, the 75% -100% range is in the very good category, it can be concluded that the developed interactive e figure 5 comparison diagrams validation i and ii by media experts the ionic equilibrium and the ph of the buffer solution edition figure 6 comparison diagrams validation i and ii by materials experts in the ionic equilibrium and the ph of the buffer solution edition journal of science learning article doi: 10.17509/jsl.v2i1.12933 25 j.sci.learn.2018.2(1).21-25 module chemistry magazine is considered positive by the respondents and can be used in the learning process. 4. conclusion this research was conducted until the phase of validation, trial, and revision, which produced a learning product in the form of ict-based teaching materials. the phases of the research that have been carried out involve investigating the target users of the product, designing, constructing and evaluating by experts to regarding the wetness of the material and the use of the media in the learning process. the results of the research on the development of interactive e-module chemistry magazine of ionic equilibrium and ph of buffer solution edition and solubility equilibrium edition are valid by material validator with a score of 93.75% and 94.45%. while the media validator obtained a score of 91.70% and 94.18%. the interactive emodule chemistry magazine is also rated very well by students and teachers with a percentage of 88.45% and 87.70% in ionic equilibrium and ph of buffer solution edition. while on an edition of solubility equilibrium obtained by percentage of 91,43% by teachers and 94,25% by students. references direktorat pembinaan sma. (2008). teaching materials development guidelines (panduan pengembangan bahan ajar). jakarta: kementerian pendidikan nasional direktorat jenderal manajemen pendidikan dasar dan menengah. direktorat pembinaan sma. (2010). ict-based teaching materials development guidelines (panduan pengembangan bahan ajar berbasis tik). jakarta: kementerian pendidikan nasional direktorat jenderal manajemen pendidikan dasar dan menengah. direktorat tenaga kependidikan. (2008). module writing (penulisan modul). jakarta: direktorat jenderal peningkatan mutu pendidik dan tenaga kependidikan departemen pendidikan nasional. kementerian pendidikan dan kebudayaan. (2016). chemistry syllabus for high school (silabus mata pelajaran kimia sekolah menengah atas/madrasah aliyah sma/ma). jakarta: kementerian pendidikan dan kebudayaan. kvisoft. (2015). kvisoft flipbook maker. retrieved from http://www.kvisoft.com/. piaget, j. (2001). the psychology of intelligence. london: routledge. rohmad, a., suhandini, p., & sriyanto. (2013). development of student worksheets based on exploration, elaboration and confirmation and disaster as geography subject learning materials for sma/ma in rembang pengembangan lembar kerja peserta didik (lks) berbasis eksplorasi, elaborasi dan konfirmasi (eek) serta kebencanaan sebagai bahan ajar mata pelajaran geografi sma/ma di kabupaten rembang. edu geography, 1(2), 1-5. rochmad, r. (2012). design of mathematics learning tool development model (desain model pengembangan perangkat pembelajaran matematika). kreano, jurnal matematika kreatif-inovatif, 3(1), 59-72. syahrowardi, s., & permana, a. h. (2016). multimedia handout design using 3d professional pageflip for learning media on android systems (desain handout multimedia menggunakan 3d pageflip professional untuk media pembelajaran pada sistem android). jpppf (jurnal penelitian dan pengembangan pendidikan fisika), 2(1), 89-96. syamsurizal, h., & chairani, n. (2015). pengembangan e-modul berbasis keterampilan proses sains pada materi kesetimbangan kimia untuk tingkat sma. semirata, 2015. watoni, a. h. (2014). chemistry for sma/ma k11 (kimia untuk sma/ma kelas xi (peminatan). bandung: yrama widya. wulandari, r., azrita, & hendri, w. (2016). development of teaching materials based on magazine-nuanced modules equipped with a glossary for k8 students at smpn 12 padang (pengembangan bahan ajar berbentuk modul bernuansa majalah dilengkapi dengan glosarium untuk peserta didik kelas viii di smpn 12 padang). bung hatta university, 5(7). yamasari, y. (2010). development of high quality ict-based mathematics teaching media (pengembangan media pembelajaran matematika berbasis ict yang berkualitas). seminar nasional pascasarjana x-its. fmipa unesa, 2010. figure 7 comparison diagrams validation i and ii by media experts in the equilibrium solubility figure 8 comparison diagrams validation i and ii by materials experts in the equilibrium solubility microsoft word layout 11995-27901-1-ce.docx a © 2018 indonesian society for science educator 9 j.sci.learn.2018.2(1).9-13 received: 16 july 2018 revised: 4 september 2018 published: 3 december 2018 an action research on enhancing grade 10 student creative thinking skills using argument-driven inquiry model in the topic of chemical environment pitukpong kumdang1*, sirinapa kijkuakul1, wipharat c. chaiyasith2 1faculty of education, naresuan university, tha pho, mueang phitsanulok district, phitsanulok 65000, thailand 2faculty of science, naresuan university, tha pho, mueang phitsanulok district, phitsanulok 65000, thailand *corresponding author. pitukpongk59@email.nu.ac.th abstract a goal of the 21st-century education is to enhance students’ creative thinking skills as the basis for construction of innovations for developing countries. generally, previous teaching tradition, teacher-centered approach, are used in many classrooms, however, failed the goal. therefore, this study aims to promote grade 10 thai students’ creative thinking by implementing argument-driven inquiry (adi) through three cycles of action research. there are 31 students participated in the study. the student data are collected using learning journals, artifacts and informal interviews then analyzed with content analysis and method triangulation. the findings indicate that students have a progression in creative thinking. they can develop skills of curiosity, originality, fluency, imagination, elaboration, and flexibility respectively. as a recommendation, it is necessary that teaching for that success needs integrations among chemical environment, geography, and art. keywords creative thinking skills, argument-driven inquiry, action research 1. introduction “framework for 21st century learning” was developed by partnership for 21st century skills (p21) to prepare all students for living in today’s and tomorrow’s world, for this framework students should have skills of creativity and innovation, critical thinking and problem solving, communication, and collaboration to succeed in work and life in the 21st century world (p21, 2017). meaningfully, the most important skill is creative thinking that is a basis for construction of innovations for developing countries. organization for economic cooperation and development (oecd) that implemented the project in the name of “teaching, assessing and learning creative and critical thinking skills in education”, also supported that the creative thinking skills should be aimed at a goal of education for present day and future generations (stéphan, 2017). nowadays, creative thinking skills are perceived as a group of six sub-skills including an alternative idea or originality, fluency, flexibility, elaboration, curiosity, and imagination (greenstein, 2012). from the pilot study in a special science classroom in lower north area school of thailand, most teachers were familiar with the traditional teaching approach, teachercentered of learning, focused on scientific contents. the school tended to appreciate rote learning for passing examination rather than active learning for using knowledge in real life situations. evidence was found in grade 10 classroom observations. the teacher appeared to use closed-ended questions which had a single answer or a single method to launch understandings of atomic theory, for example. the students were not promoted to meaningfully deep learning and linking between scientific concepts and things around them (thompson, 2017). even though reflection is a critical part of the creative process (resnick & mitchel, 2007), the students were not familiar with reflecting themselves after work. as the results, the students had little competency in creative thinking skills. this situation strongly indicated that the traditional teaching approach was blocking the creative thinking skills of students. one of the effective approaches that possibly promotes creative thinking skills is learning through scientific argumentation approach. generally, argumentation is known as a social activity of at least two parties, e.g. two journal of science learning article doi: 10.17509/jsl.v2i1.11995 10 j.sci.learn.2018.2(1).9-13 groups of students, to present the process of constructing a logical explanation that needs theory and empirical evidence to support for making a group decision and accepting or rejecting any claims (van eemeren, 1995). in other words, the argumentation seems like convincing other people through credible communication and spoken and written proof (erduran, 2007; kuhn & udell, 2011). moreover, scientific argumentation represents an attempt to establish the validity of the claims that require reasoning skill (norris, philips, & osborne, 2007) and scientific’s conceptions. the scientific argumentation is not shouting to fight each other but it is a dispute involved understanding scientific knowledge, demonstrating personal perspective towards circumstances or claims. the scientific argumentation is also framing students to practice critical thinking and creative thinking skills in order to give appropriate scientific reasons to support the claims (berland & reiser, 2011). as mentions above, the process of constructing scientific argumentation possibly promotes the student’s creative thinking skills. therefore, the researcher aims to implement the effective approach base on scientific argumentation to promote creative thinking skills of grade 10 students in the topic of chemical environment. 2. method 2.1 participants thirty-one students in the 10th-grade classroom, in phitsanulok province in thailand, had participated in the study. there were three fifty-minute periods of teaching per week, and they had a lot of learning facilities in the school such as wi-fi, laptops, and smartphones. also, student parents were willing to support all aspects related to student learning, e.g. stationery, technology, and payment. 2.2 action research this study implemented action research as a method to develop own teaching practice within conditions of the 10th-grade classroom. basically, action research is a spiral process of practices solving a particular problem (kemmis, mctaggart, & nixon, 2014). the process consists of 4 steps: 1) plan, 2) action, 3) observation, and 4) reflection (kemmis, 2009) as figure 1. 2.3 argument-driven inquiry (adi) model this study used the principles of argument-driven inquiry (adi) of the previous studies (sampson & grooms, 2010; sampson, grooms, & walker, 2011; sampson & walker, 2012) that concerned scientific argumentation approach. generally, adi is a teaching model emphasizes student-centered learning. it provides students with more opportunities to construct knowledge and scientific explanations through self-inquiry. by teacher facilitation, students learn how to develop an approach for generating data, carrying out an investigation, and simplifying data to answer a problematic issue, as well as reflecting own work and others. in this study, the researcher adaptively implemented the adi principle as the following steps: 1) to identify tasks base on a problematic issue; 2) to plan and design how to investigate data; 3) to analyze data and develop a tentative argument; 4) to present the tentative argument of each group as a group work; 5) to collaboratively revise that argument and then create final argument; 6) to create group artifacts and individual learning journals; and 7) to revise the final report. this aid model was modified through three cycles of action research, for three weeks. 2.4 research instruments and data collection the student data were collected using the artifacts, informal interviews and learning journals. each week of the adi implementation, the students were assigned to create an artifact, i.e. a poster, a model and a painting, to solve environmental issues. before the students would create the artifact, they were challenged to inquire table 1 example of categories and codes. categories codes definitions/levels sample fluency flu1 level 1: no variety or perspective s1315: “energy shortage will happen when the nonrenewable energy is low.” flu2 level 2: some variety but unclearly perspective s2321: “renewable energy could be endless.”, “renewable energy is still limited in use.” flu3 level 3: some variety and clearly describe the perspective s2317: “the area has a lot of wind or solar would be appropriate, it will help to generate energy” flu4 level 4: many varieties and clearly the perspective s2308: “chat-trakan district should be suited to renewable energy sources because of the high mountain area, and strong wind.” figure 1 research process of action research journal of science learning article doi: 10.17509/jsl.v2i1.11995 11 j.sci.learn.2018.2(1).9-13 knowledge and data in the account to the steps of the adi model. during class, the researcher, as a participant observer, checked and reflected the students’ inquiry through informal interviews. they also were asked to illustrate what they learned in the learning journal. 2.5 data analysis this study analyzed the student data using content analysis method. it is the way to search for the truth by making inference about non-definite features of the content through definite features of the fact. the method is used to gain replicable and valid results from the data regarding its content (krippendorff, 2013). table 1 shows examples of data categories and codes constructed by the analysis of the learning journals and artifacts. this study also used method triangulation for trustworthiness. consequently, the student creative thinking skills would be arranged into 4 levels as definitions in table 1. in addition, the skill was identified as six sub-skills including originality, fluency, flexibility, elaboration, curiosity, and imagination. the results will be shown in the next section. 3. result and discussion the findings of the study revealed that adaptively implementation of adi model in the chemical environment topic was more effective in improving the students’ creative thinking skills. figure 2 provides information about the percentage of students who developed creative thinking skills in each level of progression from cycle 1 to cycle 3. as an overview in cycle 1, most students (41.94%) illustrated their progression in the imagination at level 4. in the classroom observation, when the students were asked to create the final argument, they acted like an environmentalist who designed and created a poster to encourage people to use renewable energy. a group of students, for example, showed their imagination through the created poster (figure 3). their perspective on the situation of increasing temperature in the world is presented by the picture of a fried-earth egg that is being heated. this requests that people need to think about figure 2. students’ creative thinking skills 87 .1 0 25 .8 1 9. 6812 .9 0 74 .1 9 90 .3 2 0 20 40 60 80 100 cycle 1 cycle 2 cycle 3 p re ce nt ag e (% ) curiosity 6. 45 48 .3 9 9. 68 6. 45 38 .7 1 22 .5 8 9. 68 6. 45 67 .7 4 83 .8 7 0 20 40 60 80 100 cycle 1 cycle 2 cycle 3 p re ce nt ag e (% ) fluency 16 .1 3 6. 45 74 .1 9 29 .0 3 9. 68 16 .1 3 16 .1 3 54 .8 4 70 .9 7 0 20 40 60 80 100 cycle 1 cycle 2 cycle 3 p re ce nt ag e (% ) elaboration 6. 45 70 .9 7 22 .5 8 3. 231 2. 90 9. 68 2 2. 58 16 .1 3 67 .7 4 67 .7 4 0 20 40 60 80 100 cycle 1 cycle 2 cycle 3 p re ce nt ag e (% ) flexibility 16 .1 3 54 .8 4 6. 459. 68 3. 23 16 .1 3 90 .3 2 83 .8 7 0 20 40 60 80 100 cycle 1 cycle 2 cycle 3 p re ce nt ag e (% ) originality 16 .1 3 3. 23 41 .9 4 32 .2 6 16 .1 3 64 .5 2 8 3. 87 0 20 40 60 80 100 cycle 1 cycle 2 cycle 3 p re ce nt ag e (% ) imagination figure 3 example of the poster. journal of science learning article doi: 10.17509/jsl.v2i1.11995 12 j.sci.learn.2018.2(1).9-13 energy conservation and suggests that people should turn to use renewable energy. this result is in line with thompson’s study (2017) which indicated that the “realworld” situations which are meaningful problems in adi model can promote student’s imagination. in addition, some students (more than 12%) were equally found to achieve in three sub-skills, flexibility, originality and curiosity, but only one student (6.45%) had development in the sub-skill of fluency. there had no student achieving in the sub-skill of elaboration. it was possible that this less achievement appeared because the students were familiar with the traditional teaching approach yet. so, when they were challenged with a problematic issue, they were confused and felt that selfinquiry was hard to practice. examples from the students’ questions in informal interviews indicated that some students had no ideas how to inquire knowledge: “what should i do before starting the investigation?” (s01) and “how should i set the objective to explore?” (s02). moreover, the previous study supported that development of elaboration skill depended on students’ achievement in the skill of originality and competency in self-reflection. if students had no high achievement in the originality and could not reflect themselves, they seemed to have less ability in the elaboration (greenstein, 2012). cycle 2, all six sub-skills were increased dramatically. most students had highly achieved in level 4 of the originality (90.32%), curiosity (74.19%), flexibility (67.74%), fluency (67.74%), imagination (64.52%) and elaboration (57.84%). the students’ learning journal revealed their progression that they could identify problematic tasks into complexed questions as follows: “what are types of renewable energy?”, “which area in phitsanulok province could be built as a power factory from renewable energy?”, and “why do i use that area?” (s03). also, the student’s model (figure 4) was another evidence of the increase of originality, imagination, flexibility, and fluency. as the study of resnick & mitchel (2007, p.3) showed that “create” is at the root of creative thinking. if we want the student to develop their creative thinking skills, we need to provide them with more opportunities to create. there, they developed the final arguments to create the model that showed the possibility of building the power factory in the area. cycle 3, when looking at the graphs as a whole, development of the students’ creative thinking skills is continued on the way of implementing the adi model. this study found that the students had significantly changed in the development of curiosity (90.32%). from the students’ learning journals, most students could create complexed questions to guide self-solution of problematic tasks. in classroom observation, the students often appeared to discuss with their group and others. perhaps these performances led them to have more development of curiosity than what they used to do in cycle 1 and 2. in addition, more than eighty percentages of students also had development in the sub-skills of fluency, originality, and imagination. as evidence from classroom observation, the students were challenged to create their residences for living in the two-degree warmer world. such a result, the students illustrated the originality and imagination through the painting (figure 5). they presented a high technology capsule as their residences which can change seawater to be drinking water and produce food by themselves. after three cycles of action research, this study found that most students had higher development in selfreflection. this meaningful reflection causes them to have rapid progression in the development of the elaboration which is consisted of creative thinking skills. for example, a student (s03) strongly supported that “…our reflections enabled me to improve myself and i would use this strategy for my future work.”. this critical reflection shows the design or strategies can help the student improve their creative thinking skill (resnick & mitchel, 2007). it is also in line with the study of awang & ramly (2008) which stated that: “creative skills must be practiced until the thought patterns in our minds become comfortable with these creative lateral thinking techniques. we can create these creative grooves in our mind so these techniques will figure 4 an example of the model. figure 5 an example of painting. journal of science learning article doi: 10.17509/jsl.v2i1.11995 13 j.sci.learn.2018.2(1).9-13 be utilized. this also can help students produce better, more satisfying and more creative.” additionally, if the students are provided the opportunities to continuously create an artifact together through self-reflection, this strategy could help the student to improve their all subskills of creative thinking skills. 4. conclusion implementation of the modified adi model in thailand developed the students’ all sub-skills of creative thinking including curiosity, originality, fluency, imagination, flexibility, and elaboration respectively. the curiosity had the most progression in the context of the chemical environment, and this curiosity is the basis of learning other sub-skills. after some consideration on the adi model, this study suggested that the priority of teaching is challenging students to identify tasks in the account to problematic issues. this would promote the students’ curiosity. also, teachers need to encourage students to reflect themselves in every activity of the adi model. this would help students to further other subskills. in addition, implementation of the adi model need time so much, so this would be better if teachers could integrate the topic of the chemical environment with other subjects, e.g. geography and art. acknowledgment i would like to thank the faculty of education, naresuan university and the institute for the promotion of teaching science and technology (ipst) which supported me for the scholarship to expand my horizons in the education field. foremost, i would like to express my gratitude to my supervisor and supervising teacher for the motivation and support. lastly, i thank my friends for facilitating and supporting me throughout the process of this study. references awang, h., & ramly, i. (2008). through problem-based learning : pedagogy and practice in the engineering classroom. international journal of human and social sciences, 2(4), 18–23. berland, l. k., & reiser, b. j. (2011). classroom communities’ adaptations of the practice of scientific argumentation. science education, 95(2), 191–216. https://doi.org/10.1002/sce.20420 erduran, s. (2007). methodological foundations in the study of argumentation in science classrooms. argumentation in science education. greenstein, l. (2012). assessing 21st century skills : a guide to evaluating mastery and authentic learning. corwin press. kemmis, s. (2009). action research as a practice‐based practice. educational action research, 17(3), 463–474. https://doi.org/10.1080/09650790903093284 kemmis, s., mctaggart, r., & nixon, r. (2014). the action research planner : doing critical participatory action research. singapore. krippendorff, k. (2013). content analysis : an introduction to its methodology. sage. kuhn, d., & udell, w. (2011). the development of argument skills. child development, 74(5), 1245–60. https://doi.org/10.1111/14678624.00605 norris, s., philips, l., & osborne, j. (2007). scientific inquiry: the place of interpretation and argumentation. science as inquiry in the secondary setting. retrieved from http://www.nsta.org/store/download.aspx?l=8jgs7/jbvfipibfc 0tl4iw== p21 partnership for 21st century learning-creativity and innovation. (n.d.). retrieved september 8, 2017, from http://www.p21.org/about-us/p21-framework/262 resnick, m., & mitchel. (2007). all i really need to know (about creative thinking) i learned (by studying how children learn) in kindergarten. in proceedings of the 6th acm sigchi conference on creativity & cognition c&c ’07 (pp. 1–6). new york, usa: acm press. https://doi.org/10.1145/1254960.1254961 sampson, v., & grooms, j. (2010). generate an argument: an instructional model. the science teacher, 77(5), 32–37. retrieved from http://search.proquest.com/openview/0abe26442b84981b65eda c230cccd331/1?pq-origsite=gscholar&cbl=40590 sampson, v., grooms, j., & walker, j. p. (2011). argument-driven inquiry as a way to help students learn how to participate in scientific argumentation and craft written arguments: an exploratory study. science education, 95(2), 217–257. https://doi.org/10.1002/sce.20421 sampson, v., & walker, j. p. (2012). argument-driven inquiry as a way to help undergraduate students write to learn by learning to write in chemistry. international journal of science education, 34(10), 1443–1485. https://doi.org/10.1080/09500693.2012.667581 stéphan, v.-l. (n.d.). teaching, assessing and learning creative and critical thinking skills in education. retrieved october 13, 2017, from http://www.oecd.org/education/ceri/assessingprogressionincrea tiveandcriticalthinkingskillsineducation.htm thompson, t. (2017). teaching creativity through inquiry science. gifted child today, 40(1), 29–42. https://doi.org/10.1177/1076217516675863 van eemeren, f. h. (1995). a world of difference: the rich state of argumentation theory. informal logic, 17(2), 144–158. a © 2018 indonesian society for science educator 116 j.sci.learn.2018.1(3).116-121 received: 29 june 2018 revised: 2 august 2018 published: 15 august 2018 doi: 10.17509/jsl.v1i3.11797 using physics education technology as virtual laboratory in learning waves and sounds shopi setiawati maulidah1, eka cahya prima1* 1department of science education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia *corresponding author. ekacahyaprima@upi.edu abstract this research was intended to analyze the use of physics education technology (phet) as a virtual laboratory in learning waves and sounds. the analysis was in terms of the implementation of waves on a string student activity as a lesson plan, the profile of students’ cognitive, and the profile of science laboratory environment. the method which is used in this research was a descriptive method with methodological triangulation as the research design. the sample was taken on the convenient situation at grade 8 in an international school in bandung. according to the analysis of the result, the waves on a string student activity can be adopted as the lesson plan with several recommendation to be improved such as in part a, changing some sentences in the data table, changing some settings in obtaining data activity, and adding clear example in determining the base and peak point in measuring the height of wave at start and at the end. moreover in part b, adding clear instruction on how to use the ruler to measure the wavelength, and changing the picture to obtain the data of wavelength with the picture of simulation with the instructed setting are important. in part c, it needs to add the instruction to do the practice session together with the teacher and to add the instruction to make the starting point in counting the wave similar in each trial. the use of physics education technology (phet) as a virtual laboratory in learning waves and sounds shows the favorable result on both the cognitive aspect and science laboratory environment. keywords virtual laboratory, physics education technology (phet), waves and sounds, students’ cognitive, science laboratory environment 1. introduction practical work is an essential feature of science education (abraham & millar, 2008). it can not be separated from the learning process of science. nieh & vaill (2005) stated that practical work is one of the ways to enhance students’ understanding. in addition, frewer & salter (2002) stated that practical work helps students in appreciating evidence as basic of science and acquiring hands-on skills. abraham & millar (2008) stated that practical work can be defined as activities in which the students manipulate and observe real objects and materials. to do this practical work, students have to deal with handson activities. it is clearly known that hands-on activity in learning science is usually done in the laboratory, although it may be held in the classroom and field as well. in this context, laboratories are an essential component of education to make students gain experience (tüysüz, 2010). physical laboratory is very useful to promote the learning process of science, yet provides several problems for both students and teacher. pyatt & sims (2012) stated that physical laboratory experiences may not always promote conceptual change. in the real world situation, students have no such experiences in conducting laboratory activity. based on tüysüz (2010), there are several problems faced in conducting laboratory activity in the physical laboratories such as limitation of facilities, limited time allocation, and insufficient laboratory condition. those problems sometimes force the teachers to perform laboratory activities in crowded groups. moreover, related to a safety concern, tatli & ayas (2013) stated that perform laboratory activities in the physical laboratory involve risks due to poisonous and unsavory gas releases. considering the problem faced by using the physics laboratory to conduct laboratory activities, a virtual laboratory may be a preferable alternative to overcome those problems (tatli & ayas, 2013). virtual laboratories simulate a real laboratory environment and processes and are defined as a learning environment in which students journal of science learning article doi: 10.17509/jsl.v1i3.11797 117 j.sci.learn.2018.1(3).116-121 convert their theoretical knowledge into practical knowledge by conducting experiments (woodfield, 2005). tiwari & singh (2011) added that it is designed and sequenced in such a manner as to give a real feel of performing the experiment. a virtual laboratory may sometimes be a preferable alternative, or simply a supportive learning environment to physical laboratories (tatli & ayas, 2013). according to candelas, et. al. (2003), in this era, educators have got the accesses to use various kind of technology to enhance the effectiveness of the instruction process. it supports the use of virtual laboratory in the instruction process. the use of virtual laboratory as an alternative to overcome the problems faced in the physical laboratory is in line with 21st-century demands. in the 21st century, technologies have become commonplace in improving and advancing the practice of science education because of its potentials of bringing about change in ways of teaching practice and learning process (srisawasdi, 2012). one of the examples of the virtual laboratory is physics education technology abbreviated as phet (finkelstein et al., 2005). phet is developed by the university of colorado, it is freely available on its website (www.phet.colorado.edu). this website consists of more than 50 simulations related to physics subject, it can be accessed both offline and online. these simulations are designed to be highly interactive, engaging, and open learning environments that provide animated feedback to the user. the simulations model physically accurate, highly visual, dynamic representations of physics principles (finkelstein et al., 2005). phet simulation is equipped with its student activity, teacher guidance, and worksheet. many researchers in science have determined that carrying out virtual laboratory in the instruction process significantly increase students’ achievement (tüysüz, 2010; tatli & ayas 2013; candelas, et. al. 2003) and have positive effect on students’ attitudes (tuysuz, 2010; candelas, et. al. 2003; pyatt &sims, 2012). in the process of increasing students’ achievement and having a positive attitude in learning, students experience an environment which can support them to gain the knowledge and have a positive attitude. luketic & dolan (2013) stated that a student’s perception of their learning environment influence how and to what extent they learn and retain knowledge. hence, the researcher decided to analyze the students’ cognitive and their perception about their science laboratory environment in learning waves and sounds using physics education technology (phet) as a virtual laboratory. prima, putri, & rustaman (2018) have implemented a phet simulation to improve students' understanding and motivation in learning the solar system. prima, oktaviani, & sholihin (2018) conducted the learning about electricity using arduino-phetto exercise stem literacy. this research was conducted to analyze the implementation of waves on a string student activity by esler (2011) a lesson plan, to analyze the profile of students’ cognitive and science laboratory environment in learning waves and sounds using physics education technology (phet) as virtual laboratory. the result of this research is expected to be used by the teacher as an information to guide attempts to improve their classroom. 2. method the method that is used in this research was descriptive. the term descriptive research refers to studies which are known to describe and to interpret what is (cohen, manion & morrison, 2007). the location of this research was international junior high school in bandung. the population in this research were 8th-grade students in school a. the samples were one class in eighth grade. the sampling technique that is used was convenience sampling. this technique sampling means taking a group of individuals who (conveniently) are available for study (fraenkel, wallen & hyun, 1993). the instructional tools which was used in this research were waves on a string.swf, it is one of the simulations from phet which can be used to learn transverse wave and it’s properties. to obtain the data, the researcher used various kind of instruments such as students’ worksheet, cognitive test, and slei questionnaire. first, to analyze the implementation of waves on a string student activity as lesson plan, the analysis was in terms of the implementation of the initial lesson plan based on field notes and the result show up as the impact of the activity done by the students such as the data result, and the answer to the question in the worksheet. as the result of this analysis, the recommendation in revising the lesson plan will be elaborated. second, to analyze the profile of students’ cognitive, the researcher used cognitive test consist of 22 questions related to waves properties concept. the result of students’ cognitive test is analyzed by relating it to the learning process, and then it is being calculated until the average of each level of cognitive is obtained, after that interpret the average of each cognitive level by referring to the criteria described in table 1. the interpretation of the result shows the students cognitive in learning waves and sounds with physics education technology (phet). third, to analyze the profile of science laboratory environment, the researcher distributed science laboratory environment inventory (slei) questionnaire to the sample of this research. this questionnaire was developed by fraser, giddings & table 1 the criteria for the average of the cognitive aspect percentage of average (%) interpretation 80-100 very good 66-79 good 56-65 fair 40-55 poor 30-39 failed (arikunto, 2013: 281) journal of science learning article doi: 10.17509/jsl.v1i3.11797 118 j.sci.learn.2018.1(3).116-121 mcrobbie (1992). this questionnaire was field tested and validated simultaneously with a sample of 5447 students in 269 classes in six different countries including usa, canada, england, israel, australia, and nigeria. it also has been cross-validated with 1594 australian students in 92 classes (fraser & mcrobbie, 1995), 489 senior high school biology students in australia (fraser & mcrobbie, 1995) and 1592 grade 10 chemistry students in singapore (wong & fraser, 1994). the data of students’ perception about their science laboratory environment which was obtained through questionnaire distribution were processed by calculating the average score of each item. the interpretation of the result shows the science laboratory environment in learning waves and sound with physics education technology (phet) as a virtual laboratory. 3. result and discussion the data obtained will be analyzed in accordance with the research objectives. the first analysis is about the implementation of waves on a string student activity as a lesson plan. according to waves on a string student activity, there were three main parts of the lesson, in each part of the lesson, part of the worksheet which was recommended to be revised will be elaborated as follows. in part a, the students had to change the scale of amplitude and measure the height of wave at the beginning and the end of the string. since the students used similar simulation and setting, the data result of each group was supposed to be similar. in fact, the data result of each group in this part was different each other but lead to the similar tendency, means the higher the scale of amplitude, the higher the height of wave at the start and at the end of the string. it was happened because of parallax error that caused by the error in determining the base and peak point of the wave, and the wave setting that is used by each group. it is found that oscillate setting is more suitable to be used in obtaining the data in investigating amplitude instead of pulse setting. in this activity, the students also found another difficulty namely did not understand some sentences in the data table. to sum up the previous explanation, it was found that students did not understand some sentences in the worksheet, had difficulties in determining base and peak point, and oscillate setting is more suitable to be used in this activity. hence it is recommended to change the sentence by easily understood sentence, give students example on how to determine base and peak point in measuring the height, and use oscillate setting instead of pulse. in part b, the students had to measure the length of the wave at the beginning & the end of the string. the data result of each group in this activity was also various because each group has a different way to determine the half point of the crest in measuring the wavelength. it was found that the picture in the worksheet was different from the real simulation of the instructed setting. therefore, it confused the students to determine which crest that was being the wavelength 2 because the gap between the crest and the trough near to the end of the string could not be seen clearly. to sum up the previous explanation, it was found that students had difficulties in determining half point of the crest to be measured as wavelength and the picture in the worksheet was different from a real condition one. hence. it is recommended to give clear instruction on how to use the ruler to measure the wavelength and change the picture in the worksheet with real condition one. in part c, in this part the students had to change the scale of frequency 3 times, in each scale, they had to measure the waves passing the certain point at ruler within interval time. in this activity, the students did not find any meaningful difficulties. each group data result was slightly different from each other because each group has a different way to obtain the data. group 1 obtained the data by directly counting the wave as the timer start without pause the simulation first, meanwhile, group 2 obtained the data by pause the simulation first, and then start the timer, after that count the wave by clicking step button. besides that, the determination of the starting point in counting the wave was also influenced by their data result, it can be seen in table 2. table 2 shows the display of different starting point within the similar time. the wave is counted when it is passing the 10 cm point at the ruler. as can be seen in table 2, before 1 second the display of starting point 1 has been table 2 comparison of different starting point in counting the waves starting point 1 starting point 2 journal of science learning article doi: 10.17509/jsl.v1i3.11797 119 j.sci.learn.2018.1(3).116-121 able to be counted as one wave passing the ruler, meanwhile in starting point 2, at a similar time the user has not started to count the wave. it has been proved that when the user using the starting point 1, the data result would be 4 waves counted. meanwhile, when the user using the starting point 2, the data result would not be 4. to sum up a previous explanation, it was found that group 2 has a more homogenous data result than group 1 and the determination of the starting point influenced the data result. hence it is recommended to do the practice session together with the teacher, since this session was intended to make the students understand about the way to obtain the data, and it is also recommended to give instruction to make the starting point in counting the wave similar in each trial. the second analysis is about students’ cognitive. the data on students’ cognitive was obtained through objective which consists of 22 questions. this test was conducted within two days at the same time as the research implementation. the recapitulation of students’ cognitive test result can be seen in figure 1. the average score of each level of cognitive in each day shown by figure 1 was obtained by dividing the average correct answer of each level by the number of the question of each level times 100 %. according to the table 1 about the criteria of cognitive test average by arikunto (2013), the students cognitive result (can be seen in figure 1) in level of c-2 (understanding) and c-4 (analyzing) were considered in the criteria of very good both in day 1 and day 2 of research implementation. meanwhile, the student's cognitive result in a level of c-3 (applying) was considered in the criteria of good on day 1, and very good on day 2 of research implementation. this favorable result of cognitive aspect was supported by the previous research which stated that the use of virtual laboratory increased students’ achievement levels (tüysüz, 2010). the previous research also suggested that virtual laboratories are at least as effective as real laboratories in terms of introducing the students with the experiment process (yu et al.; tatli & ayas, 2013). the third analysis is about science laboratory environment. the data of science laboratory environment is obtained through the science laboratory environment inventory (slei) questionnaire which was distributed to each sample of this research. the average score of each scale as the result can be seen in table 3. table 3 shows the result of the slei questionnaire distributed to the sample of this research. it shows the average item score of each scale. the first scale is students cohesiveness, this scale means the extent to which students know, help, and are supportive of one another (fraser, giddings & mcrobbie, 1992). the average score obtained for this scale is 4.75 with the interpretation of often happened. it means the learning process using physics education technology (phet) as virtual laboratory supported the students’ cohesiveness. this virtual laboratory activity demanded the students be able to know, help, and support one another since they had to work within the group. for example, in one of the activities using this virtual lab, the students helping one another by dividing the job-desk. the second scale is open-mindedness. this scale has the meaning of the extent to which the laboratory activities emphasize an open-ended divergent approach to experimentation (fraser, giddings & mcrobbie, 1992). the average score on this scale is as much as 3.73, with the interpretation of sometimes happened. this scale is related with students’ exploration in conducting the virtual laboratory activity. in this virtual laboratory activity there was instruction given to the students in conducting the virtual laboratory activity, hence sometimes the teacher decide the best way to carry out the virtual laboratory activity and sometimes the teacher asks the students to explore the simulation by themselves. this result is in line with the previous research which stated that the students were given the opportunity to explore and manipulate experimental variables by virtual experiences (pyatt & sims, 2012). the third scale in the slei questionnaire is integration. this scale has the meaning of the extent to which the laboratory activities are integrated with non-laboratory and theory classes (fraser, giddings & mcrobbie, 1992). the average score of this scale is as much as 4.77, it was the table 3 the average score for each scale of slei no attitude scale max item score average item score 1 students' cohesiveness 5 4.75 2 open-endedness 5 3.73 3 integration 5 4.77 4 rule clarity 5 4.56 5 material environment 5 4.74 figure 1 recapitulation of the cognitive test result 93.75 78. 125 100 93.75 0 50 100 150 day 1 day 2 a c h ie v e m e n ts implementation days c-2 c-3 c-4 journal of science learning article doi: 10.17509/jsl.v1i3.11797 120 j.sci.learn.2018.1(3).116-121 highest average score compared to the other scale. it has an interpretation of often happened. it means the students often used a theory which has been learned in regular science class to conduct virtual laboratory activity. even, this virtual laboratory activity helped them to understand the theory covered in regular science class as described in one of the statements in the questionnaire. the fourth scale is rule clarity, with the meaning of the extent to which behavior in the laboratory is guided by formal rules (fraser, giddings & mcrobbie, 1992). the average score on this scale is as much as 4.56, with the interpretation of often happened. the students were given the rule before they conducted the virtual laboratory activity. since they would not deal with dangerous materials, hence they were only given rule about using the laptop as the instructional tools in conducting virtual laboratory activity. the rule forbids them to open any other application besides adobe flash player within the learning process. it made them focus on the virtual laboratory activity. the last scale is material environment which asks the students about the adequacy of laboratory equipment and materials (fraser, giddings & mcrobbie, 1992). since this virtual laboratory activity only used laptop and phet simulation as the equipment, the average score of this scale is 4.74 with the interpretation of often happen. means, the equipment used in this virtual laboratory activity was adequate enough to support the learning process using physics education technology (phet) as a virtual laboratory. 4. conclusion based on the result and discussion elaborated, it can be concluded that waves on string student activity by esler (2011) can be adopted as a lesson plan with several aspects which are recommended to be improved as follows. in part a, changing some sentences in the data table is required in order to make it easier to be understood by the students, changing some settings in obtaining data activity is required, and adding a clear example is required on how to determine the base and peak point in measuring the height of wave at the start and at the end. in part b, adding clear instruction is required on how to use the ruler to measure the wavelength, and changing the picture is required to obtain the data of wavelength with the picture of simulation with the instructed setting. in part c, adding instruction is required to do the practice session together with the teacher, and adding instruction to make the starting point in counting the wave similar in each trial. to sum up, the use of physics education technology (phet) as a virtual laboratory in learning waves and sounds shows the favorable result on both the cognitive aspect and science laboratory environment. references abrahams, i., & millar, r. (2008). does practical work really work? a study of the effectiveness of practical work as a teaching and learning method in school science. international journal of science education, 30(14), 1945-1969. arikunto, s. (2013). dasar – dasar evaluasi pendidikan. jakarta: bumi aksara. candelas, f. a., puente, s. t., torres, f., ortiz, f. g., gil, p., & pomares, j. (2003). a virtual laboratory for teaching robotics. complexity, 1(10), 11. cohen, l., manion, l., & morrison, k. (2007). research methods in education. new york: taylor & francis e-library. esler, j. (2011). waves on a string students activity. [online]. retrieved from http://phet-downloads.colorado.edu/files/activities/zip/ accessed on january 1, 2018. finkelstein, n. d., adams, w. k., keller, c. j., kohl, p. b., perkins, k. k., podolefsky, n. s., ... & lemaster, r. (2005). when learning about the real world is better done virtually: a study of substituting computer simulations for laboratory equipment. physical review special topics-physics education research, 1(1), 010103. fisher, d., henderson, d., & fraser, b. (1995). interpersonal behaviour in senior high school biology classes. research in science education, 25(2), 125-133. fraenkel, j. r., wallen, n. e., & hyun, h. h. (1993). how to design and evaluate research in education (vol. 7). new york: mcgraw-hill. fraser, b. j., giddings, g. j., & mcrobbie, c. j. (1992). assessing the climate of science laboratory classes. national key centre for school science and mathematics, 8, 1-13. fraser, b. j., & mcrobbie, c. j. (1995). science laboratory classroom environments at schools and universities: a cross‐national study∗. educational research and evaluation, 1(4), 289-317. frewer, l., & salter, b. (2002). public attitudes, scientific advice and the politics of regulatory policy: the case of bse. science and public policy, 29(2), 137-145. luketic, c. d., & dolan, e. l. (2013). factors influencing student perceptions of high-school science laboratory environments. learning environments research, 16(1), 37-47. nieh, j., & vaill, c. (2005). experiences teaching operating systems using virtual platforms and linux. acm sigcse bulletin, 37(1), 520-524. prima, e. c., oktaviani, t. d., & sholihin, h. (2018). stem learning on electricity using arduino-phet based experiment to improve 8th grade students’ stem literacy. journal of physics: conference series. 1013(1), 012030. prima, e. c., putri, a. r., & rustaman, n. (2018). learning solar system using phet simulation to improve students' understanding and motivation. journal of science learning, 1(2), 60-70. pyatt, k., & sims, r. (2012). virtual and physical experimentation in inquiry-based science labs: attitudes, performance and access. journal of science education and technology, 21(1), 133-147. srisawasdi, n. (2012). student teachers’ perceptions of computerized laboratory practice for science teaching: a comparative analysis. procedia-social and behavioral sciences, 46, 4031-4038. tatli, z., & ayas, a. (2013). effect of a virtual chemistry laboratory on students' achievement. journal of educational technology & society, 16(1),159-170. tiwari, r., & singh, k. (2011). virtualisation of engineering discipline experiments for an internet-based remote laboratory. australasian journal of educational technology, 27(4), 671-692. tüysüz, c. (2010). the effect of the virtual laboratory on students' achievement and attitude in chemistry. international online journal of educational sciences, 2(1). 37-53. wong, a. f., & fraser, b. j. (1994). science laboratory classroom environments and student attitudes in chemistry classes in singapore. annual meeting of the american educational research association. journal of science learning article doi: 10.17509/jsl.v1i3.11797 121 j.sci.learn.2018.1(3).116-121 woodfield, b. (2005). virtual chemlab getting started. pearson education website. 25, 2005. a © 2018 indonesian society for science educator 110 j.sci.learn.2018.1(3).110-115 received: 29 june 2018 revised: 2 august 2018 published: 11 august 2018 doi: 10.17509/jsl.v1i3.11796 edmodo as web-based learning to improve student’s cognitive and motivation in learning thermal physics pia fildzah vania1, wawan setiawan2, agus fany chandra wijaya3* 1department of science education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia 2department of computer science education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia 3department of physics education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia *corresponding author. agus_ipba@upi.edu abstract this research aims to investigate the use of edmodo as web-based learning in learning thermal physics in junior high school. the analysis of this research focused on students’ cognitive and motivation in the classroom. the method used in this research was quasi-experiment with the pretest-posttest design. one class taken as a sample in this research comes from one of an international school in padalarang. the quantitative data of this research was gained through the objective test (pretestposttest) based on bloom's taxonomy and questioner of motivation based on arcs motivational design. data processing was done by spss 20.0. the result of this research shows that there is an improvement of student cognitive c2 until c5 after learning using edmodo. the average score in pretest was 45.89 and improve to 72.63. the result of normalized gain was 0.49, it included as a medium improvement. edmodo also increases student motivation in learning thermal physics. keywords web-based learning, edmodo, thermal physics, students’ cognitive, students’ motivation, arcs model 1. introduction technology always develops every time, technology helps us to do everything, even in education. technology today can influence the education system, the way of teaching and how a student gets the information. using technology hopefully can make every student has a chance to get a proper education. the current generation of students, commonly referred to as “netgen” learners, grew up surrounded by technology and bombarded by continuous technological advancements. computers, multimedia, the internet, cell phones, and computer games were and continue to be a central and intuitive part of their life (pergola & walters, 2011). that situation did not happen in all school, some school has not yet used technology in their learning process. in indonesia, only some school has integrated the learning process and technology using the internet. the development of multimedia technology has changed the way people learn, obtain various information as well as in interpreting information (ziden & rahman, 2013). the use of the interactive learning environment begins with an instructor uploading a set of lecture slides and setting up an interactive lethe morning session. this task may include adding quiz questions and the lecture session preferences prior to entering the classroom. the instructor has an expanded user interface that provides access to these administer rating functions such as uploading slideshows, creating quizzes, and enabling or disabling in interactive features (chao, parker & fontana, 2011). a student can access the materials or information every time and everywhere as long as there is an internet. this is a good idea if we develop this concept into indonesia education system because children in indonesia can access the materials before they come into the classroom. in learning science concept, sometimes the student find difficulty in learning. by using this web-based learning, the student can choose the time to learn and they can use their own gadget to access the materials. web-based learning also can increase the student motivation in learning. today, some student lazy to learn because the traditional method that uses is not matched with the development of technology. the previous work has been conducted in journal of science learning article doi: 10.17509/jsl.v1i3.11796 111 j.sci.learn.2018.1(3).110-115 order to improve students’ motivation. prima, putri, & rustaman (2018) implemented phet simulation during learning solar system. putri, rusyati, & rochintaniawati (2018) conducted problem-solving model in learning heat to improve students’ motivation. in this research, the authors will find out the use of edmodo towards student cognitive and motivation while learning thermal physics. edmodo is an educational technology that the teacher, student, and parent can use. by this application, the teacher can share information, materials of teaching, a task for the student, and also an exam. the teacher can track student performance and give a score to the student. the student can access the materials every time, chat with their friends and doing online quiz. this research will try to measure student students’ cognitive that is measured in this research involves level cognitive, understanding (c2), applying (c3) and analyzing (c4) based on taxonomy bloom (anderson, et. al., 2001). to teach thermal physics, edmodo can be used as an additional media, the teacher can put all materials on the edmodo and the student can access it before starting the class or at home. in this topic, the teacher will do some classroom activity and quiz. there are some sources that the teacher will put in edmodo, there are video, articles, question, and quiz. the teacher will ask the student to access edmodo before coming to the classroom. a student can easily access the source of the lesson and can repeat it every time a student need. there are several advantages if the teacher uses edmodo than the other media, edmodo is easy to use for teacher and student, the teacher can put a note, quiz, polling, alert and student can access it every time. by using edmodo, it is expected that the student can improve their motivation as well as the student cognitive in learning thermal physics. 2. method this research using the quasi-experiment method, according to fraenkel & wallen (2007), the weak experimental design does not have built-in controls for the threat to internal validity. in addition to the independent variable, there are a number of other plausible explanations for any outcomes that occur. this research design using one-group pretest-posttest design according to fraenkel & wallen (2007). the location of this research was in one international school in bandung barat. the school used cambridge curriculum combine with 2006 curriculum of indonesia. this school using english as the main language in the teaching-learning process. the population in this research was 7th-grade students. the sample was one class in seventh grade. the sample was all of cognitive abilities and motivation on a thermal physics chapter in one class. sampling was selected by purposive sampling technique. 3. result and discussion 3.1 implementation of edmodo the implementation of edmodo conducted both in school and at the home. in school, the student got materials about thermal physics using traditional method combine with edmodo as shown in table 1 about the learning scenario. the implementation of edmodo was conducted in five meeting. the result of implementation edmodo can be seen in the activity in edmodo and in the classroom. in the classroom, the student can access edmodo using handphone and laptop. they should bring this media to access edmodo in the classroom. the other result can be seen from student activity in edmodo. the student can access edmodo to see the materials and doing exercise or quiz. the student can see their score and performance in their own profile. in the implementation of edmodo, the researcher found some effect of edmodo, the effect can be positive and also negative. in the first meeting of edmodo implementation, the students are interested in edmodo and asking many things about it. the teacher distributed booklet how to use edmodo so the students can learn and use it at the house. edmodo also was used as media for the students doing pre-test at home. they are accessed edmodo and doing their own pre-test at home, while they doing the pre-test, some students found the difficulty while doing their first pre-test. there is some reason that can make the student difficult while doing it, there are, the student cannot get internet access at home, the student inadvertent click did so their score is zero, and the student did not see the time limitation while doing a pre-test. for table 1 learning activity date school activity home activity day 1  distribute booklet to the student  ask the student to login & doing pre-test  doing pre-test day 2  introduction to thermal physics  learning-process in the classroom  doing exercise  reading powerpoint about the thermometer day 3  learning-process in the classroom  reading powerpoint about thermal capacity  learning worksheet calorimeter day 4  laboratory activity: calorimeter  announcement about post-test day 5  post-test journal of science learning article doi: 10.17509/jsl.v1i3.11796 112 j.sci.learn.2018.1(3).110-115 the student for that reason, the teacher giving the same question but using a paper test. in the school, the students learn as usual but combine with edmodo. the role of the teacher cannot be replaced with edmodo, the teacher still need to guide the student in the classroom. in the second meeting, the student brings their own laptop and handphone to the classroom so they can access edmodo in the class. the teacher also checks student problems while they're using edmodo at their home. in edmodo, the teacher put exercise that student must fill in the classroom. at the end of the class, the teacher announces student to access edmodo at home and see the next topic, so they can learn at home. the teacher also combines the learning process in the classroom using classroom activity and laboratory. in the third meeting, the teacher continues teaching in the classroom and put the materials in edmodo as usual. the teacher also put worksheet calorimeter for the student. the student should learn the worksheet before comes to the laboratory. in forth meeting, the teacher asks the student to bring their own handphone and laptop to see and fill out the worksheet. some student also found the problem while doing this, some student that did not have pdf application cannot open the worksheet using their own handphone but since this laboratory was grouped assignment, so the student can share their phone and laptop with their friends. the last meeting used to post-test and fill motivation questioner. 3.2 student cognitive student cognitive in this research is obtained by using multiple choice questions based on the level of cognitive by blooms taxonomy. this test is given to the student before learning (pre-test) using edmodo and after learning (post-test) after learning using edmodo. this test used to measure the improvement of student cognitive towards the implementation of edmodo as web-based learning in learning thermal physics are pre-test and post-test. the parameter that used was revised of bloom’s cognitive domain from c2 until c4. c2 is the representative of the low cognitive level and c3 until c4 as representative of the high cognitive level. the improvement of student cognitive was determined by calculating the score of normalized gain <g>. the normalized gain value was processed through data of pretest and post-test score which has been executed by all participant students. the result of the normalized gain <g> calculation is presented in table 2. it can be seen that the score of the student has increased from 45.89 to 72.63. the result comes from 27 samples. data pre-test taken before the student get treatment meanwhile the data of post-test collected after a student got treatment using edmodo. there were 18 questions that distributed into learning objective in thermal physics. the standard deviation was 2.915 for pre-test and 3.121 for post-test. n-gain score from this research was 0.49. the normalized gain results as much as 0.49. based on hake’s index it included as a medium category of improvement. to analyze the whole improvement profile of student cognitive, the instrument was designed to interpret the student's capability in a cognitive domain based on bloom’s taxonomy. there are three different level of cognitive that used in this research. start from c2 (understanding), c3 (applying); c4 (analyzing). the result of each cognitive domain from student answer in pre-test and post-test is shown in table 3. the improvement of the student in each domain got medium category based on hake (1999). the preliminary objective test has been conducted to see the validity, reliability, discriminating power and difficulty level of the objective test. each cognitive level has a different result, the biggest significant difference is in c4 (applying) cognitive level, after that c3 (applying). generally, all the cognitive level showing the improvement between pre-test and post-test. table 3 recapitulation of student cognitive domain based on bloom's taxonomy cognitive domain amount of question participants max score pre-test post-test <g> category c2 8 27 100 54.16 74.07 0.43 medium c3 4 36.11 66.66 0.47 medium c4 6 41.97 74.69 0.56 medium table 2 summary of student cognitive result no component pretest posttest <g> category 1 participant 27 0.49 medium 2 max score 100 3 �̅� 45.89 72.63 4 lowest score 5.6 44.4 5 highest score 77.8 100 table 4 the summary of student’s statistical test components results kolmogorov-smirnov normality test sig. ≥0,05 = normal sig. 0.110 conclusion normal wilcoxon test (asymp sig (2tailes) < 0.05, h0 = rejected) sig. 0.001 conclusion h0rejected h1retained, journal of science learning article doi: 10.17509/jsl.v1i3.11796 113 j.sci.learn.2018.1(3).110-115 to determine whether the hypothesis is accepted or rejected, the normality and wilcoxon test were conducted. the summary of normality and wilcoxon is shown in table 4. it shows that the normality test (kolmogorov-smirnov) has 0.110 (sig. ≥0.05 = normal) that is mean that the result is normal and possible using parametric statistic. to use parametric statistics, the normality should be sig. ≥0,05, but the samples that taken should be more than 30 samples. in this research, there are 27 samples that are mean less than 30 and this research cannot use parametric statistics. samples that less than 30 will use non-parametric statistic, this research use one classroom as a control class and experiment class, wilcoxon test was chosen because of that reason. in wilcoxon test, the significant difference that resulted in 0.001 (asymp sig (2tails) < 0.05 ) that is mean h0 rejected and h1 retained, or there is a difference of students cognitive and students’ motivation in learning thermal physics using edmodo as web-based learning. the implementation as treatment of edmodo has been conducted in five meeting in the classroom and edmodo also accessed by the student at home, the pre-test as well as post-test. pre-test and post-test showed the improvement of student score. the average score of pre-test was 45.89 and post-test was 72.63 the normalized gain of this research was 0.49 and based on hake (1999) is included in the medium category. using the educational networks as integration in teaching has a positive effect on secondary school students (nee, 2014). the use of edmodo is not only in the classroom, the implementation of edmodo also conducted out of school or at home. in the first meeting, the student got the induction and guiding book. the student should read the materials of learning before coming to the classroom in edmodo also doing a pre-test. pre-test conducted in edmodo, the student has a deadline to do the pre-test. some students found the difficulty while doing pre-test using edmodo. they said that they cannot turn in the test. more than half student finished pre-test in edmodo, the other doing pretest in the classroom because of the due of the pre-test. in a traditional classroom, students concentrate on school learning, the learning time and space are greatly restricted. in the educational network, all resources are shared on the network at any time and anywhere (nee, 2014). in the first meeting, the student interest with edmodo, they were asking how to use it, how they score will be upload. the students feel exciting while they know they can bring their own handphone into the classroom. a combination of traditional in-class instruction and an online classroom community via edmodo is significantly verified to be an effective factor in generating more positive attitudes. in the implementation of edmodo, some student cannot answer the pretest. the question in pre-test comes from 11 learning objective and has been tested before the test giving to the student. the difficulty level of the question also arranged before the question uploaded to edmodo. the question can be easy, medium and difficult. in the result, most of the score of pretest lower than the standard score at school. the student who do pre-test never learn thermal physics concept before, maybe just some indicator that taught in elementary school such as a thermometer. at the end of the class, the teacher giving the task to access edmodo and see what will be learned tomorrow. there is some factor that can make the student has a low score, the factor can be from internal and from external. for the student that did not have willing to learn, the score of student can be lower than standard. learning situation force the student to study at home and improve their performance. learning materials are available anytime and anywhere. that can increase their motivation to gain more score. a teaching method that used in this experiment was expository and lecturing, the student got homework in edmodo, it forces them to accessed edmodo at home. in the school student can bring their own handphone or laptop to access edmodo. handphone is one of the media to access edmodo, they can access learning materials in the classroom. that kind of situation can stimuli the student to have a higher score in their post-test. technology helps the student find another way to learn. in the classroom, the student activities in asking and answering teacher question about thermal physics. they were willing to learn. in laboratory activity about calorimeter, the students access the procedure and question in edmodo. they collected the answer in a paper. from the point of view of cognitive theory domain, the implementation of edmodo has increased their cognitive domain. in each cognitive domain, there was an improvement, the improvement of the cognitive domain because the student has already got the materials of thermal physics. the student learns at school and at home, it increases their knowledge about thermal physics, which is the reason they can increase their performance but, some student did not improve their score. based on the wilcoxon test, the value is 0.001 that means hypothesis h0 was rejected, so there was the significant difference after learning using edmodo. edmodo was helping the teacher to give the source to the student, to make a student learn at home and make the student increase their score. journal of science learning article doi: 10.17509/jsl.v1i3.11796 114 j.sci.learn.2018.1(3).110-115 3.3 student motivation the second variable in this research is student motivation towards learning thermal physics using edmodo. the data of student motivation collected from questioner of motivation, arcs model motivational design by keller (2000). keller (2000) arranged 36 questioners into four situations, there are attention, relevance, confidence, and satisfaction. each number has its own score. the result of student motivation questioner present in table 5. according to table 5, in each condition, the student felt motivated enough. attention has average score 3.28, means that the student giving attention while learning in the classroom and they are willing to invest their time, pay attention, and find out more (poulsen, lam, cisneros & trust, 2008). relevance has an average score of 3.49, this score higher than attention condition, same with attention, relevance also showing that student has been motivated enough but in different condition. the third one is confidence, the score was 3.23 this was the lowest score compared with the other condition. the score means that the expectations of the student was good, and feel motivated. the last condition is satisfied, in satisfaction, the average score was 3.54 this was the highest score in the condition that means that student motivated. and in the last column, the average from all categories, the average score of arcs was 3.38 from the maximum score was 5. this means that the student motivated enough. the analysis will do in each situation. comparison of each situation based on a score that gains by each condition shown in figure 1. satisfaction shows the highest part. the implementation of edmodo in the classroom can affect student motivation in learning. motivation refers to the reasons individuals take action. motivation to learn willingness or desire to gain information, develop skills and attain mastery (blazer, 2010). their interest in learning with this kind of software was greater than their interest in learning in the traditional classroom (wang & reeves, 2007). many students are not motivated to learn in the classroom (blazer, 2010). to see student motivation, this research used the arcs model by keller (2000). to see student motivation also can be seen from student post-test score. motivated student earns higher grades and achievement test scores than the extrinsically motivated student (blazer, 2010). the analysis of student motivation was conducted in general and also personal. the data from student motivation questioner were processed using microsoft excel 2010. the score from each situation is calculated to find the mean of each situation. the highest improvements are shown in relevance condition while the lowest improvement is shown in the attention and satisfactory condition. the result comes from calculating the average score of a combination of positive and negative criteria for each condition, then determine the categories with the provisions of the average score. the first category was attention, the average score of attention was 3.28, with percentage 60.68%, that was mean that teacher can get class attention and defend it, the student was motivated enough to learn. it is really possible since this is the first time for student used edmodo in the classroom. usually, the teacher teaches the student using the traditional method, since teacher trying using edmodo in the classroom, the teacher can get student attention and increase student motivation in learning science. the potential that technology offers to positively affect student achievement is sufficient reason to integrate technology as a means to motivate students to learn (heafner, 2004). the second category was relevance, the average score of relevance was 3.49, and the percentage was 69.9%. the score means that the teacher can give the example that related to daily life. this score means that the student motivated to learn because of the learning materials related to daily life. while learning in the classroom, teacher showing example that related to daily life, the teacher also provides several things that familiar with the student like a thermometer. in learning thermal physics, a teacher showing the video about children that got the fever to guide the student find the function of a thermometer. the third one was confidence. the average score of this categorize was 3.23, this includes in motivated enough. the percentage was 64.6 %, confidence means that teacher should trust the student and give student believes. the teacher can motivate the student to improve their performance. teacher showing the edmodo in the last session of the meeting, the teacher show student score to table 5 average score in each condition condition average definition attention 3.28 motivated enough relevance 3.49 motivated enough confidence 3.23 motivated enough satisfaction 3.54 motivated arcs 3.38 motivated enough figure 1 comparison arcs situation 3.29 3.50 3.23 3.54 0.00 1.00 2.00 3.00 4.00 5.00 attention relevance confidence satisfaction s c o re o f s tu d e n t m o ti v a ti o n situation based on arcs model journal of science learning article doi: 10.17509/jsl.v1i3.11796 115 j.sci.learn.2018.1(3).110-115 increase their motivation to gain more and to participate more in edmodo and in the classroom. the last categorize was satisfaction, the score of satisfaction was 3.58, this is the highest score among all categories, this score included as motivated. the percentage of this categorize was 62.9%. satisfaction means that the student feeling satisfied after got high score or achievement. after the teacher showing the score in edmodo, some student that got a high score express their happiness in the classroom, they are feeling satisfied with their result. these conditions in the classroom increase another student to gain more score. they were asking about the question that they cannot answer while doing a pre-test. 4. conclusion the implementation of edmodo as web-based learning has been conducted systematically. it is acquired some conclusions as follows. first, the implementation of edmodo in the classroom resulted in a positive effect on student cognitive and student motivation. second, the implementation of edmodo as web-based learning in learning thermal physics can improve student cognitively, can be noticed by the results of each cognitive level that increase from pre-test to post-test and also the normalized gain is shown medium improvement. third, the implementation of edmodo also affected student motivation in learning thermal physics, this can be noticed from the result of student motivation questioner. the result from the motivation questioner included as satisfied motivated. references anderson, l. w., krathwohl, d. r., airasian, p. w., cruikshank, k. a., mayer, r. e., pintrich, p. r., ... & wittrock, m. c. (2001). a taxonomy for learning, teaching, and assessing: a revision of bloom’s taxonomy of educational objectives, abridged edition. white plains, ny: longman. blazer, c. (2010). twenty strategies to increase student motivation. information capsule. research services, miami-dade county public schools. 0907. chao, j. t., parker, k. r., & fontana, a. (2011). developing an interactive social media based learning environment. issues in informing science and information technology, 8, 323-334. fraenkel, j.r., & wallen, n.e. (2007). how to design and evaluate research in education, sixth edition. new york: mc graw hill. hake, r. r. (1999). analyzing change/gain scores. unpublished.[online] url: http://www. physics. indiana. edu/~ sdi/analyzingchange-gain. pdf. heafner, t. (2004). using technology to motivate students to learn social studies. contemporary issues in technology and teacher education, 4(1), 42-53. keller, j. m. (2000). how to integrate learner motivation planning into lesson planning: the arcs model approach. vii seminario, santiago, cuba, 1-13. nee, c. (2014). the effect of educational networking on students’ performance in biology. in tcc worldwide online conference (pp. 7397). tcchawaii. pergola, t. m., & walters, l. m. (2011). evaluating web-based learning systems. journal of instructional pedagogies, 5. prima, e. c., putri, a. r., & rustaman, n. (2018). learning solar system using phet simulation to improve students’ understanding and motivation. journal of science learning, 1(2), 60-70. poulsen, a., lam, k., cisneros, s., & trust, t. (2008). arcs model of motivational design. retrieved march, 21, 2011. putri, a. u., rusyati, l., & rochintaniawati, d. (2018). the impact of problem-solving model on students’ concept mastery and motivation in learning heat based on gender. journal of science learning, 1(2), 71-76. wang, s. k., & reeves, t. c. (2007). the effects of a web-based learning environment on student motivation in a high school earth science course. educational technology research and development, 55(2), 169192. ziden, a. a., & rahman, m. f. a. (2013). the effectiveness of webbased multimedia applications simulation in teaching and learning. international journal of instruction, 6(2), 211-222. microsoft word layout 13104-31140-1-ce erning.docx a © 2019 indonesian society for science educator 58 j.sci.learn.2019.2(2).58-64 received: 19 september 2018 revised: 9 january 2019 published: 11 january 2019 improving students’ sustainability awareness through argument-driven inquiry erning rahmadini salsabila1*, agus fany chandra wijaya2, nanang winarno3 1al-azhar syifa budi parahyangan, bandung, indonesia 2department of physics education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia 3international program on science education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia *corresponding author. erningrahmadini@gmail.com abstract sustainability awareness is one of the things that the student should have to help in a caring environment. sustainability awareness of the student can be built by knowledge and awareness of what should be done or not. the student will be more aware of the students involved and explore more in building a concept about the environmental issue itself. the aim of this study is to investigate the impact of argument-driven inquiry toward students' sustainability awareness in learning global warming. the method used in this research was experimental research. this study used two different groups of student. the first group learned using argument-driven inquiry and the second group learned using inquiry-based learning. the population was seventh-grade students in one of junior secondary school in bandung, indonesia. the purposive sampling technique was used to choose the sample. the participants consist of 52 students from both groups. each group consists of 26 students. the data were collected by giving the questionnaire of sustainability awareness. the questionnaire consists of 15 items. the result of this study shows the different percentage in the level of students' sustainability awareness between the two groups. the most different percentage is on the sustainability practice awareness aspect. the group that used argument-driven inquiry has percentage 40.7% which means as medium and the group that used inquiry-based learning has percentage 37.6% which means as low. based on the result, argument-driven inquiry gives a better impact on students’ sustainability awareness. argument-driven inquiry can be considered as one of the alternative teaching models that can give a better understanding in building the awareness of junior secondary school student. keywords sustainability awareness, argument-driven inquiry, global warming 1. introduction argument-driven inquiry is the learning model that conducts argumentation session in the part of the learning process. argument-driven inquiry has the purpose to give more place of argumentation in the inquiry process in order to improve the learning outcome (hasnunidah, susilo, irawati & sutomo, 2015). argument-driven inquiry has seven steps of learning which are the argumentation session is included. argumentation is one of the ways to facilitate the students in gaining the concept and gaining more reason behind the process of phenomenon happens in nature. the inquiry is often used in the learning activity and argumentation is one of the important things that support the inquiry in the learning process (aulia, poedjiastoeti, & agustini, 2018). according to demircioglu & ucar (2015) stated that argumentation is not really often to be implemented in learning science. the implementation of argumentation in learning global warming is one of the important things to support and help a student in explaining the phenomenon. by argumentation, the student can explore more the idea about how and why global warming happens. global warming is a topic that discusses the environmental issue that happens in nature. the participation of students in environmental issue has been basically built at the school. besides the theoretical knowledge, the awareness of the environmental issue is also one thing that should the students have (hamid, ijab, sulaiman, md. anwar & norman, 2017). students are most likely to do things for a personal material prize or awards because the most student is not doing something where they do not get any direct journal of science learning article doi: 10.17509/jsl.v2i2.13104 59 j.sci.learn.2019.2(2).58-64 impact or advantage for them. according to sammalisto, sundström, von haartman, holm, & yao (2016) stated that students tend to not doing something if there is no advantage for themselves. sustainability awareness is one of the prerequisites for environmental attitude and behavioral change in caring the natural environment to impede climate change and global warming (hamid, ijab, sulaiman, md. anwar & norman, 2017). the students probably have a lack of awareness because they do not really understand what is happening in the environment or how global warming damages the environment. this can be a challenge for educators to build or improve the sustainability awareness of students. according to hodson (2009) stated that the students should be able to have a reason in science, so they can relate the concept and daily phenomenon. in recent years, many studies have been carried out the implementation of argument-driven inquiry in learning science. demircioglu and ucar (2015) have investigated students’ argumentation level and science process skill by using argument-driven inquiry (adi) in learning physic. the previous research from cetin & eymur (2017) has investigated students’ writing skill and scientific presentation skill by using argument-driven inquiry (adi) in learning chemistry. another research from walker, sampson, southerland, & enderle (2016) has investigated students’ academic achievement by using argumentdriven inquiry (adi) in learning chemistry. however, the research that investigates students' sustainability awareness by using argument-driven inquiry (adi) has not been researched yet. therefore, this study engaged the concept of climate change, greenhouse effect, and the impact of global warming that utilizes argumentation during the learning process to build the awareness of students about the environment. the aim of this study is to investigate the impact of argument-driven inquiry on students' sustainability awareness in learning global warming. 2. method 2.1 research method this study used the quasi-experiment method. quasiexperiment is where the treatment is administered to only one of two groups whose members were randomly assigned (creswell, 2012). this study was conducted in two different groups of student. the first group learns using argument-driven inquiry and the second group learns using inquiry-based learning. 2.2 population and sample the location of this study was in one of junior secondary school in bandung, indonesia. the school used curriculum 2013. the population of this study was seventhgrade students with aged ranged between 13-14 years old. the sample consists of 52 students: 26 (12 female, 14 male) in the first group and the 26 (13 female, 13 male) in the second group. the sample was chosen by purposive sampling technique. fraenkel, wallen, & hyun (2012) stated that purposive sampling is defined where the researcher uses the judgment to select a sample that they believe, based on prior information, will provide the data they need. the data of the sample can be seen in table 1. 2.3 research instrument the data was gathered by giving the questionnaire of sustainability awareness to the students. the questionnaire is adopted from hassan, noordin, & sulaiman (2010) that consist of three aspects of sustainability awareness which table 1 data of population and sample group population sample n percentage (%) total (%) first 7th grade male 14 53.80 100 female 12 46.20 second 7th grade male 13 50.00 100 female 13 50.00 table 2 questionnaire sustainability awareness no statement response yes no 1 i read about environmental issues in the mass media 2 i concern about environmental problems at my place 3 i always discuss environmental problems with my friends 4 i feel disappointed with air pollution 5 i feel disappointed with river pollution 6 i appreciate biodiversity 7 i concern about smoke that is emitted by vehicles 8 i try to reduce the amount of waste at home by collecting materials that can be recycled 9 i composting the food residue to become fertilizer 10 i do not use a plastic bag to wrap things 11 i conserve the use of electric energy at home 12 i conserve the use of water supply 13 i deliver information on the environment to my family members 14 i involve in the environmental awareness activities in school 15 i aware of my responsibility towards the environment table 3 sustainability awareness interpretation sustainability awareness value criteria of percentage response 0.0–39.9 practices that seldom or dislike being done/low 40.0–69.9 practices that are done/ happened moderate/medium 70.0–100 practices/feelings that are most likely one/happened/high journal of science learning article doi: 10.17509/jsl.v2i2.13104 60 j.sci.learn.2019.2(2).58-64 are sustainability practice awareness, behavior, and attitude awareness, and emotional awareness. the questionnaire consists of 15 items with “yes” and “no” choices. the questionnaire of sustainability awareness is shown in table 2. items 3, 9, 10, 13, 14, represent students’ sustainability practice awareness. items 1, 6, 7, 8, 11, 12 represents students’ behavioral and attitude awareness. items 2, 4, 5, 15 represent students’ emotional awareness. 2.4 research procedure in this study, argument-driven inquiry was implemented in the first group. there is seven steps of argument-driven inquiry which are identifying the task and guiding question, design a method and generate data, production of tentative argument, argumentation session, write an investigation report, double-blind group peer review, revise and submit a report. inquiry-based learning was implemented in the second group. the step of inquirybased learning includes identification of the problem, questioning, making a hypothesis, collecting data, analyzing data, and making a conclusion. both groups were given the topic of global warming. the concept of global warming is limited based on indonesian curriculum 2013. the research was conducted in four meetings. the first meeting until the third meeting was for the treatment of argumentdriven inquiry and inquiry-based learning, and the fourth meeting was for giving the questionnaire to the students. 3. result and discussion students’ sustainability awareness data was collected by conducting questionnaire that consists of 15 items with “yes” and “no” answer options. students’ sustainability awareness has three aspects and it is analyzed by calculating the percentage of each aspect. the questionnaire consists table 4 sustainability awareness in the first group aspect : sustainability practice awareness statement number of data yes no item 3 12 14 item 9 5 21 item 10 9 17 item 13 14 12 item 14 13 13 total 53 77 130 percentage response 40,76923 aspect : behavioral and attitude awareness statement number of data yes no item 1 18 8 item 6 22 4 item 7 24 2 item 8 12 14 item 11 22 4 item 12 22 4 total 120 36 156 percentage response 76,92308 aspect : emotional awareness statement number of data yes no item 2 22 4 item 4 25 1 item 5 26 0 item 15 24 2 total 97 7 104 percentage response 93,26923 table 5 sustainability awareness in the second group aspect : sustainability practice awareness statement number of data yes no item 3 11 15 item 9 1 25 item 10 6 20 item 13 15 11 item 14 16 10 total 49 81 130 percentage response 37,69231 aspect : behavioral and attitude awareness statement number of data yes no item 1 16 10 item 6 23 3 item 7 24 2 item 8 9 17 item 11 22 4 item 12 22 4 total 116 40 156 percentage response 74,35897 aspect : emotional awareness statement number of data yes no item 2 18 8 item 4 24 2 item 5 24 2 item 15 24 2 total 90 14 104 percentage response 86,53846 figure 1 percentage on each aspect of sustainability awareness journal of science learning article doi: 10.17509/jsl.v2i2.13104 61 j.sci.learn.2019.2(2).58-64 of items that represent all aspect of sustainability awareness. the level of sustainability awareness is determined based on the percentage score from hassan, noordin, & sulaiman (2010) that is interpreted in table 3. the percentage of students’ sustainability awareness was analyzed using microsoft excel. sustainability awareness has three aspects, there are sustainability practice awareness, behavioral and attitude awareness, and emotional awareness. in each aspect, the number of “yes” and “no” answer is calculated and the number of “yes” answer will be divided by the number of all the students’ answer both “yes” and “no” on the items of a certain aspect. the analysis of the percentage of sustainability awareness in the first group is shown in table 4, and the analysis of the percentage of sustainability awareness in the second group is shown in table 5. the result of students’ sustainability awareness in the first and the second group can be seen in figure 1. based on figure 1, those three aspects of both groups have a different percentage. the percentage of sustainability practice awareness aspect in the first and second group has a different category. in the first group is 40.7% which categorized as medium, while the second group is 37,6% which categorized as low. the result means that the first group student has better practice awareness such as discussing or delivering information about table 6 students’ activity in the first group first meeting the step of argumentdriven inquiry description of activities observed activities identification of the task and guiding question the students observe picture and video about climate change. most of the students are a very enthusiast in watching video designing method and generate data the students are divided into 5 groups and each group is designing the procedure of experiment about the impact of co2 toward temperature. some student did not bring tools and materials for the experiment, so other groups have to share and sometimes the number of tools and material is not enough. production tentative argument the students analyze the data from the experiment and finding the evidence to support their findings. the students write the tentative argument on the worksheet argumentation session each group delivers argumentation based on the problems that are given on the worksheet. among groups can perceive, criticize, oppose, and support the statement during this session. some students are not conducive, so it disturbs other students who are presenting. write an investigation report the students are making a report by completing the worksheet. the students are completing the worksheet submit the report the students submit the worksheet as their practical report. the students submit the worksheet second meeting the step of argumentdriven inquiry description of activities observed activities identification of the task and guiding question the students observe picture and video about the greenhouse effect. most of the students are a very enthusiast in watching video designing method and generate data the students are divided into 5 groups and each group is designing the procedure of experiment about the greenhouse effect. the experiment needs sunlight as the heat energy. there was lack of sunlight so the result of the experiment not really clear. production tentative argument the students analyze the data from the experiment and finding the evidence to support their findings. the students write the tentative argument on the worksheet argumentation session each group delivers argument based on the problems that are given on the worksheet. among groups can perceive, criticize, oppose, and support the statement during this session. some students are not conducive, so it disturbs other students who are presenting. write an investigation report the students are making a report by completing the worksheet. the students are completing the worksheet submit the report the students submit the worksheet as their practical report. the students submit the worksheet double-blind peer review the student submitted the assignment of making an article and reviewed and given feedback by their classmate the teacher asked the student to make a handwriting article, but some student makes it in the type-writing article. journal of science learning article doi: 10.17509/jsl.v2i2.13104 62 j.sci.learn.2019.2(2).58-64 environmental problems and doing environmental awareness activity. based on the number of percentage of behavioral and attitude awareness and emotional awareness aspect in both class have almost the same percentage. in the first group is 76,9% and the second group is 74,3%. the behavioral and attitude awareness aspect in both groups is categorized as a medium. the result means that both groups of student have the same level of awareness to implement activities such as conserve energy and water, and recycling. the emotional awareness aspect of the first group is 93,2% and the second group is 86,5% which is categorized as high. the result means that the students in both classes are very aware of the response to environmental issues that happen. from the data, it shows that the students from both groups are really aware of the responsibility to the environment. students were very aware of their responsibility to environmental problems around them. they felt “very disappointed” about air and river pollutions. this is in line with the research from hassan hassan, noordin, & sulaiman (2010) that the emotional awareness aspect of the student is high. the students understand and know that there must be a balance between environments. the behavioral and attitude awareness aspect in both classes is medium. for behavioral and attitude awareness aspect, students are required to implement actions such as reading environmental issues, appreciating, recycling, and conserving energy and water. the result of behavior and attitude awareness in research from hassan hassan, noordin, & sulaiman (2010) is also medium. it said that the students may not really have high awareness in reading environmental issues and appreciating environmental condition around them. the sustainability practice awareness aspect has the lowest percentage among another aspect. this aspect is very difficult to be achieved. this is in line with the study from hassan, rahman, & abdullah (2011) that students had a good awareness of environmental problems but yet had not changed in practice. the student should have awareness in discussing or delivering information about environmental problems and doing environmental awareness activity. the students might get environmental and awareness lesson from home and school. knowledge is the thing that affects practice. the different percentage of two groups is caused by the model that was implemented in the class. the activity of the first group in all meetings table 7 students’ activity in the second group first meeting the step of inquirybased learning description of activities observed activities identification of the problem the students observe picture and video about climate change. some students are an enthusiast in watching video questioning the students are asked to make question-based on the problem on the video most of the students are keep silent and not give any question. making hypothesis the students are stimulated to make a hypothesis about the cause of climate change. the students write the hypothesis on the worksheet collecting data the students are divided into 5 groups and each group conducting an experiment about the impact of co2 toward temperature students are not conducive. some students do not want to discuss with the group because they feel uncomfortable with each other. analyzing data each group discusses to analyze the data that is obtained from the experiment on the worksheet and deliver the result of the class. only a few students try to analyze and solve the question on the worksheet. some students are just quiet or very noisy. making a conclusion the students are led to make a conclusion some students try to give the conclusion second meeting the step of inquirybased learning description of activities observed activities identification of the problem the students observe picture and video about the greenhouse effect. some students are an enthusiast in watching video questioning the students are asked to make question-based on the problem on the video most of the students are keep silent and not give any question. making hypothesis the students are stimulated to make a hypothesis about the cause of greenhouse effect. the students write the hypothesis on the worksheet collecting data the students are divided into 5 groups and each group conducting an experiment about the greenhouse effect. students are not conducive. some students do not want to discuss with the group because they feel uncomfortable with each other. analyzing data each group discusses to analyze the data that is obtained from the experiment on the worksheet and deliver the result of the class. only a few students try to analyze and solve the question on the worksheet. some students are just quiet or very noisy. making a conclusion the students are led to make a conclusion some students try to give the conclusion journal of science learning article doi: 10.17509/jsl.v2i2.13104 63 j.sci.learn.2019.2(2).58-64 can be described in table 6. the activity of the second group in all meetings can be described in table 7. from the table 6 and 7, it can be seen that the first group that used argument-driven inquiry, the students are asked to have argument about what and why they have to do, such as arrange the procedure of experiment, filling the solution of the problem on the worksheet and argue it in class, while in the second group is different. the inquiry process itself gives more benefit to students in improving knowledge (wardani & winarno, 2017). the inquiry process that is supported by argumentation gives more space to the student to explore the idea. in the first group, there is also a step called double blind-peer review. the group of students in each class is asked to make an article that discusses alternative energy, what activity that can be done to maintain our surrounding at home and school, and what simple activity that can be done routinely to reduce the global warming. then, the article will be reviewed or given any feedback by their classmate without knowing whose article they assessed and who assessed their article, then the article will be given back to the group author to be revised and submitted. while in the second group there is no double-blind peer review. the students are only asked to make the same theme article and then submit it directly to the teacher. by having those arguments in learning global warming, the students can integrate the environmental condition, social, and economic dimensions of sustainability issues. the students can relate their past, present, and future condition so that the students gain a better understanding. at the same time, by arguing allows students to have a different point of views and opinions to be solved. the students can reflect on what is happening in global warming, what is being learned, and critically make the decision to participate in handling the problem of the topic which is learned in the classroom. this is in line with the result from pauw, gericke, olsson, & berglund (2015) that as the teacher integrate the environmental condition on the past, present, and future, the students gain the more knowledge and critically reflect on what is happening and participate in decision making to practice their knowledge. the previous study from labog (2017) found that the learning process on concept and practice at school can help in strengthening the linkage to the environment to achieve sustainability awareness. the more knowledge will increase the students’ practice awareness. this is in line with the statement from sivamoorthy, nalini, & kumar (2013) that knowledge does not influence emotional and behavior, but influence the students’ practice awareness. this level of sustainability practice awareness needs to be built more in education at school. besides theoretical knowledge, the school needs to give more practical experience to trigger the intended lifestyle or behavioral change among students, so that the students will have more sustainability practice awareness (hamid, ijab, sulaiman, md. anwar, & norman, 2017). the teacher should develop the criteria that still lack in students to develop sustainability awareness. however, cottrell & graefe (1997) stated that behavior in terms of environmental practices was a complex task and influenced by many other factors. comparing the activity in both groups, the first group and second group have different activities. based on table 6, in the first group which used argument-driven inquiry, there is argumentation session in the activity, while in the second group there is no argumentation session. in the first group, the students participated more in a group discussion during the argumentation session. the students have more opportunity to explore the idea. each group delivered argument based on global warming problems. among groups perceive, criticize, oppose, and support the statement during this session. then, the teacher clarified the students’ opinion. in argumentation session, students had more opportunity to exchange the ideas in class. in the first group, there was also a double-blind peer review. the students submit the assignment of making an article and the article was reviewed and given feedback by their classmate. these two activities are the most influenced the level of sustainability awareness of students. by increasing more knowledge of students about global warming, the students’ will learn more about how to be aware of the environment. 4. conclusion argumentation in the learning process helps the student to explain how the phenomenon happens. argumentation in the inquiry process also helps the student to gain more reason behind a process of the phenomenon itself. the students in the experimental group have a higher percentage in all aspects. the difference can be seen clearly in the sustainability practice awareness aspect which is the first group that used argument-driven inquiry has the medium percentage, while the second group has a low percentage. it can be concluded that argument-driven inquiry helps the students in gaining the higher percentage on sustainability awareness in all aspects. argument-driven inquiry can be considered as one of the alternative teaching models that can give a better impact on students’ sustainability awareness. acknowledgment the author acknowledged the principal of the school mr. roberto leonardo, s.pd who gave permission to the author in conducting this research. the author also acknowledges the science teachers who also have a role as the authors’ supervisor mrs. wiwin sriwulan, s.pd, and mrs. rika siti jahara, s.pd who helped the author in finishing the research about argument-driven inquiry in several meetings. journal of science learning article doi: 10.17509/jsl.v2i2.13104 64 j.sci.learn.2019.2(2).58-64 references aulia, e. v., poedjiastoeti, s., & agustini, r. (2018). the effectiveness of guided inquiry-based learning material on students’ science literacy skills. journal of physics: conference series, 947(1), 012049. cetin, p. s., & eymur, g. (2017). developing students’ scientific writing and presentation skills through argument-driven inquiry: an exploratory study. journal of chemical education, 94(7), 837–843. cottrell, s. p., & graefe, a. r. (1997). testing a conceptual framework of responsible environmental behavior. the journal of environmental education, 29(1), 17-27. creswell, j. w. (2012). educational research: planning, conducting and evaluating quantitative and qualitative research (fourth edi). lincoln, nebraska: pearson. demircioglu, t., & ucar, s. (2015). investigating the effect of argumentdriven inquiry in laboratory instruction. kuram ve uygulamada egitim bilimleri, 15(1), 267–283. fraenkel, j. r., wallen, n. e., & hyun, h. h. (2012). how to design and evaluate research in education (eight edit). new york: mcgraw-hill. hamid, s., ijab, m. t., sulaiman, h., md. anwar, r., & norman, a. a. (2017). social media for environmental sustainability awareness in higher education. international journal of sustainability in higher education, 18(4), 474–491. hasnunidah, n., susilo, h., irawati, m. h., & sutomo, h. (2015). argument-driven inquiry with scaffolding as the development strategies of argumentation and critical thinking skills of students in lampung, indonesia. american journal of educational research, 3(9), 1185-1192. hassan, a., noordin, t. a., & sulaiman, s. (2010). the status on the level of environmental awareness in the concept of sustainable development amongst secondary school students. procedia social and behavioral sciences, 2(2), 1276–1280. hassan, a., rahman, n. a., & abdullah, s. i. s. s. (2011). the level of environmental knowledge, awareness, attitudes and practices among ukm students. https://doi.org/10.1016/j.foodcont.2013.09.036. hodson, d. (2009) teaching and learning about science: language, theories, methods, history, traditions, and values. canada: sense publisher. labog, r. a. (2017). teachers’ integration of environmental awareness and sustainable development practices. asia pacific journal of multidisciplinary research, 5(3), 102–110. pauw, j. b. d., gericke, n., olsson, d., & berglund, t. (2015). the effectiveness of education for sustainable development. sustainability, 7(11), 15693-15717. sammalisto, k., sundström, a., von haartman, r., holm, t., & yao, z. (2016). learning about sustainability—what influences students’ self-perceived sustainability actions after undergraduate education?. sustainability, 8(6), 510. sivamoorthy, m., nalini, r., & kumar, c. s. (2013). environmental awareness and practices among college students. international journal of humanities and social science invention, 2(8), 11–15. walker, j. p., sampson, v., southerland, s., & enderle, p. j. (2016). using the laboratory to engage all students in science practices. chemistry education research and practice, 17(4), 1098–1113. wardani, t. b., & winarno, n. (2017). using inquiry-based laboratory activities in lights and optics topic to improve students' understanding about nature of science (nos). journal of science learning, 1(1), 28-35. microsoft word layout 11997-29770-1-ce.docx a © 2018 indonesian society for science educator 14 j.sci.learn.2018.2(1).14-20 received: 16 july 2018 revised: 23 september 2018 published: 3 december 2018 using science, technology, society, and environment (stse) approach to improve the scientific literacy of grade 11 students in plant growth and development kultida chanapimuk1*, sureeporn sawangmek1, pranee nangngam2 1faculty of education, naresuan university, tha pho, meueang phitsanulok district, phitsanulok 65000, thailand 2faculty of science, naresuan university, tha pho, meueang phitsanulok district, phitsanulok 65000, thailand *corresponding author. kultidach59@email.nu.ac.th abstract the environmental issues surrounding agrochemical products facing people today include serious health and ecological problems. scientific literacy is necessary for students to understand scientific knowledge and get ready for the future world. therefore, this action research aims to promote scientific literacy, in the area of plant growth by using the science, technology, society and environment (stse) approach that consists of 4 steps: 1) motivation; 2) exploration; 3) brainstorming; and 4) decision making. the participants are 35 special program students in grade 11. the pisa-like test and worksheets were used to collect data. content analysis and triangulation were used to indicate the development of scientific literacy. the findings show that the students have better scientific literacy and higher competencies in explaining phenomena scientifically, evaluating and designing scientific inquiry and interpreting data and evidence scientifically. this study suggests that student collaboration is essential to improve the scientific literacy of students. keywords science, technology, society and environment approach, scientific literacy, plant growth, and development 1. introduction in the twenty-first century, science and technology progressed greatly. while uses of scientific knowledge have been of great benefit to humankind, it also led to negative effects on society and the environment. for the example, harmful raw materials, which used in food and drink production process, have caused many people unhealthy. some of the environmental issues cause of chemical products using. accordingly, many countries prepare their citizens by including scientific knowledge in the school curriculum and enabling people to adapt to the future world (organization for economic co-operation and development; oecd, 2016a). therefore, young people should be able to use science to identify and solve problems in the real world because scientific knowledge is the base of development in the modern nation. scientific literacy is one of the necessary components of education that drives society (ogunkola, 2013). not only teachers teach the student to use scientific knowledge to explain phenomena, but also should guide students to apply related knowledge and draw appropriate conclusions based on scientific evidence. the opportunity to use science in everyday life comes from appreciating scientific processes (klainin, datesri, & pramodnee, 2008). moreover, the environmental issues in thailand include the use of chemicals caused by development technology which requires people to understand technology and its possibility to impact on the environment. this is the reason why students should be prepared to develop their scientific literacy. however, the result from the program for international education systems assessment (pisa) in 2015 indicated that thai students’ scored were 421 points in science literacy which far lower than the average of oecd countries (oecd, 2016b). it shows that thai students lack scientific literacy competencies. such competencies include the explanation of phenomena, the evaluation, and design of scientific inquiry and the interpretation of data. they can use their basic scientific knowledge to give some explanation of familiar situations such as they may encounter in the classroom, but they cannot integrate their knowledge to real-world situations (the institute for the journal of science learning article doi: 10.17509/jsl.v2i1.11997 15 j.sci.learn.2018.2(1).14-20 promotion of teaching science and technology, 2017). this result is similar to the observations of the researcher’s monitoring in a high school biology class, where the students could not draw a diagram of a plant structure after observing it under a microscope. students are unable to draw in the proportion of a diagram from what they observed. these observations show that they have low scientific inquiry skill and rarely able to interpret the information from observation. many students could explain the nature of the plant tissue, but they could not explain with evidence why monocotyledonous plants have differences in structure. this result indicates that students lack the ability to provide evidence to support their explanations. from these observations, it is apparent that the students in this study need to improve their scientific literacy. science, technology, society, and environment (stse) approach focuses on using the result of science and technology that affects society and the environment in the science lesson. it can be used for improving the students’ ability to apply their scientific knowledge in order to comprehend the relationship between what they learn in the classroom and what occurs in their daily life and also make meaningful scientific learning (pedretti et al., 2008; pedretti & nazir, 2011). in addition, students have the opportunity to practice asking scientifically valid questions, designing experiments, exploring, analyzing and interpreting data to find solutions to solve the problem. students also need to recognize the social and physical environment through the socio-scientific context (pedretti & nazir, 2011). in another research, stse process was used to promote students’ scientific literacy in ecosystem topics (gresch, hasselhorn, & bogeholz, 2015). in this study which used the stse approach modified from lau (2013) could improve students’ ability to develop their scientific literacy. this approach consists of 4 steps; 1) motivation, encourage students to be aware of important environmental issues and contexts. in the context of this study, plant growth was examined in the context of chemical pollutants; 2) exploration, students were encouraged to examine this issue and to find possible solutions; 3) brainstorming, brainstorming was used to collect information and ideas, then interpret the information; 4) decision making, students had to decide the best solution of the class from presentation and discussion. figure 1 a research methodology journal of science learning article doi: 10.17509/jsl.v2i1.11997 16 j.sci.learn.2018.2(1).14-20 specifically, the research question in this study was “how the impact of the science, technology, society, and environment approach affect to improve the scientific literacy of grade 11 students in the topic of plant growth and development” 2. method 2.1 methodology this research design is based on the concepts of action research. the developmental model used in this study refers to kemmis & mctaggart (2014). there are four main phases in a cycle of action research including planact-observe-reflect (paor). this study includes three cycles which have different topics that each cycle lasted 4 hours. the research methodology as shown in figure 1. 2.2 science, technology, society and environment approach the science, technology, society, and environment approach was adopted in this study from lau (2013). this process consists of four steps: motivation, exploration, brainstorming, decision making. the detail of stse approach in this study was described as following. first, the step is motivation, the teacher presented an environmental issue and context to the class. then, each group of students was asked to determine what scientific questions they would ask in the same environmental issues which different environmental issues in each learning cycle following by 1) the effect of using chemical coating in corn seed; 2) the effect of using large amounts of insecticide and gmos plants; 3) the effect of chemical plant growth hormones on banana planting. the second step is exploration, the participants work in a collaborative group in order to design a method for exploration, collecting information, searching for evidence and analyzing data to answer the questions that they asked in the previous step. the third step is brainstorming, the members of each group shared possible solutions and choose the best way to solve the problem according to the data collected and interpreted. then, they prepared to present their solution. the presentation could be a poster presentation, diagram, brochures, powerpoint, etc. the final step of the approach is decision making. all the groups present a solution and discuss the pros and cons of the solution. after each group has finished their presentation, the students will decide which is the best solution overall. 2.3 participants the research explored in this study involved grade 11 students of phitsanulok province in northern thailand. the samples comprised 35 students (31 boys and 4 girls) in a special program in the sciences and mathematics. 2.4 data collection the instruments used for data collecting were the pisalike test and student worksheets. the pisa-like test has 12 items consisting of multiple choices, complex multiple choices and open-ended questions in order to assess scientific literacy before and after the learning activities with stse. the quality of the test to its validity by using index of consistency (ioc) from three experts. the experts consisted of a professor from a faculty of agriculture natural resources and environment, a professor from a faculty of science education and a biology specialist teacher from the school where collected the data. the index of consistency of 12 items in the test was between 0.67 1. it demonstrated that the test consistent with content. student worksheets were used to collect information about student scientific literacy during learning activities in each cycle total of three cycles. by dividing students into eight group, each group has to collaborate to complete student worksheet. then, submit to a researcher at the end of each cycle. 2.5 data analysis the data was collected from two instruments: 1) assessing the pisa-like tests by using criteria similar to the pisa 2015 framework (oecd, 2016a); 2) assessing the table 1 scientific competencies description scientific competencies sub-competencies explaining phenomena scientifically (6 points) indicate, use and make some representations for explanation describe scientific knowledge in terms of its potential implications for society evaluating and designing scientific inquiry (15 points) ask scientific questions distinguish questions to be explored offer a way of exploration estimate the way of exploration describe and estimate how scientists ensure the credibility of data interpreting data and evidence scientifically (15 points) convert data to a different representation (e.g. chart, diagram, etc.) analyze, interpret and draw conclusions from the gathered data identify the supposition, evidence, and reasoning based on scientific knowledge distinguish between scientific evidence, justification, and other considerations estimate scientific justification and evidence from different sources note. adapted from “pisa 2015 assessment and analytical framework: science, reading, mathematics, financial literacy, and collaborative problem solving,” by oecd, 2016, p. 24-25. journal of science learning article doi: 10.17509/jsl.v2i1.11997 17 j.sci.learn.2018.2(1).14-20 student worksheets by given mark scientific competencies and sub-competencies that students demonstrated during the cycles. the scientific competencies are component of scientific literacy to indicate that students are able to handle with science-related issues (see table 1). the rubric score comprises as follow; 0exhibit no competencies; 1exhibit low competencies; 2exhibit more good competencies; 3 exhibit good competencies with content analysis. then, the results of both instruments were evaluated with basic statistics include of percentage and mean, then categorized them to scientific literacy levels developed from pisa 2015 framework into 7 levels as following; level 6 (87.5-100%); level 5 (75-87.4%); level 4 (62-74.5%); level 3 (50.5-63%); level 2 (37.5-50%); level 1b (25.5-37%); level 1a (12.5-26%), (oecd, 2016). next, scientific competencies were compared between three phases including before, during and after study with the stse approach. finally, the result’s trustworthiness was established by using method triangulation that compares the results from the pisa-like test and student worksheets. 3. result and discussion the analysis and discussion of the results will be presented as percentages of the scientific competencies and the level of scientific literacy from the pisa-like test and the student worksheets. the results are clarified in table 2. an increasing of students’ competencies before and after learning was shown in table 2 which were measured by the pisa-like test. in the pre-test, students indicated their highest level of development in c3, at 61.22%. after the students studied through stse approach, the percentage of total scores at the post-test was higher than in the pre-test, at 84.19% and 57.90% respectively. the best-developed competency was c1 at 89.29% in post-test. consistent with the result, teaching with stse had been able to improve the competencies of scientific literacy after learning. an improvement of students’ competencies was demonstrated in table 2 which were assessed by student worksheets during the learning activity. from this table, the percentage of student competency scores continually raised from cycle 1 to 3, at 60.78% and 87.65% respectively. especially, c3 was significantly increased to 90% in cycle 3, while c1 slightly increased to 89.58%. it can be seen from table 2 that the students increased their competencies in scientific literacy when they learned with stse for three cycles. from these results evaluating by pisa-like test and student worksheets indicated that student improved their scientific competencies of scientific literacy though stse approach. the results of this competencies as descript below. in an aspect of c1, the students’ competencies were evaluated by students given explanations in class. in cycle 1, the students explained the chemical using in corn seed’s solution. their findings can be presented with a simple solution for explaining how a new chemical that they use was useful. as group 1 commented in their report: “using polyethylene glycol 600 is good for the environment because it is low toxic, can protect the seed and low absorb into a human.” (group 1, cycle 1). this comment has shown that student created a poor explanation without scientific knowledge. in cycle 2, students’ explaining competency significantly improves from cycle 1 as shown in table 2. the students were able to draw a representation from the presented problem and explaining to the others in class after they had the additional time for brainstorming with members of their group. their illustrations and findings can be presented with possible benefits of their solution to society as a whole, which their evaluation of the advantages of the gmos plant issue according to the potential solutions. group 1 commented in their report: “the benefits of using physostigmine instead of glyphosate products are that they are watersoluble chemicals that are safe for the environment and plants. the weakness of this solution is that it can lead to acidic soil. this problem can be solved by adding lime (cao) to the soil.” (group 1, cycle 2). in cycle 3, the students had a good progression of explaining competency. as group 7 reported in a worksheet: “using effective microorganisms makes soil degradation and soil has more oxygens. it activates microbial resistant for soil and plant will be growth by nitrogen fixation. providing the chances for students to present their finding and brainstorming with their member group though stse approach would promote the ability to explain the phenomena. according to ladachart & yuenyoung (2016) table 2 the percentage of scores in each competency and scientific literacy level of students percentage of scores in each competency level of scientific literacy explaining phenomena scientifically (c1) evaluating and designing a scientific inquiry (c2) interpreting data and evidence scientifically (c3) total scores score % score % score % score % pre-test 3.34 55.71 8.63 57.50 9.18 61.22 21.15 57.90 3 cycle 1 3.75 62.50 8.75 58.33 9.75 65.00 22.25 60.78 3 cycle 2 5.25 87.50 11.75 78.33 11.50 76.67 28.50 80.06 5 cycle 3 5.38 89.58 13.25 88.33 13.50 90.00 32.13 87.65 6 post-test 5.36 89.29 12.05 80.36 12.18 81.22 29.59 84.19 5 journal of science learning article doi: 10.17509/jsl.v2i1.11997 18 j.sci.learn.2018.2(1).14-20 which stated that building the opportunity for students to discuss their explanations was developed the ability to explain phenomena competently. furthermore, hidayanti, pochintaniawati, & agustin (2018) also proved that brainstorming can attribute the student to give an accurate explanation and estimation possible way to solve the problem. in an aspect of c2, it can be seen in the students’ work that they could improve their competency during the exploratory process in which they began to organize and evaluate possible ways to solve the presented problem. they would need to consider the problem and appropriate questions that would need to be answered to solve the given problem. from there, they can decide what exploratory steps to take. in cycle 1, most of the students had poorly designed their own literature review before finding the solution. the knowledge that they want to know was slightly comprehensive with the topic. as group 6 commented in a worksheet: “we start to review about biochemical that safe for environment and human. then, we will find what chemical is used in the seed coat? and what is the effect when we use in the plant?” as the comment in the worksheet, it can be seen that the students’ competency is developed. consequently, teacher motivated students by using environment issue about gmos plant and encourage them to give questions about the issue. in cycle 2, teacher guided by questions for specific designing led to the improvement of students’ competency from cycle 1. the following quotation reveals the students designed in their worksheet: “we think the knowledge that should review to answer our question is what chemical that non-toxic. organophosphate is safe, so we should search what does it work? and how it uses in the plant?” in cycle 3, the students had good development of evaluating and designing competency. they could search and review appropriate literature on the subject. for example, they could collect data by reviewing the literature about plant growth and some experiments from existing research papers. in the comment written of group 8’ s worksheet: “the knowledge that is necessary to exploration included 1) ethylene was planted hormone that associated with fruit senescence, 2) fruit ripening was the process of changing methionine to ethylene that could lead to a sweet flavor in ripe fruit, 3) acetaldehyde could inhibit the synthesis of ethylene by bonding with the protein function group that could delay the fruit ripening...” (group 8, cycle 3). training the students through the exploration step leads the students encouraging scientific literacy. in line with ladachart & yuenyoung (2016) which stated that students who are trained in the scientific procedure (i.e. identify question, scientific inquiry, appraise ways of investigation) tended to draw on their own understanding and exploratory methods. moreover, eliyawati, sunarya, & mudzakir (2017) said that the issues or situation in society that occur around the students can give a response them, attempt to find solutions and pay attention to explanation carefully. in an aspect of c3, the students had the opportunity to transform large amounts of complicated information into a form that is easier to understand and present it in class. student’s report in cycle 1 (group 8) student’s report in cycle 3 (group 8) figure 2 an example of scoring for competencies in scientific literacy of students in cycle 1 and 3 journal of science learning article doi: 10.17509/jsl.v2i1.11997 19 j.sci.learn.2018.2(1).14-20 they presented their findings based on the scientific evidence, discussed other findings and considered choosing the best alternative solution of the class. students’ work is presented in figure 2. practicing the students to transform and discuss based on the scientific evidence leads the students developing the competency of interpreting data and evidence scientifically. according to ladachart & yuenyoung (2016) who state that: the teacher should train students to modify the data by turning it into a graph or other form of representation. moreover, gresch, hasselhorn, & bogeholz (2015) indicated that the decision-making process could be enhanced or justified because students had the chance to identify the scientific evidence and use it to distinguish the claims of others in the class and consider choosing the best solution through the process of argumentation. figure 2 shows cycle 1 and 3 from group 8’s worksheets. the initial and final representations of group 8 included information on the explored problem. the students examine the positive and negative implications in the situation for the environment and society. additionally, students could associate the following solutions to solve the problem and how this solution may help the environment and society. moreover, they can create a simple model to explain their solution based on group findings through the process of stse. in cycle 1, students created a simple representation by using investigated documents to be a short word. they did not represent the data as a diagram, etc. rather, the competency of the interpretation of data and evidence significantly improved from cycle 1. for the evidence as shown in figure 2, the students generated an appropriate representation that was shown by their own understanding and linked scientific knowledge to explain how the solution worked as diagram about changing of substances in banana when it ripens. for overall results, students’ scientific literacy improved during the learning cycle because they had opportunities to draw their scientific exploration, analyze investigated data and construct their scientific explanation through three cycles of stse approach. the researcher would conclude that learning though stse could promote scientific literacy of students, according to the increase of scientific literacy during stse activity in three cycles and the enhancing after learning score in post-test. similar to the level of scientific literacy developing in each cycle, the level of scientific literacy increased progressively from level 3 in cycle 1 to level 6 in cycle 3. in the same way, the scientific literacy level of students in post-test (level 5) was higher than in pre-test (level 3). however, the percentage of total scores in cycle 3 was higher than post-test, at 87.65% and 84.19% respectively. this is because the students collaborated with others in their group through the process of exploration and brainstorming. thus, the process of group work included chances to assist students in exchanging complicated information from different sources, which promoted group work where friends verified the information. the researcher suggests that group collaboration will promote the students’ scientific literacy. it is in line with a previous study of rosario (2009) which stated that: “…focus-group discussion will lead students to prefer the freedom given to them to choose their activities rather than simply accept prepared activities provided by the teacher…”. based on a result of scientific literacy developing during the study from students’ worksheets and a result of the pretest and post-test from the pisa-like test, it shows that the scientific literacy level of both was increased. this indicates that student scientific literacy progressed because of the process of stse though environmental topics related to society for relevant science learning. it is in line with yoruk (2010) who stated that: “the stse teaching approach promotes students in recognizing their own skill and enables them to learn more meaningfully than by traditional teaching”. 4. conclusion based on the results and discussion the stse approach is supportive of the conclusion that stse is able to improve the scientific literacy of students which applied their knowledge in the topic of plant growth and development. they understand the effect of the issues and why they might solve these issues and find solutions through the process of stse. most of the students could design an experiment and find a solution to environmental issues. they are also able to consider the effect of the solution on the environment and society as a whole. these results also suggest that student collaboration is essential to fully develop student scientific literacy. acknowledgment the researcher is grateful for the support from the institute for the promotion of teaching science and technology that supported the scholarship and the faculty of education, naresuan university, thailand. the researcher appreciates the interest and participation of my advisor, the students and my boyfriend who made this work possible. references eliyawati, sunarya, y., & mudzakir, a. (2017). solar cell as learning multimedia to improve students’ scientific literacy on science and nanotechnology. journal of science learning, 1(1),36-43. gresch, h., hasselhorn, m., and bögeholz, s. (2015). enhancing decision-making in stse education by inducing reflection and self-regulated learning. science education, 47(1), 95-118. hidayanti, w., i., rochintaniawati, d., & agustin, r. r. (2018). the effect of brainstorming on students’ creative thinking skill in learning nutrition. journal of science learning, 1(2),44-48. kemmis, s., mctaggart, r., & nixon, r. (2014). the action research planner: doing critical participatory action research. singapore. journal of science learning article doi: 10.17509/jsl.v2i1.11997 20 j.sci.learn.2018.2(1).14-20 klainin, s., datesri, p., and pramodnee, a. (2008). knowledge and scientific competency for the future world. thailand: the institute for the promotion of teaching science and technology) ladachart, l. and yuenyoung, c. (2016). what thai science teachers sould learn from the programme for international student assessment. parichart journal, thailand: thanksin university, 28(2) 108-37 lau, k. (2013). impacts of a stse high school biology course on the scientific literacy of hong kong students. asia-pacific forum on science learning and teaching, 14(1), 1-25. ogunkola, b. (2013) scientific literacy: conceptual overview, importance and strategies for improvement. journal of educational and social research, 3(1), 265-274. organisation for economic co-operation and development (oecd). (2016a). pisa 2015 assessment and analytical framework: science, reading, mathematic, financial literacy and collaborative problem solving, revised edition. paris: oecd publishing. organisation for economic co-operation and development (oecd). (2016b). pisa 2015 pisa results in focus. retrieved march 5, 2017, from https://www.oecd.org/pisa/pisa-2015-results-in-focus.pdf. pedretti, e. and nazir, j. (2011). currents in stse education: mapping a complex field, 40 years on. science education, 95(601), 601-626. pedretti, e. g., bencze, l., and hewitt, j. (2008). promoting issuesbased stse perspectives in science teacher education: problems of identity and ideology. science education, 17(1):941-960. rosario, b. i. d. (2009). science, technology, society and environment (stse) approach in environmental science for conscience students in a local culture. ched accredited research journal, 6(1), 269-283. the institute for the promotion of teaching science and technology. (2017). the basic information summary of pisa 2015. thailand: the institute for the promotion of teaching science and technology yoruk, n., morgil, i., secken, n. (2009). the effects of science, technology, society and environment (stse) education on students’ career planning. us-china education review, 6(8), 68-74. microsoft word layout 12878-30282-1-ed.docx a © 2018 indonesian society for science educator 26 j.sci.learn.2018.2(1).26-32 received: 6 september 2018 revised: 16 december 2018 published: 18 december 2018 the effect of steam-based learning on students’ concept mastery and creativity in learning light and optics gita ayu wandari1, agus fany chandra wijaya2, rika rafikah agustin1* 1international program on science education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia 2departement of physics education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia *corresponding author. rikarafikah@upi.edu abstract the integrated knowledge should be implemented to face the 21st-century era. beside the integrated knowledge, mastery the concept and creativity also must be involved in order to enhance the quality of education. thus, this research was aimed to investigate the effect of steam-based learning on students’ concept mastery and creativity in learning light and optics. the method that used was a mixed method with convergent parallel design. the population in this research was 8th-grade students in private junior high school in west bandung and the sample was one class of 8th grade. the school implemented indonesian curriculum 2013 in the teaching-learning process. the sampling techniques were convenience sampling. the number of participants in this research was 27 students. the quantitative data in this research was obtained through an objective test. the objective test was made based on bloom’s taxonomy revision by anderson. the qualitative data was obtained through the creativity rubric adopted from creative product semantic scale (cpss) developed by o’quinn and bessemer. the dimension that was in creativity is novelty, resolution, and elaboration and synthesis. according to the research, students’ concept mastery improved as much as 0.78 with category high improvement after the implementation of steam-based learning. for students’ creativity achievement, in every dimension gained different result: 1) novelty is categorized into good with 75.6%, 2) resolution is categorized into good with 77.8%, and 3) elaboration and synthesis are categorized into enough with 65.3 %. overall, students’ concept mastery and creativity in the implementation of steam-based learning in learning light and optic is categorized as good. keywords steam-based learning, students’ concept mastery, students’ creativity, light and optics 1. introduction the new paradigm of the 21st century science education explores a wide range of possibilities that can foster students’ interest in science and creative convergent thinking. in indonesia, education is supported by the national curriculum of 2013. the purpose of using national curriculum 2013 is to prepare indonesian to be a creative, productive, innovative, affective, and also give a contribution to the environment, country, and the world. steam (science, technology, engineering, art, and mathematics) education has been implemented to enhance scientific literacy to use the integrated knowledge in the newly revised korean science education curriculum (kong & huo, 2014). recently steam education has emerged to develop human resource with creativity in mind and see and understand human society in the future. steam education is defined as education in which the students’ understanding and interest in related subjects such as science, technology, engineering, etc., foster of conversions of thinking and problem solving based on science and technology (baek & yoon, 2016). hence, the purpose of national curriculum 2103 of indonesia and steam has the similarity which is to make the student be creative and implement the knowledge in daily life. learning activities cannot be separated from the mastery concept. ability in mastering the material can be seen from the mastery concept. mastery concept is the students’ ability to understand the meaning of learning and apply it in their daily life (shidiq, rochintaniawati, & sanjaya, 2017). mastery concept is very important, anderson and krathwohl (2001) state that mastery concept can improve their intellectual skills and help them solve the problem they face and lead them to meaningful learning. journal of science learning article doi: 10.17509/jsl.v2i1.12878 27 j.sci.learn.2018.2(1).26-32 concept mastery should be completed by creativity and other skills to enhance the quality of students. deep understanding of a concept is highly needed to maximize the students’ creativity. creativity has long been recognized as a powerful force in shaping human society progress and knowledge (henriksen, mishra, & fisser, 2016). creativity and innovation concern to the process of creating and applying new knowledge (gurteen, 1998). concept mastery not only should be completed with the creativity but also should be related or implemented in daily life. light and optics is something that used in daily life. it is an essential concept that student is difficult to understand. light and optics concept is a complex area for students and if it is not connected even implemented to daily life the student may not grasp the concept easily. the concept of light and optics is a relay on daily life such as camera, microscope, telescope, etc. the concept of the optic can be developed to make a valuable product in the future. this concept consists of science and mathematical explanation. besides, it also could be integrated with technology, engineering, and art to produce a better product. light and optics topic also in line with steam and national curriculum 2013 of indonesia because the topic makes the student be a creative one to make something useful in daily life. thus, the present study proposed the research to see the effect of steam-based learning on students’ concept mastery and students’ creativity in learning light and optics. this research will analyze two variables they are students’ concept mastery and students’ creativity towards steam-based learning. 2. method the research method used in this research is the mixed method. according to creswell (2012) stated that mixed method is a type of research design which collects, analyzes, and mixes both quantitative and qualitative methods in a single study or series of studies to understand research problem. the quantitative data focuses on students’ concept mastery that is measured by an objective test in form multiple choices test while the qualitative analysis focuses on students’ creativity that is measured by the rubric of cpss developed by o’quin and bessemer (2006). there are several sub aspects in cpss rubric. after the students finish their project, the researcher gives a score to the project done by the student and find the average of the score. all data are analyzed separately based on the indicator and compared to obtain better understanding and interpretation regarding the effect of steam-based learning on students’ concept mastery and creativity in learning light and optics. then, the researcher finds the gain to see the improvement of a student using steam based learning. besides, the researcher relates the result of the concept mastery and creativity and find the relation of both variables. the location of this research was in private junior high school in bandung. the school was using indonesian curriculum 2013 in the teaching-learning process. bahasa indonesia was the daily language used. the population in this research are 8th students in private junior high school. the researcher took one class in eight grades as the sample consist of 27 students. the sampling technique was convenience sampling technique. fraenkel, wallen, and hyun (2013) stated that convenience technique is used because there is a group of individuals who (conveniently) are available for study (fraenkel, wallen, & hyun, 2013). the research problem and question rather than only a method (creswell, 2012). based on the research method which has been adjusted to the research objective, therefore the research design which was implemented in this research is the convergent parallel design. the rationale for this design is to complete the understanding of research problem result from collecting both quantitative and qualitative data at the same time (creswell, 2012). besides researcher chose one class to make the researcher easier to conduct the research. the school has 3 classes of 8 grade. the researcher selected one class of 27 students because compare to the other classes, it was the available class and the amount of the student is not too much. in this research, the concept of light and optics is limited by core competence no. 3 and basic competence no 3.12 and 4.12 that are attached in kurikulum 2013. the focus of subtopics that will be investigated by students such as (1) light properties, (2) the image formation of lenses, and (3) telescope. there are two types of instruments used in this research. there are objective test and rubric. first, an objective test based on bloom’s revised edition was used to measure students’ concept mastery before and after implementing steam-based learning. it consists of pretest and post-test. a pre-test is conducted to find the prior knowledge, while the post-test is conducted to identify whether the cognitive mastery is increasing or not. the cognitive level that will be tested in this objective test is c4 (analyzing), c5 (evaluating), and c6 (creating). concept mastery test consists of forty-one questions before judgment by experts. it is used to measure students’ concept mastery. after judging by the expert the objective is only twenty questions as a representative for each learning indicators. then, the test was distributed to students in grade 8 as a test. the next step after conducting the test, the result is analyzed the objective test using anates software to measure the validity, reliability, difficulty level, discriminating power and distractor. second, creative product semantic scale (cpss) developed by besemer and treffinger (1981) was adapted to analyze students’ creativity. this analysis was used to assess students’ product at the end of the class. the journal of science learning article doi: 10.17509/jsl.v2i1.12878 28 j.sci.learn.2018.2(1).26-32 adapted rubric of cpss constructed by the author has been judged by two experts. 3. result and discussion the results show quantitative and qualitative data. the pre-test and the post-test are conducted to determine the students’ concept mastery before and after treatments. qualitative analysis will describe the students’ product of the project at the end of the class. 3.1 students’ concept mastery the profile of students’ concept mastery was obtained from objective test consisted of 20 multiple choice question that has been given as pre-test and post-test while implementing steam-based learning in chapter light and optics. the test item has been tested in term of validity, reliability, discriminating power, and difficulty level. besides, it also has been judged by some experts and revised so it is appropriate to be used as a research instrument to obtain the data of students’ concept mastery. the improvement of students’ concept mastery is determined by the calculation of the normalized gain <g>. normalized gain is calculation processed through data of pre-test and a post-test score of students. the result is presented in table 1. the average of the pre-test conducted before the implementation of steam-based learning is 43.14 and it is improving into 87.40 in the post-test. for the score of the students, the lower score of the student in pre-test is 15 and the highest is 70. merely, the lower score of post-tests is 70 and the highest is 100. to get the improvement score, a normalized gain <g> was calculated. based on the result, the students’ concept mastery is improving with the normalized score 0.78 that assume as high improvement according to hake (1998). in order to analyze the profile of students’ concept mastery, the improvement of each test item in every cognitive domain should be processed. the test item used in this research was developed based on the bloom’ taxonomy revision by anderson and krathwohl (2001). there are three levels considered to be used in this research based on the basic competence that is used. those are analyzing (c4), evaluation (c5), and creating (c6). the result is presented in table 2. in order to analyze the improvement of concept mastery in each group, the test item was analyzed based on table 1 the result of students’ concept mastery no component pretest posttest g <g> category 1 participant 27 27 44.26 0.78 high 2 average 43 87 3 lowest score 15 70 4 highest score 70 100 figure 1 concept mastery’s improvement of each group table 2 the summary of students’ concept mastery based on bloom’ taxonomy revised edition cognitive domain amount of question number of students average score g <g> category pre-test post-test c4 8 27 52.8 88.4 35.6 0.75 high c5 7 27 36.5 89.4 52.9 0.83 high c6 5 27 33.3 82.2 48.9 0.73 high c4 c5 c6 c4 c5 c6 c4 c5 c6 c4 c5 c6 c4 c5 c6 1 2 3 4 5 pre (%) 45.8 40.5 23.3 56.3 31 26.7 40 31.4 48 60 37.1 32 62.5 42.9 40 post % 97.9 92.9 83.3 89.6 83.3 80 85 85.7 76 82.5 97.1 84 85 88.6 88 <g> 100 90 80 80 70 70 80 80 50 60 100 80 60 80 80 0 20 40 60 80 100 120 s co re cognitive level of group journal of science learning article doi: 10.17509/jsl.v2i1.12878 29 j.sci.learn.2018.2(1).26-32 the students’ group. the result of each cognitive domain in each group is presented in figure 1. the result of steam-based learning implementation shows the average of pre-test is 43.35, meanwhile the average of post-test is 87.42. it is showing the improvement of concept mastery by processed the pre-test and post-test results the gain as much as 44.26. in order to categorize the students’ concept mastery improvement, the normalized gain is used. the result of the normalized gain is 0.78 which is categorized as a high improvement. therefore, the hypothesis is accepted that steam-based learning improves students’ concept mastery. this result is supported by previous findings by kim, ko, han, and hong (2014) that is steam education influenced the improvement of academic achievement, creative problemsolving abilities, and scientific attitude. the research compared the control and experimental group in post-test and analyze it quantitatively and found out there is differences in the results. another research by kim and park (2012) resulted that the steam teaching model is stimulated the students understanding related the activities in steam itself. henriksen (2014) also stated in their research that steam education significantly influenced the improvement of academic achievement, basic scientific process skills, and affective domain. this result showed the improvement of students’ assessment in content knowledge from 30% to 40%. the instrument was arrange based on bloom’s taxonomy revised edition by anderson and krathwohl (2001). there is three cognitive levels used in this research which is analyzing (c4), evaluating (c5), and creating (c6). the result shows that n-gain for c4 level is 0.78 which categorized as high, c5 level is 0.83 which categorized as high, and c6 level is 0.73 which categorized as high. the higher value of n-gain is evaluating (c5). the result shows the students are easier to checking, critiquing, testing something, or making a judgment. light and optics are though in indonesian curriculum 2013 and it delivered when implementation of steambased learning. in the process of the implementation, the students find a problem in learning light and optics. they said the topic is too abstract and many concepts deliver in this chapter. the strategy of the teacher to make the student understand the topic is to divide the class into groups and give them a problem related to the concept. the students are pro-active. they always ask the question outside the class if they do not understand the concept. the students have high curiosity makes the students give more attention while the implementation. there are steps of steam-based learning implemented in this research. there are 3 steps, those are 1) presentation of the situation; 2) creative design; 3) emotional experience. the first steps implemented at the beginning of the meeting until all the concept has been delivered to make the student understand the relationship between the concept and real life (baek & yoon, 2016). compare to the other method, in pratiwi, rochintaniawati, & agustin (2018) stated that the improvement of students’ concept mastery after implemented the multiple intelligent-based learning resulted in medium improvement as much as 0.61. hence, every method will have a different result. 3.2 students’ creativity in this research, students’ creativity is measured by telescope they made as a final project. students’ creativity is assessed by using cpss rubric adapted from besemer and treffinger (1981). besemer and treffinger (1981) suggested that product creativity is grouped into three creative dimensions which are novelty, resolution, and elaboration and synthesis. each aspect has several criteria and also sub-criteria to make easier to assess the product of creativity. the criteria of novelty used are original, germinal, and surprising; for the resolution are valuable, logic, and useful; and the last, resolution and synthesis are organic, wellcrafted, elegant, understandable, and complex. the result of creativity is stated in figure 2. figure 2 presents the result of every creativity dimension. for novelty dimension is 75.6%, resolution dimension is 77.8 %, and elaboration is 65.3 %. in order to see the whole profile of creativity in each group, the creativity in each group is necessary to be processed. the category of all dimensions is good. the students were more attractive and excited when they make the project. every group has a good collaboration because every group consists of the students who have the same characteristic, thus they enjoyed when making the project. the result of the creativity in each group is presented in figure 3. based on figure 3, every dimension in each group has a different result. group 1 got 89% for novelty, 56% for resolution, and 73% for elaboration in average 73%. group 2 got 100% for novelty, 89% for resolution, and 93% for figure 2 result of creativity dimension 76% 78% 65% 60% 65% 70% 75% 80% 85% novelty resolution elaboration p er ce n ta g e creativity dimension journal of science learning article doi: 10.17509/jsl.v2i1.12878 30 j.sci.learn.2018.2(1).26-32 elaboration in average 94%. group 3 got 67% for novelty, 78% for resolution, and 100% for elaboration in average 82%. group 4 got 89% for novelty, 56% for resolution, and 93% for elaboration in average 79%. the last, group 5 got 78% for novelty, 67% for resolution, and 73% for elaboration in average 73%. creativity dimensions of a product consist of three aspects which are novelty, resolution, and elaboration and synthesis (besemer, 2000). each aspect has some criteria. the criteria of novelty are original, surprising, and germinal; for resolution are valuable, logic, and useful; for elaboration and synthesis are elegant, well-crafted, organic, complex, and understandable. each criterion of creativity is used to assess students’ creative product as the final result of steam-based learning implementation. overall, the students’ creative product is categorized as good (henriksen, 2014). this result is supported by research conducted by kim, ko, han, and hong (2014) that significant improvement was found in students’ creativity by using steam-based learning compared to the control group. it happens because in steam-based learning consist of the creative process in every step of the implementation. in line with the research of kim, chung, woo, & lee (2012) that steam leads to processes that result in creativity, innovation, and continued growth and exploration of the world. another research supported this result conducted by kim, ko, han, and hong (2014) that there is significant improvement found in the creativity and scientific interest in elementary students. 3.3 the relation between student’s concept mastery and creativity in order to compare and relate the result of students’ concept mastery and creativity, the charts of the comparison between student’s concept mastery and creativity is made. the chart is shown in figure 4. the relation between students’ concept mastery and creativity could be seen through the cognitive level of concept mastery and dimension of creativity. here, the level of cognitive used was analyzing, evaluating, and creating. the dimension of creativity used here was novelty, resolution, and elaboration and synthesis. anderson and krathwohl (2001) stated that analyzing is finding, integrating, something, and organizing something or breaking material into its constituent parts and detecting how the parts relate to one another and to an overall structure or purpose. evaluating is checking, critiquing, of testing something or making judgments based on criteria and standards. creating is designing, constructing, planning, or making something or putting elements together to form a novel, coherent whole or make an original product. the dimension of creativity used here novelty, resolution, and elaboration and synthesis. besemer and treffinger (1981) stated that novelty is the newness of the product, new techniques, new process, and another element of newness. resolution is how well the product does what it is supposed to do. elaboration and synthesis consider product’s presentation style. there is a similarity between the cognitive level and the dimension of creativity. if we relate the level of cognitive and the dimensions of creativity, we can see that analyzing is closest to resolution. both analyzing and resolution are to make something in order, relate to one another, and how something does what it is supposed to do. evaluating is closest to elaboration and synthesis which is testing or judging something or product and how to describe it. creating is closest to novelty which is constructing or making something new. figure 3 result of creativity in each group 1 2 3 4 5 novelty 89% 100% 67% 89% 78% resolution 56% 89% 78% 56% 67% elaboration 73% 93% 100% 93% 73% average 73% 94% 82% 79% 73% 0% 20% 40% 60% 80% 100% 120% p er ce n ta g e dimension in each group journal of science learning article doi: 10.17509/jsl.v2i1.12878 31 j.sci.learn.2018.2(1).26-32 based on the explanation, the students’ concept mastery and creativity relate each other. it is proven with the pattern of the relationship between students’ concept mastery and creativity result. the data of the students’ concept mastery and creativity is shown in figures 4. from the data, it is clear that every cognitive level paired with a dimension of creativity will be the same. the higher the value of c4, the novelty value will be high. as well as elaboration and c5 also resolution and c4. it also can be related also with the implementation of steam-based learning where the students more excited to prove something whether it is right or wrong and in the presentation session, they were so excited to explain the product. it is proven by kim and park (2012) stated that the steam improves the student understanding and interest. this condition is proven with the value both of evaluating and elaboration and synthesis are higher than other cognitive level and dimension of creativity. different from the result of analyzing and novelty which gains the lowest value. the students are hard to analyze something and making the product different with the example given. 4. conclusion based on the result of steam-based learning implementation that has been conducted, steam-based learning effect significantly to students’ concept mastery and creativity in learning light and optics. there are some other conclusions gained: implementation of steam-based learning on light and optics concept improves students’ concept mastery. it can be noticed by the gain of pre-test and post-test score that is 0.78 which included as high improvement category. implementation of steam-based learning can be used to profile students’ creativity through the project. students’ creativity is assessed based on cpss rubric focuses on tree dimension which is novelty, resolution, and elaboration and synthesis. students’ creativity on novelty gain 76%, on the resolution is 78%, while on the elaboration and synthesis is 69%. all creativity is categorized as good. acknowledgment the authors acknowledge mrs. rika rafikah for stimulating discussion about educational assessment. references anderson, l. w., & krathwohl, d. r. (2001). a taxonomy for learning, teaching and assessing (a revision of bloom’s taxonomy of educational objectives). united states: addison wesley longan, inc. baek, j.-e., & yoon, m.-b. (2016). development and application of steam education program based on robots: through a themebased robot soccer. international journal of computer science and information technology for education, 1(1), 27–34. besemer, s. p. (2000). creative product analysis to foster innovation. design management journal (former series), 11(4), 59-64. besemer, s. p., & treffinger, d. j. (1981). analysis of creative products: review and synthesis. the journal of creative behavior, 15(3), 158-178. bloom, b. s. (1984). the 2 sigma problem: the search for methods of group instruction as effective as one-to-one tutoring. educational researcher, 13(6), 4-16. creswell, j. w. (2012). educational research: planning, conducting, and evaluating quantitative and qualitative research. educational research (vol. 4). united states: pearson. https://doi.org/10.1017/cbo9781107415324.004 fraenkel, j. r., wallen, n. e., & hyun, h. h. (2011). how to design and evaluate research in education. new york: mcgraw-hill humanities/social sciences/languages. gurteen, d. (1998). knowledge, creativity and innovation. journal of knowledge management, 2(1), 5-13. figure 4 comparison between students’ concept mastery and creativity 1 2 3 4 5 c4 52.1 33.3 45 22.5 22.5 c5 52.4 52.3 54.3 60 46.7 c6 60 53.3 28 52 48 resolution 56 89 78 56 67 elaboration 73 93 100 93 73 novelty 89 100 67 89 78 0 20 40 60 80 100 p er ce n ta g e group journal of science learning article doi: 10.17509/jsl.v2i1.12878 32 j.sci.learn.2018.2(1).26-32 hake, r. r. (1998). interactive-engagement versus traditional methods: a six-thousand-student survey of mechanics test data for introductory physics courses. american journal of physics, 66(1), 6474. henriksen, d. (2014). full steam ahead: creativity in excellent stem teaching practices. the steam journal, 1(2), 15. henriksen, d., mishra, p., & fisser, p. (2016). infusing creativity and technology in 21st century education: a systemic view for change. journal of educational technology & society, 19(3). kim, d. h., ko, d. g., han, m. j., & hong, s. h. (2014). the effects of science lessons applying steam education program on the creativity and interest levels of elementary students. journal of the korean association for science education, 34(1), 43-54. kim, s. w., chung, y. l., woo, a. j., & lee, h. j. (2012). development of a theoretical model for steam education. journal of the korean association for science education, 32(2), 388-401. kim, y., & park, n. (2012). development and application of steam teaching model based on the rube goldberg’s invention. in computer science and its applications (pp. 693-698). springer, dordrecht. kong, y. t., & huo, s. (2014). an effect of steam activity programs on science learning interest. advanced science and technology letters, 59, 41–45. o'quin, k., & besemer, s. p. (2006). using the creative product semantic scale as a metric for results‐oriented business. creativity and innovation management, 15(1), 34-44. pratiwi, w. n. w., rochintaniawati, d., & agustin, r. r. (2018). the effect of multiple intelligence-based learning towards students' concept mastery and interest in matter. journal of science learning, 1(2), 49-52. shidiq, a., rochintaniawati, d., & sanjaya, y. (2017). the use of self construction animation learning software to improve the students concept mastery on structure and functions of plants. pancaran pendidikan, 6(3). microsoft word 53-59_the profile of students’ science process skill in learning human muscle tissue experiment at secondary s a © 2018 indonesian society for science educator 53 j.sci.learn.2018.1(2).53-59 received: 20 february 2018 revised: 26 march 2018 published: 31 march 2018 the profile of students’ science process skill in learning human muscle tissue experiment at secondary school shofwa widdina1*, diana rochintaniawati1, lilit rusyati1 1international program on science education, faculty of mathematics and science education, universitas pendidikan indonesia *corresponding author. nahrulshofwa@gmail.com abstract it is important for people absorbs and understand the information of how world their live in works, as human basic form of learning process, science process skills serve as the tools it is not used only by the scientist in the process of their discovery, but also by the people as tools to understand information about the world. the aim of this study was to profile students’ basic science process skills, to profile students’ integrated science process skills, and the profile of students’ science process based on gender. in gaining the data this study uses observation method in students’ laboratory activity. the results of the study indicated that students’ basic science process skills used in the study categorized in a high category with index value above 60%, and integrated science process skills indicators science process used in this study categorized into two categories with; low (23%) and very high (90.7%). there are no differences more than 24% in the profile students science process skills based on gender other than male students did outperform female students. keywords science process skill, gender, human muscle tissue 1. introduction nowadays, a variety of technologies are more advances to assist or aid humans for them easier in doing their works. not only with advanced technology in other fields, technology in the educational field also experience into better and will still developing to assist educator or teacher in delivering the subject to the students, so the students understand what teachers taught, either in teaching methods or the instrument teacher used. particularly in science, science educations experienced great change. in an era where everything is progressing, cause the emergence of high competition between countries especially in education to improve the quality of its citizen. indonesia is one of the countries which should be able to take part in the competition (jirana and damayanti, 2016). improving the quality of the citizens is an important goal of all countries, as improving the quality of citizen becomes an absolute necessity of country and to realize this goal education serves as a weapon (mulyasa, 2006). however, in realizing every goal is not easy as jirana and damayanti (2016) stated that in realizing the goal of good education as a weapon it is facing a difficulty which is citizens’ lack of skills related to education is lack of quality of skills which is one of them is science process skills. as the quality of skill is really important for every country especially in education, particularly in science an international study center institution located in america conducts a regular international comparative assessment of student achievement in mathematics and science called as timss. through the institution, the participate countries able to make the decision for improving their own educational policy such as measuring the effectiveness of their educational system, pinpointing any areas of weakness in their system (timss, 2015). timss (trends in international mathematics and science study) is a study center that aims to support and promote the use of data by researchers, analysts, and others interested in improving education (timss, 2015). timss has monitored student achievement in mathematics and science at fourth and eighth grades every four years since 1995. it is well positions to provide an overview of countries’ performance in mathematics and science and how that performance has evolved (mullis, martin, and loveless, 2016). the latest result of timss 2015 the top performing countries in the science of the eighth grader are singapore, japan, chinese taipei, republic of korea, and journal of science learning article 54 j.sci.learn.2018.1(2).53-59 slovenia. comparing the data, indonesia has to occupy the low ranking in place 45 (rahmawati, 2016). through the data that was obtained researcher take interest in figuring out students’ science level. based on american heritage dictionary, the profile defines as a biographical account presenting the subject’s most noteworthy characteristics and achievements. it is important to do profiling so the teacher or educator get to know the students. through profiling, teacher gets to know how far students ability and achievement, interest, weaknesses, and strength in learning, then teachers able to help students to be a success in their academic. science process skills are thinking skills that scientist uses to construct knowledge in order to solve problems and formulate results (ozgelen, 2012). in the same way, nwosu and okeke (1995) also stated science process skills have been described as mental and physical abilities and competencies which serve as tools needed for the effective study science and technology as well as problem-solving, individual and societal development. just as akinbobola and afolabi (2010) stated science process skills as cognitive and psychomotor skills employed in problem-solving, problem identification, data gathering, transformation, interpretation, and communication. aka, guven, and aydogdu (2010) stated science process skills consist of basic science process skills and integrated science process skills. basic science process skills which include: observing, asking questions, classifying, measuring, and predicting. the second group was integrated science process skills which include; namely identifying and defining variables, collect and transform data, create data tables and graphs, describing the relationship between variables, interpret the data, manipulating materials, recording the data, formulating hypotheses, designing investigations, make inferences and generalization (karamustafaoğlu, 2011). the process skill assessment one of an effective method for testing student achievement, and provide the teacher with feedback on student performance. the assessment designs to be flexible enough or easy to be administrated individually, in a small group, or with the entire class, more importantly, it meets the needs of both teachers and students (oslund, 1992). refers to peraturan menteri pendidikan dan kebudayaan republik indonesia nomor 22 tahun 2016 about elementary and secondary education standard process in making lesson plan teacher should consider about the difference of every students based on aptitude, level of intelligence, talents, learning styles, gender, ability level, ethics, motivation and interest of students. due to the statement, it is important in understanding the diversity of student to improve the quality of learning especially for based gender differences. 2. method the location of the research is a private junior high school in tangerang of the academic year 2016/2017. the school uses bilingual languages english and bahasa indonesia in teaching learning activity and applies national curriculum 2013. the subject of the research was 8th-grade students. the students who participated in this research were 78 students. the students consist of 39 male students and 39 female students. in this research, the concept of muscle based on indonesian curricula 2013 by core competence number three understanding and applying knowledge (factual, conceptual and procedural) based on student inquiry about science, technology, art, and culture that is related with natural phenomena in daily life, basic competence no 3.1, as attached in kementerian pendidikan dan kebudayaan (2013) understanding the movement of living things, human movement, and the effort in maintaining health of movement. the analysis of curriculum about core competence and basic competence indicates the subtopics that had been investigated by students such as (1) structure and function of muscles, (2) experiment structure of human muscle tissue activity, (3) muscle tissues shape, (4) the location of each human muscle types. there were several materials that will be mentioned in this research that is structure and function in animal tissue and four types of tissues in the animal body. material that will be discussed in this research will be focused on muscle tissue in the human body. there are three types of instrument used in this research. there are observation sheet, performance assessment, and rubric. first, observation sheet used to measure each student through the laboratory experiment where the process skills will be observed by the observer. observation of the students starts from the beginning of teaching-learning activity to experiment activity. in this research, the researcher used observation sheet in the form of the rating scale. rating gives a numerical value to some kind of judgment (arikunto, 2003). the observation format that is used is in five categories 0, 1, 2, 3, and 4. the scale as follow; scale 4 with the statement of very high, scale 3 with the statement of high, scale 2 with the statement of sufficient, scale 1 with the statement of low, and scale 0 with the statement of very low. the observers give the scale based on the given rubric. second, performance assessment used to assess how the students in handling and using a microscope, is it poor, moderate, or good. third, rubric as the guidance for the observer to give a desirable score of students’ process skills in observation sheet through experiment activity. the rubric that used in the research customized with students’ worksheet. journal of science learning article 55 j.sci.learn.2018.1(2).53-59 3. result and discussion the result of profiling students’ science process skills was obtained through conducting observation; an observation that was conducted in private school is analyzing science process skills that appear in conducting laboratory activity in biology experiment of muscle tissue. the observation was conducted in a group. where the group being formed in teaching-learning laboratory activity. the data was obtained by three observers when teaching learning takes place. before conducting observation, the observers were given the guideline and instructions how to fill the observation sheets. the observers stand behind in each group of students and observing their activity of human muscle, in order to give the suitable score that shown in the activity observers gave a sign () with the indicators that show from the student. the observation process was made in order to not disturb the activity. 3.1 the profile of students’ basic science process skills process skills’ measuring has an aspect in magnifying the image of slides with different magnification that suitable specific dimension. the table shows most of the students’ science process skills in measuring categorized sufficient with the result 55%, follow with the high category with 32%. process skills classifying has two aspects which first, taking notes in comparison to skeletal muscle, cardiac muscle, and smooth muscle, and second, classifying the slide of muscle based on a characteristic of muscle. table 1 shows most of the students categorized in the very high category in two aspects with 72% and 45% respectively. the resulting study shows most students’ basic science process in observing skill is sufficient. observation skill is one of important basic skill. however, even it is insufficient category it is still below expectation. there are a lot of factors that can affect the results such as learning instrument, classroom environment and characteristics of students. learning instruments such as learning media; worksheet and teacher’s teaching method. the ability of students in formulating testable questions, recording accurate and relevant data in an inquiry, and making inferences based on the outcomes of experiment affecting the process scientific observation (kelly, 2013). while doing observation in laboratory activity students tend to rush in the using laboratory equipment which is the microscope to get the answer quickly. this pace is contrary to the required pace of observation; careful and precise observation (campbell, 2010). communicating skills of the students in three aspects show insufficient very high and high category respectively. in the first aspect, many students make the description of all three muscle tissues in their worksheet by drawing the tissues that were shown under the microscope lens. rezba, sparague, and fiel (2002) stated that drawing is one of many communicating tools that can be used to share what we know. far before alphabet and numbers were develop drawing used to record the data. katz (2017) stated that science requires the recording of data to seek insights and patterns’, drawing is an act of recording data. the second aspect of communicating skills shows very high category. after doing an observation of specimen using microscope they sit in their own group to discuss the data they obtain through a microscope. by doing the discussion students share what they know by oral description and documented it in their worksheet. the oral or verbal description is one of the communication tools to communicate their observations and ideas (rezba, sparague and fiel, 2002). the third aspect of communicating skill was put most of the students in the high category. in making the conclusion of the experiment, students doing a discussion verbally table 1 category of students’ basic science process skills no indicator category percentage 1 observing a. observing the specimen by using a microscope very high 14 % high 40 % sufficient 41 % low 5 % very low 0 % 2 communicating a. make the description of the skeletal, smooth, and cardiac muscle tissue as it is shown under a microscope very high 6 % high 12 % sufficient 67 % low 15 % very low 0 % b. discussing the results of observation within a group very high 58 % high 31 % sufficient 6 % low 1 % very low 4 % c. make a conclusion at the end of the experiment very high 28 % high 35 % sufficient 14 % low 4 % very low 19 % 3 measuring a. magnify the image of slides with different magnifications from low to high magnifications that suitable specific dimension very high 10 % high 32 % sufficient 55 % low 3 % very low 0 % 4 classifying a. taking notes about the comparison the skeletal, smooth, and cardiac muscle tissue very high 72 % high 14 % sufficient 9 % low 5 % very low 0 % b. classifying the slide of the muscle based on a characteristic of muscle that shows under the microscope very high 45 % high 5 % sufficient 8 % low 3 % very low 40 % journal of science learning article 56 j.sci.learn.2018.1(2).53-59 before put it into written language in exchanging information about the topic. through the finding, students’ measuring skill resulted in category sufficient. during laboratory activity, students were tried out to find the object focus of the specimen by using a microscope. the microscope has three different objectives lenses resulted in different magnifications. through the finding, students were asked to find the focus from the lowest magnifications after they found the focus with the objectives they calculate the lens with the magnification. classifying skill has two aspects, during the observation using a microscope and when students making notes of the comparison the three muscles. during the laboratory activity teacher prepared several prepared specimens of muscle tissues for students to observe, while doing the observation using a microscope with several prepared specimen students classified the prepared specimen into a category that has similar characteristics. as stated by teo (2003) classifying allows pupils to organize their observations and make sense out of them based on recognizable patterns, and observed similarities and differences. there, students taking note the characteristics of the specimen after they observed and classified them into certain similarities and differences to record the data after their finding. 3.2 the profile of students’ integrated science process skills in the process of obtaining data, students were asked whether they already understand the instruction in making design investigation on their own. some of the students were understood the instructions but not used to make the design on their own because usually it is already available, prepared by the teacher. they just have to implement the designated investigation or experiment. the process skill designing investigation resulted in the students in category very low because students are not accustomed to designing their own investigations or experiments. designing or planning and carrying selfdesigned investigation is important. it helps students learn how to engage in science practices and help them understand and identify science as a way of improving the world (wingert and bell, 2015). the experimenting skill of the students from table 2 showed that the majority of students’ experiment skill in the very high category. students understand what they do inside the laboratory, the following step by step of the instructions. students accustomed to doing laboratory activity following the instructions of the prepared procedure on the worksheet. they prepared the tools and materials needed for the laboratory activity and handling the tools and materials with care. through the finding, even the students accustomed with the procedures of laboratory activity, researcher found that after the experiment not many students following the procedures steps by steps, some of them not handling the set of tools and materials with care, this happened with students put the tools and materials carelessly on the edge of the table, before they warned by the teacher. the low score of students’ science process skills naturally will have an impact on students’ ability to perform various activities such as experiments, laboratory activity, especially in inquiry and discovery. though the experiment is an essence of science itself that science as a process (sukarno, permanasari, and hamidah, 2013). however, science will not run in the right way if the subjects do not have decent science skills. therefore, it can be presumed there are lacking or not optimal activity-based learning during the process of acquiring data due to the lack of students’ science process skills. 3.3 the profile of students’ science process skills based on gender integrated science process skills, designing investigation female outperformed male students with 38% in category sufficient, while male students acquire 59% with category very low. experimenting skill male and female students table 2 category of students’ integrated science process skills no indicator category percentage 1 designing investigation a. designing the type of muscle that will be investigated in “muscle investigation” very high 5 % high 15 % sufficient 5 % low 15 % very low 59 % 2 experimenting a. experiment by following the instruction given very high 63 % high 37 % sufficient 0 % low 0 % very low 0 % figure 1 comparison of students’ basic science process based on gender journal of science learning article 57 j.sci.learn.2018.1(2).53-59 acquire very high category with percentage 74% and 51%, male students outperformed female. the comparison results of science process skill based on gender presented in figure 1 and figure 2. the results of the study show that there are no big differences. there are many factors that affect the results. in the intellectual area, generally male and female generally same there is no difference because they learn in the same room at the same time unlike in the past where male students get to go to school and learn. but in observing skill, the male did outperform females, this might happen because of some factors such as visual-spatial abilities. visual-spatial abilities are the ability that enables people to aware of and locate the objects and their relationship. many old types of research resulted in favoring male students better in their visual-spatial abilities. but, the data showed just slight difference result between male and female, it can be said as lips (2008) stated that there are no reliable gender differences in spatial-visualization. in communicating skill three aspects have same results between male students and female students. even they have same results but female students’ results higher than male students. it could be that females have better developed communicating skill. human brain organizations, the cerebral cortex divided into two; left hemisphere and right hemisphere. speech, writing, language, and calculation located in the left hemisphere, while spatial construction and nonverbal ideation located in the right hemisphere. the explanation seems simple, but refer to the result of a study that has no big difference results, it can be explained that brain organization not really affect the results, because in the left hemisphere calculation located there (lips, 2008). it can be concluding that there are no differences in communicating or verbal skill even though female students did slightly better. in-depth analysis is also carried out by overlooked with the score of chapter test of students in learning science. there are no significant differences between male students and female students. in the score of chapter test female students did outperform male students, but in the research finding male students outperformed female students. 3.4 the profile of science process skills after each indicator of students’ science process skills assorted, then the indicators for each science process skills categorized as the index result of indicators presented in table 3. table 3 shows the results of students’ science process skills at the time of research observation when doing an experiment of muscle tissues. from table 3, the percentage students’ science process skills for each indicator are; observing 65.7% (high), communicating 67. 9% (high), classifying 70.6% (high), measuring 62.5% (high), designing investigation 23% (low), and experiment 90.7% (very high). table 3 shows four indicators of students’ basic process skills categorized as high. as stated by rauf et al. (2013) basic science process skills are the intellectual foundation in scientific inquiry. two indicators of students’ integrated science process skills have two different categories, the category of designing investigation is low it might be affected by some factors, such as students not understanding the instruction given and they not accustomed to making or designing their investigation and it was revealed that students did not accustom in designing their own investigations, while experiment indicator is categorized in very high category. students accustom doing experiment or laboratory activity with procedure investigations provided by the teachers. the results of students’ science process skills in junior school quite satisfying, but it also showed that the ability of students’ science process relatively still low. this happened due to various factors found through the finding. first, the lack of human resources of science teachers in teaching students science process skills. second, there is lack of science materials and tools in supporting teacher in teaching the students to improve and develop process skills. third, lack of guidance for teachers how to do the assessment and the development of teaching in science process skills to the students, because in some opportunity, teaching and learning activity the teacher still use traditional way. table 3 result of students science process skill no indicator index category 1 observing 65.7% high 2 communicating 67.9% high 3 classifying 70.6% high 4 measuring 62.5% high 5 designing investigation 23% low 6 experiment 90.7% very high figure 2 comparison of students integrated science process based on gender journal of science learning article 58 j.sci.learn.2018.1(2).53-59 therefore, to improve students’ science process skills, the teachers who one of the main resources of information and knowledge of students need to be trained and have a more deep understanding of science process skill. according to sukarno, permanasari and hamidah (2013) by the training of science teacher is expected to have thorough understanding and knowledge in terms of types of science process skills, science process skill development methods, and science process skill assessment. with the deep understanding and knowledge of teachers about science process skills, it is expected that teacher will able to apply the knowledge and understanding science process skills in their teaching-learning activity to the students. the teachers can explore the science process skills in their teaching materials, learning models, and exploration activities. 4. c onclusion referring to research finding in the profile of students’ science process skills in learning muscle tissue experiment at secondary school, it can be concluded that based on the research finding data, students’ basic science process skills can be concluded in category level of sufficient. students’ integrated science process skills result concluded in category level of high. the results of students’ basic science process skills based on gender shows that male students did outperform female students in some indicators and categories but it is not guaranteed that male is better than females because in learning the concept they learn it in the same class at the same time with the same teacher. there are no differences in learning to state that which one is better than the others. references abungu, h. e., okere, m. i. o., and wachanga, s. w. (2014). the effect of science process skills teaching approach on secondary school students’ achievement in chemistry in nyando district, kenya. journal of educational and social research mcser publishing, rome-italy. 4 (6), 359-372. aka, e. i., guven, e., and aydogdu, m. (2010). effect of problem solving method on science process skills and academic achievement. journal of turkish science education. 7(4), 13-25. akinbobola, a. o., and afolabi, f. (2010). analysis of science process skills in west african senior secondary school certificate physics practical examinations in nigeria. american-eurasian journal of scientific research. 5 (4), 234-240. aktamis, h., and ergin, o. (2008). the effect of scientific process skills education on students’ scientific creativity, science attitudes, and academic achievements. asia-pacific forum on science learning and teaching. 9 (-), 1-21. amelink, c. (2009). literature review: gender differences in science achievement. swe-awe casee overview. american heritage dictionary of the english language, fifth edition, (2011). retrieved march 23, 2018, from http://www.thefreedictionary.com/profile. arikunto, s. (2003). dasar – dasar evaluasi pendidikan. bandung: bumi aksara. arikunto, s. (2007). manajemen penelitian. jakarta: rinerka cipta. ary, d., jacobs, l. c., irvine, c. k. s., and walker, d. a. (2010). introduction research education. ninth edition. usa: wadsworth. atikah. (2014). gender differences and role of visuo-spatial representation to improve student’s conceptual mastery in learning human urinary system. (thesis). retrieved from library of universitas pendidikan indonesia. bandung: universitas pendidikan indonesia. barman, c. (1992). science, children, & learning. school of education. indianapolis. [online]. retrieved from http://castle.eiu.edu/~scienced/3290/science/process/crb.html. [accessed on september 14, 2017]. betts, j. g., young, k. a., kruse, d. h., desaix, p., poe, b., korol, o., wise, j. a., johnson, e., johnson, j. e., and womble, m. (2016). anatomy and physiology: openstax. texas: houston. [online]. retrieved from http://solr.bccampus.ca:8001/bcc/file/f4873e49e09c-469e-9ee8-9f14ca5a4e00/1/anatomyphysiologyopenstax.pdf blakstad, o. (2008). experimental research. [online]. retrieved from https://explorable.com/experimental-research. [accessed on september 11, 2017]. campbell, m. (2010). teaching observation skill with a science journal. retrieved from: http://blog.teachersource.com. [accessed on november 2017] creswell, j. w. (2012). educational research. boston: pearson education. cohen, l., mamion, l., and morrison, k. (2007). research method in education. new york: routledge taylor and francis group. dehart, b. g., sraufe, l. a., and cooper, r. g. (2004). child development its nature and course fifth edition. new york: mcgraw-hill. dewi, s. (2008). keterampilan proses sain. bandung: tinta emas. feyzioglu, b. (2009). an investigation of the relationship between science process skills with efficient in laboratory use and science achievement in chemistry education. journal of turkish science education. 6 (3), 114-132. fraenkel, j. r., and wallen, n. e. (2007). how to design and evaluate research in education, sixth edition. new york: mc-graw hill. gurian, m., henley, p., and trueman, t. (2001). boys and girls learn differently! a guide for teachers and parents. san fransisco, ca: jossey-bass. harlen, w. (1999). purposes and procedures for assessing science process skills. proquest education journals. assessment in education. 6 (1), 129. [online]. retrieved from: https://sunnyscience.wikispaces.com/file/view/purposes+and+ procedures+for+assessing+science+process+skills+wynne+h arlen.pdf. [accessed on august 20, 2017]. houser, r. (2009). counseling and educational research. evaluation and application. second edition. the united states of america. california: sage publications, inc. jirana, j., and damayanti, m. (2016). an analysis of science process skills of pre service biology teachers in solving plants physiology problems. international conference on education. 477-480. karamustafaoglu, s. (2011). improving the science process skills ability of science students teachers using i diagrams. eurasian journal of physics and chemistry education. 3, 26-38. katz, p. (2017). introduction: drawing and science are inseparable. drawing for science education. 1-8. sense publisher. kelly, k. m. (2013). science journals in the garden: developing the skill of observation in elementary age students. dissertation and theses. portland state university. kementrian pendidikan dan kebudayaan. (2013). kurikulum 2013: kompetensi dasar sekolah menengah pertama (smp)/ madrasah tsanawiyah (mts). jakarta: kemdikbud. khan, n. a., jan, s., and amin, i. (2014). research trends in select science faculties of the university of jammu. library philosophy and practice (e-journal). university of nebraska – lincoln. [online]. retrieved from http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=2644 &context=libphilprac. [accessed on august 11, 2017]. koscik, t., o’leary, d., moser, d. j., andreasen, n. c., and nopolous, p. (2009). sex differences in partial lobe morphology: journal of science learning article 59 j.sci.learn.2018.1(2).53-59 relationship to mental rotation performance. national institute of health. brain and cognition. 69 (3). 451-459 lancour, k. l. (2012). process skills for life science training guide. science olympiad. [online]. retrieved from https://scioly.org/wiki/images/d/d6/pslsl_training_hammond0 4.pdf. [accessed on september 11, 2016]. lin, h. h. (2015). gender differences in science performance. macrothink institute. journal of studies in education. 5 (4), 181-190. http://dx.doi.org/10.5296/jse.v5i4.8526. lips, h. m. 2008. sex and gender: an introduction. six edition. new york: mcgraw-hill. mulyasa, e. (2006). menjadi guru profesional: menciptakan pembelajaran kreatif dan menyenangkan. bandung: remaja rosdakarya. nwosu, a.a., and okeke, e, a.c. (1995). the effects of teachers’ sensitization of students’ acquisition of science process skills. journal of the science teachers’ association of nigeria. 30 (1&2), 39-45. ongowo, r. o., and indoshi, f. c. (2013). science process skills in the kenya certificate of secondary education biology practical examinations. scientific research. 4 (11), 713-717. oslund, k. l. (1992). science process skills assessing hands-on student performance. united states of america: addison-weasley innovative division. ozgelen, s. (2012). scientist' science process skills within a cognitive domain framework. eurasia journal of mathematics, science & technology education. 8, 283-292. permendikbud. (2016). peraturan menteri pendidikan dan kebudayaan nomor 22 tahun 2016 tentang standar prosess pendidikan dasar dan menengah. purwanto, n. (2010). prinsip – prinsip dan teknik evaluasi pengajaran. bandung: pt. remaja rosdakarya. rahmawati. (2016). hasil timss 2015 trends in international mathematics and science study. suspend. kemdikbud. rezba, r. l., sparague, c., and fiel, r. l. (2002). learning and assessing science process skills. iowa: kendall/hunt publishing company. rambuda, a. m., and fraser, w. j. (2004). perceptions of teachers of the applications of science process skills in the teaching of geography in secondary schools in the free state province. south african journal of education. 24 (1), 10-17. rauf, r. a. a., rasul, m. s., mansor, a. n., othman, z., and lyndon, n. (2013). inculcation of science process skills in a science classroom. asian social science. 9 (8), 47-57. reece, j. b., taylor, m. r., simon, e. j., and dickey, j. l. (2012). campbell biology. san fransisco: pearson benjamin cummings. santrock, j. w. (2012). child development thirteenth edition. new york: mcgraw hill. sukarno., permanasari, a., and hamidah, i. (2013). the profile of science process skill (sps) student at secondary high school (case study in jambi). international journal of scientific and research (ijser). 1(1), 79-83. teo, n. (2003). science teaching in primary school. singapore: c.o.s. printers pte ltd. timss. (2015). timss & prils. [online]. retrieved from timss & prils international study center: http://timssandpirls.bc.edu/timss2015/international-database. [accessed on october 22, 2015]. trefil, j., and hazen, r. m. (2013). sciences. seventh edition. united states: john wiley & sons. wingert, k., and bell, p. (2015). why should students learn to plan and carry out investigations in science and engineering?. stem teaching tool 19 planning and conducting. zeidan, a. h., and jayosi, m. r. (2015). science process skills and attitudes toward science among palestinian secondary school students. world journal of education. vol 5 (1). 13-24. zubaidah, s., mahanal, s., yuliati, l., and sigit, d. (2014). ilmu pengetahuan alam smp/mts. jakarta: pusat kurikulum dan perbukuan, balitbang, kementrian pendidikan dan kebudayaan. a © 2020 indonesian society for science educator 196 j.sci.learn.2020.3(3).196-204 received: 14 february 2020 revised: 01 july 2020 published: 31 july 2020 smartchem: an android application for learning multiple representations of acid-base chemistry eliyawati1*, rika rafikah agustin1, yustika sya’bandari2, rossy andini herindra putri1 1international program on science education (ipse) study program, faculty of mathematics and science education, universitas pendidikan indonesia, bandung, indonesia 2department of science education, kangwon national university, chuncheon-si, republic of korea *corresponding author eliyawati@upi.edu abstract an android application named smartchem was developed to explain multiple representations of acid-base chemistry. this paper is a description of an android-based media (smartchem) through the stages of design, development, validation and revision, and finally, limited trials with pre-service science teachers. smartchem is intended to aid students in understanding through media explanations of symbolic, macroscopic, and submicroscopic representations of acid-base material. the validity of the tool was assessed through the ratings of a panel of five expert judges using aiken’s validity index (aiken’s v). the results show that some parts of the smartchem media needed to be revised, especially in linking submicroscopic level content with symbolism. from the trials of this application with trainee teachers, help in understanding multiple chemical representations of acids and bases was demonstrated; however, the low achievement group was more concerned with technical features, while the higher achieving group appreciated the content and learning experience. keywords smartchem, android application, chemical multiple representation 1. introduction human life these days is inseparable from the development of knowledge, information, and communication technology. the rapid growth of information and communication technology in the era of the industrial revolution 4.0 is inevitable and is impactful on all sectors of life (umachandran, jurcic, & ferdinandjames, 2018). industrial revolution 4.0 supports the development of information and communication technology that provides the opportunity to adopt and apply the latest information and communication technology in all sectors of life, including education (liao, loures, venâncio, brezinski, & deschamps, 2018). global demands encourage the education sector to continually adjust the technological developments for the efforts in improving the quality of education, mainly in adopting the use of technology for the education sector, especially in the learning process. utilization of technology can be used as a supporting tool in understanding certain subjects, one of which is chemistry (helsy & andriyani, 2017). chemistry is a natural science branch that studies natural phenomena, especially those related to the structure, composition, properties, and changes in matter (istijabatun, 2008). in addition to studying changes in matter, chemistry also explores the energy that accompanies these changes. in other words, chemistry is a science full of concepts, ranging from simple to more complex concepts, and from concrete to abstract concept. the understanding in chemistry is tied directly with understanding the representation of macroscopic, submicroscopic, and symbolic levels, and the appropriate connections these three levels of representation have (gkitzia, salta, & tzougraki, 2011). the macroscopic level is a chemical phenomenon that can truly be observed by the unaided eye, including students' experience every day (treagust, chittleborough, & mamiala, 2003). the microscopic level is a chemical phenomenon that cannot be observed directly, such as electrons, molecules, and atoms. the symbolic level is a representation of chemical phenomena using a variety of media, including models, images, algebra, and mailto:eliyawati@upi.edu journal of science learning article doi: 10.17509/jsl.v3i3.23280 197 j.sci.learn.2020.3(3).196-204 computational forms. the inability to represent one level of representation can hinder the ability to solve chemical problems for other levels of representation (kamkhou & yuenyoung, 2019). most students find it difficult to master the concepts of chemistry. this is reinforced by research data that shows the difficulty of students understanding the concept of chemistry, mainly at the microscopic, and symbolic levels (roche allred & bretz, 2019; wu, krajcik, & soloway, 2001). this difficulty is caused by the characteristics of these levels that are invisible and abstract. abstract chemical concepts will become a problem when these concepts are key concepts used to understand a certain occurring phenomenon. analogy and the use of appropriate models are needed to understand the chemical concepts (helsy & andriyani, 2017; sunyono & meristin, 2018). one effort that can be made to analogize an abstract concept to be more concrete is to use teaching media. teaching media can improve the understanding of the three levels of chemical representation, especially at the symbolic and submicroscopic levels (eliyawati, rohman, & kadarohman, 2018). one of the learning media that is in line with the development of the industrial revolution 4.0 era is the use of teaching media based on information and communication technology in the form of an android application (andriani, 2016). teaching media in the form of an android application can help students understand the learning materials and help teachers and parents present the learning materials (mewengkang, eunike, liando, ngodu, & stefanus, 2019). android applicationbased teaching media that can be used by students to understand the concepts of chemistry is smartchem. this media is a multimedia device that can display text, images, animations, videos, and audio simultaneously in the form of an android application. smartchem, an android application-based media, is expected to help students visualize chemical processes. multimedia can help students visualize concepts at the molecular level and stimulate memories of facts, concepts, or principles. this microscopic level of representation is also inseparable from the other two levels of understanding, namely the macroscopic level and the symbolic level (ardac & akaygun, 2004). chemical understanding can be achieved by increasing the ability to explain and describe macroscopic, microscopic, and symbolic levels and the ability to connect between the three appropriately (raviolo, 2001). one topic of chemistry that must be understood through all three levels of representation is acid-base. there is a lot of research on teaching media that addresses the topic of acid-base, such as virtual laboratories (ibrahim, handoyo, & ikhsan, 2017); macromedia flashes (ibrahim et al., 2017); worksheet development (cahyani & nasrudin, 2019); and android-based games (kumar, 2013). however, no one has developed an android-based learning media that discusses the three levels of chemical representation on the topic of acid-base. therefore, it is necessary to create an android-based media, smartchem, to explain the multiple representations of chemical understanding on the topic of acid-base. this smartchem media was created to help pre-service science teachers understand multiple chemical representations at the macroscopic, submicroscopic, and symbolic levels on the topic of acid-base, mainly addressing the problem of understanding chemistry, especially for abstract concepts, and abstract concepts with concrete examples. this research will discuss the stages of developing android application-based smartchem media, characteristics of android application-based smartchem media, and the validation and testing of android application-based smartchem media. 2. method this research is a descriptive research of the development of android-based smartchem media to explain multiple chemical representations on the topic of acid-base. descriptive research describes a situation, problem, phenomenon, service, and program systematically, and provide information about the living conditions of a community, or describe the attitudes towards a particular problem (kumar, 2011). with this smartchem media, it is expected that pre-service science teachers can understand multiple chemical representations on the topic of acid-base. some stages in this research included (1) designing android app-based smartchem media; (2) developing android app-based smartchem media; (3) validating and revising android app-based smartchem media; and (4) limited trials to determine preservice science teachers' responses towards the android app-based smartchem media. the design of the android-based smartchem media begins with concept analysis and making the storyboard. at the development stage of smartchem media based on android applications, researchers made android applications using dart programming language and supported by the flutter framework using the microsoft visual studio code editor. five experts then validated smartchem that was developed. these five experts have the same background in science with different specific expertise, which are material content experts, it experts, and language experts. the validity was assessed by obtaining ratings by using aiken’s validity index (aiken’s v). validated aspects of the smartchem include display aesthetics, program analysis, learning applications, and smartchem contents. after being validated, smartchem media is revised and limited testing. there were 45 preservice science teachers involved in the smartchem media trial. the trial participants were pre-service science teachers in one science education majors in indonesia. journal of science learning article doi: 10.17509/jsl.v3i3.23280 198 j.sci.learn.2020.3(3).196-204 the pre-service science teachers are final year students who have studied the content of the material. this trial was conducted to find out their response to the smartchem media that was developed. several questions were asked, especially regarding the content clarity of the multiple chemical representations of acid-base topics presented and the ease of use of this android-based media application. 3. result and discussion 3.1 designing android app-based smartchem media the designing stage of the android-based smartchem media is done by analyzing acid-base concepts and creating acid-base storyboards. the first step is the analysis of the acid-base concept. the concept described must be understood and explained using words that are not necessarily similar to the description contained in the textbook. analysis of the concepts that have been carried out always emphasizes macroscopic, submicroscopic, and symbolic representation levels on the acid-base topic. the results of the concept analysis are one concept included in the concept of apperception, and six concepts that are essential in acid-base material, including acid-base theory (arrhenius, brønsted-lowry, and lewis), acid nomenclature, basic nomenclature, the strength of acidbase, acid, and base. each concept was analyzed of macroscopic, submicroscopic, and symbolic levels, and the connections of the three levels. the concepts examined can be used to make the media scripts in the form of storyboards. storyboards that have been made are then developed into an android application-based media. the figure 1 a-d are the example of a storyboard. 3.2 developing android app-based smartchem media the storyboard that has been made used as a basis for making animated videos. the prototype of this smartchem application is a compilation of video animation that was inserted into the application. these compiled videos can be accessed through the androidbased use. therefore, the central aspect of developing this application is making animated videos. the animated video created explains the understanding of multiple chemical representations consisting of macroscopic, submicroscopic, and symbolic understanding of the acidbase topic. the following are the steps for making the animated video for smartchem. the first step in making animated videos is to create assets or objects that will be used in making animation, such as animated characters, molecular forms, electrical circuit forms, and other purposes. asset creation is done using coreldraw software. coreldraw is a software for pc used to create vector shaped objects. then, the next step is making the animation itself. macromedia flash software is used to create the animation. after all, assets were created, the assets are figure 1a storyboard of what is a base? figure 1b storyboard of acid and base nomenclature figure 1c storyboard of acid electrical conductivity figures 1 storyboard figure 1d story board video journal of science learning article doi: 10.17509/jsl.v3i3.23280 199 j.sci.learn.2020.3(3).196-204 entered into flash to be designed and moved as desired. this animation movement used motion tween. each scene is made into an animation and converted from the animation into a video format. this step takes quite a lot of time, because of the rendering process from animation to the video format. because the rendering process is quite a demanding for the computer, the resulting file size is quite large. therefore, the videos were compressed using a video editor called wondershare filmora. in this step, video compressing and text additions, such as titles and captions in the video were made. afterward, the animated video is ready to be inserted into the application. the application is created using the dart programming language and is supported by the flutter framework. the use of flutter is different from making android native. the resulting application can only be run on the android platform if java/kotlin language, which is used as the basis for making android, was used. but if you use flutter, applications can be rendered into multiple platforms. the development of the application was done using the microsoft visual studio code editor. it starts with the development of the basis/prototype of the application, followed by the page and video controller system, until it ends with the display's finalization. after the application has reached the final stage, the application is compiled, and the build process is carried out. the process resulted in an installation file in the form of an "apk" (android application extension) that can be used to install applications. 3.3 validating and revising android app-based smartchem media the android app-based smartchem media need to be accomplished with the validation process. this stage is a primary concern that measures the extent to which the content of smartchem matches three aspects of a qualified digital teaching media. five validators gave their expert judgment on three aspects of the application: 1) audio/visual aesthetic display, 2) program features, and 3) pedagogical features. the validity was assessed by obtaining ratings from a panel of five expert judges using aiken's validity index (aiken's v) (1985). the procedure for determining aiken's v index starts either with the rating of a single smartchem aspect by n judges or the ratings of m aspects by a single judge. the validators were also asked to give recommendations and suggestions on parts that need to be improved and revised. the aiken's v index is computed using the formula v = ∑s [n(c − l)] s stands for the difference between the judge's validity ratings of the item (r) and the lowest validity category (l). this rule provides s = r – l, and c states the highest table 1 validation score of smartchem aspect judges’ ratings aiken’s v a b c d e audio/visual aesthetic display type and size of font 5 4 5 5 5 0.95 valid color composition 5 4 4 2 4 0.7 invalid graphical clarity 5 4 4 5 4 0.85 valid animation 5 4 4 5 4 0.85 valid sounds 5 2 5 3 3 0.65 invalid screen display 4 2 4 5 4 0.7 invalid clarity of terms 5 5 5 5 4 0.95 valid language usage 5 3 4 5 5 0.85 valid characters placement 5 5 4 5 4 0.95 valid interface flow 4 1 4 5 4 0.7 invalid program features complexity 5 5 4 4 2 0.75 invalid algorithm design 5 3 5 5 4 0.85 valid program interactivity 4 3 3 5 2 0.6 invalid response to input 5 2 3 5 4 0.7 invalid space efficiency 4 3 4 5 1 0.6 invalid the ease of program 4 3 5 5 5 0.85 valid pedagogical features learning material 5 1 5 4 4 0.7 invalid the appropriateness of content within animation 5 3 5 2 3 0.65 invalid the clarity of learning material 5 4 5 4 4 0.85 valid the clarity of examples 5 3 5 4 4 0.8 valid the clarity of exercise 4 2 4 3 3 0.55 invalid journal of science learning article doi: 10.17509/jsl.v3i3.23280 200 j.sci.learn.2020.3(3).196-204 validity category. this study has magnitudes extending from 1 to 5 so that l = 1 and c = 4. the aiken's v provides an index of raters affirmation that ranges from 0 to 1. the accepted value of aiken's v index is authorized on the exact binomial test represented by a table containing v's critical values by the number of judges and the number of ratings (aiken, 1985; dunn, 1999). the number of judges and ratings in this study designated 0.8 critical value. aiken's v index of smartchem is displayed in table 1. table 1 shows that overall, the lowest aiken's v index is 0.55 obtained on the clarity of exercise, whereas the highest index is 0.95 achieved on the type and size of the font as well as clarity of terms. the table also shows that the smartchem is valid in ten aspects, invalid at 11 aspects. aiken's v index of each aspect is shown in figure 2, figure 3, and figure 4. based on figure 2, the lowest score v is obtained for sounds and interface flow in the aesthetic display aspect. the index of these two aspects is 0.65. moreover, the validators recommend that several revisions need to be made. the revision included: 1) the quiz menu needs to be improved because it cannot be appropriately executed 2) the interface flow can be improved by adding guidelines for using apps in the beginning 3) some negligible sound effects need not be minimized 4) exploration of interface flow needs to be clarified 5) the speed of sounds should be uniform. figure 3 shows the average score for program features. based on the figure, it can be seen that the v values range from 0.6 to 0.85. the highest score on this aspect is lower than the highest score on the aesthetic display. the highest score v index is achieved for algorithm design and the ease of the program. the lowest v is acquired for the aspect of program interactivity and space efficiency, which is 0.6. several recommendations were generated to improve the application performance: 1) some questions need to be added at the end of each material accomplished with feedback 2) program interaction with users need to be more dynamic. based on figure 4, it is known that the smartchem aiken's v index on pedagogical features ranges from 3.2 to 4.4. the lowest v is obtained on the clarity of exercise, while the highest v is achieved on the learning material. suggestions from the validators related to these two aspects include 1) wider space is important to improve exercise performance 2) some questions measuring higher-order thinking skills (hots) need to be added into the exercise 3) questions on the exercise should be adjusted with the content within smartchem. the interface of smartchem media that has been created and revised is as follows. 3.4 limited trials a limited trial was conducted with 45 pre-service teachers majoring in science education. a questionnaire was administered to evaluate the feasibility of the developed smartchem. mainly, it assesses the pre-service teachers’ perception regarding the mobile connectivity, learning materials, and mobile layout of the smartchem, as well as to assess their experience after learning with this media. the result of the trial is presented in table 2. table 2 the result of limited trials no construct statement sa (%) a (%) d (%) sd (%) 1 mobile connectivity the application is a convenient and effective time to consume 13.33 62.22 22.22 0 the application is easy to connect 24.44 62.22 13.33 0 the materials are accessible 22.22 64.44 13.33 0 2 materials the symbolic representations (chemical formula, chemical reaction, phase) are understable 33.33 64.44 2.22 0 the submicroscopic representations (molecular representations) are understable 24.44 71.11 4.44 0 the macroscopic representation (real phenomenon) are understable 35.56 64.44 0 0 i can connect the symbolic, submicroscopic, and macroscopic level in learning acid-base 13.33 80.00 6.67 0 3 mobile layout the appearance is attractive 17.78 57.78 17.78 6,67 all text is clearly seen and readable 42.22 46.67 11.11 0 all medias are support the materials comprehension 22.22 57.78 17.78 0 4 learning experience mobile learning can help to understand the concept 31.11 57.78 11.11 0 mobile learning can help to give meaningful learning 31.11 57.78 11.11 0 mobile learning can help to motivate students 33.33 60.00 6.67 0 sa = strongly agree, a = agree, d = disagree, sd = strongly disagree journal of science learning article doi: 10.17509/jsl.v3i3.23280 201 j.sci.learn.2020.3(3).196-204 the results show that roughly around 62% of preservice teachers agreed that the smartchem is convenient and effective, easy to connect, and accessible (table 2). in regards to the learning materials provided smartchem, 64.44% agreed that the symbolic and macroscopic representation is understandable. whereas, around 71.11% of pre-service teachers agreed that submicroscopic is understandable. as many as 80% agreed, they could connect the content of symbolic, submicroscopic, and macroscopic levels in learning acidbase. most of the pre-service teachers also respond positively to the layout of the smartchem, where 57.78% agreed that it is attractive and can support the comprehension of multiple representations of the acidbase topic. additionally, 46.67% agreed that all the text is seen and readable. furthermore, the pre-service teachers evaluate smartchem after they experience learning with the media. as many as 57.78% agreed that smartchem could help to understand the concept and give meaningful learning. sixty percent of pre-service teachers noted to decide that smarchem can help to motivate them in learning. furthermore, around 30% of pre-service teachers strongly agreed that smartchem assists positively on their learning. the use of computer-based media is essential in figure 2 average score on audio/visual aesthetic display 0.95 0.7 0.85 0.85 0.65 0.7 0.95 0.85 0.9 0.65 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 a v e ra g e s c o re audio/visual aesthetic display figure 3 average score on program features 0.75 0.85 0.6 0.7 0.6 0.85 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 complexity algorithm design program interactivity response to input space efficiency the ease of program a v e ra g e s c o re program features journal of science learning article doi: 10.17509/jsl.v3i3.23280 202 j.sci.learn.2020.3(3).196-204 enhancing the effectiveness of the learning process due to it can provide many components such as text image, graphics, animation, audio, and video, which complement each other and construct a powerful system (rachmadtullah, zulela, & sumantri, 2018). notably, in the chemistry course where the concepts of the abstract and submicroscopic level at the same time presented in certain topics (calik & ayas, 2005), such as in the acid and base. the results of this study show most of the preservice teachers perceived positively to the use of smartchem. it can be inferred that smartchem is feasible to be employed as the media in learning multiple representations of acid-base. research conducted by liu, liao, & pratt (2009) recognized that the ease of the media being used is in line with the usefulness of media itself, which consequently increases people's intention to use. the robust content presented is also correlated positively with a higher-level concentration of the user. furthermore, the research by owston, york, & murtha (2013) noted a strong correlation between the students' perception of using blended learning and students' grades. besides, the use of computers in learning is a powerful strategy to improve students' motivation (leow & neo, 2014; lin, chen, & nien, 2014). therefore, feasible smartchem can be further performed to investigate the students' learning improvement and motivation in the chemistry course. the pre-service science teacher who participated in a limited trial was the student who had enrolled in the general chemistry course. based on the result of their academic performance on that course, 13 students had low achievement (group 1), 19 students had medium achievement (group 2), and 13 students had high achievement (group 3). additionally, we analyzed students with different learning achievements in the general chemistry courses in response to the feasibility of the figure 4 average score on pedagogical features 0.7 0.65 0.85 0.8 0.55 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 learning material the appropriateness of content within animation the clarity of learning material the clarity of examples the clarity of exercise a v e ra g e s c o re pedagogical features figure 5 pre-service teacher perception on smartchem based on learning achievement on general chemistry course 3.2 3.3 3.3 3.3 3.0 3.1 2.9 3.1 2.9 3.4 3.2 3.4 1 2 3 4 mobile connectivity materials mobile layout learning experience a v e ra g e s c o re the construct of smartchem assessment on limited trials group 1 group 2 group 3 journal of science learning article doi: 10.17509/jsl.v3i3.23280 203 j.sci.learn.2020.3(3).196-204 smarchem. the result is depicted in figure 5. generally, the three groups assessed positively to the feasibility of the smartchem, since the average score of each group at four constructs is more than two from four likert-scale. based on figure 5, the group with low achievement in the general chemistry course gave a higher score in assessing the mobile connectivity and mobile layout of the smartchem. meanwhile, the group with high achievement gave a higher score for the learning material and learning experience with smartchem. the low achievement group was more concerned about the technical features such as the accessibility, effectiveness, and the attractiveness of the smartchem in learning. they did not give a higher number to the content of the materials might because they were predicted to have less comprehension of the learning materials, especially the multiple representations in acidbase. therefore, they also had a lower score than the high achievement group in learning experience using smartchem. nevertheless, the high achiever group was more concerned with the content materials and learning experience. the students in this group had more comprehension of the materials, could compare the material with the reference, and could assess and give smartchem with a higher score than other groups. consequently, they also gave a higher score to the learning experience due to they perceived it is more meaningful learning compare with their previous experience. we also analyze pre-service teachers' perceptions regarding the use of smartchem based on the academic year. the students in the third year of academic study, which is batch 2017 consists of 23 students. meanwhile, the students in the fourth year of academic study, which is batch 2016, consists of 22 students. the comparison between the two batches in assessing the feasibility of smartchem is presented in figure 6. students with more extended experience in college study gave lower scores to the feasibility of the smartchem on all constructs. it might be caused they have experienced to have more complex material from the chemistry or other courses. they also might experience learning with more varied teaching media. therefore, they have a broader view of assessing the smartchem, and they scored lower instead of batch 2017. 4. conclusion multiple understanding of the chemical representation of science students still lacks, especially in symbolic and submicroscopic understanding. one way to improve the understanding of multiple chemical representations of prospective science teacher students is by developing android-based smartchem media. characteristics of smartchem media based on android applications that have been developed are media that can explain symbolic, macroscopic, and submicroscopic representations together to understand acid-base material. this smartchem media is validated by five experts so that there are parts that need to be revised, especially in terms of linking submicroscopic level content with symbolic. the results of smartchem media trials show this media can assist students in understanding multiple chemical representations on the topic of acid-base. the low achievement group was more concerned about the technical features such as the accessibility, effectiveness, and the attractiveness of the smartchem in learning. nevertheless, the high achiever group was more concerned with the content materials and learning experience. the recommendation for this research is that smartchem when testing electrical conductivity, pays more attention to the electrolysis process. media apps like this need to be developed for other topics, especially those related to scientific literacy and multiple chemical representations. figure 6 pre-service teachers’ perception on smartchem based on academic year of study 2.9 3.2 2.8 3.0 3.2 3.3 3.3 3.4 0 1 2 3 4 mobile connectivity materials mobile layout learning experience a v e ra g e s c o re the construct of smartchem assessment on limited trials batch 2016 batch 2017 journal of science learning article doi: 10.17509/jsl.v3i3.23280 204 j.sci.learn.2020.3(3).196-204 references aiken, l. r. (1985). three coefficients for analyzing the reliability and validity of ratings. educational and psychological measurement, 45(1), 131-142. andriani, t. (2016). sistem pembelajaran berbasis teknologi informasi dan komunikasi. sosial budaya, 12(1), 117-126. ardac, d., & akaygun, s. (2004). effectiveness of multimedia-based instruction that emphasizes molecular representations on students’ understanding of chemical change. journal of research in science teaching, 41(4), 317–337. https://doi.org/10.1002/tea.20005 cahyani, i., & nasruddin, h. (2019). the development of students worksheet oriented problem solving to train creative thinking skills in acid base matter for 11th grade. unesa journal of chemical education, 8(2). calik, m., & ayas, a. (2005). a comparison of level of understanding of eighth‐grade students and science student teachers related to selected chemistry concepts. journal of research in science teaching, 42(6), 638-667. dunn, j. g. (1999). assessing item content-relevance in sport psychology scale-construction research: issues and recommendations. measurement in physical education and exercise science, 3(1), 15-36. eliyawati, rohman, i., & kadarohman, a. (2018). the effect of learning multimedia on students’ understanding of macroscopic, sub-microscopic, and symbolic levels in electrolyte and nonelectrolyte. journal of physics: conference series, 1013(1). https://doi.org/10.1088/1742-6596/1013/1/012002 gkitzia, v., salta, k., & tzougraki, c. (2011). development and application of suitable criteria for the evaluation of chemical representations in school textbooks. chemistry education research and practice, 12(1), 5-14. helsy, i., & andriyani, l. (2017). pengembangan bahan ajar pada materi kesetimbangan kimia berorientasi multipel representasi kimia. jurnal tadris kimiya, 2(1), 104. https://doi.org/10.15575/jta.v2i1.1365 ibrahim, f., sugiyarto, k. h., & ikhsan, j. the development of html5-based virtual chemistry laboratory (vich-lab) media on acid-base material to improve high school students' selfefficacy. istijabatun, s. (2008). pengaruh pengetahuan alam terhadap pemahaman mata pelajaran kimia. jurnal inovasi pendidikan kimia, 2(2), 323–329. https://doi.org/10.1007/978-0-387-72659-5_2 kamkhou, p., & yuenyoung, c. (2019). magnet and pin kit: connection symbolic and submicroscopic representations of lewis dot structure and molecular geometry. journal of physics: conference series, 1340(1). https://doi.org/10.1088/17426596/1340/1/012070 kumar, r. (2011). research methodology third edition a step by step guide for beginner (3rd ed.). sage publications. kumar, s. (2013). international journal on new trends in education and their implications (ijonte), 4(4), 214. leow, f. t., & neo, m. (2014). interactive multimedia learning: innovating classroom education in a malaysian university. turkish online journal of educational technology-tojet, 13(2), 99-110. liao, y., loures, e. r., venâncio, a., brezinski, g., & deschamps, f. (2018). the impact of the fourth industrial revolution: a crosscountry/region comparison. production, 28(0). https://doi.org/10.1590/0103-6513.20180061 lin, h. m., chen, w. j., & nien, s. f. (2014). the study of achievement and motivation by e-learning-a case study. international journal of information and education technology, 4(5), 421. liu, s. h., liao, h. l., & pratt, j. a. (2009). impact of media richness and flow on e-learning technology acceptance. computers & education, 52(3), 599-607. mewengkang, a., liando, o. e. s., ngodu, m. r., moningkey, e. r. s., & wantania, t. (2019). android based application for children learning with indonesian and mongondow language. 5th upi international conference on technical and vocational education and training (ictvet 2018). atlantis press. owston, r., york, d., & murtha, s. (2013). student perceptions and achievement in a university blended learning strategic initiative. the internet and higher education, 18, 38-46. rachmadtullah, r., zulela, ms., & sumantri, m. s. (2018). development of computer-based interactive multimedia: study on learning in elementary education. international journal of engineering & technology, 7(4), 2035-2038. raviolo, a. (2001). assessing students ’ conceptual understanding of solubility equilibrium. journal of chemical education, 78(5), 629– 631. roche allred, z. d., & bretz, s. l. (2019). university chemistry students’ interpretations of multiple representations of the helium atom. chemistry education research and practice. https://doi.org/10.1039/c8rp00296g sunyono, s., & meristin, a. (2018). the effect of multiple representation-based learning (mrl) to increase students’ understanding of chemical bonding concepts. jurnal pendidikan ipa indonesia, 7(4), 399–406. https://doi.org/10.15294/jpii.v7i4.16219 treagust, d. f., chittleborough, g., & mamiala, t. l. (2003). the role of submicroscopic and symbolic representations in chemical explanations. international journal of science education, 25(11), 1353– 1368. https://doi.org/10.1080/0950069032000070306 umachandran, d. k., jurcic, i., & ferdinand-james, d. (2018). gearing up education towards industry 4.0. international journal of computers & technology, 17(2), 7305–7311. https://doi.org/10.24297/ijct.v17i2.7754 wu, h. k., krajcik, j. s., & soloway, e. (2001). promoting understanding of chemical representations: students’ use of a visualization tool in the classroom. journal of research in science teaching, 38(7), 821–842. https://doi.org/10.1002/tea.1033 a © 2021 indonesian society for science educator 185 j.sci.learn.2021.4(2).185-191 received: 08 october 2020 revised: 22 february 2021 published: 25 march 2021 android-based animation for chemical elements and experiments as an interactive learning media dedi saputra1*, burcu gürbüz2, haryani haryani1 1sistem informasi, universitas bina sarana informatika, jakarta, indonesia 2johannes gutenberg-university of mainz, germany; ̈usk ̈udar university, ̇istanbul, turkey *corresponding author. dedi.dst@bsi.ac.id abstract the use of smartphone technology with various operating system platforms and android is also widely used in education. various types of applications were created to support the learning process at school and outside of school. the purpose of this research is to create an alternative learning media in the form of an interactive animation application that utilizes smartphone technology on material or topic discussion of elements and chemical experiments in the eyes of chemistry lessons. this android-based interactive animation application is focused on periodic table material that provides necessary information about elements and chemical experiments that include ph solutions, electrolyte solutions, and synthetic reactions in them. the application is designed using the construct 2 application software, html 5 programming language, the intel xdk compiler with the waterfall software development method, and blackbox testing as a software testing method. this study's results are in the form of an android-based learning application to test chemical elements hoped that this application could be implemented for chemistry subjects. especially for high school students, as an attractive alternative learning media to make it easier to understand chemistry. keywords interactive animation, smartphone, android, construct 2 1. introduction the learning process should make students enthusiastic about participating in learning to participate in learning activities thoroughly and continuously. the most comfortable learning atmosphere is one of the essential factors in improving the classroom's effectiveness; learning possible to take place optimally (syafaruddin, mesiono, butar-butar, & assingkily, 2020). the development of information technology and communication significantly impacts instructional media learning in schools and other educational institutions. during the covid-19 pandemic, where the education process switches to online methods, information technology, and communication change the learning location from class to anywhere and anytime students can study. communication technology encourages the evolution of learning locations and times. learning no longer only occurs in school and class, but learning can occur anywhere as long as there are teaching materials and students feel comfortable with the situation (rusdi & yunus, 2016). many students at every level of education consider chemistry to be a lesson containing quite tricky concepts because of the many elements and abstract reactions. chemical elements that consist of many elements often make students lazy with chemistry, especially in terms of memorizing the periodic system of chemical elements., studying the composition and nature of an object as well as changes in the formation of that substance. chemistry comes from the egyptian "keme" which is means "earth" is the study of composition, structure, and properties of matter, and all changes that accompany chemical reactions (pratama, yuanita, & susantini, 2016). chemistry is the study of the composition and properties of an object and its changes and formation. chemistry is a branch of natural science (ipa). among branches of science has a close relationship. as technology advances, chemistry continues to develop into a new, more specialized science. according to schwartz et al. (rahayu, 2017), chemistry is an experimental discipline. chemists conduct scientific inquiry, make generalizations, and propose theories to explain natural phenomena. it can be concluded, chemistry is the study of the composition, structure, composition, journal of science learning article doi: 10.17509/jsl.v4i2.28787 186 j.sci.learn.2021.4(2).185-191 and properties of substances to find answers to how natural phenomena are related to a composition, the structure, properties, change, dynamics, and substance energetics. abstract chemical concepts will be a problem when these concepts are the key concepts used to understand a phenomenon that occurs (eliyawati, agustin, sya’bandari, & putri, 2014). solving problems result in low student interest in learning even though chemistry as basic science is considered to have essential values applied in various life spheres. students would better understand chemistry and understand difficult chemistry concepts if they could make deeper connections between concepts and realities (rastegarpour & marashi, 2012). students can improve their understanding of chemistry and like chemistry as fun learning through learning media (lubis & ikhsan, 2015). one way to help these students can be done by utilizing gadget technology that is already widely owned (hafidha, 2014). the development of science, information technology, and communication provides significant changes to the learning process development. information and communication technology (ict) in education continues to evolve in various strategies and patterns. it can be grouped into an e-learning system as a form of learning that utilizes electronic devices and digital media and mobile learning (m-learning) as a particular form of learning to utilize mobile communication technology and devices (kurniawati & priyanto, 2018). that was also followed by mobile phone technology (mobile phone), which is currently known as a smartphone with practical characteristics and can be carried anywhere. a mobile application is a software or a program that runs on a mobile device whose job is to perform specific tasks for its users (sunarya, prima, & wihardi, 2020). with the increasing number of mobile device users or smartphones, especially among students in indonesia, the mobile learning method can be used as an alternative to solve problems in education. this research refers to previous research that discusses (1) the design of an android-based telecommunication quiz application (budiharjo & yulianto, 2014), (2) the making of an android-based "pontianak punye" quiz game application (saputra & rafiqin, 2017; aksad, 2018), (3) solar system based android (nuqisari & sudarmilah, 2019), (4) atwood machine-based android (shabrina, warsono, & kuswanto, 2017), (4) android based math & trash educational game using scirra construct 2 and adobe phonegap (widaningrum, prasetiyo, & astuti, 2020), (5) arduinoandroid based game (yasin, prima, & sholihin, 2018), and (6) smartchem: an android application for learning multiple. representations of acid-base chemistry (eliyawati, agustin, sya’bandari, & putri, 2014). based on the problem mentioned above and relevant research to make chemistry learning media in the form of an androidbased interactive animation application, it can also be used as a learning method (mobile learning). hence, the two terms discussed in this article are mobility and learning. the two terms under consideration in this article are therefore mobility and learning. on the one hand, "mobility" refers to the technology's capabilities within the students' physical contexts and activities as they participate in higher learning institutions. on the other hand, it refers to the learning process's activities and the learners' behavior as they use the technology to learn. it also refers to students' highly mobile attitudes as they use mobile technology for learning purposes (el-hussein & cronje, 2010; busran & yunanda, 2015). the purpose of mobile learning is to make learning easier for students wherever and whenever they are so that they are not limited to space, time, and place (nasution, 2016; fudholi, 2015). the design of an interactive animation application for learning chemical elements based on android. 2. method the methodology used in this research is the descriptive analysis method, which aims to get a clear description of what is needed. the needs for completing this research are functional needs and non-functional needs. the summary of some of these opinions is that a storyboard is a series of illustrations or images displayed sequentially containing story ideas to visualize a story in detail from each scene. some examples of the design of the storyboard in this study can be seen in figure 1 and figure 2 creating the program code: stages in making chemical elements learning animation: (1) defining the problem by analyzing the problem and then solving the problem through a program or application based on the need of view. (2) planning and system design. at this stage, the researcher makes a program flowchart to describe the animation display and program logic. (3) implementation. at this stage, the researcher makes program codes using the construct 2 programming language. (4) documentation stage. at this stage, the programming code figure 1 storyboard splash screen design journal of science learning article doi: 10.17509/jsl.v4i2.28787 187 j.sci.learn.2021.4(2).185-191 has reached 80%. furthermore, documentation or comments are made on the program. (5) testing. this stage tests the unit or program module and tests the program input. at this stage, the software design is realized as a series of programs or program units. then unit testing involves verification that each program unit meets its specifications. the test method used is blackbox testing. "black-box testing is testing the software in terms of functional specifications to find out whether the functions, inputs, and outputs of the software are by the required specifications (rosa & shalahuddin, 2013). at the test stage of the program, researchers use black box testing by testing the program's function, testing the program's validation, whether the input is by its output. the logic of the program has been described through flowchart diagrams. it can be concluded that black box testing is testing software by running programs with unknown internal performance and finding out whether the functions, inputs, and outputs of the software match the required specifications. support or maintenance: the support or maintenance stage can repeat the development process from specification analysis to existing software changes, but not to create new software. 3. result and discussion mobile learning is defined by clark quinn (fujiawati & raharja, 2019) as the intersection of mobile computing and e-learning: accessible resources wherever you are, robust search capabilities, rich interaction, powerful support for effective learning, and performance-based assessment. the mobile learning method is a learning model that utilizes flexible information and communication technology in both time and place, contains many multimedia content features, material exploration capabilities, multi-interaction, effectiveness, and performance-based assessments. e-learning is independent of location in time or space. mobile learning can perhaps be defined as 'any educational provision where the sole or dominant technologies are handheld or palmtop devices. this definition may mean that mobile learning could include mobile phones, smartphones, personal digital assistants (pdas), and their peripherals, perhaps tablet pcs and perhaps laptop pcs, but not desktops in carts and other similar solutions (traxler, 2005). the conclusion from this definition is mobile learning is a learning model that utilizes information and communication technology. in the concept of mobile learning, its utilization includes the availability of teaching materials that can be accessed at any time and the visualization of exciting material. mlearning or mobile learning refers to using handheld devices such as pdas, smartphones, laptops, and other information technology devices that will be widely used in the learning process; this study focused on cell phones (cellphones) or smartphones (smartphones). the goal is that the long-life learning process of students/students can be more active, time-efficient and can increase student interest in learning with an exciting application. cognitive, affective, and psychomotor students are taught multi-representation learning better than conventional (meidayanti, sunyono, & tania, 2015). the use of android-based interactive animation as a learning medium has changed the learning paradigm, starting from how a person learns, how to obtain information, and adjust information. this android-based interactive animation learning media also makes it easier for students to understand and practice to achieve the learning process's objectives. the application design results can be explained with several display samples implemented to explain the basic learning material and chemical experiments to achieve the expected objectives. the application's design has been created and explained by several display samples that are implemented to explain figure 2 main menu storyboard design figure 3 splash screen view journal of science learning article doi: 10.17509/jsl.v4i2.28787 188 j.sci.learn.2021.4(2).185-191 necessary learning materials and chemical experiments so that the expected goal can be achieved. 3.1 application display 3.1.1 splash screen display this display contains a splash screen image for a few seconds to enter the main menu layout page. the splash screen display is an initial description of the menu in this application to get an overview of this application. the image shown is symbolic of the application. the splash screen display can be seen in figure 3. 3.1.2 main menu display this main menu layout contains (a) a title, (b) a periodic table button to display the periodic table layout, (c) an element list button displays the element layout based on a group from the periodic table, (d) a chemical experiment button to display animated layout options, (e) a play button to display game layout menu options such as quizzes and guess the picture that will be played. exit button to display the exit confirmation options. the main menu display can be seen in figure 4. 3.1.3 periodic table menu display the periodic table menu layout contains the periodic table where each box contains a picture of each element, the shape of the periodic table position is landscape. if this box is selected, it will go to the following layout, which contains an explanation of the periodic table elements. the back button functions to return to the main menu layout. the periodic table menu display can be seen in figure 5. 3.1.4 explanation of the periodic table display in this periodic table, the explanation layout contains images of the elements selected from the periodic table. then there is an explanation of each element selected from the periodic table. explanation of drag-drop-shaped elements can be pulled down and up. the back button returns to the periodic table menu layout. the periodic table explanation display can be seen in figure 6. 3.1.5 element list display the element list layout contains a list of elements based on the periodic element group, where each box contains an image of the element based on its respective group. if this box is selected, it will go to the following layout, which contains an explanation of the elements. the next button figure 4 main menu view figure 5 periodic table menu view figure 6 periodic table explanation view journal of science learning article doi: 10.17509/jsl.v4i2.28787 189 j.sci.learn.2021.4(2).185-191 (>>) goes to the other group elements layout. the back button functions to return to the main layout. the display of the elements list can be seen in figure 7. 3.1.6 chemical experiment display in this chemical experiment menu layout, three animation buttons can be selected where each animation has a different animation. each animation has its animation characteristics—the back button (<<) functions to return to the main menu layout. the chemical experiment menu display can be seen in figure 8. 3.2 testing testing in this study was conducted to obtain validation of the application design and its use. testing on the application aims to determine whether the application is running correctly according to its function before the respondent carries out an assessment. testing is part of system development that has been planned and very systematic to test or evaluate the desired truth of software quality (aziz, setiawan, khanh, nurdiyansyah, & yulianti, 2020). testing of application designs that are made using blackbox testing focuses on the process of program input and output. black box testing focuses on software's functional specifications (mustaqbal, firdaus, & rahmadi, 2015). the test results can be seen in supplementary information table s1 . testing of use in its implementation is carried out by distributing questionnaires to users, namely high school students with 30 respondents. ten questions are divided figure 7 element list menu view figure 8 chemical experiment menu view table 1 application view questionnaire answer recapitulation app view strongly agree agree disagrees very disagree number q 1 24 6 0 0 30 q 2 26 4 0 0 30 q 3 26 4 0 0 30 total 76 14 0 0 90 percentage 84.44% 16% 0 0 100% table 2 application purpose questionnaire answer recapitulation application objectives strongly agree agree disagrees very disagree number q 4 24 5 1 0 30 q 5 28 2 0 0 30 q 6 25 4 1 0 30 q 7 28 2 0 0 30 total 105 13 2 0 120 percentage 87.50% 11% 2% 0 100% table 3 recapitulation of user ease questionnaire answers user friendly strongly agree agree disagrees very disagree number q 8 27 3 0 0 30 q 9 28 2 0 0 30 q 10 28 2 0 0 30 total 83 7 0 0 90 percentage 92.22% 8% 0 0 100% journal of science learning article doi: 10.17509/jsl.v4i2.28787 190 j.sci.learn.2021.4(2).185-191 into three groups of questions: three questions about application views, four questions about application objectives, and three questions about ease of use. the questionnaire results can be seen in tables 1, 2, 3 and the graph of the processing results shown in figure 9. 4. conclusion from the description of the results and discussion of research on interactive animation applications and chemical experiments on an android-based learning media with a mobile learning model and its implementation it can be concluded: (1) this application can be an alternative solution for learning media, especially for high school students in learning and understanding the basic material of chemistry; (2) the features in the interactive animation application and chemical experiments based on android such as the periodic table, list of elements, chemical experiments, and games are made very interactive, exciting and user-friendly, (3) based on the category of the interactive animation application of elements and chemical exper iments android-based, it is obtained an average value of 84.44% strongly agree and 16% agree; this means that the appearance of the application is considered very good by the user (4) in the recapitulation of the objective of interactive animation applications of elements and chemical experiments based on android, the average value of strongly agree is 87.5%, agrees 11%; and disagree 2%, this means that the objective of the application is considered good by the user (5) the ease of user application of interactive animation of elements and chemical experiments based on android obtained an average value of 92.22% strongly agree and 8 % agree, this means the applications are very user-friendly. acknowledgment previously, the authors would like to thank all those who have helped and contributed to this research. in particular, the authors thank all students of sma kemala bhayangkari 1 sungai raya. in this study, the authors collected data and information by making direct observations at sma kemala bhayangkari 1 sungai raya, especially the science department, as the object of research. the author conducted a question and answer process with several educators or several authoritative persons related to this study's material. the data collection process was carried out by distributing questionnaires to 30 respondents containing ten questions to generate a conclusion or feedback from the application. references aksad, h. (2018). rancang bangun game adventure burung enggang berbasis android menggunakan scirra construct 2. progresif: jurnal ilmiah komputer, 13(2), 1771–1782. aziz, i. a., setiawan, b., khanh, r., nurdiyansyah, g., & yulianti, y. (2020). pengujian black box pada aplikasi sistem kasir berbasis website menggunakan teknik equivalence partitions. jurnal teknologi sistem informasi dan aplikasi, 3(2), 82–89. budiharjo, s., & yulianto, a. d. (2014). rancang bangun aplikasi quiz telekomunikasi berbasis android. journal ict, 5(8), 70–79. busran, b., & yunanda, n. d. (2015). rancang bangun aplikasi pembelajaran iqra untuk anak usia dini berbasis android. jurnal momentum, 17(1), 78–83. el-hussein, m. o. m., & cronje, j. c. (2010). defining mobile learning in the higher education landscape. journal of educational technology & society, 13(3), 12–21. eliyawati, e., agustin, r. r., sya’bandari, y., & putri, r. a. h. (2014). smartchem: an android application for learning multiple representations of acid-base chemistry. journal of science learning, 3(3), 196–204. fudholi, a. (2015). animasi interaktif pembelajaran pengenalan dan perancangan jaringan komputer. penelitian ilmu komputer sistem embedded dan logic, 3(1), 28–40. fujiawati, f. s., & raharja, r. m. (2019). analisis kesiapan mahasiswa pendidikan seni mengaplikasikan pembelajaran berbasis online (e-learning & mobile learning). jpks (jurnal pendidikan dan kajian seni), 4(2), 150–164. hafidha, p. n. w. (2014). augmented reality sistem periodik unsur kimia sebagai media pembelajaran bagi siswa tingkat sma berbasis android mobile. komuniti: jurnal komunikasi dan teknologi informasi, 6(2), 122–131. kurniawati, l. s., & priyanto. (2018). the effect of mobile learning on senior high school (sma): case study at public senior high school (sma negeri) in yogyakarta. journal of physics: conference series, 1140, 12017. https://doi.org/10.1088/17426596/1140/1/012017 lubis, i. r., & ikhsan, j. (2015). pengembangan media pembelajaran kimia berbasis android untuk meningkatkan motivasi belajar dan prestasi kognitif peserta didik sma. jurnal inovasi pendidikan ipa, 1(2), 191–201. meidayanti, r., sunyono, s., & tania, l. (2015). pembelajaran simayang tipe ii untuk meningkatkan self-efficacy dan keterampilan berpikir kritis. jurnal pendidikan dan pembelajaran kimia, 4(3), 856– 867. mustaqbal, m. s., firdaus, r. f., & rahmadi, h. (2015). pengujian aplikasi menggunakan black box testing boundary value analysis (studi kasus: aplikasi prediksi kelulusan smnptn). jurnal ilmiah teknologi infomasi terapan, 1(3), 31–36. nasution, m. i. p. (2016). strategi pembelajaran efektif berbasis mobile learning pada sekolah dasar. iqra’: jurnal perpustakaan dan informasi, 10(1), 1–14. figure 9 questionnaire data processing graph 24 26 26 24 28 25 28 27 28 28 6 4 4 5 2 4 2 3 2 2 0 0 0 1 0 1 0 0 0 0 0 10 20 30 40 50 r e s p o n d e n ts questionnaire questions strongly agree disagrees very disagree journal of science learning article doi: 10.17509/jsl.v4i2.28787 191 j.sci.learn.2021.4(2).185-191 nuqisari, r., & sudarmilah, e. (2019). pembuatan game edukasi tata surya dengan construct 2 berbasis android. emitor: jurnal teknik elektro, 19(2), 86–92. pratama, a., yuanita, l., & susantini, e. (2016). pengembangan perangkat pembelajaran kimia berbasis strategi belajar peta konsep untuk meningkatkan pemahaman konsep dan kemampuan berpikir kritis siswa sma. jpps (jurnal penelitian pendidikan sains), 5(2), 1023–1031. rahayu, s. (2017). mengoptimalkan aspek literasi dalam pembelajaran kimia abad 21. prosiding seminar nasional kimia uny, 21, 1–16. rastegarpour, h., & marashi, p. (2012). the effect of card games and computer games on learning of chemistry concepts. procedia-social and behavioral sciences, 31, 597–601. https://doi.org/10.1016/j.sbspro.2011.12.111 sukamto, r. a., & shalahuddin, m. (2013). rekayasa perangkat lunak terstruktur dan berorientasi objek. bandung: informatika, 3. rusdi, h., & yunus, m. (2016). pengembangan media pembelajaran berbasis android “chembird” pada materi kimia kelas xi di sman 17 makassar. jurnal ecosystem,16(2),290–301. saputra, d., & rafiqin, a. (2017). pembuatan aplikasi game kuis “pontianak punye” berbasis android. jurnal khatulistiwa informatika, 5(2), 71–85. shabrina, s., warsono, w., & kuswanto, h. (2017). android used in the learning innovation atwood machines on lagrange mechanics methods. international journal of science and applied science: conference series, 2(1), 338–345. sunarya, e. n., prima, e. c., & wihardi, y. (2020). the development of ‘e-layer’android mobile application as interactive multimedia in earth layer topics for junior high school. pervasivehealth: pervasive computing technologies for healthcare, 1, 1437-1444. syafaruddin, s., mesiono, m., butar-butar, a., & assingkily, m. s. (2020). manajemen pembelajaran pendidikan agama islam di sdit bunayya pandan kabupaten tapanuli tengah. auladuna: jurnal pendidikan dasar islam, 7(1), 32–45. traxler, j. (2005). defining mobile learning. iadis international conference mobile learning, 261–266. widaningrum, i., prasetiyo, h., & astuti, i. p. (2020). android based math & trash educational game using scirra construct 2 and adobe phonegap. jurnal resti (rekayasa sistem dan teknologi informasi), 4(1), 37–49. yasin, a. i., prima, e. c., & sholihin, h. (2018). learning electricity using arduino-android based game to improve stem literacy. journal of science learning, 1(3), 77–94. microsoft word layout 13271-31201-1-ce sofi hanif.docx a © 2019 indonesian society for science educator 50 j.sci.learn.2019.2(2).50-57 received: 5 october 2018 revised: 26 december 2018 published: 11 january 2019 enhancing students’ creativity through stem project-based learning sofi hanif1*, agus fany chandra wijaya2, nanang winarno1 1international program on science education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia 2department of physics education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia *corresponding author. hanifsofi@gmail.com abstract in some school, teacher-centered is commonly found in the learning process. the learning process itself is still in the form of direct transfer of knowledge from teacher to students. actually, students will learn better if they are engaged in a meaningful learning activity. stem project-based learning is one of the alternative teaching strategies that engaged students in meaningful learning. the aim of this study is to investigate the impact of stem project-based learning on students' creativity in the topics of light and optics. the study used qualitative research with the narrative design. data collection technique that used is observation. the population is eight grade students in one of junior secondary school that is located in bandung, indonesia. the sample consists of 25 students that chosen based on purposive sampling technique. the data is obtained through creativity product analysis matrix (cpam). there are three creativity dimensions that used in this study which are resolution, elaboration and novelty dimension. students’ creativity is obtained as much 76% which categorized as good. based on the result, stem project-based learning give a good impact on students' creativity. stem project-based learning can be used as alternative teaching strategies in junior secondary school. keywords stem project-based learning, students’ creativity, light and optics 1. introduction the advance of technology can produce competition in several life aspects, especially in education. the competition makes some countries change their education system and strategies that involving technology in the learning process. the use of technology in the learning process can be seen in the utilization of technology in making students’ project. students are more excited when making a project that involves the technology by constructing the power point which includes the byte, video clips, picture, text, and animation in the slide. students identified that working with technology is easier and possible for students to work quickly and efficiently (heafner, 2004). regarding that technology is important in this modern era, the education system should prepare students with the skills that need in facing the advance of technology. the students’ skills in indonesia do not really satisfy the skills needed in facing the advance of technology if the learning process only based on teacher-centered. in some school, teacher-centered is commonly found in the learning process. the learning process itself is still in the form of direct transfer of knowledge from teacher to students. students will learn better if they are engaged in meaningful learning activity (fortus et al., 2005). science, technology, engineering and mathematics (stem) education has potential to improves the quality of education. stem education integrates the contents and skills with science, technology, engineering, and mathematics. therefore, asmuniv (2015) stated that stem education can improve the quality of human resource with interdisciplinary in preparing students career. stem gives an opportunity for students to understand real-world problems based on those interdisciplinary subjects (dugger, 2010). interdisciplinary stem aims to emphasize the importance of 21-century skill development such as adaptation skill, social skill, communication skill, problem-solving skill, and self-development (bybee, 2010). science (s) explains the existence of objects and events, the laws and principle of these objects and events, and the relationship between them (capraro, capraro, & morgan, 2013). technology is an innovation and modification of natural environment to produces things that needed and journal of science learning article doi: 10.17509/jsl.v2i2.13271 51 j.sci.learn.2019.2(2).50-57 desired by a human (dugger, 2010). technology (t) maybe e-books, or online encyclopaedia which gives students direct access to find any information or sources; probes, sensors and experiments sets that enable students to collect data; social networking or websites that enables students access or contact the experts via online communication tools; presentation or video editing software that facilitate students in making presentation; and recording or analysis software that enables students to extend their capabilities (bruce & levin, 1997). engineering (e) is research and development based on science in order to manufacture certain products to solve problems (capraro, capraro, & morgan, 2013). engineering in stem project-based learning can be called the design process. mathematics (m) defined as an abstract representational system used in the study of numbers, shape, structure and change, and the relationship between these concepts (capraro, capraro, & morgan, 2013). several studies have been observed about the stem field in some cases. previous studies have been measured pre-service science teachers interest in stem career by interest survey (winarno, widodo, rusdiana, rochintaniawati, & afifah, 2017); the attitude of preservice science teacher regarding stem area (winarno et al., 2018); students’ problem-solving skill and students’ creativity based on girls’ interest in stem subject field (cooper & heaverlo, 2013); students’ stem literacy by conducting stem learning using arduino-android game (yasin, prima, & sholihin, 2018); and students’ science process skills, students’ science concept and students’ science content knowledge for gifted elementary students in the involvement of stem (robinson, dailey, hughes, & cotabish, 2014). stem project-based learning is one of learning model that can be used to satisfy the needs of stem education and prepare students in facing the advance of technology. stem project-based learning is the project-based model that integrate science, technology, engineering and mathematics (stem) in curriculum design (lou, tsai, & tseng, 2011). the design process and interdisciplinary of instruction make stem project-based learning is unique. the design process of stem project-based learning starts with preparing well-defined outcome by setting the objective and planning the summative assessment of the project. then, students will be given the ill-defined task that expresses their ideas to solve a complex problem with a different solution (capraro, capraro, & morgan, 2013). based on the study of lou, chou, shih, & chung (2017), there are 5 stages of stem project-based learning that can be adopted for the teacher. the stage of preparation is guiding students to understand the theme, scope, and problem. the stage of implementation required students to produce a project according to their design drawings and conducted the actual test. the stage of presentation is requiring students to present the project result. the stage of evaluation required the teacher to gives the evaluation or suggestion regarding students’ project. the stage of correction was encouraged students to make the correction according to the evaluation. creativity is one of 21st-century skill that needed by students in facing the advance of technology and preparing their future career. based on teacher interview, there still many teachers who measure the cognitive aspects. from this case, there is an indication that students have a lack of skills, especially in creativity. teachers have not trained students to strengthen their creativity. even though, the curriculum that was developed is more emphasized in the creativity aspect. creativity is one of important skill that should be fostered by students (dawes & wegerif, 2004). creativity refers to the creation of a novel and appropriate response, product, or solution to an open-ended task (amabile, 2012). if the creativity relates to the learning and technology it will produce a high quality of work. in the recent study show that technology allows the students to construct several media that can help them to produce high quality of work in the creativity context (loveless, 2002). stem project-based learning has a chance to give a positive impact in creativity because students will develop their own idea to create the product. various studies have been proved that stem projectbased learning gives effects in several aspects. stem project-based learning has been measured students’ creativity in aspects of adventurousness, curiosity, imagination and challenge (lou, chou, shih, & chung, 2017); students' learning attitude through multi-function electric vehicle (tseng, chang, lou, & chen, 2013); students’ science achievement through the implementation of latent growth modelling (erdogan & capraro, 2016); students’ imagination and stem knowledge development for female high school students (lou, tsai, tseng, & shih, 2014); academic achievement for high, middle, and low achievers (han, capraro, & capraro, 2014); and perception of pre-service and in-service teachers regarding the implementation of stem project-based learning in science class (siew, amir, & chong, 2015). students creativity through stem project-based learning has been conducted in the previous study. the previous study investigated four dimensions of creativity in aspects of adventurousness, curiosity, imagination and challenge in the concept of density, buoyancy, fluid, heat transfer and thermal energy (lou, chou, shih, & chung, 2017). for further identification, this study will investigate students creativity with another three dimensions of creativity. the three dimensions of creativity that used are resolution, elaboration, and novelty, while the concept that was chosen is light and optics. therefore, the aims of this study are to investigate the effect of stem project-based learning on students’ creativity in the concept of light and optics. journal of science learning article doi: 10.17509/jsl.v2i2.13271 52 j.sci.learn.2019.2(2).50-57 2. method 2.1 research method the study used qualitative research with the narrative design. narrative research designs are one of the qualitative procedures where researcher describes the things that happened during class, then collects and explains stories about students’ lives and experience in the form of narratives (creswell, 2012). data collection technique that used is observation. in collecting data, the researcher has a role as non-participant in the study. in non-participant observation study, the researcher only watches and observe the activities in the class and not directly involves in the observed situation. 2.2 population and sample this study is conducted in one of junior secondary school that is located in bandung, indonesia. kurikulum 2013 is implemented as the curriculum in this school. the population of this study is 8th-grade students between 14 until 16 years old. about 25 students that consist of 13 males and 12 females are selected as sample. the sampling technique that used was purposive sampling which requires the researcher to uses a personal judgment and believes to choose the samples (fraenkel, wallen, & hyun, 2012). 8thgrade students in this school are categorized as the high, medium and low achievement. thus, researcher considered sample who have the medium achievement. the sample and population are represented in table 1. 2.3 research instrument the research instrument was used to collect the data needed in this study. research instrument that used is creativity product analysis matrix (cpam) that was developed by besemer and treffinger (1981). the data that was collected from students’ creativity is based on a creative product that was made by students during stem project-based learning activity. the students’ creativity is scored by 1 until 3 scales for each criterion of creativity. the criterion that used is valuable, useful, well-crafted, expressive, original and novelty. the creativity product analysis matrix (cpam) can be shown in table 2. table 1 population and sample population sample n percentage (%) total (%) 8th grade students male 13 52 100 female 12 48 table 2 instrument for creative product analysis matrix (cpam) creative dimension criterion score 1 2 3 novelty germinal the lower level of germinal: the product is inspiring others with the creation medium level of germinal: the product is inspiring others to try something new high level of germinal: the product is inspiring others to try something new by directly give ideas to develop more product design original the lower level of originality: students mostly use the previous finding as their product idea medium level of originality: students use the previous finding as their idea, but they make a modification of the product high level of originality: the product idea comes from their own understanding resolution valuable the lower level of valuable: the product is not compatible with the purpose and not relates to the concept medium level of valuable: the product is compatible with the purpose and not relates to the concept high level of valuable: the product is compatible with the purpose and relates to the concept useful the lower level of usefulness: the product can be used once medium level of usefulness: the product can be used continuously with a certain requirement high level of usefulness: the product can be used continuously without any requirement elaboration well crafted the lower level of well crafted: the product is done well medium level of well crafted: the product is done well with the good looking design high level of well crafted: students take an effort to give interesting product design by using some materials expressive the lower level of expressive: the product is presented with lacking body language and need to control speaking tone, not understandable medium level of expressive: the product is presented with lacking body language and need control speaking tone, but understandable high level of expressive: the product is presented in a communicative way (using effective body language and clear voice) and understandable manner journal of science learning article doi: 10.17509/jsl.v2i2.13271 53 j.sci.learn.2019.2(2).50-57 2.4 research procedure the stages used in this study consist of preparation, implementation, presentation, evaluation, and correction (lou, chou, shih, & chung, 2017). this study needs fourth meeting to finish all stages of stem project-based learning, i.e. (1) first meeting, researcher conducted preparation stage which leads students to understand the theme and scope, (3) second meeting, researcher conducted implementation stage which let students to create the product based on their design drawing, (4) third meeting, researcher conducted presentation and evaluation stage that give opportunities for other to give suggestion regarding the project that are presented, and (5) fourth meeting, researcher conducted correction stage which give students opportunity to improve their product. the learning activities of each stage stem project-based learning can be represented in table 3. 3. result and discussion the implementation of stem project-based learning is related to the integration of science (s), technology (t), engineering (e), and mathematics (m). in this study, students will make the mini projector based on the stem field. the integration of stem in making mini projector activities can be presented in table 4. science (s) field in this study discusses the concept of the image that was formed in the lens. before making the mini projector, students should recognize how is the image formation in both of convex lens and concave lens. if students already understand about the characteristic of the image, they are able to decide the proper lens that used in constructing the mini projector. in this concept, students were expected to determine the correct length or room of lens, so the mini projector will produce a real and enlarged image. technology (t) field in this study can be sawed in preparation stage when students used the internet to find any information that was needed in making a mini projector. furthermore, students should make their decision to select the suitable tools and materials. technology (t) field also can be found in the implementation stage when students conducted the actual test, this activity requires students to check whether their mini projector is worked or not. engineering (e) in this study can be observed in the preparation stage when students made their own design drawing. design drawing that was made by students should be suitable with the concept of image formation in the lens. in order to make students easier to construct the mini projector, students were expected to put detail information in their design drawing, such as focal length, type of lens and distance of the object. mathematics (m) in this study refers to the magnification of image that was produced by the mini projector. students apply the formula to calculate the magnification of a mini projector. the result shows the qualitative data that was obtained based on creativity rubric. students’ creativity measured based on students’ product which is making a simple projector. the students’ creativity is assessed by using the creative product analysis matrix (cpam) that adapted from besemer and treffinger (1981). cpam is grouped into three creative dimensions which are resolution, elaboration, and novelty. the data is obtained based on the criterion of each creativity dimension. each criterion is scored with a rubric scale from 1 until 3 based on several requirements. creativity is the creation of a novel and appropriate response, product, or solution to an open-ended task (amabile, 2012). two criteria have been selected for each three dimensions of creativity based on besemer and treffinger (1981). useful and valuable criteria have been selected for the resolution dimension. valuable criteria table 3 the activities of stem project-based learning for each stage meeting stage activity 1st preparation students recognize the project theme and scope students find the information from the internet regarding the basic concept in making project students discuss tools and materials that will be used students produce design drawing 2nd implementation students make the project based on the design drawing students conduct an actual test of their product 3rd presentation each group present their product and basic concept behind the product evaluation teacher gives an evaluation regarding students’ product students conduct peer evaluation regarding another groups’ product 4th correction students make self-correction about the product according to suggestion and feedback table 4 the integration of stem in making the mini projector science (s) technology (t) engineering (e) mathematics (m) the formation of image in lens find information from the internet design drawing magnification calculation decide the tools and materials conduct an actual test journal of science learning article doi: 10.17509/jsl.v2i2.13271 54 j.sci.learn.2019.2(2).50-57 refers to how the product is judged worthy by others because the product fills the financial, physical, social, and psychological needs by the judgment, while useful criteria refers to how the product has clear and meet the practical application. then, well-crafted and expressive criteria have been selected for elaboration dimension. well-crafted criteria refers to how the product appears and has been worked or reworked with care which idea developed, while expressive criteria defined as how should the product is presented with the communicative way and understandable manner. for the last, germinal and original criteria was chosen for novelty dimension. germinal criteria defined as the product is likely to suggest an additional for the future creative product, while original criteria is how the product is unusual and rare to find with the same product idea in a similar experience. the result is obtained based on the criterion of each creativity dimension. each criterion is scored with a rubric scale from 1 until 3 based on several requirements. the creative rubric of cpam is presented in table 5. all criteria of each creativity dimension are used to assess a student’s project product after implementing stem project-based learning. the recapitulation of students’ creativity in this study can be seen in table 6. the result is shown that each creativity dimension has different attainment. resolution dimension obtained 77%, elaboration dimension obtained 87% and novelty dimension obtained 63%. the comparison of students’ creativity result for each dimension can be seen clearly in figure 1. the average score of each dimension creativity after implementing stem project-based learning is obtained 76% which categorized as good based on purwanto (2009). based on the result of this study, students who learn light and optics through stem project-based learning has good creativity. students are trained to realize their ideas by designing and constructing the product in stem projectbased learning. thereby, students were given the opportunity to develop their idea by using several tools and materials that can improve the quality of the product. it can be inferred that students’ who learned science by using stem project-based learning have good creativity. the result is in line with the study that is conducted by lou, chou, shih, & chung (2017) who stated that the implementation of stem project-based learning gives the positives influence on the effective development of creativity. the result of this study also in line with the previous finding which stated that stem approach, especially in hands-on activity through project-based learning, requires students to think critically and creatively (siew, amir, & chong, 2015). in the preparation stages, students are freely given an opportunity to investigate the problems and find some information that needed to solve the problems from the internet. this is appropriate with munandar (2004) who stated that creativity can be developed in a free situation to conduct an investigation. preparation stages also give an opportunity for students to discuss with their group in determining the project based on information that is obtained from the internet. the discussion is used to stimulate students in delivering their idea. it is in line with rustaman, et. al. (2003) who stated that discussion gives several advantages to stimulates students’ courage and creativity in expressing their idea, students also have responsibilities for the result of group discussion. table 5 creative product analysis matrix (cpam) rubric creative product criteria criterion group 1 group 2 group 3 group 4 group 5 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 novelty germinal      original      resolution valuable      useful      elaboration well crafted      expressive      table 6 students’ creativity result creativity dimension average category resolution elaboration novelty 77% 87% 63% 76% good figure 1 students creativity for each dimension 77 87 63 0 20 40 60 80 100 resolution elaboration novelty p er ce n ta g e (% ) creativity dimension journal of science learning article doi: 10.17509/jsl.v2i2.13271 55 j.sci.learn.2019.2(2).50-57 in implementation stages, students conduct an experiment to create the product that has been designed. furthermore, students conduct an actual test to make sure that their product is working. munandar (1999) stated that creative thinking skill can be developed through experiment and discussion activity between students. in presentation stages, students try to communicate their product and also their design. students express some obstacles that they are faced in making the project. as it is known that expressive is one criterion of elaboration dimension. students should think creatively, how to draw attention when present the product. expressive criteria refer to the product is presented with the communicative way and understandable manner (besemer & treffinger, 1981). in the evaluation and correction stages, students made a repairment to improve their product. these stages become a reflection for students to find how is the best way to improve the quality of the product. one of effective teaching should give students opportunities for reflecting their own thinking, receiving feedback from other students, and revising the ones’ thinking as a result of new information freely. capraro, capraro, & morgan, (2013) stated that the effective instruction should provide the opportunities for students in evaluating scientific evidence based on their own understanding, connecting the theory with their own explanation, and participating active learning. in this case, students’ creativity plays a role in creating an effective solution to repair students’ product. when conducted stem project-based learning, the class is divided into five groups that consist of five students to create the project. all group members should cooperate with each other in making a simple projector. the recapitulation of students’ creativity for each group is presented in table 7. based on the result in table 7, there are different achievements of creativity for each group. group 1 obtained 83.33%, group 2 obtained 50 %, group 3 obtained 94.43%, group 4 obtained 83.33%, and group 5 obtained 66.67%. based on the result, there is a distant gap in creativity result between group 2 with another group, because group 2 has the lowest percentage of creativity. the comparison of students’ creativity results for each group can be shown in figure 2. the distant gap can be found between group 3 and group 2. group 3 has the highest percentage of creativity which obtained as much 83% categorized very good. meanwhile, group 2 has the lowest percentage of creativity which obtained as much 50% categorized as very lack creativity. group 2 also has the lowest percentage of three creativity dimension if compared with other groups. when making the creative product, the class is divided into groups. thus, each group worked and discussed together to developed their idea in making the creative product. based on the result, each group has different attainment of creativity. however, there is a distant gap in creativity result between group 3 which categorized as very good and group 2 which categorized as very lack. the condition happened because group 2 always bicker among member when making the product. they blame each other if there is a member who is negligent with responsibilities. as the result, group 2 not take an effort to improve the quality of the product, while other groups attempted to improve their product quality. the students’ product after implementing stem project-based learning can be shown in figure 3. table 7 students creativity result for each group group creativity dimension average resolution elaboration novelty 1 83.33% 100% 66.67% 83.33% 2 50% 66.67% 33.33% 50% 3 100% 83.3% 100% 94.43% 4 83.33% 100% 66.67% 83.33% 5 66.67% 83.33% 50% 66.67% figure 2 students’ creativity result for each group 83 .3 3 50 94 .4 3 83 .3 3 66 .6 7 0 20 40 60 80 100 120 group 1 group 2 group 3 group 4 group 5 p er ce n ta g e (% ) groups figure 3 students’ product in making a simple projector journal of science learning article doi: 10.17509/jsl.v2i2.13271 56 j.sci.learn.2019.2(2).50-57 three creativity dimension of all groups have a different percentage. group 2 has the lowest percentage of three creativity dimension compared with other groups. to make the comparison of each dimension creativity between all groups can be seen clearly, the result also changes into graphical form. the creativity result for each dimension creativity can be shown in figure 4. 4. conclusion the students who implemented stem project-based learning in the concept of light and optics have good creativity in the dimension of resolution, elaboration, and novelty. the creativity result that obtained as much as 76% which is categorized as good. stem project-based learning can be used as alternative teaching strategies in junior secondary school. acknowledgment authors acknowledge principal of the school who already allowed to conduct the research about stem project-based learning. references amabile, t. m. (2012). componential theory of creativity. harvard business school, 1–10. asmuniv. (2015). pendekatan terpadu pendidikan stem upaya mempersiapkan sumber daya manusia indonesia yang memiliki pengetahuan interdisipliner dalam menyongsong kebutuhan bidang karir pekerjaan masyarakat ekonomi asean (mea). retrieved october 23, 2017, from http://www.vedcmalang.com/pppptkboemlg/index.php/menuut ama/listrik-electro/1507-asv9 besemer, s. p., & treffinger, d. (1981). analysis of creative products: review and synthesis. the journal of creative behavior, 15(3), 158– 178. bruce, b. c., & levin, j. a. (1997). educational technology: media for inquiry, communication, construction, and expression. journal of educational computing research, 17(1), 79–102. bybee, r. w. (2010). advancing stem education: a 2020 vision. technology and engineering teacher, 13(6), 30–35. capraro, r. m., capraro, m. m., & morgan, j. (2013). stem project-based learning: an integrated science technology engineering and mathematics (stem) approach. rotterdam: sense publishers. cooper, r., & heaverlo, c. (2013). problem solving and creativity and design : what influence do they have on girls ’ interest in stem subject areas ? american journal of engineering education, 4(1), 27–38. dawes, l., & wegerif, r. (2004). thinking and learning with ict: raising achievement in primary classrooms. london: routledge falmer. dugger, w. e. (2010). evolution of stem in the united states. in 6th biennial international conference on technology education research in australia. retrieved from http://www. iteea. org/resources/pressroom/australiapaper. pdf. erdogan, n., & capraro, r. m. (2016). viewing how stem projectbased learning influences students’ science achievement through the implementation lens: a latent growth modeling. eurasia journal of mathematics, science & technology education, 12(8), 2139–2154. fortus, d., krajcik, j., dershimer, r. c., marx, r. w., & mamlok‐ naaman, r. (2005). design‐based science and real‐world problem‐ solving. international journal of science education, 27(7), 855–879. fraenkel, j. r., wallen, n. e., & hyun, h. h. (2012). how to design and evaluate research in education (vol. 53). new york: mcgraw-hill. https://doi.org/10.1017/cbo9781107415324.004 han, s. y., capraro, r. m., & capraro, m. m. (2014). how science, technology, engineering, and mathematics (stem) project-based learning (pbl) affects high, middle, and low achievers differently: the impact of student factors on achievement. international journal of science and mathematics education, 13(5), 1089– 1113. heafner, t. (2004). using technology to motivate students to learn social studies. retrieved july 3, 2018, from http://www.citejournal.org/volume-4/issue-1-04/socialstudies/using-technology-to-motivate-students-to-learn-socialstudies. lou, s.-j., tsai, h.-y., tseng, k.-h., & shih, r.-c. (2014). effects of implementing stem-i project-based learning activities for female high school students. international journal of distance education technologies, 12(1), 52–73. lou, s. j., chou, y. c., shih, r. c., & chung, c. c. (2017). a study of creativity in cac2 steamship-derived stem project-based learning. eurasia journal of mathematics, science & technology education, 13(6), 2387–2404. lou, s. j., tsai, h. y., & tseng, k. h. (2011). stem online projectbased collaborative learning for female high school students. kaohsiung normal university journal, 30, 41–61. loveless, a. m. (2002). literature review in creativity. england. munandar, u. s. c. (1999). mengembangkan bakat dan kreativitas anak sekolah. jakarta: gramedia widiasarana indonesia. munandar, u. s. c. (2004). pengembangan kreativitas pada anak berbakat. jakarta: rineka cipta. purwanto, m. n. (2009). prinsip-prinsip dan teknik evaluasi pengajaran. bandung: remaja rosdakarya. robinson, a., dailey, d., hughes, g., & cotabish, a. (2014). the effects of a sciencefocused stem intervention on gifted elementary students ’ science knowledge and skills. journal of advanced academics, 25(3), 189–213. rustaman, n. y., dirdjosoemarto, s., yudianto, s. a., achmad, y., subekti, r., rochintaniawati, d., & nurjani, m. (2003). strategi belajar mengajar biologi. bandung: jica imstp fpmipa upi. siew, n. m., amir, n., & chong, c. l. (2015). the perceptions of preservice and in-service teachers regarding a project-based stem approach to teaching science. springerplus, 4(1), 1–20. tseng, k. h., chang, c. c., lou, s. j., & chen, w. p. (2013). attitudes towards science, technology, engineering and mathematics (stem) in a project-based learning (pjbl) environment. international journal of technology and design education, 23(1), 87-102. winarno, n., widodo, a., rusdiana, d., rochintaniawati, d., & afifah, r. m. a. (2017). profile of pre-service science teachers based on figure 4 creativity dimension for each group 83 .3 3 50 10 0 83 .3 3 66 .6 7 10 0 66 .6 7 8 3. 3 10 0 83 .3 3 66 .6 7 33 .3 3 10 0 66 .6 7 50 0 30 60 90 120 group 1 group 2 group 3 group 4 group 5 p er ce n ta g e (% ) groups resolution elaboration novelty journal of science learning article doi: 10.17509/jsl.v2i2.13271 57 j.sci.learn.2019.2(2).50-57 stem career interest survey. journal of physics: conference series, 895(1) 012170. winarno, n., widodo, a., rusdiana, d., rochintaniawati, d., afifah, r. m. a., & putra, m. (2018). investigation of pre-service science teachers’ attitudes towards science, technology, engineering and mathematics (stem). advanced science letters, 24(4), 28192822. yasin, a. i., prima, e. c., & sholihin, h. (2018). learning electricity using arduino-android based game to improve stem learning electricity using arduino-android based game to improve stem literacy. journal of science learning, 1(3), 77–94. microsoft word layout 12879-27768-2-ed.docx a © 2018 indonesian society for science educator 1 j.sci.learn.2018.2(1).1-8 received: 6 september 2018 revised: 20 november 2018 published: 2 december 2018 the effect of predict-observe-explain (poe) strategy on students’ conceptual mastery and critical thinking in learning vibration and wave dandy furqani1*, selly feranie 2, nanang winarno1 1international program on science education, faculty of mathematics and science education, universitas pendidikan indonesia 2department of physics education, faculty of mathematics and science education, universitas pendidikan indonesia *corresponding author. dandyfurqani@gmail.com abstract scientific learning in schools requires not only students’ ability to understand concept, but also critical thinking abilities of the students. however, the current scientific learning process is still focused on only cognitive aspects. therefore, a teaching model or strategy that is able to support students to understand concept as well as develop students’ critical thinking abilities is needed. one of the existing needed strategies is predict-observe-explain (poe). the aim of this research is to identify the effects of predict-observe-explain (poe) strategy on students’ conceptual mastery and critical thinking in learning vibration and wave. the method that was used in this research was weak experiment and the design was one-group pretest-posttest. the population of this research was 8th grader students in a junior high school in bandung. the sample of the research was 18 students. the technique that was used was purposive sampling. the results of the research were: there was enhancement in students’ conceptual mastery, indicated by average normalized gain of 0,29; there was enhancement in students’ critical thinking abilities from level 1,30 (challenged thinker) to 2,07 (beginning thinker). students can easily predict, observe and explain waves concept have difficulties on transversal waves and longitudinal waves concepts. for the next research it is recommended that predict-observe-explain (poe) is to be tried on motion, electricity and ecosystem. keywords predict-observe-explain (poe), conceptual mastery, critical thinking abilities, vibration and wave 1. introduction the indonesian national curriculum of 2013 stated that learning activities should be focused on students with actively seeking learning patterns and also critical learning (indonesian ministry of education and culture, 2015). based on the statement of the indonesian ministry of education and culture, we can infer that the learning is now emphasized on critical thinking skills of the students. however, according to an interview with a teacher of a middle school in indonesia, physics topics such as vibration and wave are considered a tough subject for the students. vibration and wave, as like as other physics topics, require not only students’ knowledge on basic understanding, but also complex thinking, especially to understand the characteristics and types of waves. therefore, it is necessary to find a more suitable learning model for topics such as vibration and wave. critical thinking ability is the ability to think to solve the problem systematically. the purpose of critical thinking is to achieve a deep understanding which is to uncover the meaning behind an event (johnson, 2010). critical thinking ability can be sharpened through laboratory activities, discoveries, homework to develop critical thinking skills, and exams designed to build critical thinking skills. critical thinking skills can be enhanced through group discussions were organized and guided directly by the teacher. highlevel questions can encourage deeper critical thinking (wardatun, dwiastuti & karyanto, 2015). critical learning emphasizes activities to analyze, interpret, and assess a case or an issue and rationally and logically. such activities are part of the critical thinking skills. these learning activities require high learning motivation from the students themselves. motivation to learn can help students develop critical thinking skills because by having the motivation to learn, the students will be more enthusiasm and always feel challenged to keep learning (ulfah, asim & parno, 2014). critical thinking as part of thinking skills must be possessed by every member in the community because a lot of problems in life that must be done and finished (wijaya, 2007). low critical journal of science learning article doi: 10.17509/jsl.v2i1.12879 2 j.sci.learn.2018.2(1).1-8 thinking skills will lead learners to have difficulty when faced with a concrete problem in everyday life. the main goal of the school is to improve the students’ critical thinking ability (slavin, 1997). however, in fact, the students in the school did not have the opportunity to develop critical thinking skills so that students' critical thinking abilities tend to be low. according to pratama (2012), the readiness of students to find the best knowledge from a context, confidence to ask, honesty and objectively seek for the information are still low. in fact, someone who has the readiness to find the best knowledge of a context, confidence to ask, honesty and objectively seeks for the information tends to have the nature of critical thinking. learning activities that are done at school are less able to develop the critical thinking skills of their students. learning activities that are designed are still lacking in providing critical thinking activities of students, such as activities to analyze, interpret, assessing a case or an issue and rationally and logically, which have been mentioned above (ulfah, asim & parno, 2014). learning is a change indeed through activities, practice, and experience. the paradigm of learning should emphasize on learning itself, is student-centered, should be shifted from "teacher” and “what is to be taught" to the direction of “students” and “what to do”. learning must also create meaningful connections with real life (hilgard & brower in hamalik, 2009). according to kibirige, osodo & tlala (2014), in order to learn, students have to construct knowledge in the learning process. therefore, the teacher must be able to provide supportive learning environments. before the learning process begins, students may have different opinion or knowledge about what they are going to learn. during the learning process, students might be not satisfied with they have learned. they may find or seek an explanation that is more acceptable, understandable and meaningful. learning can be done by using the previously existing knowledge for new knowledge. the teacher may let the students accommodate, assimilate or replace the previously existing knowledge with the new one. the accommodation, assimilation or replacement, however, often causes misconception among students. students’ prior knowledge should also be considered to make it more of a meaningful learning experience for the students (kala, yaman & ayas, 2012). in regard to science learning, teachers can involve students to make hypotheses, investigate, and analyze data to develop students’ thinking (wardani, 2017). one model of learning that is capable of developing students’ thinking optimally is predict, observe and explain (poe) learning model. poe learning models can include ways that can be taken by a teacher to assist students in improving the understanding of the concept and their psychomotor. poe learning model engages students in predicting a phenomenon, observations through demonstrations or experiments, and finally explain the results of the demonstration as well as their hypothesis. by doing this way, acquired knowledge will be preserved in students’ memory and increase students' science processing skills (zulaeha, darmadi & werdhiana, 2014). to make an active teaching-learning process, students need to be able to clearly express themselves in written form and verbal form; teachers need to introduce a new teaching strategy like the predict-observe-explain (poe) that can be used in association with demonstrations and hands-on activities that can help to enhance classroom practice by identifying the learner’s conception (hilario, j.s., 2015). poe is also suited to be applied in physics subjects that can mostly be observed in experiments, and help to solve misunderstanding (nana & sajidan & akhyar & rochsantiningsih, 2014). according to the given statements, predict-observe-explain (poe) should be able to be applied as one of the solutions to solve the problem at school regarding the topic of vibration and wave. the novelty of this research is, this research measures conceptual mastery and critical thinking also analyses students’ ability to predict, observe and explain in poe stages. from the problems, it is seen that the teachinglearning process in the school is still not very effective to bring out students’ capacity to its full potential, including in topics such as vibration and waves. therefore, the aim of this study is to investigate the effect of predict-observeexplain (poe) strategy on students’ conceptual mastery and critical thinking in learning vibration and wave. 2. method 2.1 research method and research design the method that was used in this study is weak experiment method. fraenkel, wallen, and hyun (2011) stated that this design is weak and do not have built-in control for threats to internal validity. in addition to the independent variable, there are a number of other plausible explanations for any outcomes that occur to find the effect of predict-observe-explain (poe) strategy on students’ conceptual mastery and critical thinking in learning vibration and wave topic on 8th-grade secondary school. table 1 one-group pretest and posttest design o x o pretest: 24 multiple choice questions and 6 essay items were given. (dependent variable) treatment: given to the students using predict-observeexplain strategy posttest: 24 multiple choice questions and 6 essay items were given. (dependent variable) journal of science learning article doi: 10.17509/jsl.v2i1.12879 3 j.sci.learn.2018.2(1).1-8 this research used one group pretest and posttest design. therefore, in this study, the researcher picked one group, conducted pretest, gave treatment, and then conducted posttest. according to fraenkel, wallen, and hyun (2011), in the pretest and posttest experiment, researcher assigns a single group and measure or observe not only after giving a treatment of some sort but also before. this design is detailed in table 1. 2.2 population and sample the population in this research was international junior high school in bandung which implements the indonesian national curriculum of 2013. the population in this research was 8th-grade students. the samples were from a class in eighth grade. the samples consist of 18 students with the ages ranging from 13 to 14 years old. 8 students (44%) of the samples were male students while the other 10 students (56%) are female students. the sampling technique that was used in this research is purposive sampling. according to fraenkel, wallen, and hyun (2011), purposive sampling is a sampling, in which researchers do not simply study whoever is available but rather uses their judgment to select a sample they believe, based on prior information, will provide the data they need. there are 18 students from one class that are assigned as samples in this research. the percentage of students’ gender is detailed in table 2. the stages in this research are represented in table 3. 2.3 research instrument the instrument is necessary to be used for gaining data. in this research, the instruments that were used include pretest and posttest. the researcher used pretest and posttest instrument to test the students’ conceptual mastery and critical thinking in vibration and wave topic for both the control group and experimental group. the pretest was held before the groups are given treatment, and the posttest was given after the treatment was applied. initially, in both pretest and posttest, the students were to be given 50 questions multiple choices tests, consisting of 26 questions to measure students’ conceptual mastery and 24 questions to measure students’ critical thinking. however, after some considerations and thorough the validation process, the questions were reduced into 24 multiple choices question items for conceptual mastery test and 6 essay question items for critical thinking ability test. the questions for the students’ conceptual mastery are based on bloom’s taxonomy (anderson et al., 2000), while the questions for the students’ critical thinking that are covered by six indicators of students’ critical thinking by ennis. 2.4 research procedure the steps of conducting this research consist of three main stages, which are the preparation stage, the implementation stage and completion stage. preparation stage includes: (1) formulating the problem and research objectives, (2) defining the dependent and independent variables of the research; (3) determining the sample and the population of the research; (4) conducting literature review about predict-observe-explain (poe) learning strategy, students’ conceptual mastery, students’ critical thinking and vibration and wave topic; (5) designing research instruments; (6) testing research instrument; and (7) making revision of research instrument. implementation stage includes (1) specifying a group for the research; (2) conducting pretest to the sample group; (3) giving treatment to sample group; (4) conducting posttest to sample group. completion stage includes (1) calculating the data; (2) analyzing the data; (3) making result and conclusion; and (4) reporting of the research paper. 3. result and discussion 3.1 analysis of students’ conceptual mastery in analyzing students’ conceptual mastery, students’ scores gained from the tests are calculated and compared using data from both pretest and posttest. the average ntable 2 the percentage of students’ gender population gender number of students percentage (%) 8th grade students male 8 44 female 10 56 total 18 100 table 3 stages of the research stage activity preparation formulating the problem and research objectives defining the dependent and independent variables of the research determining the sample and the population of the research conducting a literature review about predict-observe-explain (poe) learning strategy, students’ conceptual mastery, students’ critical thinking, and vibration and wave topic designing research instruments testing research instrument making a revision of research instrument implementation specifying group for the research conducting pretest to the sample group giving treatment to sample group conducting posttest to sample group completion calculating the data analyzing the data making result and conclusion reporting of the research paper table 4 interpretation of indicator of conceptual mastery pretest posttest gain n-gain interpretation 29.17 50.00 20.83 0.29 low journal of science learning article doi: 10.17509/jsl.v2i1.12879 4 j.sci.learn.2018.2(1).1-8 gain scores are calculated and then interpreted using the criteria based on hake’s. the result is shown in table 4. based on the research, out of 100 score rating, students obtained an average score of pretest of 29.17. in the posttest, students obtained an average score of 50. using the average scores obtained in the pretest and posttest, a gain of 20.83 is obtained. furthermore, an average n-gain score of 0.29 is obtained. this value is interpreted as low according to hake. the result is shown in figure 1. based on figure 1, there is an enhancement in the posttest compared to the pretest. the initial score obtained by the students measured by the pretest is shown to be 29.17 out of 100 score rating. the students were already informed about holding the pretest before and were not taught by the teacher about the concept before to test their prior knowledge. after the implementation of poe strategy, the posttest was held to measure students’ conceptual mastery about the concept. the result of score 50 out of 100 score rating was obtained from the posttest. as shown by the data, there is an enhancement in students’ conceptual mastery measured in the pretest and the posttest. there is a gain of 20.83 value obtained by comparing students’ prior knowledge in the pretest and students’ knowledge after being treated by poe strategy by the posttest. it is seen that the value of n gain is shown to be 0.29. according to hake (1999), this value means there is a low gain in students’ conceptual mastery in learning vibration and wave using poe strategy. by using students’ score from pretest and posttest, the researcher analyzed the data using spss ver.20 to determine whether the data is normally distributed or not. the result is detailed in table 5. in the table, it is shown from the shapiro-wilk test that the significance is less than 0.05. thus, the data is not normally distributed. in comparison, the significance in the pretest has a value of 0.010 while the posttest has a significance value of 0.35. since there is only one class used for the sample and the data is compared to its pretest and posttest score, wilcoxon test is used to determine the difference. the result is described in table 6. based on the table, the level of significant value in the test is 0.05. since the significant value is 0.00, which is less than 0.05, it means there is the difference between students’ pretest and students’ posttest. the result is supported by kibirige, osodo, and tlala (2014), which found enhancement in students’ understanding of the concept. although, based on the research, different levels of cognitive skills have a different gain. as the cognitive level rise, the more complex the thinking level will be, and less effective predict-observe-explain becomes. thus, predictobserve-explain shows the linear result, supported by kala, yaman, and ayas (2012). the enhancement is likely due to students’ capability in learning through predict-observe-explain stages. in the predict stage, students were required to gather information with their prior knowledge, assisted with reliable sources. in this stage, students seemed to pass it without much difficulty. in the second stage, which is observed, students had to carefully observe the moving pendulum and slinky which and find any information to be obtained as knowledge. in the third stage, students were required to put the knowledge they obtained into words or any other methods to describe, which also require communication skills and comprehensive understanding. the enhancement in students’ conceptual mastery is also analyzed from the average of for each aspect. in this calculation, each aspect of conceptual mastery (c1, c2, and c3) is calculated using multiple choices test in the pretest and posttest. the data obtained are detailed in table 7. based on table 7, the c1 level has an n-gain value of 0.65, which is obtained using pretest and posttest. according to hake, the n-gain value of the c1 level is categorized into fair. however, the c2 level has an n-gain value of 0.25, which is categorized into low. the c3 level has an n-gain value of 0.12, which is categorized into low as well. the data is also shown in figure 2. from the figure, it is seen that c1 cognitive level has higher n-gain than c2 and c3 cognitive levels. it is then table 5 statistical analysis for conceptual mastery test instrument kolmogorov-smirnov shapiro-wilk statistic df sig. statistic df sig. pretest .220 18 .021 .855 18 .010 posttest .235 18 .010 .888 18 .035 table 6 statistical result of wilcoxon non-parametric test ty posttest pretest z -3,742b asymp. sig. (2-tailed) ,000 table 7 n-gain scores for each aspect of conceptual mastery level pretest posttest gain ngain inter pretation c1 30.56 75.93 45.37 0.65 fair c2 30.56 47.92 17.36 0.25 low c3 27.22 36.11 8.89 0.12 low figure 1 students’ average score on conceptual mastery test journal of science learning article doi: 10.17509/jsl.v2i1.12879 5 j.sci.learn.2018.2(1).1-8 followed by a c2 cognitive level. in comparison, the c3 cognitive level has the least value of n-gain. from the analysis, we see that poe strategy is more effective for the c1 cognitive level (understanding), less effective for the c2 cognitive level (understanding), and least effective for the c3 cognitive level (applying). this happened most likely because c1, c2 and c3 cognitive levels require different thinking levels. according to clark (2015), the levels can be thought of as degrees of difficulties. that is, the first ones must normally be mastered before the next one can take place. c3 cognitive level requires a higher thinking process than c2 cognitive level, while c2 cognitive level requires a higher thinking process than c1 cognitive level. this result is in accordance with hilario (2015) that stated that the poe strategy helped students to understand the concept better 3.2 analysis of students’ critical thinking in analyzing critical thinking ability, the researcher uses rubrics to determine the score, which is then interpreted to determine the development of critical thinking skill. the data obtained is as detailed in table 8. based on table 8, students obtained a score with an average value of 1.30, which is interpreted as challenged thinker according to paul and elder. from the posttest, students obtained a score with an average value of 2.07, which is interpreted as beginning thinker. these scores are obtained out of 4 score rating using the rubrics. using the scores, a gain of 0.78, which is an increase of roughly 20% from pretest, is obtained by calculating the score of posttest minus the score of the pretest. the example of students’ answers is shown in figure 3, figure 4 and figure 5. figure 4 represents the students’ answer in the pretest, while figure 5 represents students’ answer in the posttest. for the pretest, the representative figure 4.4 obtained score 1, because the answer is correct but the reason is related to the answer according to the rubric for the scoring, which is “the answer is incorrect and the reason is not relevant to the question”. meanwhile, for 4.5 that is presented in the posttest obtained score 4, because the answer is correct that the reason given is acceptable according to the rubric, which is “the question is answered correctly and the reason is significant to the question”. both figures are from the same student. during the pretest, students seemed to have many ideas about the concept of vibration and wave with their prior knowledge. during the posttest, students seemed to have set an idea according to the knowledge they obtained and then applied it to solve the problem. table 8 scoring and interpretation of students’ critical thinking ability name pretest posttest gain score (average) interpretation score (average) interpretation student 1 0.83 unreflective thinker 2.33 beginning thinker 1.50 student 2 1.50 challenged thinker 2.50 practicing thinker 1.00 student 3 1.33 challenged thinker 1.83 beginning thinker 0.50 student 4 0.83 unreflective thinker 1.17 challenged thinker 0.33 student 5 1.67 challenged thinker 2.50 practicing thinker 0.83 student 6 1.00 unreflective thinker 2.33 beginning thinker 1.33 student 7 1.50 challenged thinker 2.00 beginning thinker 0.50 student 8 1.50 challenged thinker 1.67 challenged thinker 0.17 student 9 1.50 challenged thinker 1.83 beginning thinker 0.33 student 10 1.17 challenged thinker 1.83 beginning thinker 0.67 student 11 1.33 challenged thinker 2.17 beginning thinker 0.83 student 12 0.67 unreflective thinker 2.50 practicing thinker 1.83 student 13 1.50 challenged thinker 2.50 practicing thinker 1.00 student 14 1.67 challenged thinker 2.67 practicing thinker 1.00 student 15 1.67 challenged thinker 1.67 challenged thinker 0.00 student 16 1.00 unreflective thinker 1.83 beginning thinker 0.83 student 17 1.67 challenged thinker 2.33 beginning thinker 0.67 student 18 1.00 unreflective thinker 1.67 challenged thinker 0.67 average 1.30 challenged thinker 2.07 beginning thinker 0.78 figure 2 n-gain scores for each aspect of conceptual mastery journal of science learning article doi: 10.17509/jsl.v2i1.12879 6 j.sci.learn.2018.2(1).1-8 the analysis shows that there is a positive result or enhancement of students’ critical thinking ability with poe strategy, which is supported by cholisoh, fatimah, and yuniasih (2015) in “critical thinking skills in integrated science learning viewed from learning motivation” that stated that there is a positive effect on the students’ critical thinking skills with poe strategy. in table 4.5, there are 17 students that have an increase in thinking level. the increase in critical skills varies from one to two levels. in the pretest, the result shows that students also have a range of critical skill indicated by the scores, with the lowest being 0.67 (unreflective thinker) while the highest being 1.67 (challenged thinker), which is shown by a number of students. in the posttest, students’ critical thinking vary with the lowest being 1.17 (challenged thinker) and the highest being 2.67 (practicing thinker). we also find that there are two students that did not raise thinking level and stay at challenged thinker. in addition, one of the two did not have gain in pretest-posttest comparison. the enhancement in students’ critical thinking is likely to be caused by each stage of poe that corresponds to the aspects of the critical thinking. in predict stage, students have to make a prediction with their prior knowledge and clarify it with other sources before observing the objects. once it is set, during observe stage, students have to adjust strategies or tactics to conduct the observation. after the information is obtained, students sum up what they have found during observation and their prior knowledge and make an inference to obtain information. during the process, students are not only required to find the right answer but also to construct the knowledge so that the answer they come with is within the reason they have. therefore, students should be able to use their further thinking skill to solve the problems during the process. 3.3 analysis of students’ capability in predict-observeexplain (poe) in addition to analyzing the data with the test, the researcher also conducted a test to analyze students’ gaining capability in poe using the worksheet. the results are shown in figure 6, figure 7 and figure 8. in the predicting stage, as represented in figure 6, students seemed to have roughly no difficulty in predicting the definition of waves. as seen in the diagram, there were no students that obtained 0 scores for the concept of waves. students mostly have a score of 1, which are obtained by 15 students (83%). the rest 3 students (17%) managed to obtain score 2 in predict stage. the opposite result of predict stage occurred for transversal waves. most students failed in predicting transversal waves, which is indicated by the fact that 10 students (56%) obtained a score of 0 for predicting transversal waves. 7 students (39%) obtained the score of 1, while 1 student (6%) managed to obtain the score of 2. as for longitudinal waves, 10 students (56%) failed to predict the concept. 5 students (28%) obtained the score of 1, while 3 students (17%) managed to obtain the score of 2. as seen on the diagram, the result on predict stage shows a roughly similar result for both transversal waves and longitudinal waves. while students seem to have roughly no difficulty on the definition of waves concept, students seem to have difficulty for an advanced concepts like transversal waves and longitudinal waves. the result is in line with hilario (2015) that found that students can make sensible prediction through poe. this result is also in line with white and gunstone (2014) that stated that prediction requires extended knowledge of the problem to solve the given problem so that more complex concept like transversal waves and longitudinal waves are more difficult than the definition of waves concept. in the observed stage, as represented in figure 7, we see that 2 students (11%) failed with the score of 0. roughly 12 students (67%) obtained the score of 1, while the rest 4 students (22%) managed to obtain the score of 2 for the concept of the definition of waves. as for transversal waves, 12 students (67%) fail with the score of 0, while 6 students (33%) obtained the score of 1, and no students managed to get the score of 2. for longitudinal waves, 11 students (61%) failed with the score of 0, while 3 students (17%) obtained the score of 1, and 22% students managed to get the score of 2. based on the diagram, students seem to be able to perform observation about the definition of waves. however, students still have difficulty for transversal waves and longitudinal waves. the result is in line with hilario (2015) that found that students can make sensible figure 3 example of questions used in critical thinking test figure 4 example of students’ answer in critical thinking pretest figure 5 example of students’ answer in critical thinking posttest journal of science learning article doi: 10.17509/jsl.v2i1.12879 7 j.sci.learn.2018.2(1).1-8 prediction through poe. this result is also in line with white and gunstone (2014) that stated that prediction requires extended knowledge of the problem to solve the given problem so that more complex concept like transversal waves and longitudinal waves are more difficult than the definition of waves concept. in the explain stage, as represented in figure 6, students seemed to be able to perform with not much difficulty. in the concept of the definition of waves, 22% of students failed with the score of 0 while the other 78% of students managed to obtain the score of 1. in the concept of transversal waves, 72% of students failed with the score of 0 while 28% students managed to obtain the score of 1. in the concept of longitudinal waves, 33% of students failed with the score of 0 while the rest 67% students managed to obtain the score of 1. based on the diagram, students have quite the grasp of wave and are able to explain it their own way easier than another concept. students were also able to grasp the concept of longitudinal waves, though not as easy as wave concept. on the other hand, students seem to have difficulty in learning about the transversal wave, unlike waves and longitudinal waves. this result is in line with white and gunstone (2014) that stated that prediction requires extended knowledge of the problem to solve the given problem so that more complex concept like transversal waves and longitudinal waves are more difficult than the definition of waves concept. the findings show that there is a possibility that students have a misconception in a transversal wave, which is in line with kala, yaman, and ayas (2012), which found that poe can be used for the teacher to find students’ misconception in a concept. 4. conclusion based on the results on the analysis in the research, it can be concluded that: (1) poe strategy shows enhancement in students’ conceptual mastery, indicated by normalized n-gain value 0.29; (2) as for critical thinking skills, poe strategy seems suited to enhance critical thinking ability. using poe strategy, the result showed that students gain increase in critical thinking from level 1,30 (challenged thinker) to 2,07 (beginning thinker); (3) poe is good to implement the knowledge into students. however, in its implementation, it is not effective as a strategy to implement the whole idea into students as students might face difficulty, especially in some stages. students can easily predict, observe and explain wave concept, but find it tough for transversal wave concept and longitudinal wave concept; (4) poe is probably not suited for some subjects or topics that require high thinking level. predict stage should go well when students are learning common knowledge, but might not be the case with rare cases. observe stage should go well with sensible knowledge, but might not suit abstract knowledge. explain should go well if students have least required communication ability, else it might cause misunderstanding instead. to put it short, poe also has limitation but can be solved by modifying the stages in poe by adding, removing or replacing one or some stages in poe to suit the subjects/topics. in its implementation, the teacher might want to consider what concept to be taught with poe strategy. figure 6 students’ score in predict stage figure 7 students’ score in observe stage figure 8 students’ score in explain stage journal of science learning article doi: 10.17509/jsl.v2i1.12879 8 j.sci.learn.2018.2(1).1-8 acknowledgment many supports have been received by the researcher so that this could have been conducted. through this study, the researcher would like to give acknowledgment to the school that had allowed for the research to take place. secondly, for those who had given supports to help provide information, guidance and so on by the judges, lecturers, university staffs, and many others that are not mentioned. references anderson, l. w., krathwohl, d. r., airasian, p. w., cruikshank, k. a., mayer, r. e., pintrich, p. r., ... & wittrock, m. c. (2001). a taxonomy for learning, teaching, and assessing: a revision of bloom’s taxonomy of educational objectives, abridged edition. white plains, ny: longman. cholisoh, l., s. fatimah, & f. yuniasih. (2015). critical thinking skills in integrated science learning viewed from learning motivation. jurnal pendidikan fisika indonesia, 11(2), 134-141. clark, donald. (2015, january 12). bloom's taxonomy of learning domains: the cognitive domain. retrieved on november 9, 2016, from http://www.nwlink.com/~donclark/hrd/bloom.html fraenkel, j. r., wallen, n. e., & hyun, h. h. (2011). how to design and evaluate research in education. new york: mc. graw hill hake, r.r. (1999). analyzing change/gain scores. dept. of physics, indiana university 24245 hatteras street, woodland hills, ca, 91367 usa. hamalik, o. 2009. psikologi belajar dan mengajar (6th edition). bandung: pt grasindo. hilario, j.s. (2015). the use of predict-observe-explain-explore (poee) as a new teaching strategy in general chemistrylaboratory. international journal of education and research, 2(3), 38-39. indonesian ministry of education and culture. (2015). republic of indonesia government regulation no. 19 year 2005 about national standard of education. jakarta: kemendikbud. johnson, e. b. (2010). menjadikan kegiatan belajar-mengajar mengasyikkan dan bermakna (indonesian translation from the original contextual teaching & learning: what it is and why it’s hero to stay). bandung: penerbit kaifa. kala, n., yaman, f. & ayas, a. (2012). the effectiveness of predict– observe–explain technique in probing students’ understanding about acid-base chemistry: a case for the concepts of ph, poh, and strength. international journal of science and mathematics education, 11(1), 570-571. kibirige, i., osodo, j. & tlala, k.m. (2014). the effect of predictobserve-explain strategy on learners’ misconceptions about dissolved salts. mediterranean journal of social sciences, 5(4), 301-304. nana, sajidan, akhyar, m., & rochsantiningsih, d. (2014). the development of predict, observe, explain, elaborate, write, and evaluate (poe2we) learning model in physics learning at senior secondary school. journal of education and practice, 5(19), 5758. paul, r.w., & elder, l. (2009). the miniature guide to critical thinking concepts & tools (6th edition). ca: the foundation for critical thinking. pratama (2012) hubungan antara kecenderungan berpikir kritis dengan indeks prestasi kumulatif (ipk) mahasiswa prodi dokter fk undip. report of research. universitas diponegoro, semarang. slavin (1997). educational psychology theory and practice, fifth edition. boston: allin and bacon. ulfah, q.n., asim & parno (2014). penerapan model pembelajaran poe (predict-observe-explain) untuk meningkatkan kemampuan berpikir kritis dan motivasi belajar siswa kelas x-mia 4 sma n 6 malang dalam materi fisika kalor. universitas negeri malang, malang. wardani, t. b., & winarno, n. (2017). using inquiry-based laboratory activities in lights and optics topic to improve students' understanding about nature of science (nos). journal of science learning, 1(1), 28-35. wardatun, h., dwiastuti, s. & karyanto, puguh (2015). the influence of predict observe explain write towards of critical thinking skills and in x degree sma n 2 sukoharjo in academic year on 2012/2013. universitas sebelas maret, surakarta. white, r., & gunstone, r. (2014). prediction-observation-explanation. probing understanding, 4. routledge, 2014. wijaya, c. (2007). pendidikan re-medial sarana pengembangan mutu sumber daya manusia. bandung: remaja rosdakarya. zulaeha, i.w., darmadi & k. werdhiana (2014). pengaruh model pembelajaran predict, observe and explain terhadap keterampilan proses sains siswa kelas x sma negeri 1 balaesang. universitas tadulako, palu. a © 2020 indonesian society for science educator 149 j.sci.learn.2020.3(3).149-155 received: 02 march 2020 revised: 27 june 2020 published: 28 july 2020 exploring prospective teachers' reasons for choosing general science as a specialization edson boy r. manalansan1, marjorie a. fogata1, danilo v. rogayan jr.1* 1president ramon magsaysay state university, san marcelino, zambales, philippines *corresponding author danrogayan@prmsu.edu.ph abstract choosing a career path is difficult for students, especially in their transition from senior high school to college. students struggle to select the most viable program that suits their interests, skills, and passion. hence, most students end up shifting among courses and, at worst, dropping out of their program. this qualitative study explores the reasons prospective teachers specialize in general science in a teacher education program. the data were gathered through interviews and were treated through thematic analysis. eight themes emerged including (1) alignment to chosen senior high school strand; (2) personal choice and interest; (3) passion for science; (4) personal knowledge and skills; (5) inspiration by teachers; (6) encouragement from family; (7) challenge to oneself; and (8) non-availability of a preferred course. the study has established important implications for admission policies in terms of the selection process for students enrolling in the general science program. higher education institutions (heis) should support the continuous improvement of the science education curriculum, campus and physical facilities, and student services, which are at the heart of education in a volatile, uncertain, complex and ambiguous (vuca) world. keywords career choice, education in vuca world, general science specialization, prospective teachers, science education 1. introduction choosing a career path is a difficult decision to make for most of the students who are in transition from senior high school to college. students are confronted with dilemmas about choosing what degree to pursue in college (fouad, ghosh, chang, figueiredo, & bachhuber, 2016; hacker, carr, abrams, & brown, 2013; lent, ezeofor, morrison, penn, & ireland, 2016). also, they are struggling with what university to enroll in that caters to their interests and needs. in other words, choosing a career to pursue in college remains to be a challenge, especially in the volatile, uncertain, complex, and ambiguous (vuca) world. the education in the modern world faces compounded uncertainties due to globalization, and extensive and contemporary use of technologies (morales, 2019), thus it is imperative to develop among students the passion and love for learning in a vuca world. vuca serves as the new normal in education, which describes the chaotic, turbulent, rapidly changing education environment (morales, 2019; waller, lemoine, mense, garretson, richardson, 2019). with this change in the educational landscape, it is imperative to guide the students on their career paths. making a career choice is a significant phase in every student's life. students must consider several factors before arriving at a decision. the influence of several factors such as parental education, profession, and income on the career (kazi & aklaq, 2017), demand of parents, mothers, in particular, are more influential in the career choice of the secondary school students compared to their peers (halim, rahman, zamri, & mohtar, 2018; hashim & embong, 2015). the reconceptualizing of science could affect the students' attitudes towards the said subject because it will be a new topic for them to understand (erduran & dagher, 2014). students' attitude towards science is also one influencing factor that can lead them to choose science as a career (razali, talib, manaf, & hassan, 2018). science teaching is one of the many careers that students can pursue in college. this profession requires a heart and a soul because those who would want to enter this job must possess the burning desire to teach (rogayan, 2018) not only the science content but also scientific attitudes and scientific values. furthermore, science mailto:danrogayan@prmsu.edu.ph journal of science learning article doi: 10.17509/jsl.v3i3.23493 150 j.sci.learn.2020.3(3).149-155 teachers showed some major influence on students' motivation towards the subject and eventually in their employment in science-related careers (bohndick, kohlmeyer, & buhl, 2017; du preez, 2018). teachers influence students through the following ways: experiences in science classrooms, extracurricular activities initiated by science departments, and information about the content of courses, and strategies for coping with advanced studies (munro & elsom, 2016). relatedly, krajcik & czerniak (2014) mentioned that teachers as the key to the doors of the students' careers also affect the reason of the students in choosing their careers. in the philippines, researches on teaching career mainly focused on preservice teachers' teaching perspectives and motivations (abulon, 2012; abulon & rungduin, 2015; lopez, & irene, 2017), high school students' motivation to pursue teaching career (mangaoil, rungduin, abulencia, & reyes, 2017), preservice teachers' attitude towards teaching and work values (torres, ballado, & catarman, 2014), and wastage in teacher preparation investments (abulon et al., 2015). likewise, local studies on students' career paths in science teaching are minimal. previous studies have focused on the driving forces of graduating college students' choice in specializing science (dela fuente, 2019), gender inequality as an influence in the students' interest in pursuing a career in science teaching (morales, avila, & espinosa, 2016), and developing students' attitude leading towards a life-changing career (laguador, 2013). the present study explores first-year college students' reasons for choosing general science as a specialization in a teacher education program. despite the offering of science education as an available degree program in colleges and universities, students who pursue this degree remains limited. ambag (2018) claimed that there is a shortage of science and math teachers in the philippines. in 2016, the department of education (deped) philippines stressed the need to hire more teachers with the implementation of the k to 12 programs. further, the government tried to solve the shortage by offering teaching posts with above entry-level salary grades to science and technology graduates under the junior level science scholarships of the department of science and technology-science education institute (dost-sei). deped also called for graduates of science, math, or engineering courses without license certification, as well accredited technical-vocational graduates with expertise in specific learning areas, to apply as part-time teachers for the k to 12 senior high school program (ambag, 2018). teaching science has always been a challenge for many since it requires rich content knowledge and innovative pedagogical knowledge. moreover, the scientific background of the facilitators of learning must also be in place. these are some encumbrances that hamper senior high school students to major in science. data on students' preferred college courses revealed an inclination towards actuarial sciences, engineering, architecture, computerbased, and management-related courses (ched, 2016). due to the dearth of local studies conducted on the students' reasons for specializing in general science teaching prompted the researchers to conduct this study. the study explored prospective teachers' reasons for choosing general science as their specialization in a teacher education program. it further described the factors and driving forces that influenced these teacher education students to pursue a career in science education. 2. method 2.1. research design the study adopted a qualitative approach in research to explore the reasons for students in specializing in general science using an in-depth structured interview guide. the qualitative approach of research relies on text and image data and has different steps in data analysis, and draws on diverse designs (creswell, 2014). qualitative researchers build their patterns, categories, and themes from the bottom up by organizing the data into increasingly more abstract units of information. this inductive process illustrates working back and forth between the themes and the database until the researchers have established a comprehensive set of themes. then deductively, the researchers look back at their data from the themes to determine if more evidence can support each theme or whether they need to gather additional information. thus, while the process begins inductively, deductive thinking also plays a vital role as the analysis moves forward (creswell, 2014). 2.2. research site and participants the study was conducted in a state-owned university in central luzon, philippines, for the academic year 20182019. the researchers selected the locale as a research site for it offers bachelor of secondary education (bsed) program major in general science under the new teacher education curriculum in the philippines. the participants of this study were first-year college students who are taking bsed major in general science at the time of the study. the participants were chosen through purposive-comprehensive sampling. purposivecomprehensive sampling refers to a sampling plan where the sampling is carried out in the criteria. purposivecomprehensive sampling can be an easy form of sampling because it is a type of sampling that is widely used in qualitative research for the identification and selection of information-rich cases related to the phenomenon of interest (palinkas, green, wisdom, & hoagwood, 2015). the researchers involved all the 13 first year bsed general science students, six males, and seven females. the researchers only involved 13 participants because they already have reached the data saturation in the study. faulkner and trotter (2017) defined data saturation as the point in the research process when no new information is journal of science learning article doi: 10.17509/jsl.v3i3.23493 151 j.sci.learn.2020.3(3).149-155 discovered in data analysis and this redundancy signals to researchers that data collection may cease. saturation means that a researcher can be reasonably assured that further data collection would yield similar results and serve to confirm emerging themes and conclusions. the participants met selection criteria: (a) first-year students, (b) taking up bachelor of secondary education major in general science, and (c) bonafide students of the university, and (d) currently enrolled in academic year 2018-2019. 2.3. data collection tools the structured interview guide served as the main instrument in gathering the data. it is composed of two parts. the first part consists of questions about the students' demographic profiles like age, sex, and religion. the second part consists of an open-ended question regarding the prospective teachers' reasons for choosing general science as specialization. sample follow-up questions were asked during the interview: (a) what is the attitude of the prospective teachers towards science?; (b) who influenced prospective teachers to enroll in science education program; (c) what are the characteristics of students in a science-related program; (d) why do prospective teachers choose science instead of choosing other fields of specialization; and (e) what career path do the prospective teachers plan to take. the tool was subjected to content and construct validity. experts in language and qualitative research were tapped to check the tool in terms of content, grammar, and coherence to the purpose of the qualitative study. after this, a pilot interview of the tool was made by the researchers. three non-participants took the pilot interview to identify some potential problems in the conduct of the interview. some improvement in the tool was made based on the problems encountered in the pilot interview. 2.4. data collection and analysis in phase 1 of the study, the researchers developed an interview guide that was validated by the experts. in the second phase, the researchers secured the permission and the approval from the campus director and the college program chair, before conducting the study. assents were also secured by the participants. the next phase was the conduct of the interview to the students taking up bsed major in general science. the interview was held in the students' classroom. the interview was conducted for about half an hour per participant. participants' assent was secured before the interview was done. the recorded interviews were individually transcribed to arrive at an extended text. transcription of the tape recordings was done as soon as the interview concluded. in analyzing the qualitative data, the researchers assigned broad code to the transcribed responses, which served as a basis for the identification of the emerging themes. the themes that emerged in the study were further subjected to member checking procedures via an informal interview. through the informal interview, the researchers assured the trustworthiness and truthfulness of the data collected. using qualitative methodology indicates an emphasis on the qualities of entities and on processes and meanings that are not experimentally tested or measured in terms of quantity, amount, or intensity (labaree, 2009). when one examines and judges accomplishments and effectiveness, one is engaged in evaluation. when this examination of effectiveness is conducted systematically and empirically through careful data collection and thoughtful analysis, one is engaged in evaluation research (patton, 1990). validation of data was conducted to establish reliability. two external reviewers checked the interview transcripts for validity and accuracy. informant feedback was likewise done to make sure the consistency of the findings. the primary role of the researchers in this study is data gatherer. the researchers served as interviewers to the participants, and they also did an in-depth analysis of the interview transcripts. 3. result and discussion eight essential themes emerged from the analysis of interview transcripts. these themes served as a guide in the identification of the reasons of the prospective teachers in specializing in a general science under the teacher education program (table 1). 3.1. alignment to chosen senior high school (shs) strand some of the participants considered their senior high school track in their choice of a college degree. they cited that they wanted to enhance and explore their previous knowledge about science by taking general science as their major or specialization. one of the participants stated: "i took up stem when i was in senior high school, so i decided to take up general science because it is inclined to that, and i think it's practical (p6)." based on the findings, students have considered their shs strand in the degree to pursue higher education. the alignment of senior high school strand to the degree in college is critical because students have been equipped with the prerequisite knowledge and skills in the program they will enroll in. this implies that several kinds of hands-on science, technology, engineering and mathematics (stem) engagement activities are likely to foster or maintain positive stem dispositions at the middle school and high school levels and that these highly positive levels of dispositions can be viewed as a target toward which projects seeking to interest mainstream secondary students in stem majors in college and stem careers, can hope to aspire. (christensen, knezek & tyler-wood, 2015). journal of science learning article doi: 10.17509/jsl.v3i3.23493 152 j.sci.learn.2020.3(3).149-155 3.2. personal choice another reason for the prospective teachers in specializing in general science is a personal choice. they stated that they enrolled in the program based on their own decision, belief, passion, curiosity, excitement, among others. in other words, the participants' choice of the program was based on their perspective. "nobody influenced me to enroll this course; it's just my interest and passion in life," said one of the prospective teachers. the prospective teachers commonly mentioned that they personally chose general science as their specialization. it implies that they are not forced to enroll in the said course. another participant shared, "i chose science because it's good to teach like math and english and because i really like science ever since i was a kid. so, i'll finish what i started." the personal choice of the participants in choosing general science as a major in college can be rooted in their pleasurable experiences when they were kids, as well as their positive experiences in the basic education program. one participant expressed, "i’m not going to choose science because it’s not my favorite subject, but when i think further, i decided to take up science.” in this response, the participant’s decision to take science is out of the picture, but when he thought about it deeper, he ended up choosing science as his major. according to wang and degol (2016), the driving force to enroll in a science-related program is current knowledge. if the students have prior knowledge in science, know how to do some investigation and explore things, know the rules and laws, and the future goals to attain, it’s where the science student is motivated to pursue a science-related program. 3.3 passion for science the love and passion for science is another reason given by prospective teachers. one of them said that she really wanted science, and she is passionate about learning more about the field. this implies that a strong desire of a person can be a significant factor in pursuing a specific career. if they are passionate about a certain area, they always find a way to pursue what they are passionate about. relatedly, enthusiasm for science can be considered as a driving force in pursuing a science career (concannon & grenon, 2016; fernet, lavigne, vallerand, & austin, 2014; insall, 2018). 3.4 personal knowledge and skills one of the participants said that “i choose science because, for me, it’s easier than the other fields of specialization.” this statement suggests that prospective teachers should consider their capabilities, knowledge, and skills in selecting a degree to take in college. they make sure that they have acquired the necessary knowledge and skills to be able to table 1 eight essential themes emerged from the analysis of interview transcripts theme significant statement frequency (n=13) theme description 1. alignment to chosen senior high school strand i took up stem when i was in senior high school, so i decided to take up general science because it is inclined to that and for it to be efficient. [p6, female] 5 this deals with the congruence of the senior high school curriculum to the current science education program that students enrolled in. 2. personal choice nobody influenced me to enroll in this course, and it’s just my interest and passion in life. [p1, male] 4 this focuses on the participants’ selection of the program based on their own perspective. 3. passion for science this[science] is my favorite; i wanted science, and i think this is my passion. [p9, female] 1 this pertains to the love of the students for science and its nature. 4. personal knowledge and skills i chose science because for me it’s easier than the other fields of specialization. [p1, male] 3 this deals with the students’ capabilities in handling science education course. 5. inspired by teachers i took up science because of my teachers when i was in senior high school. our teachers are very professional. [p3, male] 5 this pertains to the influence of teachers and mentors in the basic education to the students’ choice of career. 6. inspired by family i am influenced by my family to take up science teaching because it is in demand, and it’s easy. [p6, female] 4 this theme focuses on the influence of family members on the students’ selection of college degree. 7. to challenge oneself i want to challenge myself to explore more. [p3, male] 11 this describes how students challenge themselves in the career they choose. 8. non-availability of the preferred course we don’t have any other courses at the university, so my second choice is to explore other courses. [p4, female] 1 this theme deals with the nonoffering of the preferred degree programs of the students. journal of science learning article doi: 10.17509/jsl.v3i3.23493 153 j.sci.learn.2020.3(3).149-155 pass their chosen specializations. this is consistent with the findings of several studies that personal knowledge and skills are influencing factors in one’s choice of specialization (mupinga, & caniglia, 2019; salonen, hartikainen-ahia, hense, scheersoi, & keinonen, 2017; salonen, karkkainen, & keinonen, 2018; stipanovic, stringfield, & witherell, 2017). 3.5 inspired by teachers half of the prospective teachers stated that they were influenced by their teachers during high school. they described their positive experiences under the tutelage of their innovative and inspiring teachers. in the case of one participant, she was inspired to take up general science because of her teacher in high school. she stated, “i was inspired by my teachers when i was in senior high school. our teachers are professional.” in this statement, it is very evident that teachers assume a big role in motivating students in their career choice. the teachers teaching style and strategy contributed to the development of students’ interest to venture science (dela fuente, 2019). several studies support the findings of the present study that teachers are one of the many factors why students choose a sciencerelated career path like science teaching (mau, & li, 2018; sorgo et al., 2018; van rooij, fokkens-bruinsma, & goedhart, 2019; wilhelm, 2010). 3.6 inspired by family the prospective teachers expressed that they were influenced by their families and relatives to pursue science teaching because it is in demand. they also cited that the program is easy for them. “my mom told me to take science. she said that a science major is very rare in schools. so, i decided to take a science major, and i think it is in-demand.” this implies that the family has a strong impact on the choice of students specializing in general science. dela fuente (2019) averred that students have direct connections with parents and relatives due to the filipino culture of “close family ties.” the environment affects students’ career paths. they commonly have a significant other that they look up and they want to be like them in the future. parents have a significant role for their children in choosing their careers. although, based on the interview, in some instances, parents dictate what career their child should take in college. parents have been very influential individuals in the career choice of the students (kazi & akhlaq, 2017; lee, lee, & dopson, 2019; muenks, peterson, green, kolvoord, & uttal, 2020; xing, & rojewski, 2018). 3.7 to challenge oneself to challenge oneself is one of the reasons why students specialize in general science. one participant shared that he wants to challenge himself to know his knowledge about the subject. he said, “i want to challenge myself and to explore more.” this suggests that students specialize in the field that they see as challenging and worth exploring, like science. hippman and davis (2016) argued that students should challenge themselves not only in terms of taking steps to achieve what they want in their career but also in their own negative self-talk. hence, science teacher educators may consider providing challenging learning tasks for prospective teachers that can be helpful in their future career in science teaching. 3.8 non-availability of the preferred course one of the prospective teachers expressed that he just enrolled in this field because his desire is not available in the university. his response, “first, we don’t have any other courses offered in the university, so my second choice is to explore other courses. this implies that students specialize in general science because they have no other choices. heis may consider offering degree programs that are aligned to the needs of the industry and responsive to the service areas. the schools should also offer other stem-related programs apart from science education as the country needs more stem professionals and more scientists and researchers. a conceptual framework of students’ reasons for specializing in general science in a teacher education program was crystallized, which may be a basis in further investigations (figure 1). eight significant themes ultimately emerged to describe the prospective teachers’ reasons to specialize in science. they specialized in general science because it is in alignment with chosen senior high school strand; a personal choice and interest; passion for science; personal knowledge and skills; inspired by teachers; inspired by family; to challenge oneself; and nonavailability of the preferred course in the university. a multitude of factors has influenced the students in taking general science as their major in college. salient findings in this study include the students’ indication of the alignment of senior high school strands in their degree to pursue in college. interestingly, the passion for science also surfaced in the findings of the study as one reason for students specializing in the field. to challenge oneself is figure 1 students’ reason for specializing in general science journal of science learning article doi: 10.17509/jsl.v3i3.23493 154 j.sci.learn.2020.3(3).149-155 another salient finding of this qualitative investigation suggesting that students also try to go out of their comfort zone and challenge themselves to try courses that they see as challenging. hence, the crystallized conceptual framework must be used as a basis for teacher education institutions (teis) to craft policies and programs, specifically in their admission and retention mechanisms. it can also serve as a significant input in the schools’ career guidance programs and career interventions, specifically in embracing the vuca world, the new normal in education. 4. conclusion the rich narrative data obtained in the study showed significant and positive reasons why prospective teachers enrolled in a teacher education program and specialized in general science. implications of the findings to the university’s admission policies in terms of the selection process of students in the program were established in the study. in terms of policies, freshmen students who wish to enroll under the bsed general science program may be given an intensive interview with emphasis on their motivations, interests, attitude towards science and skills. the university administrators may further review and reformulate the admission and retention policies to better select students in the science teacher education program. furthermore, a review and updating of the science education curriculum, enhancement of the university’s plant and physical facilities, as well as improving the student services may be considered by higher education institutions (heis). heis must ensure the relevance of the program outcomes in the new industrial era in which machine intelligence and disruptive technologies are transforming the educational setting (reusia, rogayan & andres, 2020). continuing professional development programs for science faculty members may also be strengthened at the heart of education in a volatile, uncertain, complex and ambiguous (vuca) world. since the study is purely qualitative with only 13 participants, findings were solely dependent on the participants’ responses and experiences. hence, a small number of participants limits the results. further research may use a mixed-method research design to elucidate further prospective teachers’ reasons for taking general science as a field of specialization under the teacher education program. other samples from different regional contexts may also be considered since the present study only focused on the central luzon area of the philippines. acknowledgments the authors would like to acknowledge the bsed general science students for the academic year 2018-2019 for taking part in this study, to dr. ferdinand v. tamoria, dr. luz n. corpuz, and miss kareen joy b. manglicmot of president ramon magsaysay state university (prmsu) for the constructive criticisms in this research article; and to the editors and anonymous peer reviewers for the valuable comments in improving the paper. references abulon, e. l. r., orleans, a. v., bedural, z. l., david, a. p., florentino, j. v., & rungduin, t. t. (2015). exploring wastage in teacher preparation investments in the philippines. the normal lights, 8(2), 8-30. abulon, e. l., & rungduin, t. t., (2015). a narrative analysis of conversations with graduates who did not pursue teaching: inputs to teacher education policies in the philippines. international journal of research studies in education, 4(2), 13-27. https://doi.org/10.5861/ijrse.2014.966 abulon, e.l.r. (2012). preservice teachers’ motivation related to career choice: the case of pnu beced and beed students. the normal lights, 6 (1), 68-79. ambag, r., (2018). teaching science in the philippines: why (and how) we can do better. https://www.flipscience.ph.news/featuresnews/features/teaching-science-philippines bohndick, c., kohlmeyer, s., & buhl, h. m. (2017). competencies and career choice motives: characteristics of high school students interested in teacher education programmes. journal of education for teaching, 43(5), 566-580. https://doi.org/10.1080/02607476.2017.1355029 christensen, r., knezek, g., & tyler-wood, t. (2015). alignment of hands-on stem engagement activities with positive stem dispositions in secondary school students. journal of science education and technology, 24(6), 898-909. https://doi.org/10.1007/s10956015-9572-6 commission on higher education (ched). (2016). higher education facts and figures. https://ched.gov.ph/2016-higher-educationfacts-figures concannon, c. & grenon, m., (2016). researchers: share your passion for science! biochemical society transactions, 44(5), 1507-1515. https://doi.org/10.1042/bst20160086 creswell, j.w. (2014). research design: qualitative, quantitative and mixed methods approaches (4th ed.). singapore: sage. dela fuente, j. a. (2019). driving forces of students’ choice in specializing science: a science education context in the philippines perspective. the normal lights, 13(2), 225-250. du preez, m. (2018). the factors influencing mathematics students to choose teaching as a career. south african journal of education, 38(2), 1-13. https://doi.org/10.15700/saje.v38n2a1465 erduran, s. & dagher, z.r. (2014). reconceptualizing the nature of science for science education contemporary. netherlands: springer netherlands. https://doi.org/10.1007/978-94-017-9057-4 faulkner, s. l., & trotter, s. p. (2017). data saturation. the international encyclopedia of communication research methods, 1-2. https://doi.org/10.1002/9781118901731.iecrm0060 fernet, c., lavigne, g. l., vallerand, r. j., & austin, s. (2014). fired up with passion: investigating how job autonomy and passion predict burnout at career start in teachers. work & stress, 28(3), 270-288. https://doi.org/10.1080/02678373.2014.935524 fouad, n. a., ghosh, a., chang, w. h., figueiredo, c., & bachhuber, t. (2016). career exploration among college students. journal of college student development, 57(4), 460-464. https://doi.org/10.1353/csd.2016.0047 hacker, j., carr, a., abrams, m., & brown, s. d. (2013). development of the career indecision profile: factor structure, reliability, and validity. journal of career assessment, 21(1), 32–41. https://doi.org/10.1177%2f1069072712453832 halim, l., abd rahman, n., zamri, r., & mohtar, l. (2018). the roles of parents in cultivating children's interest towards science learning https://doi.org/10.5861/ijrse.2014.966 https://www.flipscience.ph.news/features-news/features/teaching-science-philippines https://www.flipscience.ph.news/features-news/features/teaching-science-philippines https://doi.org/10.1080/02607476.2017.1355029 https://link.springer.com/article/10.1007/s10956-015-9572-6 https://link.springer.com/article/10.1007/s10956-015-9572-6 https://ched.gov.ph/2016-higher-education-facts-figures https://ched.gov.ph/2016-higher-education-facts-figures https://doi.org/10.1042/bst20160086 https://doi.org/10.15700/saje.v38n2a1465 https://doi.org/10.1007/978-94-017-9057-4 https://doi.org/10.1002/9781118901731.iecrm0060 https://doi.org/10.1080/02678373.2014.935524 https://doi.org/10.1353/csd.2016.0047 https://doi.org/10.1177%2f1069072712453832 journal of science learning article doi: 10.17509/jsl.v3i3.23493 155 j.sci.learn.2020.3(3).149-155 and careers. kasetsart journal of social sciences, 39(2), 190-196. https://doi.org/10.1016/j.kjss.2017.05.001 hashim, h. m. & embong, a. m., (2015). parental and peer influences upon accounting as a subject and accounting as a career. journal of economics, business and management, 3(2), 252-256. https://doi.org/10.7763/joebm.2015.v3.189 hippman, c., & davis, c. (2016). put yourself at the helm: charting new territory, correcting course, and weathering the storm of career trajectories. journal of genetic counseling, 25(4), 720-730. https://doi.org/10.1007/s10897-016-9936-y insall, r. (2018). science careers-improve your effectiveness but keep your passion. current biology, 28(18), r1078-r1079. https://doi.org/10.1016/j.cub.2018.07.079 kazi, a. s., & akhlaq, a. (2017). factors affecting students’ career choice. journal of research and reflections in education, 11(2), 187-196. krajcik, j. s., & czerniak, c. m. (2014). teaching science in elementary and middle school: a project-based approach. routledge. https://doi.org/10.4324/9780203113660 labaree, r. v. (2009). research guides: organizing your social sciences research paper: types of research designs. http://libguides.usc.edu/writingguide/researchdesigns laguador, j. m. (2013). developing students’ attitude leading towards a life-changing career. educational research international, 1(3), 28-33. lee, p. c., lee, m. j., & dopson, l. r. (2019). who influences college students’ career choices? an empirical study of hospitality management students. journal of hospitality & tourism education, 31(2), 74-86. https://doi.org/10.1080/10963758.2018.1485497 lent, r. w., ezeofor, i., morrison, m. a., penn, l. t., & ireland, g. w. (2016). applying the social cognitive model of career selfmanagement to career exploration and decision-making. journal of vocational behavior, 93, 47-57. https://doi.org/10.1016/j.jvb.2015.12.007 lopez, n. l., & irene, e. a. (2017). motivation and commitment to teaching among preservice teachers of a state university in samar, philippines. journal of academic research, 2(3), 18-29. mangaoil, a. b., rungduin, t. t., abulencia, a. s., & reyes, w. m., (2017). why i want to teach: exploring factors affecting students’ career choice to become teachers. the normal lights, 11(2), 236263. mau, w. c. j., & li, j. (2018). factors influencing stem career aspirations of underrepresented high school students. the career development quarterly, 66(3), 246-258. https://doi.org/10.1002/cdq.12146 morales, m. p. e., avilla, r. a., & espinosa, a. a. (2016). does gender inequality influence interest in pursuing a career in science or mathematics teaching? issues in educational research, 26(1), 65-81. morales, m.p.e. (2019). education in a vuca (volatile, uncertain, complex, ambiguous) world. the normal lights, 13(2), v-x. muenks, k., peterson, e. g., green, a. e., kolvoord, r. a., & uttal, d. h. (2020). parents’ beliefs about high school students’ spatial abilities: gender differences and associations with parent encouragement to pursue a stem career and students’ stem career intentions. sex roles, 82, 570-583. https://doi.org/10.1007/s11199-019-01072-6 munro, m. & elsom, d. (2016). choosing science at 16: the influence of science teachers and careers advisers on students’ decisions about science subjects and science and technology careers. cambridge: nicec briefing. mupinga, d., & caniglia, j. (2019). what middle school students know about careers and the influences surrounding their choices. journal of technology studies, 45(1), 36-45. palinkas, l. a., green, c. a., wisdom, j. p., & hoagwood, k.e. (2015). purposeful sampling for qualitative data collection and analysis in mixed method implementation research. administration and policy in mental health and mental health services research (5), 533-44. https://doi.org/10.1007/s10488-013-0528-y patton, m. q., (1990). qualitative evaluation and research methods (2nd ed.). beverly hills, ca: sage publications, inc. razali, f., talib, o., abd manaf, u. k., & hassan, s. a. (2018). students attitude towards science, technology, engineering and mathematics in developing career aspiration. international journal of academic research in business and social sciences, 8(5), 946-960 https://doi.org/10.6007/ijarbss/v8-i5/4242 reusia, d.h.r., rogayan, d.v. jr., & andres, k.p. (2020). science education graduates of a state university from 2008-2018: a tracer study. the normal lights, 14(1), 56-79. rogayan, d. v. jr. (2018). why young filipino teachers teach?. asia pacific higher education research journal, 5(2), 48-60. salonen, a., hartikainen-ahia, a., hense, j., scheersoi, a., & keinonen, t. (2017). secondary school students’ perceptions of working life skills in science-related careers. international journal of science education, 39(10), 1339-1352. https://doi.org/10.1080/09500693.2017.1330575 salonen, a., karkkainen, s., & keinonen, t. (2018). career-related instruction promoting students’ career awareness and interest towards science learning. chemistry education research and practice, 19(2), 474-483. https://doi.org/10.1039/c7rp00221a sorgo, a., dojer, b., golob, n., repnik, r., repolusk, s., pesek, i., ... & spur, n. (2018). opinions about stem content and classroom experiences as predictors of upper secondary school students' career aspirations to become researchers or teachers. journal of research in science teaching, 55(10), 1448-1468. https://doi.org/10.1002/tea.21462 stipanovic, n., stringfield, s., & witherell, e. (2017). the influence of a career pathways model and career counseling on students' career and academic self-efficacy. peabody journal of education, 92(2), 209221. https://doi.org/10.1080/0161956x.2017.1302217 torres, t. c., ballado, r. s., & catarman, n. s. (2014). attitudes towards teaching and work values of preservice teacher education students. international journal of education and research, 2(12), 33-42. van rooij, e. c. m., fokkens-bruinsma, m., & goedhart, m. (2019). preparing science undergraduates for a teaching career: sources of their teacher self-efficacy. the teacher educator, 54(3), 270-294. https://doi.org/10.1080/08878730.2019.1606374 waller, r. e., lemoine, p. a. , mense, e. g., garretson, c. j., & richardson, m. d. (2019). global higher education in a vuca world: concerns and projections. journal of education and development, 3(2). https://doi.org/10.20849/jed.v3i2.613 wang, m. t., & degol, j. l. (2015). school climate: a review of the construct, measurement, and impact on student outcomes. educational psychology review, 28(28), 315-352. https://doi.org /10.1007/s10648-015-9319-1 wilhelm, w.b. (2010). the relative influence of published teaching evaluation and other instructor attributes on course choice. journal of marketing education, 26(1), 17-30. https://doi.org/10.1177/0273475303258276 xing, x., & rojewski, j. w. (2018). family influences on career decisionmaking self-efficacy of chinese secondary vocational students. new waves-educational research and development journal, 21(1), 48-67. https://doi.org/10.1016/j.kjss.2017.05.001 https://doi.org/10.7763/joebm.2015.v3.189 https://doi.org/10.1007/s10897-016-9936-y https://doi.org/10.1016/j.cub.2018.07.079 https://doi.org/10.4324/9780203113660 http://libguides.usc.edu/writingguide/researchdesigns https://doi.org/10.1080/10963758.2018.1485497 https://doi.org/10.1016/j.jvb.2015.12.007 https://doi.org/10.1002/cdq.12146 https://doi.org/10.1007/s11199-019-01072-6 https://doi.org/10.1007/s10488-013-0528-y https://doi.org/10.6007/ijarbss/v8-i5/4242 https://doi.org/10.1080/09500693.2017.1330575 https://doi.org/10.1039/c7rp00221a https://doi.org/10.1002/tea.21462 https://doi.org/10.1080/0161956x.2017.1302217 https://doi.org/10.1080/08878730.2019.1606374 https://doi.org/10.20849/jed.v3i2.613 https://link.springer.com/article/10.1007/s10648-015-9319-1 https://link.springer.com/article/10.1007/s10648-015-9319-1 https://doi.org/10.1177/0273475303258276 a © 2019 indonesian society for science educator 79 j.sci.learn.2019.2(3).79-84 received: 27 january 2019 revised: 28 may 2019 published: 22 july 2019 the effect of guided inquiry laboratory activity with video embedded on students’ understanding and motivation in learning light and optics tina afriani1, rika rafikah agustin1, eliyawati1* 1international program on science education, faculty of mathematics and science education, universitas pendidikan indonesia, bandung, indonesia *corresponding author. eliyawati@upi.edu abstract this research aims to investigate the effect of guided inquiry laboratory activity with video embedded on students’ understanding and students’ motivation in learning lights and optics topic. the method used in this research was pre-experiment. the sampling technique used in this research was convenience sampling, and the samples were taken from grade 8 in one of junior high school in bandung. the sample was 20 students. the class implemented guided inquiry laboratory activity with video embedded in learning light and optics. the students’ understanding was measured using test given at pretest and post-test while students’ motivation was calculated using software ministeps (rasch model). the t-test paired sample also was performed on the average level of 95% to identify the significant difference of students’ understanding before and after the implementation of guided inquiry laboratory activity with video embedded. the results of this research show that the use of guided inquiry laboratory with video integrated gives an improvement of students understanding. even though the value of n-gain is 0,264 (categorized as low level), the statistical test shows that there is a significant difference between students understanding before and after the implementation of guided inquiry laboratory activity with video embedded. there are 15 students from 20 students who are motivated in learning light and optics by using guided inquiry laboratory activity with video embedded. students are motivated by the implementation of guided inquiry laboratory activity with video embedded. keywords guided inquiry laboratory activity, video embedded, students’ understanding, students’ motivation, light, and optics 1. introduction physics is a branch of science that studies about natural phenomena that are physically by observation, experiment, and theory. physics is taught through learning activities in schools by a set of activities that are designed to support student learning (prima, utari, chandra, hasanah, & rusdiana 2018). the principle of learning physics is to prioritize scientific processes to produce products and to be based on scientific attitudes. the scientific process in learning physics is identical to the implementation of activity with the scientific method (sari & sunarno, 2018). there are lots of learning models, approach, and learning methods that can support the process of learning physics. based on the 2013 curriculum of indonesia, current science learning need to use a scientific approach. scientific learning is learning that adopts scientific steps in building knowledge through the scientific method (puspitasari, lesmono, & prihandono, 2015). in line with curriculum 2013, teachers must have learning support devices in the form of teaching materials, worksheets, discussion sheets, and media used to facilitate students in understanding the material (sunarya, & mudzakir, 2017; prima, putri, & sudargo, 2017). students’ learning outcomes for physics subjects are relatively still low. when learning physics, it is found that the lack of seriousness of most students in learning physics is that students do not play an active role in the class and still focus to teacher center (sari & sunarno, 2018). most students’ do not have right learning motivation and a positive attitude in learning physics (prima, putri, & rustaman, 2018). this indicated by the lack of the seriousness of students’ in learning physics and the negligence of students’ to do assignments from the teacher. in addition, there are several factors that affect students’ learning result; which are in the instrument, learning device such as curriculum, program structure, learning facilities and learning media, and teachers as learning designers. mailto:eliyawati@upi.edu journal of science learning article doi: 10.17509/jsl.v2i3.15144 80 j.sci.learn.2019.2(3).79-84 there are several ways to support the physics learning process is using media is one kind of innovative learning. by watching the video, students’ emotions are triggered by specific visual scenes, the actors, and background music. a video can have a substantial effect on students’ mind (berk, 2009). video is a combination of images and sound, create a powerful medium for an explanation of concepts while instructing learners with content that provides multiple senses (pratiwi, rochintaniawati, & agustin, 2018). laboratory activity is significant in learning science. the purpose of laboratory work is to developing students understanding related to scientific content, problemsolving skill, and science processes skills. students have to know the connection between experiment and scientific theory. by scientific inquiry, students determine the problems, developing solutions, and alternative solutions for these problems, search for information, evaluate the information, and communicate with their friends (salsabila, wijaya, & winarno, 2018). learning with guided inquiry is an effective way to vary the atmosphere of classroom teaching patterns. guided inquiry learning is group learning where students are given the opportunity to think independently and help each other with friends and guide students to have individual responsibilities and responsibilities in groups or partners (ambarsari, santosa, & maridi, 2013). in guided inquiry laboratory activity, student search for an experiment by the given problem. usually, guided inquiry experiments are based on discovery, the procedure is predetermined (kumdang, kijkuakul, & chaiyasith 2018). the use of guided inquiry laboratory activity, in which students have principal autonomy in the design and execute the experiment (fakayode, 2014). teachers should move away from traditional lecturing and cookbook-style laboratories to active learning strategies that help students to develop their cognitive processes (tessier & penniman, 2006). in the guided inquiry laboratory method, student search for an experiment by a given problem. in this activity, the experiments are similar to the expository experiments, but a lab manual is not given to the students. students search for the experiment process and reach scientific information by the test (ural, 2016a). guided inquiry laboratory enhanced students' learning. using video have an advantage which is can help students to observe the phenomena that too fast or too long happened. the use of animation video can attract students to more focus on the explanation of the concept and can saving time in explaining the concept. learning with video animation can give the opportunity to student and teacher to teaching and learning faster and combine active learning and technology. the characteristic of the video is in the form of animation with colorful and cheerful visual and audio, the source of the video is from youtube. there is a youtube channel named peek a boo that contains some educative video for any subject, peek a boo make a character that is used to explain the material named dr. binocs with the title of the video is the dr. binocs show. in this research, the topic of light and optics is chosen because based on the information from the student learning method that is used in the class still teacher center. the teacher only delivers the concept based on the handbook, and then students were asked to answer the questions from the book. and also still lack laboratory activity in learning light and optics, the teacher even uses lecturing method to delivers the concept. students are too difficult to focus and pay attention to the teacher during teaching-learning activity. it caused students learning outcome are low. previously, there are several research conducted to improve students’ understanding in light and optics topic by implementing several approach, method or models, such as virtual laboratory flash animation (permana, widiyatmoko, & taufiq, 2016), interactive module with lcds program by arbai, edie, and pamelasari (2014) and using direct instruction model by anggraini, zainuddin, and miriam (2017). however, there is no research about guided inquiry laboratory activity with video embedded in learning light and optics. such research could further suggest changes in educational practices. for this reason, this research was initiated to investigate students’ understanding and motivation by using video embedded guided inquiry laboratory activity among 8th grade students. it is thought that by using guided inquiry laboratory activity with video embedded will increase students' understanding of light and optics topics and motivate students’ in learning light and optic. such research could further suggest changes in educational practices. for this reason, this research was initiated to investigate students’ understanding and motivation by using guided inquiry laboratory activity with video embedded among 8th grade students. it is thought that by using guided inquiry laboratory activity with video embedded will increase students' understanding of light and optics topics and students’ motivations and students’ understanding of learning light and optic. 2. method the research method that was used in this research was pre-experiment. pre-experiment a type of research that uses a single subject to determine the causal relationship between the independent variable and dependent variable without any extraneous variable (anderson, & krathwohl, 2001). this method is appropriate with the purpose of this research that investigates the effect of guided inquiry laboratory activity with video embedded on students’ understanding and motivation in learning light and optics. it can determine the change of independent variable but not due to extraneous factors. the pre-experimental journal of science learning article doi: 10.17509/jsl.v2i3.15144 81 j.sci.learn.2019.2(3).79-84 method may able to approach the true experimental process (cohen, manion & morrison, 2007). the research design that was used in this research was one group pre-test and post-test as seen in table 1, which means that the researcher only takes an experimental group to measures the groups’ dependent variable (o1), that usually called as a pre-test. the pre-test was given to the subject using an instrument in the form of multiple choices. the next step was giving an experimental manipulation (x), by learning with video embedded with guided inquiry laboratory activity, before conducting post-test. the research was taken in private junior high school in bandung. the school uses bahasa indonesia in their learning activity of science. the curriculum of the junior high school is national curriculum of 2013. the population in this research was 8th-grade students’ of the public junior high school in bandung. the samples were 8th-grade students from one class consist of 20 students. consist of 12 male and 8 female with age around 14 years old. the sampling technique that was used was convenience sampling, which is a selection of the subjects that selected by the willingness of the researcher and its available to be studied (cohen, manion, & morrison, 2007). the objective test was developed to measure students’ understanding of light and optics. this test was given to the students in pre-test that was before any implementation of guided inquiry laboratory with video embedded, and post-test that was after the treatment given the students. the purpose of post-test was given to measure students’ understanding after the treatment. the type of question that was given in the pre-test and post-test are multiple-choice, contained 25 questions. all test items were judged by the experts and tested to the students’ that have learned about light and optics. to determine students’ motivation in learning light and optics with video embedded with guided inquiry laboratory activity that is used in this research, a questionnaire is developed by using 4 point likert-scale (strongly agree, agree, disagree, strongly disagree). the students’ motivation toward science learning (smtsl) was used to assess students’ motivation in learning light and optics and the students’ answer was to analyze by statistical software, namely rasch model. 3. result and discussion 3.1 the effect of guided inquiry laboratory activity with video embedded on students understanding in learning light and optics the objective test was given to the subject of the research to determine their understanding. the instrument was distributed twice, pre-test and post-test, the concept that was used are the same, which is light and optics. moreover, the score that has been obtained from all research subjects were analyzed to determine the normality of items, homogeneity test of items, also hypothesis of the research. the spss software was used to analyze the data that consist of 25 questions in the form of multiple choices. table 2 shows that the average score of post-test is higher than the pre-test. from 20 students in the pre-test, the maximum score is 55. it is proving that the students’ prior knowledge is still low. in this research, the average is too low to be categorized as a good score in line with the minimum criteria of mastery learning in the school, which is 65. there only three students who get score 50. the other scores are below 50; even the worst is 20. meanwhile, after the implementation of learning using the guided inquiry laboratory with video embedded, the maximum rating is 75. the post-test score is higher than pre-test, it is because after the post-test and re-check their answer most students can answer the questions that they cannot respond in the pre-test. this is because during teaching-learning activity students more active and more explore their activity. some students relish learning with guided inquiry laboratory activity with video embedded, it makes students easier to absorb new knowledge. it is because guided inquiry laboratory activity with video embedded improves students understanding about light and optics by involving students in a scientific atmosphere where they are allowed to try how a scientist works. during the teaching-learning process with guided inquiry laboratory activity, they are stimulated to pose scientific questions, collect and analyze the data from hypothesis, design and conduct a scientific investigation, formulate the explanation and communicate the arguments. the normality test result of pretest and posttest shows that the data are typically distributed because the result table 1 research design of one group pre-test and post-test pre-test treatment post-test o1 x o2 table 2 descriptive statistic of objective test, pre-test and posttest component pre-test post-test n average score maximum score minimum score 20 37,25 55 20 20 53,25 75 20 significant (sig 𝛼=0,05) information normality 0,134 normally distributed 0,472 normally distributed significant (sig 𝛼=0,05) information homogeneity 0,032 the data are homogen significant (sig 𝛼=0,05) information hypothesis test 0,000 h1 accepted, there is a significant difference journal of science learning article doi: 10.17509/jsl.v2i3.15144 82 j.sci.learn.2019.2(3).79-84 shows the values which are more than 0,05. the method of calculating normality using shapiro-wilk because the sample is only 20, which was a small number and this method is effective to reduce the error of normality in a few amounts (cohen, manion, & morrison, 2007). the improvement of students understanding can be indicated by the average gain score which is 16.00. the average normalized gain score is 0.264. it means that the average normalized gain score is at a medium level. the same with the normality test result, the homogeneity test result of pretest and posttest shows a value which is more than 0,05, it can be said that the data is homogeny. because the data were normal and homogeny, than it decided to compare the mean used parametric statistics which was paired sample t-test. obtained that the sig. value 2 –tailed of t-test paired sample is 0.000. compared to the significance (𝜶), which has the value 0.05, the sig. amount 2-tailed is smaller than the significance (𝜶) value. then it can be concluded that h1is accepted, which means that there is a significant difference in students understanding between pre-test and post-test using guided inquiry laboratory with video embedded in learning light and optics. in line with the other research about guided inquiry, the students in the guided-inquiry condition demonstrated significant improvements in both conceptual understanding and their levels of explaining the concept (ambarsari, santosa, & maridi, 2016). the result of implementing guided inquiry shows that there has been a significant increase in students' attitudes, and their academic achievement (ural, 2016b). 3.2 students’ motivation in learning light and optics students’ motivation in learning light and optics, in this research, determined using questionnaire using 4 – likert scale. in the questionnaire only use a 4-likert scale because the neutral option is meaningless, a neutral option does not express whether it positive or negative opinion. the neutral option was excluded as a strategy to encourage students to express either a positive or negative opinion, rather than remaining neutral. the questionnaire was given to research subject after conducting post-test. the overview of this questionnaire is the motivation of students in learning science using guided inquiry laboratory with video embedded, especially in learning light and optics topic. the racsh model was used to analyze the instrument that consists of 25 statements in the form of the questionnaire. the neutral option was excluded as a strategy to encourage students to express either a positive or negative opinion, rather than remaining neutral. the questionnaire was given to research subject after conducting post-test. the overview of this questionnaire is the motivation of students in learning science using guided inquiry laboratory with video embedded, especially in learning light and optics topic. after the questionnaire was given to the subject, then the score that has been obtained were analyzed used ministep software from racsh model to determine the profile of students motivation in learning light and optics. the racsh model was used to analyze the instrument that consists of 25 statements in the form of a questionnaire as shown in figure 1. the result shown in the form of variable maps, there is two areas in the maps which is person area and item area. person map indicates students meanwhile item indicates the students’ motivation statements. this variable map arranges the students from the students who have high motivation for the students who have low motivation in learning light and optics (topbottom). the top item indicates the hardest students’ motivation statement to approve and the bottom item indicates the easiest students’ motivation statement to approve. from the data 09l is the person who stands higher in the top than other people, that means the person with code 09l is the most person that approves with the statement in the questionnaire or 09l is the most motivated students that other subject of the research. in the result, person 09l can answer all the states in the survey even the hardest statement to be approved. then, there was 19l that stand in the bottom than other people, that is mean the person 19l is the hardest to agree with the statements in the questionnaire or 19l is the less motivated students after the treatment. the data shows there were 15 persons that more motivated than the other person, 15 persons consisted of 9 male and 6 female. and there were 2 persons that less motivated that consist of 1 male and 1 female. based on the limits of the data above, there are 3 students whose figure 1 the result of students’ motivation journal of science learning article doi: 10.17509/jsl.v2i3.15144 83 j.sci.learn.2019.2(3).79-84 learning motivation is so-so. that consists of 2 male and 1 female. on the data not only person who more motivated or less motivated, but there also obtained data about the item motivation that easiest to approve and the item motivation that hardest to approve. based on the data shows that there was an item that hardest to approve that is m17 or item number 17, which states “i participated in learning light and optics so i would look smarter than other students”. there also shows an item that easiest to approve that is m20 or item number 20, which is a state “i feel delighted when i can solve complex problems”. based on the data persons 04p, 08p, 16l, 17 l, and 01p cannot approve the statement m17 or item number 17 as stated above. then, persons 02l, 10l, and 20l cannot agree with statement m17 and m16, but they approve statement m18 and the rest of the statements. next are persons 03l, 05l, 07p, and 11p they cannot approve m17, m16, and m18 but they can approve m14 and the rest statement. contrast with 10l, 03l, 05l, 07p, and 11p recommend statement m14 that has been stated above, it may indicate they prefer to ask the answer to other as an easy way to answer the assignment rather than they think and find the answer by themselves. then, persons 14l and 18l cannot approve statement m17, m16, m18, m14, and m5 but they agree statement m12 and m13 and the rest of statements. m5 stated “i want to take part in learning light and optics because it is exciting”, they did not approve the statement may indicate learning light and optics cannot attract their interest or they not attract in learning physics. persons 12l and 13l only approve m7 and the rest of the statements and cannot approve m12 and m13, m12 stated “i believe i can do test in chapter light and optics” and m13 “if there is assignment in light and optic that have difficult part, i will pass it and try the easiest one”. it may indicate a lack of confidence in answering questions from the test or assignment. statement m7, m6, and m9 cannot be approved by person 15p, m6 stated “when i was learning a new concept in chapter light and optics i relate the concept with my experiences” and m9 “during learning light and optics i try to relate every concept i’ve learned before.” but person 15p can approve the rest of statements, this may indicate person 15p lack of previous mastering concept. person 06l cannot approve statement m11, m25, m4, m2, m23, m24, and m8, but 06l can approve m15 and m3. m15 stated, “i participated in learning light and optics to get good scores” and m3, “i think it is important to learn how to solve the problem in chapter light and optics.” it may indicate, a person during the teaching-learning process 06l try to learn how to solve a problem in chapter light and optics to get a good score in the final test. not an only variable map, there also scalogram as the result of data analyzed using rasch model. scalogram shows the response pattern systematic between students’ motivation (sorted from high to low ability vertically, top to bottom) and items’ motivation (classified from manageable to severe horizontally, from left to right) as seen in figure 2. both scalogram and variable maps. the students who have high motivation is 09l, and the students who have low motivation is 19l. there are a number of students who have unique response patterns. some students have unique patterns such as 17l, 18l, 06p, and 19l. students with 17l code, on motivation item 4 answered disagree even though for the following items that had a higher difficulty level (more difficult to agree), he actually answered strongly agree. likewise, students with code 06p, inconsistent because on motivation items 21 and 22 answered no agree, even though there should be a pattern and showing from variable maps that 06p must answer agree. the last unique thing that can be analyzed is that students with the 19l code where students are less motivated can be seen from the answers, which most patterns disagree with the statement of motivation in learning light and optics. from the data, it was obtained that guided inquiry laboratory activity with video embedded in learning light and optics can motivate students. the improvement of students motivation with guided inquiry laboratory activity can happen because students feel challenged or new things to find the concept by the experiment, then students look enthusiastic and interested during the teaching-learning process. learning with guided inquiry laboratory activity can make learning activity more meaningful for students because the experience that they get from the experiment activity can help students to relate and explain the event that connects with daily life. the experiment that was figure 2 scalogram showing the response pattern systematic between students’ motivation (sorted from high to low ability vertically, top to bottom) and items’ motivation (classified from manageable to severe horizontally, from left to right) journal of science learning article doi: 10.17509/jsl.v2i3.15144 84 j.sci.learn.2019.2(3).79-84 conducted to get the data can increase students selfconfidence. discussion activity that was conducted after the experiment activity can make students more confident for the asking and deliver their opinion. this is in line with previous research, teaching using guided inquiry can increase students motivation in learning science (yuniastuti, 2013). science laboratories in schools support and promote student motivation and learning strategy use (milner, templin, & czerniak, 2011). 4. conclusion be rooted in data analysis, the use of guided inquiry laboratory activity with video embedded gives an improvement of students understanding. even though the value of n-gain categorized as low level, the statistical test shows that there is a significant difference between students’ knowledge before and after the implementation of guided inquiry laboratory activity with video embedded. students are motivated by the implementation of guided inquiry laboratory activity with video embedded in learning light and optics. there are 15 students from 20 students who are motivated in learning light and optics by using guided inquiry laboratory activity with video embedded. almost all statements related to students motivation toward science learning can be approved by students. acknowledgment author’s wishing to acknowledge the principal of the school for the permission to conduct the research (teaching in the class for several meetings). references anderson, l. w. & krathwohl, d. (2001). taxonomy for learning, teaching, and assessing: a revision of bloom’s taxonomy of educational objectives. new york, ny: longman. almuntasheri, s., gillies, r. m., & wright, t. (2016). the effectiveness of a guided inquiry-based, teachers' professional development programme on saudi students' understanding of density. science education international, 27(1), 16-39. ambarsari, w., santosa, s., & maridi, m. (2013). penerapan pembelajaran inkuiri terbimbing terhadap keterampilan proses sains dasar pada pelajaran biologi siswa kelas viii smp negeri 7 surakarta. pendidikan biologi, 5(1). anggraini, a. n., zainuddin, & miriam, s. (2017). meningkatkan keterampilan prosedural siwa melalui model pengajaran langsung pada materi ajar cahaya dan alat-alat optik kelas viii f smpn 26 banjarmasin. berkala ilmiah pendidikan fisika, 1(3), 154–168. arbai, s. z., edie, s. s., & pamelasari, s. d. (2014). pengembangan modul ipa terpadu bermuatan mind mapping pada tema cahaya dan penglihatan untuk kelas viii smp/mts. unnes science education journal, 3(1). berk, r. a. (2009). multimedia teaching with video clips: tv, movies, youtube, and mtvu in the college classroom. international journal of technology in teaching & learning, 5(1). cohen, l., manion, l., and morrison, k. (2007). research method in edcuation sixth edition. usa: routledge. fakayode, s. o. (2014). guided-inquiry laboratory experiments in the analytical chemistry laboratory curriculum. analytical and bioanalytical chemistry, 406(5), 1267–1271. kumdang, p., kijkuakul, s., & chaiyasith, w. c. (2018). an action research on enhancing grade 10 student creative thinking skills using argument-driven inquiry model in the topic of chemical environment. journal of science learning, 2(1), 9-13. milner, a. r., templin, m. a., & czerniak, c. m. (2011). elementary science students’ motivation and learning strategy use: constructivist classroom contextual factors in a life science laboratory and a traditional classroom. journal of science teacher education, 22(2), 151–170. permana, n. a., widiyatmoko, a., & taufiq, m. (2016). pengaruh virtual laboratory berbasis flash animation terhadap pemahaman konsep dan keterampilan berpikir kritis peserta didik tema optik kelas viii smp. unnes science education journal, 5(3). pratiwi, w. n. w., rochintaniawati, d., & agustin, r. r. (2018). the effect of multiple intelligence-based learning towards students' concept mastery and interest in matter. journal of science learning, 1(2), 49-52. prima, e. c., putri, a. r., & rustaman, n. (2018). learning solar system using phet simulation to improve students' understanding and motivation. journal of science learning, 1(2), 60-70. prima, e. c., putri, c. l., & sudargo, f. (2017). applying pre and post role-plays supported by stellarium virtual observatory to improve students’ understanding on learning solar system. journal of science learning, 1(1), 1-7. prima, e. c., utari, s., chandra, d. t., hasanah, l., & rusdiana, d. (2018). heat and temperature experiment designs to support students’ conception on nature of science. jotse: journal of technology and science education, 8(4), 453-472. puspitasari, r., lesmono, a. d., & prihandono, t. (2015). pengaruh model pembelajaran poe (prediction , observation and explanation) disertai media audiovisual terhadap keterampilan kerja ilmiah dan hasil belajar siswa dalam pembelajaran ipafisika di smp. jurnal pembelajaran fisika, 4(3), 211–218. salsabila, e. r., wijaya, a. f. c., & winarno, n. (2018). improving students’ sustainability awareness through argument-driven inquiry. journal of science learning, 2(2), 58-64. sari, n., sunarno, w., & sarwanto, s. (2018). analisis motivasi belajar siswa dalam pembelajaran fisika sekolah menengah atas. jurnal pendidikan dan kebudayaan, 3(1), 17-32. schoffstall, a. m., & gaddis, b. a. (2007). incorporating guided-inquiry learning into the organic chemistry laboratory. journal of chemical education, 84(5), 848. sunarya, y., & mudzakir, a. (2017). solar cell as learning multimedia to improve students’ scientific literacy on science and nanotechnology. journal of science learning, 1(1), 36-43. tessier, j. t., & penniman, c. a. (2006). an inquiry-based laboratory design for microbial ecology. bioscene, 32(4), 6–11. ural, e. (2016a). the effect of guided-inquiry laboratory experiments on science education students’ chemistry laboratory attitudes, anxiety and achievement. journal of education and training studies, 4(4), 217–227. ural, e. (2016b). the effect of guided-inquiry laboratory experiments on science education students ’ chemistry laboratory attitudes , anxiety and achievement, 4(4), 217–227. van zile-tamsen, c. (2017). using rasch analysis to inform rating scale development. research in higher education, 58(8), 922-933. yuniastuti, e. (2013). peningkatan keterampilan proses, motivasi, dan hasil belajar biologi dengan strategi pembelajaran inkuiri terbimbing pada siswa kelas vii smp kartika v-1 balikpapan. jurnal penelitian pendidikan, 13(1). microsoft word layout 13262-31141-1-ce aulianuriy.docx a © 2019 indonesian society for science educator 42 j.sci.learn.2019.2(2).42-49 received: 28 september 2018 revised: 2 january 2019 published: 9 january2019 using brain-based learning to promote students’ concept mastery in learning electric circuit aulianuriy sani1*, diana rochintaniawati1, nanang winarno1 1international program on science education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia *corresponding author. aulianuriy@gmail.com abstract teaching approaches in some school are still concentrating on memorizing. teachers have to make the learning is meaningful for the students. one of the alternative tools is by using brain-based learning. the purpose of this study is to investigate the effect of brain-based learning on students’ concept mastery in learning electric circuit for 8th-grade students. the method used in this research was experimental research. the research design that used is pretest and posttest design. the sample was taken by random sampling in class. participants were 49 students at one of international secondary school in bandung, west java, indonesia. experiment group learns with brain-based learning (n=26) while the control group learning with lectured based learning (n=23). the results of students’ concept mastery that learned using brain-based learning is better than students’ concept mastery that learned using lectured-based learning. the improvement of students’ concept mastery can be noticed by independent t-test with significant 0.003. based on the analysis of students’ concept mastery results, the n-gain score in experiment group is 0.43 which categorized as a medium improvement while in the control group is 0.25 which categorized as a low improvement. based on these results showed that brain-based learning can be an alternative tool to improve students’ concept mastery significantly. keywords brain-based learning, students’ concept mastery, electric circuit 1. introduction currently, technological advances in the world are growing rapidly. the instructional method in school needs to adjust with the more unpredictable learning condition instead of previously (saleh, 2012). aziz (in shabatat & altarawneh, 2016) stated that teaching approaches are still concentrating on memorizing. these make the students only act as a receiver of information sent by the teacher without relating to students’ interest and these approaches make students’ receive the information without thinking independently and processing although the students have imagination and active thinking. according to altarawneh (2016), the educators and psychologists, refinement of teaching and learning process using the neurocognitive concept to bring up brain-based learning (shabatat & al-tarawneh, 2016). dissimilar to conventional techniques of schooling, which is frequently said to restrain learning by overlooking the brain's regular learning forms, the brain-based learning is accepted to support learning because of its allencompassing methodology towards the students. it is a way to deal with realizing which supports the brain's best common operational standards, with the goal of attaining maximum attention, understanding, meaning, and memory (jensen, 2008). brain-based learning is a student-focused and instructor encouraged methodology that uses students' intellectual gifts and accentuating important learning, it is not the only memorization. brain-based learning recommends that educators must submerge students in perplexing, intuitive encounters that are both rich and genuine. personally, the meaningful challenge can stimulate students’ mind to the desired state of alertness that must have by the students (uzezi & jonah, 2017). one of the important things in learning is comprehending the concept. students’ concept mastery is important that has to be gained by the students. as we know that physics is one of the difficult subjects for junior high school students (saleh, 2012). one of the topics in physics of junior secondary school is an electric circuit. the journal of science learning article doi: 10.17509/jsl.v2i2.13262 43 j.sci.learn.2019.2(2).42-49 concepts investigated include electric diagrams, current, potential difference at battery terminals, and resistance. it has been repeatedly shown that students and even teachers, make misconceptions. one of these misconceptions is the confusion between potential difference and current (liégeois, chasseigne, papin, & mullet, 2003). evidence suggests that with the right kind of learning environment, physics can be a valuable learning experience for the majority of students (redis & steinberg, 1999). there are five groups of factors that influence the level of learning achievement other than previous knowledge which has the biggest influence on learning success (klauer, 1988). these are intellectual capability, environmental components, motivational factors, and the application of learning strategies (klauer, 1988). one of the efforts to improve students’ concept mastery is by using brain-based learning approach. teachers have to make learning is meaningful for the students because brain-based learning involves accepting the rules of how the brain processes, and then organizing instruction bearing these rules in mind to achieve meaningful learning (shabatat & al-tarawneh, 2016). according to saosa (1995) stated that a brain-based approach integrates the engagement of emotions, nutrition, enriched environments, music, movement, meaningmaking and the absence of threat for maximum learner participation and achievement. it is a good strategy to make students’ motivate in learning. brain-based learning can be seen as appropriate for school students. brain-based learning will make the students experiencing in the different learning environment as well as the steps is a good strategy to apply in school. research on brain-based learning has been conducted in the past two decades (haghighi, 2013). a previous study by saleh (2012) it has been found and proven that brainbased learning was effective in encouraging conceptual understanding towards physics among students. in another study, brain-based learning was measured achievement of the female students in chemistry subject (shabatat & altarawneh, 2016); students’ attitudes level and motivation in science class (akyürek & afacan, 2013); students’ academic achievement and retention of knowledge in science course (ozden & gultekin, 2013); students’ academic achievement, attitude, motivation and knowledge retention in electrochemistry (uzezi & jonah, 2017); and academic achievement of students with different learning styles (duman, 2010). therefore, this study has investigated the difference between students' concept mastery in both control class and experimental class in learning physics, especially for electric circuit topic. there are three teaching materials about the electric circuit, which are the circuit component, series circuit, and parallel circuit. the aim of this study is to investigate the effect of brain-based learning on students’ concept mastery in learning the electric circuit. 2. method the research method which was used in this study is quasi-experiment. quasi-experiments include assignment, but not a random assignment of participants to groups (creswell, 2012). the experiment design is shown in table 1. the location of this research was held in one of international secondary school in bandung, west java, indonesia. the school used the cambridge curriculum. the population in this research was 8th-grade students. the samples are 8th-grade students from two different classes in international secondary school in bandung. students in both groups come from similar educational and socio-economic backgrounds. their ages ranged between 13-14 years old. the sampling technique was cluster random in class. fraenkel, wallen, & hyun (2012) stated that cluster random sampling is defined where one is obtained by using groups as the sampling unit rather than individuals. the data of the sample can be seen in table 2. in this study, the topic of the electric circuit is limited based on igcse physics syllabus for secondary students. the subtopics that investigated are (1) circuit component, (2) series circuit, (3) parallel circuit. the experimental group was given brain-based learning. according to saosa (1995), there are seven steps to conduct brain-based learning which are activation means to activate the students prior knowledge, clarification means clarify the objective and students have their personal performance target, making the connection means they connect their previous understanding with the new information, doing the learning activity means the students are digesting, thinking about, and experiencing multisensory, demonstration of student understanding means the students are in brain active-processing, review of student recall and retention means the students strengthen the transfer process and summarize the knowledge or information, and previewing the new topic means the students are prepared for the new topic. while the control group followed the lectured-based learning that includes lecturing and discussion. students’ concept mastery from both groups was measured before and after the intervention to determine table 1 experiment design control class pre-test lectured based learning post-test experimental class pre-test brain based learning post-test table 2 data of the sample group popul ation sample n percent age (%) total (%) control 8th grade male 13 56.52 100 female 10 43.48 experiment 8th grade male 15 57.70 100 female 11 42.30 journal of science learning article doi: 10.17509/jsl.v2i2.13262 44 j.sci.learn.2019.2(2).42-49 the effectiveness of the implemented brain-based learning. the research was done in five meetings. the first meeting was for pre-test, the second meeting was for circuit component subtopic, the third meeting was for series and parallel subtopic, the fourth meeting was for practical action, and the fifth meeting was for post-test. students’ concept mastery was measured using an objective test of 20 multiple choice questions which consists the cognitive level c1 (remembering), c2 (understanding), c3 (applying), and c4 (analyzing) based on bloom taxonomy (anderson and krathwohl, 2001). all of the test items were analyzed in the process of judgment from the expert and tested to the students. the result of the test items after the process of judgment will be used, revised, or either deleted. the objective test analyzed using anates. the reliability score is 0.78 which is high reliability. 3. result and discussion the results show quantitative data. the pre-test and the post-test are conducted to determine the students’ concept mastery before and after treatment. 3.1 implementation of brain-based learning the research was done in five meetings. the first meeting and last meeting was fore pretest and posttest. in this research, experiment class was treated learn with brainbased learning, while the control class was treated learned with lectured based learning. the research was done in one of international junior high school in bandung in april 2018 with the samples students from 8th-grade in two classes. the instructional process was done in three meetings with the duration in each meeting was 70 minutes. both the control group and experiment group have the same duration. the pretest was held on april 11th, 2018 and posttest was held on april 25th, 2018. the implementation of brain-based learning was investigated by observation sheet during the lesson. the percentage of brain-based learning implementation is presented in table 3. according to table 3 above about the percentage of brain-based learning implementation, teacher and students implemented or done all activities that have been determined in the lesson plan. the average implementation percentage is 100% which according to arikunto (2013) is categorized as very good. the result of table 3 will be elaborated in the following explanation. first treatment  the first treatment was held on april 12th, 2018 in control group while in the experiment group was held on april 13th, 2018. both groups learned circuit component subtopic. the main difference as general from both group shown in table 4 at first, the teacher relates the previous topic about current and the topic that they would learn which was about circuit component by showing them a picture of an electric circuit including circuit component. the teacher stated the learning objectives of the lesson and showed the mind map to the students so that the students can develop new knowledge and the teacher gave instruction to do brain gym. all students were joined. the main activity began, the teacher showed the picture of the circuit component as shown in powerpoint and the students have to predict. they also discussed how the lamp can be turned on/off. the students were given the worksheet and they work with their partner. they had to discuss the difference of closed circuit and open circuit and predict the function of a circuit component in the worksheet. after that, the students discussed in their kagan’s group (a group that has arranged by the school). there was six groups. one of the group was asked to presents the results of discussion and teacher clarified the answer and gave then question example. the next activity was demonstrating understanding, the students played the game. each group would have one card. in the card, there are questions about draw the electrical circuit based on the circuit component provided in the questions. all the students were excited because the reward was offered to the group that has the best score. while they were doing their mission, the teacher sets the time so it would be more challenging and classical music was played. the next activity, the students swap their answer to another student. the answer is shown on the board, the representative of the students wrote it on the board and teacher clarify the answer using colorful board marker. in the end, the teacher reviews the activity and the lesson by asking some questions to the students. rewards were given to the group that has the best score. there are three groups table 3 percentage of brain-based learning implementation meeting topic percentage of implementation criteria 1 circuit component 100% all activities implemented 2 series circuit 100% 3 parallel circuit 100% table 4 student activities in the first treatment class description of activities experiment start students are shown mind map about circuit component and did brain gym. main students did discussion and exercise in the group when the music is played. close teacher review the lesson and give the reward. control start students are shown learning objectives of learning circuit component. main students did discussion and exercise in group. close teacher review the lesson. journal of science learning article doi: 10.17509/jsl.v2i2.13262 45 j.sci.learn.2019.2(2).42-49 that have the same score. the teacher asked students to prepare the next lesson about series and parallel circuit. second treatment  the second treatment was held on april 18th, 2018 in both control group and experiment group. both groups learned series and parallel circuit subtopic. the main difference as general from both group shown in table 5. at first, the teacher relates the previous topic about circuit component and the topic that they would learn which was about series and parallel circuit by showing them a picture of series an parallel circuit. the teacher stated the learning objectives of the lesson and showed the mind map to the students so that the students can develop new knowledge and the teacher gave instruction to do brain gym. all students were joined. the main activity was begun, the teacher showed the picture of series and parallel circuit as shown in powerpoint and the students have to predict what is the difference between both circuits. one of the students shared the answer. the students were given the worksheet and they work with their partner. they had to discuss the difference of voltage, current, and resistance in series and parallel circuit in their worksheet with a partner. after that, the students discussed in their kagan’s group (a group that has arranged by the school). there was six groups. one of the group was asked to presents the results of discussion and teacher clarified the answer and gave them question example. the next activity was demonstrating understanding, the students played the game. each group would have one card. in the card, there are questions about identifying series and parallel circuit. the questions relate also about current, voltage, and resistance. all the students were excited because the reward was offered to the group that has the best score. while they were doing their mission, the teacher sets the time so it would be more challenging and classical music was played. some of the group looked hard to answer the question and need more time but in the end, they can do the question. the next activity, the students swap their answer to another student. the answer is shown on the board, the representative of the students wrote it on the board and teacher clarify the answer using colorful board marker. in the end, the teacher reviews the activity and the lesson by asking some questions to the students and conclude the lesson. rewards were given to the group that has the best score. there are two groups that have the same score. the teacher asked students to prepare the next lesson about the practical activity of series and parallel circuit. third treatment  the third treatment was held on april 19th, 2018 in control group while in the experiment group was held on april 20th, 2018. both groups did the practical activity of series and parallel circuit. the main difference as general from both group shown in table 6 at first, the teacher relates the previous topic about series and a parallel circuit including some formula and the topic that they would learn which was about the practical activity of series and parallel circuit by showing them some tools of the activity. the teacher stated the learning objectives of the lesson and showed the mind map to the students so that the students can develop new knowledge and the teacher gave instruction to do brain gym. all students were joined but some of them looked more enthusiastic to do the activity. the main activity began, the teacher showed the phet (physics education technology) simulation to the students and they have to predict which circuit would have greater current and greater resistance, and also vice versa. they were shown phet simulation so that they can predict the result of the activity that they would do. one of the students shared the answer. the students were given the worksheet and they work with their group. all of the group make the series and parallel circuit based on the question on the worksheet. the students discussed in the group about the results of ammeter reading, voltmeter reading, and the rheostat affecting the current. one of the group was asked to presents the results of discussion and teacher clarified the answer and gave them question example related to current, voltage, and resistance. the next activity was demonstrating understanding, the students played the game. each group would have one card. in the card, there are questions about identifying and table 5 student activities in second treatment class description of activities experiment start students are shown mind map about series and parallel circuit and do brain gym. main students did discussion and exercise in the group when the music is played. close teacher review the lesson and give the reward. control start students are shown learning objectives about series and parallel circuit. main students did discussion and exercise in group. close teacher review the lesson. table 6 student activities in third treatment class description of activities experiment start students are shown mind map about the practical activity and do brain gym main students did discussion and exercise in the group when the music is played. close teacher review the lesson and give the reward. control start students are shown learning objectives main students did discussion and exercise in group. close teacher review the lesson. journal of science learning article doi: 10.17509/jsl.v2i2.13262 46 j.sci.learn.2019.2(2).42-49 calculating the voltage, current, resistance in series and a parallel circuit (lamp and resistor). all the students were excited because the reward was offered to the group that has the best score. while they were doing their mission, the teacher sets the time so it would be more challenging and classical music was played. it is not too conducive since some of the students were playing with the tools, the teacher keeps remind the students not to play with the tools. the next activity, the students swap their answer to another group. the answer is shown on the board, the representative of the students wrote it on the board and teacher clarify the answer using colorful board marker. in the end, the teacher reviews the activity and the lesson by asking some questions to the students and conclude the lesson. rewards were given to the group that has the best score. there are two groups that have the same score. the teacher asked students to prepare the next meeting which is posttest. 3.2 students’ concept mastery the results of pretest and posttest score were calculated and the data were analyzed using spss version 20 to know whether the data is normally distributed or not. then, the data is analyzed using a parametric or nonparametric test based on the result of the normality test. the statistic test was done in order to know the difference of concept mastery between control and experiment group. the recapitulation of the statistical test result of control and experiment group is shown in table 7. from the results in the shapiro-wilk test, the significance value (α) on the control group is 0.819 and 0.315 for experiment group. if compared with the value of α result in sig. > α = 5% then h0 is accepted which means that the data on the control group and experiment group are normally distributed. then, continue to homogeneity test. based on the results of levene statistics test, the result of the homogeneity test is 0.332 so if compared with α, resulting in 0.332 > α = 5%, then the data are homogeneous. since the data is normally distributed and homogenous, then continue with a parametric test which is independent-samples t-test. the level of significant value used in the test is 0.05. the results of the test show that the significant value is 0.003 or less than 0.05, it means that there is the difference in students’ concept mastery in learning electric circuit after using brain-based learning (bbl) or there is a significant effect. the average score of students’ concept mastery in pretest and posttest is shown in figure 1. based on figure 1 it can be found that the average of pretest score in the control group was 38.91 and for the experiment, the group was 33.84. while the average of posttest score in control and experiment group was 55.21 and 63.84. it means there is an improvement on students’ concept mastery in learning electric circuit after the treatment. it can be concluded that students from the table 7 recapitulation of hypothesis test on students’ concept mastery hypothesis test result normality test experiment class signification (sig. = 0.05) conclusion 0.315 normally distributed control class signification (sig. = 0.05) conclusion 0.819 normally distributed homogeneity test signification (sig. = 0.05) 0.332 conclusion homogenous independent t-test signification (sig. = 0.05) 0.003 (asymp. sig.(2-tailes) < 0.05, h1 = accepted) conclusion h1 = accepted, h0= rejected there is a significant difference figure 1 average score in experiment and control group 38.91 55.21 33.84 63.84 0 10 20 30 40 50 60 70 80 pretest posttest a ve ra g e sc o re group control experiment figure 2 n-gain score in experiment and control 0.25 0.43 0.0 0.1 0.2 0.3 0.4 0.5 control group experiment group n -g ai n s co re group journal of science learning article doi: 10.17509/jsl.v2i2.13262 47 j.sci.learn.2019.2(2).42-49 experiment group that learned using brain-based learning have a difference in improvement of learning outcomes than the control group in learning the electric circuit. the difference between n-gain for experiment group and the control group can be seen in figure 2. the analysis of n-gain on control group and experiment group shows that the treatment gives the impact on the improvement of students’ concept mastery in learning the electric circuit. the n-gain score from control group is 0.25 which is categorized as a low improvement while the experiment group got 0.43 which is categorized as a medium improvement (hake, 1999). 3.3 students’ cognitive domain data continued analyzed from the average of n-gain from c1, c2, c3, and c4 in both groups. to know the improvement of students understanding in each level, test of n-gain was done by first grouping the questions based on its cognitive level. then, find the average of pretest and posttest score, and after that calculating the n-gain from the control group and experiment group. the results of pretest and posttest for each cognitive dimension in each group is shown in table 8. from table 8, it can be seen that each cognitive domain shows different results of students’ concept mastery either in pretest and posttest. in the control group shows that the average n-gain on remembering (c1) is 0.43 which is categorized as a medium. the average of n-gain on understanding (c2) is 0.18 which is categorized as low. the average of n-gain on applying (c3) is 0.06 which is categorized as low, and the average of n-gain on analyzing (c4) is 0.29 which is categorized as low. the results in the experiment group show that the average n-gain on remembering (c1) is 0.12 which is categorized as low. the average of n-gain on understanding (c2) is 0.22 which is categorized as low. the average of n-gain on applying (c3) is 0.45 which is categorized as a medium, and the average of n-gain on analyzing (c4) is 0.75 which is categorized as high. it can be concluded from, in the experimental group has higher improvement in the c4 domain while in control group has higher improvement on c1 domain. based on the results, the data can support that the fact that students in the experiment group can more comprehend the topic than in the control group. the improvement for each cognitive level is presented in figure 3. according to figure 4.3, we can see that generally, the experimental group got higher n-gain in cognitive domain c2 (understanding) is 0.22, c3 (applying) is 0.45, and c4 (analyzing) is 0.75. however, for domain c1 (remembering), control group got higher n-gain because for the control group in the pretest they got a lower score than experiment group. most of the students in experiment group answered correctly for domain c1 (remembering) in the pretest so that the n-gain for domain c1 (remembering) is 0.12 which is lower than a control group that got 0.43. in c2 (understanding) domain, the experimental group got higher n-gain than control group although both of them categorized as a low improvement. the low gain was obtained because most of the questions in the test are c2 (understanding) domain. this can be one of the factors because the number of the question if more than other domain. in c3 (applying) domain, the experiment group got higher n-gain than the control group. in the question are mostly about the picture of series and a parallel circuit including ammeter and voltmeter. although both of the group were exercised about this question, in experiment group, they showed phet simulation and practical activity so that it is easier to understand and more interesting while in control group they only experienced in practical activity. in c4 (analyzing) domain, experiment group got higher n-gain than the control group. the questions are mostly provided a picture of series and parallel circuit. only one question requires calculation. the experiment group got higher n-gain because the students in the class were paying attention compared to the control group. in the control group, not all the students paying attention when table 8 pretest and posttest for each cognitive domain group aspect cognitive level score c1 c2 c3 c4 experiment pretest 69.23 34.61 30.77 27.88 posttest 80.76 53.84 65.38 83.65 g 11.53 19.23 34.61 55.76 <g> 0.12 0.22 0.45 0.75 category low low medium high control pretest 52.17 36.95 41.73 34.78 posttest 95.65 50.43 54.78 55.43 g 43.47 13.47 13.04 20.65 <g> 0.43 0.18 0.06 0.29 category medium low low low figure 3 comparison of the cognitive level of blooms’ taxonomy between experiment and control group 0.43 0.18 0.06 0.29 0.12 0.22 0.45 0.75 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 c1 c2 c3 c4 s co re n -g ai n o n c o g n it ri ve l ev el cognitive level control group experiment group journal of science learning article doi: 10.17509/jsl.v2i2.13262 48 j.sci.learn.2019.2(2).42-49 the teacher explained this type of question. the questions would be easy if they have known the concept. the cognitive domain result can show that the use of brain-based learning can improve students’ ability in all cognitive domain level that was measured. the researchers attributed this result to the benefits of using the brainbased learning that was summarized by caine & caine (1997) and jensen (2012). the strategies harmonized with principles, such as work in groups, encourage cooperation among students, indicate that teaching process will be more efficient when it is conducted within a teaching environment that provides students with chances to exchange experiences within collaboration groups. the use of concept charts which leads to link the major and minor themes together, and organize the knowledge hierarchically makes learners more capable to use, retain the knowledge easily. the note writing (taking notes) help in memorizing the written material. marshall (2002) and obeidat and abussameed (2013) emphasize that taking notes helps knowledge memorization, retrieving, developing, enhancing with more information. taking notes also make new relations between previous and new knowledge. the use of brainstorming helps in generating creative ideas by encountering a new situation and problem by the students. the review of the previous lesson is important to these strategies whether work in groups, concept charts, notes taking, and brainstorming activates the previous information in the brain. jensen (2008) discovered that information is not static so it needs frequent review and repetition, else it will be lost or leaked. these skills need more time to apply. the positive impact is seen by the application of several strategies especially on increasing the interaction of students to the class situations. by applying varied activities and techniques can meet the needs and interests of students and take into account the individual differences, it also helps more relaxation, active processing and improves achievement (aljorani, 2008). another difference between experiment group and control group is brain gym. brain gym facilitates the process of waking up the mind/body system, and learning readiness. through simple integrative movements that focus on specific sensory aspects, brain gym activates the full mind/body function across the body midline (klinek & indiana, 2009) the improving of students’ concept mastery that has been found is in line with the research done by saleh (2012) which found that brain-based teaching approach was more effective in developing students’ conceptual understanding as compared to the conventional method. it is also in line with other research by shabatat & altarawneh (2016) which found the level of achievement has been improved by using brain-based learning. furthermore, the students who learned with brainbased learning tend more active in class especially when they had to discuss in a group. they were encouraged with a challenging environment and rewards. this is in line with the finding from shabatat & al-tarawneh (2016) that based on the results, the researchers found the benefits of using brain-based method and principles which are the students work in groups, encourage cooperation among students, environmental support that provides students with chances to exchange experience within collaboration groups. this result in line with a previous study (ozden & gultekin, 2013) that brain-based learning approach appears to be more effective than the traditional teaching procedures in science courses in terms of improving students’ academic achievement. another difference between experiment group and control group was the utilization of mind map. the experiment group was shown mind map at the beginning of the lesson, it is helpful to improve the achievement of students. this result in line with finding from jbeili (2013) that using digital mind maps had a significant effect on students’ science achievement. the implementation of inquiry activities in experiment group also in line with finding from wardani (2017) that inquiry-based laboratory activity can improve students’ understanding. 4. conclusion brain-based learning can improve students’ concept mastery on electric circuit topic. it can be proved by the acceptance of h1 and the result of significance is 0.003 which means that there is a significant difference in learning using brain-based learning on students’ concept mastery. the improvement also supported by the results of n-gain in experiment group is 0.43 which is categorized as medium improvement and n-gain in control group is 0.25 which is categorized as a low improvement. it can be concluded also that brain-based learning improved students’ concept mastery in all cognitive level. brainbased learning can be one of the alternative teaching approaches that can improve students’ concept mastery in learning the electric circuit. acknowledgment author’s wishing to acknowledge the principal of the school for the permission to conduct the research (teaching in the class for several meetings). references akyürek, e., & afacan, ö. (2013). effects of brain-based learning approach on students ’ motivation and attitudes levels in science class. mevlana international journal of education, 3(1), 104–119. al-tarawneh, m. (2016). the impact of a teaching-learning program based on a brain-based learning on the achievement of the female students of 9th grade in chemistry. higher education studies, 6(2), 162. aljorani, y. (2008). teaching design according to brain-based learning and its impact on the achievement of 9th-grade students in biology and development of their scientific thinking. baghdad, iraq: university of baghdad. journal of science learning article doi: 10.17509/jsl.v2i2.13262 49 j.sci.learn.2019.2(2).42-49 anderson, l. w., & krathwohl, d. r. (eds.). (2001). a taxonomy for learning, teaching, and assessing. new york: addison wesley longman, inc. arikunto, s. (2013). dasar-dasar evaluasi pendidikan. jakarta: bumi aksara. assalti, n. (2004). brain-based learning (1st ed.). amman, jordan. caine, r., & caine, g. (1997). making connection: teaching and brain. alexandria, va: ascd. creswell, j. w. (2012). educational research fourth edition. boston: pearson. duman, b. (2010). the effects of brain-based learning on the academic achievement of students with different learning styles. educational sciences: theory & practice educational sciences: theory & practice, 10(4), 2077–2103. fraenkel, j. e., wallen, n. e., & hyun, h. h. (2012). how to design and evaluate research in education. new york: mc graw hill. haghighi, m. (2013). the effect of brain-based learning on iranian efl achievement and retention. procedia social and behavioral sciences, 70, 508–516. hake, r. r. (1999). analyzing change/ gain score. retrieved from http://www.physics.indiana.edu jbeili, i. m. a. (2013). the impact of digital mind maps on science achievement among sixth grade students in saudi arabia. procedia social and behavioral sciences, 103, 1078–1087. jensen, e. (2008). brain-based learning (second). california: corwin press. jensen, e. (2012). teaching with the brain in mind, workshop held. usa, san antonio. retrieved from http://www.jennsenlearning.com/pdf/10mosteffectivetips.pdf klauer, k. j. (1988). teaching for learning-to-learn: a critical appraisal with some proposals. instructional science, 17(4), 351–367. klinek, s. r., & indiana. (2009). brain-based learning: knowledge, beliefs, and practices of college of education faculty in the pennsylvania state system of higher education. vasa, (may), 1– 188. retrieved from http://medcontent.metapress.com/index/a65rm03p4874243n. pdf liégeois, l., chasseigne, g., papin, s., & mullet, e. (2003). improving high school students’ understanding of potential difference in simple electric circuits. international journal of science education, 25(9), 1129–1145. marshall, b. (2002). the secrets of getting better grades: study smarter not harder (2nd edition). park avenue: jist publishing, inc. obeidat, t., & abussameed, s. (2013). brain function-based learning. retrieved from http://www.anawatifly.blasspo ozden, & gultekin. (2013). the effects of brain-based learning on academic achievement and retention of knowledge in science course. electronic journal of science education, 12(1), 1–17. redis, e. f., & steinberg, r. n. (1999). teaching physics: figuring out what works. physics today, 52. retrieved from https://www.physics.umd.edu/perg/qm/qmcourse/newmodel/ research/whatwork/index.htm saleh, s. (2012a). the effectiveness of brain-based teaching approach in dealing with the problems of students’ conceptual understanding and learning motivation towards physics. educational studies, 38(1), 19–29. saleh, s. (2012b). the effectiveness of the brain-based teaching approach in enhancing scientific understanding of newtonian physics among form four students. international journal of environmental and science education, 7(1), 107–122. shabatat, k., & al-tarawneh, m. (2016). the impact of a teachinglearning program based on a brain-based learning on the achievement of the female students of 9th grade in chemistry. higher education studies, 6(2), 162. sousa, d. 1995. how the brain learns. a classroom teacher’s guide. reston, va: national association of secondary school principals. uzezi, j. g., & jonah, k. j. (2017). effectiveness of brain-based learning strategy on students ’ academic achievement, attitude, motivation and knowledge retention in electrochemistry. journal of education, society and behavioural science, 21(3), 1–13. https://doi.org/10.9734/jesbs/2017/34266 wardani, t. b., & winarno, n. (2017). using inquiry-based laboratory activities in lights and optics topic to improve students' understanding about nature of science (nos). journal of science learning, 1(1), 28-35. a © 2021 indonesian society for science educator 134 j.sci.learn.2021.4(2).134-139 received: 30 september 2020 revised: 27 january 2021 published: 4 february 2021 comparison of the 7th grade students’ accomplishments in skill and acquisition based assessment-evaluation ali kolomuc1*, zafer karagölge2 1faculty of education, artvin coruh university, turkey 2faculty of education, atatürk university, turkey *corresponding author. alikolomucscr@artvin.edu.tr abstract this study compares the successes of numbers 94 7th grade students in science education with assessmentevaluation questions based on skill and acquisition. assessment-evaluation achievement tests based on skill and acquisition were developed by researchers towards the measurement of force and acquisitions in friction units taking place in the 7th-grade curriculum. while skill-based questions are being prepared, high school entrance exam (hsee) and international exams such as programme for international student assessment (pisa) were developed due to inspiration from trends in international mathematics and science study (timss) exam questions and acquisition-based assessment and evaluation questions were taken from pre-exam books. it was found that the reliability coefficient for the traditional test is 0.70 and for the skill-based test is 0.72. at the same time, students’ opinions were taken to detect students' perspectives against skill and acquisition-based questions after application. according to the statistical results gathered from research, the academic achievements of students who are prepared with the same acquisitions and applied with skill and acquisition-based assessment-evaluation questions were analyzed. it was determined that despite the lack in solving skill-based questions, students were successful in acquisition-based assessmentevaluation. the students' average success in the acquisition-based test was 76 out of a hundred, and the average score in the skillbased test was 44. besides, it can be considered to extend skill-based questions for turkey to go beyond successes. keywords science, context-based assessment, gain based assessment, and assessment, force and energy 1. introduction in our rapidly growing information age, countries spend lots of effort increasing science education quality and benefiting from its results (kaya, balay & göçen, 2012). for this reason, while some countries are reviewing their current program and restructuring them in line with developments, others are leading their way to the development of new science programs (şenel, çepni, yıldırım & er nas, 2007). in parallel with this purpose, in our country, primary education science teaching program that was put into action in 2000 was changed to science and technology teaching program in 2004-2005 school years. one of the main reasons for this change is the failure of our countries in international exams ministry of education (bakanlığı, 2018); organisation for economic co-operation and development, 2016). it can be indicated that similar situations are valid for other countries' education systems (breakspear, 2012; bonal & tarabini, 2013; harus & davidovitch, 2019; jürges, schneider, & büchel, 2005; woessmann, 2018). for example, hopkins, pennock, ritzen, ahtaridou & zimmer (2008) put forward the idea that pisa has a strong influence on the educational policies of countries such as honk kong-china, spain, canada, norway, and poland. therefore, countries are giving importance to the skills assessed through pisa. the most important reason for students' failure is the result of the education and evaluation of students. students can’t relate the science and math terms with real life, so they have hard time-solving questions in these exams. teachers think that current teaching programs do not adequately guide both teachers and students about solving skill-based questions. in this manner, teachers draw attention to the dimensions of assessment-evaluation of turkish, math, and science teaching programs and remarking that these dimensions contain weaker content compared to the other dimensions of program (erden, 2020). in both teaching and evaluation process, relating the science topics with real life can lead the success with it in global scale exams at the mailto:alikolomucscr@artvin.edu.tr journal of science learning article doi: 10.17509/jsl.v4i2.28618 135 j.sci.learn.2020.4(2).134-139 same time, it can increase the quality of teaching and contribute meaningful and long-termed information as well (bellocchi, king, & ritchie, 2016; nentwig, demuth, parchmann, gräsel & ralle, 2007; sak & kaltakçı gürel, 2018). as a consequence of the change in our countries’ programs, our country slowly but indeed starts to climb the steps of success in these exams (pisa, timms). in turkey’s 2023 education vision document, activities about reforming the education system and the purpose of all exams, content, and structure according to question types and their benefit are planned for rearrangement. moreover, it is aimed to test reasoning, critical thinking, exposition, guessing, and related mind skills (erden, 2020). in the meantime, it is aimed to switch from an assessment perceptiveness including various concepts, facts, and memorization of formulas to an assessment perceptiveness including basic mind skill quantification (meb, 2016). hence, it can be referred that the ministry of education took steps in the direction of two goals with the “high school entrance exam.” ministry of education, in its recent year exams, started to concentrate on skill-based questions. when skill-based questions are compared with previous exam questions, significant distinctness can easily be mentioned (güler & ülger, 2018). while previous exam questions are asked mostly in the level of knowledge and comprehension (özden et al., 2014), on the other hand in the high school entrance exam, questions are asked in advance level (batur, ulutaş & beyrut, 2018; berber & anılan, 2018; ekinci & bal, 2019; kılkapan & nacaroğlu, 2019). within this scope, the ministry of education aims at measuring high-level skills such as reading comprehension, exposition, deduction, problem-solving, analysis, critical thinking, and scientific process skills through skill-based questions. furthermore, it can be stated that questions are being prepared on a hard ground by relating with daily life. ministry of education slowly begins to adapt secondary school students by publishing skill-based questions. ministry of education tries to popularize these questions by preparing courses for teachers aiming for skill-based questions. because of skill-based evaluation methods, educators draw attention to the rise of students' successes in relating their cognitive development through critical thinking, communication, and content with real-life facts. using skill-based questions in assessment-evaluation activities is quite important from the point of using the information on real-life issues and problem-solving (elmas & eryılmaz, 2015; i̇lhan & hoşgören, 2017; sak & kaltakçı gürel, 2018). skill-based questions involve short stories, including reasons for calculations about real objects or facts (heller & hollabaugh, 1992). in these short stories, enriched questions from real life, experiences, and attractive issues are directed to students. in this way, when students try to solve by relating the problem with their life or facts under observation, they will be able to look for different solutions while thinking at a high cognitive level and improve their approach toward problem-solving in real life (rennie & parker, 1996; tekbıyık & akdeniz, 2010). science educators also point out different evaluation methods by defining that traditional evaluation tests aren’t enough for assessing science education success. another feature of skill-based assessment and evaluation approaches focuses on high-level cognitive learning and problem-solving skills. thus, contextual questions measure deeper understanding instead of remembering the information, and this ensures the research of more detailed information about students’ learning of the topic (akpınar, 2012; ültay & usta; 2016). while traditional problem necessitates the knowledge's remembrance, the contextual problem requires deeper understanding (wilkinson, 1999; ültay & usta; 2016). in addition to that, students can find contextual problems more interesting and tempt them to solve problems. in educational fields, it was observed that students have more attention to skill-based questions. it was observed that students with less attention to class try to answer skill-based questions. as it is seen in the previous researches, both teachers and candidate teachers are insufficient for the use of skillbased questions. when the literature was examined, limited researches were obtained related to skill-based evaluation. this research aimed to compare students' academic achievement by developing skill-based and acquisitionbased evaluation questions appropriate with the same acquisitions of the topic “force and energy” found in science education course program. in continuation of the research, students’ views were taken to detect their perspectives against skill-based questions. it was considered that this research would contribute to the field and fill the gap in the assessment and evaluation field. 2. method 2.1. sample the sample of this research was 94 students studying in 7th grade in artvin, a small province situated in the northeastern part of turkey. among the respondents, 54 were female, and 40 were male. the socioeconomic conditions of students are at a medium level, and the average age was 13. 2.2. conducted researches in this research, while skill-based questions were developed by the inspiration from sbs, pisa, timms, and postgraduate students, as for traditional assessment and evaluation questions, they were developed by investigating high school entrance exam (hsee) books. postgraduate students developed skill-based questions, and pilot studies gave their final shape. while preparing the tests, 20 pieces of the same subject were prepared for the same gains. the same teacher gave the subject to all classes, and after the end of the subject, the students were given an exam. the students had not seen the exam questions before, and it was their first time in the exam. acquisitions of skill-based and journal of science learning article doi: 10.17509/jsl.v4i2.28618 136 j.sci.learn.2020.4(2).134-139 traditional assessment and evaluation questions in the test were shown in table 1. as with the last given shape, the test was investigated under five teachers and five academicians regarding its content validity. this kind of proceedings increases the test's validity and reliability (çalık & ayas, 2002; ayas & demirbaş, 1997; peterson & treagust, 1989). the reliability score of the test was found as 0.72. lgs preparation books prepared traditional assessment and evaluation questions with 20 questions, and the reliability score of this test was found as 0.70. sample questions suitable for the skill-based assessment questions used in the study are shown in appendix a. questions which are appropriate to traditional assessment and evaluation used in the research are given in appendix b 3. result and discussion as a consequence of skill and acquisition-based tests applied to the students, average points out of 100 based on class were shown in table 2. according to the statistical process; it was seen in table 3 that in all classes success rate in the skill-based test was lower compared to the traditional test. according to the independent t-test results, when students' success scores were compared, there was no significant difference between tests. depending on these results, it was observed that students were less successful in the skill-based test (p < 0.001). 3.1. perspectives of students toward skill-based questions another aim of the research was to determine the perspectives of students toward skill-based questions. findings of students’ thoughts in skill-based questions were given in table 4. when findings were examined, it was seen that students were more successful in traditional tests. when students’ opinions about skill-based evaluation questions were examined, between students, additional comments were revealed. the average time of finishing the acquisition based test was 8 to 9 minutes. for a skill-based test, it was 35 to 40 minutes. students mostly had difficulty in the skill-based test, but they found the question style entertaining even though the success rate was lower than the acquisitionbased test. the common result of many studies in the literature show that assessment-evaluation deeply affected the learning process (metin & demiryürek, 2009; bellocchi, king, & ritchie, 2016; sak & kaltakçı gürel, 2018) and when assessment-evaluation activities used properly, it became effective in the direction of increasing learning level and quality of students (clarke, 2001; black & william, 2002). in consequence of reformed programs, in several studies related to the application process, it was stated that there becomes a problem in the application of the assessment-evaluation process in terms of time issue and knowledge deficiency. students expressed the lack of teachers in skill-based assessment-evaluation. s16, s67, s73, and s87 underlined the situation as, in my opinion, these questions are mind-confusing and never seen before. i think these questions are non-sense “. in this sense, it is easy to conclude that teachers didn’t use this type of question. failure of students in skill-based assessmentevaluation can be related to the lack of teachers in this topic. this defect can be fulfilled by taking teachers into seminars about skill-based assessment-evaluation. thus, it was observed that the ministry of education sometimes opened courses about preparing and applying this kind of question. in parallel with this result, in erden’s (2020) table 1 learning outcome of the subject learning outcome name of the subject 1. measurement of force 1.1. it measures the magnitude of force by a dynamometer. newton (n) is used as the absolute unit of force. 1.2. it designs a dynamometer by using simple devices. measurement of force and friction 2. friction force 2.1. it gives daily life examples of friction force. 2.2. it explores by experience the movement effect of friction force in various places. experiments are done about the movement effect of friction force in rough and slippery surfaces. 2.3. it generates new ideas about increasing or decreasing friction in daily life. table 2 success rates in students’ tests classes context-based assessment success rates (%) traditional assessment success rates (%) a 44 76 table 3 group statistics tester n mean std. deviation std. error mean t p traditional test skill based test 94 94 75.74 44.04 18.96 17.21 2.76 2.51 27.37 17.54 .000 https://tureng.com/tr/turkce-ingilizce/learning%20outcome https://tureng.com/tr/turkce-ingilizce/learning%20outcome journal of science learning article doi: 10.17509/jsl.v4i2.28618 137 j.sci.learn.2020.4(2).134-139 study, teachers found course books inadequate for not including the recent topics, being simple at theme evaluation, not having rich visual content, and being inadequate to prepare students for lgs (çarkıt, 2019). conducted researches state that students also want to see lecturing and tests in coursebooks toward central exams (fidan, 2018). according to kutlu (2018) and erg (2019), the quality of questions placed in coursebooks is confronted as one of the education issues in today’s world. in teaching science education concepts within structural learning, one of the most important aims is to help students develop an understanding at a conceptual level and make them use them in new situations (çalık, 2003; ward & herron, 1990). solving mathematical and formula-based questions may not prove that students understand the concept (prima, putri & rustaman, 2018). the solution to some problems may end up at the final of instrumental learning (ayas, çepni, johnson & turgut, 1997; çepni, 2019). therefore, questions should be given which involve numerical operations to a bare minimum to improve students’ scientific discernment and give prominence to concept teaching. when viewed from this aspect, numerical operations weren’t given in the skill-based assessment-evaluation test. as students coded as s33, s49, s22, s64 mention, “questions are good in this way. interpretation questions being integrated with knowledge questions are excellent, to begin with. even interpretation questions should be increased in number. the acquisitionbased test was easy and knowledge-focused. skill-based questions compelled me a little bit, and it was more interpretation less knowledge involved” in their sayings, it also matches with the aspect above. in this way, the problem-solving and scientific discernment skills of students will burst into prominence. depending on the skill-based test development, it was paid attention to whether students can build a bond between daily life experiences and microscopic level of events. hence, systems including the microscopic level of demonstrations were given in the test. when students can revive microscopic operations in sciences, they can structuralize the knowledge more meaningfully and maintain the learning permanently. also, they can easily conceptualize other knowledge types and build an appropriate bond between them. as it is known, to detect the actions in science, macroscopic, microscopic, and symbolic levels are used (özmen, ayas & coştu, 2002; maulidah & prima, 2018; kızkapan & nacaroğlu, 2019; çepni, 2019). actions under the macroscopic process are direct observations of students. science actions at a microscopic level are explained using molecules, atoms, theoretical concepts, and models. science actions under symbolic level are shown with symbols, numbers, formulas, equations, and structures (özmen, ayas & coştu, 2002; güler & ülger, 2018; batur, ulutaş & beyrut, 2018). thus for understanding a concept table 4 perspectives of students toward skill-based questions students students’ thoughts about skill-based questions s2, s25 skill-based questions were difficult. acquisition based tests were much more comfortable. s34, s63 in my opinion, skill-based questions were much more difficult. s9, s17, s78 the acquisition-based test was easy. i liked both tests. s33,s49 questions are right in this way. interpretation questions being integrated with knowledge questions are excellent to begin. even interpretation questions should be increased in number. s22, s64 the acquisition-based test was easy and knowledge-focused. skill-based questions compelled me a little bit, and it was more interpretation less knowledge involved. s22, s64 the acquisition-based test was easy and knowledge-focused. skill-based questions compelled me a little bit, and it was more interpretation less knowledge involved. s41, s49 s55, s79 i think this kind of question should be asked, both good for interpretation and visual quality. s5, s37 questions were strange, straightforward, and surprising. in these questions, the possibility of making a mistake is very high. these questions are mind-confusing and never seen before. in this test, more logic and calculation were seen. in my opinion, s84 all of the questions were interpretational and figurative. however, questions were asked straightforwardly. thus we need to know the topic thoroughly. s5, s78 s83, s86 questions are both interpretational and knowledge-focused. i think they were right. s77, s89 questions were comfortable but needed lots of attention. we solve questions with many operations. however, in this test, there was none. s2, s21, s33,s43 i think skill-based questions were so simple, but i always fail at straightforward win at difficult questions. i liked the exam very much, and the acquisition-based questions were easy. questions can be qualified, but a form of asking was what i didn’t like. questions were right nonetheless. journal of science learning article doi: 10.17509/jsl.v4i2.28618 138 j.sci.learn.2020.4(2).134-139 adequately, it is necessary to develop a connection according to these three levels. when considered from this point, students' failures in questions prepared according to skill-based assessment-evaluation methods are higher than usual because of the traditional education they take. students raised under traditional education have their assessment-evaluation with traditional tests. result in this, the reason for staying at the bottom of international exams such as pisa and timms is traditional assessmentevaluation. one of the failures is that they face this kind of questions for the first time (ö16, ö67, ö73, and ö87) (table 4). also, it can be related to the lack of application teachers possess for a skill-based assessment-evaluation topic. 4. conclusion this study aims to prepare skills and acquisition-based assessment and evaluation questions suitable for the same gains of the "force and energy" unit in the science course program and compare students' academic achievements in these tests. another aim of the study is to determine the thoughts of the students about skill-based questions. when looking at the results of two different tests applied to students to compare their academic achievements in the tests, the students' moderate success in the acquisitionbased test was 76 out of 100, and the average score in the skill-based test was 44. according to these outcomes, the students' success rate in the skill-based test was found to be lower. for the students' thoughts about skill-based questions, students' opinions are recorded and shown in table 4. the students' opinions stated that they had just encountered skill-based questions (s16, s67, s73, s87) (table 4). one of the reasons students fail the skill-based test can be attributed to their new encounters with questions. when results gained from the test and application are considered, there are suggestions for fulfilling the purpose of planning the education being more systemized. rather than teaching based on mathematical operations and formulas, concept teaching should be brought front to understand the courses better. a thus more in-depth understanding of concepts by students can be provided. questions related to concept studies in the data collection tool can be provided clear enough to qualify microscopic thinking and intellectual discernment. by applying skillbased assessment-evaluation questions into other units, teachers can be resourceful in terms of assessmentevaluation. furthermore, teachers can be taken into seminars about the assessment-evaluation topic to be informed about skill-based assessment-evaluation. references akpınar, m. (2012). bağlam temelli yaklaşımla yapılan fizik eğitiminde kavramsal değişim metinlerinin öğrenci erişisine etkisi. gazi üniversitesi, eğitim bilimleri enstitüsü, ankara. ayas, a., çepni, s., johnson, d., & turgut, m. f. (1997). kimya öğretimi, öğretmen eğitimi dizisi. yök/dünya bankası milli eğitimi geliştirme projesi yayınları. ankara. ayas, a. & demirbaş, a. (1997). turkish secondary students’ conception of introductory chemistry concepts. journal of chemical education, 74(5), 518-521. batur, z., ulutaş, m. & beyret, t. n. (2018). lgs türkçe sorularının pisa okuma becerileri̇ hedefleri açısından incelenmesi. milli eğitim dergisi, 48(1), 595–615. bellocchi, a., king, d. t., & ritchie, s. m. (2016). context-based assessment: creating opportunities for resonance between classroom fields and societal fields. international journal of science education, 38(8), 1304-1342. berber, a. & anılan, b. (2018). son on yıldaki ortaöğretime geçiş sınavlarındaki fen bilimleri alan soruları ile ilgili öğretmen adaylarının görüşlerinin incelenmesi. electronic turkish studies, 13(27), 203-224 black, p., & william, d. (2002). improved standards achieved by transforming assessment for learning. news archive: kings college london. bonal, x. & tarabini, a. (2013). the role of pisa in shaping hegemonic educational discourses, policies and practices: the case of spain. research in comparative and international education, 8(3), 335–341. breakspear, s. (2012). the policy impact of pisa: an exploration of the normative effects of international benchmarking in school system performance. oecd education working papers, no. 71. oecd publishing (nj1). clarke s. (2001). unlocking formative assessment: practical strategies for enhancing pupil’s learning in the primary classroom. london: hodder & stoughton educational. çalık, m., & ayas, a. (2002). öğrencilerin bazı kimya kavramlarını anlama seviyelerinin karşılaştırılması, 2000’li yıllarda i. öğrenme ve öğretme sempozyumu, 29-31. çalık, m. (2003). farklı öğrenim seviyesindeki öğrencilerin çözeltilerle i̇lgili kavramları anlama seviyelerinin karşılaştırılması. yayınlanmamış yüksek lisans tezi. trabzon: k.t.ü. fen bilimleri enstitüsü. çarkıt, c. (2019). 2018 türkçe dersi öğretim programı çerçevesinde hazırlanan 8. sınıf türkçe ders kitabının değerlendirilmesi. electronic journal of social sciences, 18(71), 1368-1376. çepni, s. (2019). pisa ve timss mantığını ve sorularını anlama. ankara: pegem a yayıncılık. ekinci, o. & bal, a. p. (2019). 2018 yılı liseye geçiş sınavı (lgs) matematik sorularının öğrenme alanları ve yenilenmiş bloom taksonomisi bağlamında değerlendirilmesi. anemon muş alparslan üniversitesi sosyal bilimler dergisi, 7(3), 9-18. elmas, r., & eryılmaz, a. (2015). how to write good quality contextual science questions: criteria and myths. journal of theoretical educational science, 8(4), 564-580. erden, b. (2020). türkçe, matematik ve fen bilimleri dersi beceri temelli sorularına i̇lişkin öğretmen görüşleri, ajer academia eğitim araştırmaları dergisi, 5(2), 81-103. erg (2019). eğitimin i̇çeriği, eğitim i̇zleme raporu 2019, 22 mart 2020 tarihinde https://www.egitimreformugirisimi.org/wp content/uploads/2010/01/eir_egitimin_icerigi.pdf web adresinden alınmıştır. fidan, m. (2018). ortaokul öğrencilerinin türkçe ders kitaplarının tasarımına yönelik görüşlerinin analizi. bayterek uluslararası akademik araştırmalar dergisi, 1(2), 178–189. güler, h, k. & ülger, b. (2018). pisa, timss ve teog sınavlarının temele aldığı öğrenme kuramları, s. çepni(ed.), pisa ve timss mantığını ve sorularını anlama içinde (s.111–153). ankara: pegema yayıncılık. harus, e. b. & davidovitch, n. (2019). the gems exams in israel— between center and periphery. international education studies, 12(10), 9-21. heller, p., & hollabaugh, m. (1992). teaching problem solving through cooperative grouping. part 2: designing problems and structuring groups. american journal of physics, 60(7), 637-644. https://www.egitimreformugirisimi.org/wpjournal of science learning article doi: 10.17509/jsl.v4i2.28618 139 j.sci.learn.2020.4(2).134-139 hopkins, d., pennock, d., ritzen, j., ahtaridou, e., & zimmer, k. (2008). external evaluation of the policy impact of pisa. paris: oecd, november, 13, 2018. i̇lhan, n., & hoşgören, g. (2017). fen bilimleri dersine yönelik yaşam temelli başarı testi geliştirilmesi: asit baz konusu. fen bilimleri öğretimi dergisi, 5(2), 87-110. jürges, h., schneider, k. & büchel, f. (2005). the effect of central exit examinations on student achievement: quasi-experimental evidence from timss germany. journal of the european economic association, 3(5), 1134–1155. kaya, a., balay, r., & göçen, a. (2012). öğretmenlerin alternatif ölçme ve değerlendirme tekniklerine ilişkin bilme, uygulama ve eğitim ihtiyacı düzeyleri. international journal of human sciences. (9)2, 12291259. kızkapan, o. & nacaroğlu, o. (2019). fen bilimleri öğretmenlerinin merkezi sınavlara (lgs) ilişkin görüşleri. nevşehir hacı bektaş veli üniversitesi sbe dergisi, 9(2), 701–719. kutlu, ö. (2018). ölçme ve değerlendirmede değişim zamanı. hürriyet. nisan 2020’de https://www.hurriyet.com.tr/egitim/olcmedegerlendirmede-degisim-zamani-40790292 adresinden ulaşılmıştır. maulidah, s. s., & prima, e. c. (2018). using physics education technology as virtual laboratory in learning waves and sounds. journal of science learning, 1(3), 116-121. meb (2016). pisa 2015 ulusal raporu . http://pisa.meb.gov.tr/?page_id=22 adresinden alınmıştır. bakanlığı, m. e. (2018). liselere geçiş sistemi (lgs): merkezi sınavla yerleşen öğrencilerin performansı. eğitim analiz ve değerlendirme raporları serisi, (3). metin, m. & demiryürek, ö. (2009). türkçe öğretmenlerinin yenilenen türkçe öğretim programlarının ölçmedeğerlendirme anlayışı hakkındaki düşünceleri. ondokuz mayıs üniversitesi eğitim fakültesi dergisi. 28, 37‐51. nentwig, p. m., demuth, r., parchmann, i., ralle, b., & gräsel, c. (2007). chemie im kontext: situating learning in relevant contexts while systematically developing basic chemical concepts. journal of chemical education, 84(9), 1439. özden, m., akgün, a., çinici, a., sezer, b., yıldız, s., & taş, m. m. (2014). merkezi sistem ortak sınav fen bilimleri sorularının webb’in bilgi derinliği seviyelerine göre analizi. adıyaman üniversitesi fen bilimleri dergisi, 4(2), 91–108. özmen, h., ayas, a., & coştu, b. (2002). fen bilgisi öğretmen adaylarının maddenin tanecikli yapısı hakkındaki anlama seviyelerinin ve yanılgılarının belirlenmesi. kuram ve uygulamada eğitim bilimleri, 2(2), 507-529. peterson, r. ve treagust, d. (1989). grade – 12 students’ misconceptions of covalent bonding and structure. journal of chemical education, 66(6), 459 – 460. prima, e. c., putri, a. r., & rustaman, n. (2018). learning solar system using phet simulation to improve students' understanding and motivation. journal of science learning, 1(2), 60-70. rennie, l. j., & parker, l. h. (1996). placing physics problems in reallife context: students’ reactions and performance. australian science teachers journal, 42(1), 55-59. sak, m., & kaltakçi gürel, d. e. r. y. a. (2018). öğrencilerin işık konusundaki bağlam temelli sorular ile geleneksel soruları cevaplama düzeylerinin cinsiyete göre karşılaştırılması. yüzüncü yıl üniversitesi eğitim fakültesi dergisi, 15(1), 672-697. şenel, t., çepni, s., yıldırım, n., & er nas, s. (2007). süreç odaklı değerlendirmede kullanılabilecek bir analitik rubriğin geliştirilmesi: yaşamımızdaki elektrik ünitesi örneği. yeditepe üniversitesi eğitim fakültesi dergisi, 2(2). tekbıyık, a. & akdeniz, a. r. (2010). bağlam temelli ve geleneksel fizik problemlerinin karşılaştırılması üzerine bir inceleme. necatibey eğitim fakültesi elektronik fen ve matematik eğitimi dergisi, 4(1), 123140. ültay, n., & usta, n. d. (2016). investigating prospective teachers’ ability to write context-based problems/öğretmen adaylarının bağlam temelli problem yazabilme becerilerinin belirlenmesi. eğitimde kuram ve uygulama, 12(2), 447-463. ward, c. r., & herron, j. d. (1980). helping students understand formal chemical concepts. journal of research in science teaching, 17(5), 387-400. wilkinson, j. w. (1999). the contextual approach to teaching physics. australian science teachers journal, 45(4), 43-50. woessmann, l. (2018). central exit exams improve student outcomes. iza world of labor http://pisa.meb.gov.tr/?page_id=22 a © 2020 indonesian society for science educator 1 j.sci.learn.2020.4(1).1-7 received: 17 august 2020 revised: 22 november 2020 published: 28 november 2020 relationships between teaching experience and teaching ability with tpack: perceptions of mathematics and science lecturers at an islamic university muhammad haviz1*, ika metiza maris2, elda herlina2 1department of biology education, faculty of science education, iain batusangkar indonesia 2department of mathematics education, faculty of science education, iain batusangkar indonesia *corresponding author. mhaviz@iainbatusangkar.ac.id abstract the purpose of this study was to measure islamic university mathematics and science lecturers (iu-msls) perception of technological pedagogical content knowledge (tpack) in 21st-century learning. this research utilized a quantitative method derived from a descriptive survey collected from 48 lecturers at a single university. confirmatory factor analysis (cfa) and alpha cronbach methods were applied to determine the quality of the instruments used in this study. furthermore, descriptive statistics and anova were used to analyze the data obtained, while correlations were used to test the hypotheses. the results showed no relationship between teaching experience (te) and teaching ability (ta) with tpack. in addition, there were no differences in lecturers' perceptions of tpack based on the material being taught. this study concluded that tpack is an essential competency for mathematics and science lecturers in an islamic university. keywords technological pedagogical content knowledge (tpack), teaching subject, teaching experience, islamic university of mathematics and science lectures (iu-msls) 1. introduction over the past two decades, educators have determined strategies to prepare students and prospective teachers to navigate the increasingly globalized world and the interconnected landscape associated with the 21st century (teo, 2019). prospective teachers need valuable skills to deal with the competitive global changes, which students need to prepare themselves after graduating from college (kaufman, 2013; larson & miller, 2011). technological pedagogical content knowledge (tpack) was needed by the teacher (elas, majid, & narasuman, 2019; mulyadi, wijayatingsih, budiastuti, ifadah, & aimah, 2020; yigit, 2014). according to aacte (2010), tpack is a 21st-century skill that teachers must possess. chai, koh & tsai (2013) used 74 articles on tpack to integrate and transform the skills needed by an information communication and technology (ict) teacher in the classroom. furthermore, chen & xie (2018) showed that prospective teachers' skill needs to be possessed due to its relationship with their characteristics. this explanation also shows that tpack is a hot topic that requires adequate studies related to mastering mathematics or science (geisinger, 2016). the study's findings show an opportunity to investigate tpack together with a variety of determinants and perspectives, such as experience, abilities, and study subjects. previous studies investigated the use of teaching strategies and determined the factors associated with learning mathematics and science. tondeur, scherer, siddiq, & baran (2020) explored the strategies' effectiveness by using the synthesis of qualitative evidence (sqd) model used to prepare preservice teachers for tpack and found that it provides recommendations to improve their potential. kan’an (2018) determined the relationship between jordanian students' 21st-century skills (cs21) and academic achievement in science and found that female urban students performed better than their rural male counterparts. tokmak, incikabi, & ozgelen (2012) investigated the effect of tpack on mathematics, science, and literacy education preservice teachers' and found no significant differences between natural and social science. mailto:mhaviz@iainbatusangkar.ac.id journal of science learning article doi: 10.17509/jsl.v4i1.27327 2 j.sci.learn.2020.4(1).1-7 however, other factors such as age, gender, number of years, and the subject area also influence their ability. according to allen, singh, & rowan (2019), a teacher's characteristics affect their professional experience. fauth et al. (2019) stated a relationship between teacher competency, teaching quality, and student outcomes. this result is because teachers with academic education have better experiences (dijkema, doolaard, ritzema, & boske, 2019). sladek, bond, & phillips (2010) reported that there are gender and age differences in the thinking process, of men, adults, and teenagers. according to warren, apps, hoskins, azmi, & boyce (2018), age is positively related to creative performance. however, the research conducted by liang, chai, koh, yang, & tsai (2013), and koh, chai, & tsai (2014) showed that it was negatively associated with technological knowledge (tk), technological pedagogical knowledge (tpk), and technological pedagogical content knowledge (tpck). in other factors, the experience was a determining factor that contributes to tpack. liu, zhang, & wang (2015) found that teachers with less experience had significantly higher technological integrative knowledge. however, senior teachers had significantly higher pk and ck than those with less experience (cheng & xie, 2018). there are indeed studies showing that teaching time is negatively associated with technological knowledge (tk), technological pedagogical knowledge (tpk), technological content knowledge (tck), and technological pedagogical content knowledge (tpck) and positively related to content knowledge (ck) and pedagogical content knowledge (pck) (koh, chai, & tay, 2014). however, in research carried out by louws, meirink, veen, & driel (2017) on teachers’ years of experience, negative linear trends were found on their learning activities. the research of evrim & feral (2004) also showed several patterns between the teaching styles of science teachers and their education majors, professional development, and years of experience. according to the results, many science teachers tend to use individual styles in their learning environments. this finding showed that teachers have the responsibility to guide each student throughout the learning process. furthermore, a study on the relationship between the teacher's chosen learning sphere and experience found that their participation gradually decreases as they become more experienced (richter, kunter, klusmann, lüdtke, & baumert, 2011). in natural, shulman (1986) introduced the pedagogical content knowledge (pack), due to the difference between pedagogical knowledge (pk) and content knowledge (ck). mishra & koehler (2006) perfected the pack by adding technological knowledge (tk) to obtain the tpack terminology to complement a teacher's expertise. therefore, tpack's emphasis lies in the effectiveness of technology, pedagogy, and content knowledge (thompson & mishra 2007). mishra & koehler (2006) stated that the tpack framework in the seven bodies of knowledge needed for technology integration as shown in figure 1are as follows (1) technological knowledge (tk) — knowledge of technology tools, (2) pedagogical knowledge (pk) —knowledge of teaching methods, (3) content knowledge (ck) —knowledge of the subject matter, (4) technological content knowledge (tck) —knowledge of subject matter representation with technology, (5) technological pedagogical knowledge (tpk) —knowledge of using technology to implement different teaching methods, (6) pedagogical content knowledge (pck) — knowledge of teaching methods concerning subject matter content, and (7) technological pedagogical content knowledge (tpack) —knowledge of using technology to implement teaching methods for different types of subject matter content (cox & graham, 2009; koehler & mishra, 2009; mishra & koehler, 2006). however, a clear gap found in the application of integrated learning in islamic universities is not yet clearly determined the type of skills needed by students. if it is related to the application in class, there are not many reports of studies on the application and type of skills needed by students. so, this study conduct to measure the perception of tpack post the implementation of integrated learning by islamic university mathematics and science lectures (iu-msls). in this study, the authors measure the islamic university mathematics and science lectures’ (iu-msls) perception on technological pedagogical content knowledge based on two; the relationship between teaching experience (te) and teaching abilities (ta) with tpack, and differences in lecturers' perceptions on tpack based on teaching the subject. figure 1 tpack framework (koehler & mishra, 2009) journal of science learning article doi: 10.17509/jsl.v4i1.27327 3 j.sci.learn.2020.4(1).1-7 according to the directorate general of islamic education ministry of religion of the republic of indonesia number 2498 in the year 2019, integrated learning was a characteristic of studying islamic university in indonesia. previously, this integrated instruction was developed independently by each islamic university in indonesia. since 2016, iain batusangkar indonesia has also implemented integrated learning. the application of integrated instruction during the learning process refers to the university's integrative learning guidelines. the application of this integrated learning improves student skills and learning outcomes. for example, research is conducted by haviz (2016) and haviz, lufri, fauzan, & efendi (2012). both studies have integrated embryology with the quran at islamic universities. although with different content are integrated instruction researches on their respective content. the differences identified between a flipped classroom and a non-flipped classroom instructional model, and the results of the study showed that the out of class activities included the sharing of short video clips uploaded to the institutional learning management system for students' access before class had successfully established the basic psychological needs of self-determination theory. 2. method this study utilized the quantitative method with a descriptive survey (gay, mills, & airasian, 2009; creswell, 2014). data were obtained from 48 lecturers that have taught at iain batusangkar, west sumatra, indonesia. these lecturers were between the ages of 30-50 years old with an average age of 38.54 and 26 women and 22 men. furthermore, a total of 9 and 39 lecturers are doctorate and masters’ degree holders, respectively. the subjects taught include mathematics (13 people or 27.08%), biology (15 people or 31.25%), physics (12 people 25.00%), and chemistry (8 people or 16.67%). data were collected using the instrument developed by koh, chai & tsai (2010), which was designed based on the previous research conducted by schmidt et al. (2009b). in line with this, koh, chai & tsai (2010) stated that many other studies also used the instrument to investigate tpack students, such as the studies conducted by archambault & crippen (2009), graham et al. (2009), lee & tsai (2010), schmidt et al. (2009a), and schmidt, sahin, thompson, & seymour, (2008). this questionnaire contains positive and negative questions with scores of 1, 2, 3, 4, and 5 in the very disagree, not agree, neutral, agree, table 1 pattern/structure coefficients for tpack factor 1 ck factor 2 pk factor 3 pck factor 4 tk factor 5 tpk factor 6 tck factor 7 tpack ck1 0.62 ck2 0.92 ck3 0.55 pk1 0.67 pk2 0.44 pk3 0.57 pk4 0.70 pk5 0.46 pk6 0.52 pck1 0.61 pck2 0.76 pck3 0.81 tk1 0.04 tk2 1.01 tk3 0.65 tk4 0.56 tk5 0.08 tk6 0.19 tpk1 0.60 tpk2 0.50 tpk3 0.87 tpk4 0.90 tpk5 0.92 tck1 0.18 tck2 1.55 tck3 0.50 tpack1 0.76 tpack2 0.73 tpack3 0.82 tpack4 1.00 journal of science learning article doi: 10.17509/jsl.v4i1.27327 4 j.sci.learn.2020.4(1).1-7 and very agree on categories. in this study, the two instruments' validity and reliability tests were carried out using the cfa and cronbach alpha tests. the data analysis technique used refers to the survey technique described by creswell (2014). the steps in analyzing the data are as follows: (1) make a report on the number of sample members surveyed/not surveyed, (2) create a table of the number of respondents and their percentages, (3) discuss and estimate the bias of respondents with the research team and their effects on the study, (4) calculate data by using descriptive statistics in the form of percentages, averages and standard deviations using spss 21 for windows. furthermore, these data are displayed in tables, graphs, or diagrams. (5) the confirmatory factor analysis (cfa) and cronbach's alpha were widely used by previous researchers such as suhr (2018), chai, deng, tsai, & koh (2015), jia, oh, sibuma, labanca, and lorentson (2016), and sang, liang, chai, dong, & tsai (2018), to prove the hypothesis, statistics used inferences with anova and correlation. the inference test results are interpreted in a tabular form, and the conclusions obtained are tested at 5% and 1% confidence levels. the validity and reliability tests used the cfa and alpha cronbach methods to determine the instrument's goodness and structure for the research data collection. in this study, the loading factor used was above and below 0.1, respectively, on the relevant and irrelevant factors. the cfa test results on 7 factors, namely ck, pk, pck, tk, tpk, tck, and tpack, are shown in table 1. the data shows that the validity score is in the range 0.04 1.55, with the tpack coefficients above 0.1. this finding showed that all the tpack questionnaire statements were valid, with the two questionnaires consisting of a good level of reliability. cronbach's alpha score for the tpack questionnaire was 0.920 with n=30 items. finally, it was concluded that the two questionnaires in this study were valid and reliable and used to collect further research data. a correlation test was used to determine the relationship between teaching experience and teaching ability with tpack and anova test was used to determine differences in teachers' perceptions about tpack based on teaching subject 3. result and discussion 3.1 the correlation between teaching experience (te) and teaching ability (ta) of islamic university mathematics and science lectures (iu-msls) with technological pedagogical content knowledge (tpack) the study results about the relationship between te and ta of iu-msls with tpack are shown in table 2. this study found the lowest te scores in the tk factor, at -0.340, and the highest in the tpack at 0.128. the study also found that the lowest ta score in ck, at -0,138, and the highest in the pck factor, at 0.680. the correlation test results showed that there is no relationship between te and ta with the tpack factors. 3.2 differences in islamic university mathematics and science lectures (iu-msls) perceptions about tpack based on teaching subject (ts) the studies on iu-msls perceptions about tpack based on teaching subject (ts) are shown in table 3. in mathematics, the lowest and highest scores of 3.09 (0.98) and 4.02 (0.98), were found in pck and pk, respectively. in biology, the lowest and highest scores of 3.24 (0.50) and 4.07 (0.54) were found in pck and ck, respectively. in physics, the lowest and highest scores of 2.67 (0.76), and 3.50 (0.50) were found in pck and the pk, respectively. in chemistry, the lowest and highest scores were 3.58 (0.52), and 3.8 (0.72), respectively. this finding also showed that there is no relationship between mathematics and science with tpack. this finding also showed that mathematics, biology, physics, and chemistry considered tpack a critical competency to be mastered by islamic university mathematics and science lectures (iu-msls). table 2 correlation between the te and ta of iu-msls and the factors of tpack ck pk pck tk tpk tck tpack teaching experience -0.202 0.032 0.184 -0.340 0.116 0.021 0.128 teaching ability -0.138 -0.111 0.680 -0.309 0.063 0.144 0.043 * p<.05, **p<.01 table 3 differences in islamic university mathematics and science lectures (iu-msls) about tpack based on teaching subject (ts) ck (m, sd) pk (m, sd) pck (m, sd) tk (m, sd) tpk (m, sd) tck (m, sd) tpack (m, sd) math 3.52 (0.74) 4.02 (0.45) 3.09 (0.98) 3.69 (0.45) 3.54 (0.61) 3.57 (0.46) 3.32 (0.67) bio 4.07 (0.54) 3.89 (0.30) 3.24 (0.50) 3.67 (0.46) 3.76 (0.53) 3.77 (0.28) 3.53 (0.55) phys 3.44 (0.35) 3.50 (0.50) 2.67 (0.76) 3.42 (0.31) 3.20 (0.94) 3.17 (0.86) 2.96 (1.01) chemist 3.78 (0.39) 3.72 (0.19) 3.44 (0.51) 3.50 (0.17) 3.80 (0.72) 3.78 (0.84) 3.58 (0.52) f (anova) 2.078 2.122 1.072 0.664 0.967 1.555 0.931 scheffe test journal of science learning article doi: 10.17509/jsl.v4i1.27327 5 j.sci.learn.2020.4(1).1-7 in this study, the results of the validity and reliability of the tpack questionnaires were valid and reliable. this finding, according to the tpack was developed in previous research. koh, chai & tsai (2010) wrote 27 items that measure 5 factors of technology: knowledge, content, pedagogy, teaching, and critical reflection. this questionnaire results from the development of previous research conducted by schmidt et al. (2009b). this study, regarding the benefits and use of instruments, is also similar to others previous studies as follows: (a) survey of preservice teacher knowledge and technology (schmidt et al., 2009a), (b) survey of pedagogical knowledge and technology content (sahin, 2011), (c) assessing students' perceptions about pck of college teachers (jang, guan & hsieh, 2009; jang & tsai, 2012), and (d) tpack in science survey questions (graham et al. 2009). these four surveys were also used as a basis by lee & kim (2017) to develop their survey questionnaire. the tpack survey modified by lee & kim (2017) contains 55 items used to measure 7 tpack knowledge domains: 16 tk items, 8 ck items, 9 pk items, 7 pck items, 6 tck items, 5 tpk items, and 4 items tpack. according to taber (2017), the results of this study showed that cronbach's alpha is relevant for reliable testing instruments used to collect the data. the results of other studies showed that surveys on students' perceptions of critical thinking, creative thinking, and authentic problem solving were dominant predictors in 21st-century learning practice (chai, deng, tsai, & koh, 2015; jia, oh, sibuma, & lorentson, 2016; ercikan & oliveri, 2016). the finding of the relationship between te and ta of iu-msls with tpack showed that different teaching abilities and experiences still consider tpack as an essential competency to be mastered by iu-msls. teaching abilities and teaching experience was no relationship between tpack. because tpack was a part of the 21st-century skill. this finding assumed that the lectures with old age show less or no competencies technology content, such as information and communication skill. thus, these results also showed that 21st-century skills and tpack need to be integrated into learning, which focuses not only on knowledge (herde, wüstenberg & greiff, 2016; silva, 2009). this finding also showed that tpack is competencies will be used to increase students' ability to master information and communication technology (ict). this finding is also found in various articles that have been written by previous researchers. for example, the articles were writing by koh, chai, & tsai (2010), koh (2013), koh & chai (2014), koh, chai, & tay (2014). cai, koh, tsai, & tan (2011) stated a clear link between both as part of the 21st-century skills. this statement has also supported the finding of this study. in this study, the result about relationships between teaching ability of iu-msls with tpack showed that factor age, numbers of years, and the subject area is negatively associated with tpack. this finding accordingly with previous research. sladek, bond, & phillips (2010) reported gender and age differences in men, adults, and teenagers' thinking processes. according to warren, apps, hoskins, azmi, & boyce (2018), age is positively related to creative performance. however, the research conducted by liang, chai, koh, yang, & tsai (2013), and koh, chai, & tsai (2014) showed that it was negatively associated with tk, tpk, and tpck. in others, liu, zhang, & wang (2015) found that teachers with less experience had significantly higher technological integrative knowledge. however, senior lecturers had significantly higher pk and ck than those with less experience (cheng & xie, 2018). studies show that teaching time is negatively associated with tk, tpk, tck, and tpck and positively related to ck and pck (koh, chai, & tay, 2014). in research carried out by louws, meirink, veen, & driel (2017) on teachers’ years of experience, negative linear trends were found in their learning activities. evrim & feral's (2004) research also showed several patterns between the teaching styles of science lectures and their education majors, professional development, and years of experience. according to the study results, many science lecturers tend to use individual styles in their learning environments. this finding showed that lecturers have the responsibility to guide each student throughout the learning process. furthermore, for this implication, a study on the relationship between the lectures chosen to learn sphere and experience found that their participation gradually decreases as they become more experienced (richter, kunter, klusmann, lüdtke, & baumert, 2011). 4. conclusion the resulting study showed that islamic university mathematics and science lectures (iu-msls) stated tpack are essential competencies for mathematics and science lectures in islamic university. the results also showed no relationship between teaching experience (te) and teaching ability (ta) with tpack and there are no differences in lecturers' perceptions on tpack based on teaching the subject. this study recommended that tpack are needed competencies for mathematics and science lectures in islamic university. acknowledgment the authors gratefully acknowledge to all participants. this work was supported by iain batusangkar references aacte. (2010). press releases & statements the american association of colleges for teacher education and p21 release paper on 21st century knowledge and skills in educator preparation. https://aacte.org/newsroom/press-releases-statements/88-the-american-association-ofcolleges-for-teacher-education-and-p21-release-paper-on-21stcentury-knowledge-and-skills-in-educator-preparation. allen, j., singh, p., & rowan l. (2019) professional experience in initial teacher education: keeping abreast of change in the 21st century. journal of science learning article doi: 10.17509/jsl.v4i1.27327 6 j.sci.learn.2020.4(1).1-7 asia-pacific journal of teacher education, 47(4), 323-326. https://doi.org/10.1080/1359866x.2019.1637599. archambault, l., & crippen, k. (2009). examining tpack among k12 online distance educators in the united states. contemporary issues in technology and teacher education, 9(1), 71-88. http://www.citejournal.org/vol9/iss1/general/article2.cfm. chai, c. s., deng, f., tsai, p. s., & koh, j. h. (2015). assessing multidimensional students’ perceptions of twenty-fistcentury learning practices. asia pacific education review, 16(3), 389–398. https://doi.org/10.1007/s12564-015-9379-4. chai, c. s., koh, j. h. l., & tsai, c. c. (2013). a review of technological pedagogical content knowledge. educational technology & society, 16(2), 31–51. https://www.questia.com/read/1g1-331807147/a-review-oftechnological-pedagogical-content-knowledge. chai, c. s., koh, j. h. l., tsai, c.-c., & l. w. tan., (2011). modeling primary school preservice teachers’ technological pedagogical content knowledge (tpack) for meaningful learning with information and communication technology (ict). computers & education, 57(1), 1184–1193. https://doi.org/10.1016/j.compedu.2011.01.007. cheng, s. l., & xie, k. (2018). the relations among teacher value beliefs, personal characteristics, and tpack in intervention and nonintervention settings. teaching and teacher education, 74, 98-113. https://doi.org/10.1016/j.tate.2018.04.014. cox, s., & graham, c. r. (2009). diagramming tpack in practice: using and elaborated model of the tpack framework to analyse and depict teacher knowledge. techtrends, 53(5), 60–69. https://doi.org/10.1007/s11528-009-0327-1. creswell, j. w. (2014). research design; quantitative, qualitative and mixed method approaches (4th ed.). california: sage publication, inc. dijkema, s., doolaard, s., ritzema, e.s., & boske, r.j. (2019). ready for take-off? the relation between teaching behavior and teaching experience of dutch beginning primary school teachers with different educational backgrounds. teaching and teacher education, 86, 102914. https://doi.org/10.1016/j.tate.2019.102914. elas, n, i, b., majid, f, b, a., & narasuman, s, a, i. (2019). development of technological pedagogical content knowledge (tpack) for english teachers: the validity and reliability. international journal of emerging technologies in learning (ijet), 14(20), 18-33. https://doi.org/10.3991/ijet.v14i20.11456 ercikan, k., & oliveri, m. e. (2016). in search of validity evidence in support of the interpretation and use of assessments of complex constructs: discussion of research on assessing 21st century skills. applied measurement in education, 29(4), 310-318. http://dx.doi.org/10.1080/08957347.2016.1209210. evrim, g., & feral, o. b. (2004). patterns in teaching styles of science teachers in florida and factors influencing their preferences. reports – research. eric number: ed490781. https://files.eric.ed.gov/fulltext/ed490781.pdf. fauth, b., decristan, j., decker, a. t., büttner, g., hardy, i., klieme, e., & kunter, m. (2019). the effects of teacher competence on student outcomes in elementary science education: the mediating role of teaching quality. teaching and teacher education, 86, 102882. https://doi.org/10.1016/j.tate.2019.102882. gay, l. r., mills, g. e., & airasian, p. w. (2009). educational research, competencies for analysis and application (9th. ed). new jersey: pearson education. geisinger, k. f. (2016). 21st century skills: what are they and how do we assess them? applied measurement in education, 29(4), 245-249. http://dx.doi.org/10.1080/08957347.2016.1209207 graham, r. c., burgoyne, n., cantrell, p., smith, l., st. clair, l., & harris, r. (2009). measuring the tpck confidence of inservice science teachers. techtrends, 53(70), 70-79. https://doi.org/10.1007/s11528-009-0328-0. haviz, m. (2016). designing and developing the integrated learning model on embryology. transylvanian review, 24(7), 1043-1052. http://transylvanianreviewjournal.org/index.php/tr/article/vie w/2998 haviz, m., lufri, fauzan, a., & efendi, z. m. (2012). pengembangan model pembelajaran integratif pada biologi perkembangan hewan: analisis kebutuhan pengembangan. ta'dib, 15(1), 1-14. http://ecampus.iainbatusangkar.ac.id/ojs/index.php/takdib/artic le/viewfile/213/212 herde, c.n., wüstenber, s & greiff, s. (2016). assessment of complex problem solving: what we know and what we don’t know. applied measurement in education, 29(4), 265-277. https://doi.org/10.1080/08957347.2016.1209208. jang, s. j., & tsai, m. f. (2012). exploring the tpack of taiwanese elementary mathematics and science teachers with respect to use of interactive whiteboards. computers & education, 59(2), 327e338. https://doi.org/10.1016/j.compedu.2012.02.003 jang, s. j., guan, s. y., & hsieh, h. f. (2009). developing an instrument for assessing college students’ perceptions of teachers’ pedagogical content knowledge. procedia social and behavioral sciences, 1(1), 596– 606. https://doi.org/10.1016/j.sbspro.2009.01.107. jia, y., oh, y. j., sibuma, b., labanca, f., & lorentson, m. (2016). measuring twenty-first century skills: development and validation of a scale for in-service and preservice teachers. teacher development; an international journal of teachers' professional development, 20(2), 229252. http://dx.doi.org/10.1080/13664530.2016.1143870. kan’an, a. 2018. the relationship between jordanian students’ 21 st century skills (cs21) and academic achievement in science. journal of turkish science education, 15(2), 82-94. https://doi.org/10.12973/tused.10232a. kaufman, k. j. (2013). 21 ways to 21st century skills: why students need them and ideas for practical implementation journal. kappa delta pi record, 49(2), 78-83. http://dx.doi.org/10.1080/00228958.2013.786594. koehler, m. j., & mishra, p. (2009). what is technological pedagogical content knowledge? contemporary issues in technology and teacher education (cite journal), 9(1), 60-70. https://citejournal.org/volume-9/issue-1-09/general/what-istechnological-pedagogicalcontent-knowledge. koh, j. h. l. (2013). a rubric for assessing teachers' lesson activities with respect to tpack for meaningful learning with ict. australasian journal of educational technology, 29(6), 887-900. https://doi.org/10.14742/ajet.228. koh, j. h. l., & chai, c. s. (2014). teacher clusters and their perceptions of technological pedagogical content knowledge (tpack) development through ict lesson design. computers & education, 70, 222–232. https://doi.org/10.1016/j.compedu.2013.08.017. koh, j. h. l., chai, c. s., & tay, l. y. (2014). tpack-in-action: unpacking the contextual influences of teachers’ construction of technological pedagogical content knowledge (tpack). computers & education, 78, 20–29. https://doi.org/10.1016/j.compedu.2014.04.022. koh, j. h. l., chai, c. s., & tsai, c. c. (2010). examining the technological pedagogical content knowledge of singapore preservice teachers with a large-scale survey. journal of computer assisted learning, 26(6), 563–573. https://doi.org/10.1111/j.13652729.2010.00372.x. koh, j. h. l., chai, c. s., & tsai, c. c. (2014). demographic factors, tpack constructs, and teachers' perceptions of constructivistoriented tpack. journal of educational technology & society, 17(1), 185e196. http://www.jstor.org/journal/jeductechsoci. larson, l. c., & miller, t. n. (2011). 21st century skills: prepare students for the future. kappa delta pi record, 47(3), 121-123. http://dx.doi.org/10.1080/00228958.2011.10516575 lee, c. j., & kim, c. (2017). a technological pedagogical content knowledge based instructional design model: a third version implementation study in a technology integration course. educational technology research and development, 65(6), 1627– 1654. https://doi.org/10.1007/s11423-017-9544-z. https://doi.org/10.1080/1359866x.2019.1637599 http://www.citejournal.org/vol9/iss1/general/article2.cfm https://www.questia.com/read/1g1-331807147/a-review-of-technological-pedagogical-content-knowledge https://www.questia.com/read/1g1-331807147/a-review-of-technological-pedagogical-content-knowledge https://doi.org/10.1016/j.compedu.2011.01.007 https://doi.org/10.1016/j.tate.2018.04.014 https://doi.org/10.1007/s11528-009-0327-1 https://doi.org/10.1016/j.tate.2019.102914 https://doi.org/10.3991/ijet.v14i20.11456 http://dx.doi.org/10.1080/08957347.2016.1209210 https://files.eric.ed.gov/fulltext/ed490781.pdf https://doi.org/10.1016/j.tate.2019.102882 http://dx.doi.org/10.1080/08957347.2016.1209207 https://doi.org/10.1007/s11528-009-0328-0 http://ecampus.iainbatusangkar.ac.id/ojs/index.php/takdib/article/viewfile/213/212 http://ecampus.iainbatusangkar.ac.id/ojs/index.php/takdib/article/viewfile/213/212 https://doi.org/10.1080/08957347.2016.1209208 https://doi.org/10.1016/j.compedu.2012.02.003 https://doi.org/10.1016/j.sbspro.2009.01.107 http://dx.doi.org/10.1080/13664530.2016.1143870 https://doi.org/10.12973/tused.10232a http://dx.doi.org/10.1080/00228958.2013.786594 https://citejournal.org/volume-9/issue-1-09/general/what-is-technological-pedagogicalcontent-knowledge https://citejournal.org/volume-9/issue-1-09/general/what-is-technological-pedagogicalcontent-knowledge https://doi.org/10.14742/ajet.228 https://doi.org/10.1016/j.compedu.2013.08.017 https://doi.org/10.1016/j.compedu.2014.04.022 https://doi.org/10.1111/j.1365-2729.2010.00372.x https://doi.org/10.1111/j.1365-2729.2010.00372.x http://www.jstor.org/journal/jeductechsoci http://dx.doi.org/10.1080/00228958.2011.10516575 https://doi.org/10.1007/s11423-017-9544-z journal of science learning article doi: 10.17509/jsl.v4i1.27327 7 j.sci.learn.2020.4(1).1-7 lee, m. h., & tsai, c. c. (2010). exploring teachers’ perceived selfefficacy and technological pedagogical content knowledge with respect to educational use of the world wide web. instructional science, 38, 1-21. https://doi.org/10.1007/s11251-008-9075-4. liang, j. c., chai, c. s., koh, j. h. l., yang, c. j., & tsai, c. c. (2013). surveying in-service preschool teachers' technological pedagogical content knowledge. australasian journal of educational technology, 29(4), 581e594. https://doi.org/10.14742/ajet.299. liu, q., zhang, s., & wang, q. (2015). surveying chinese in-service k12 teachers' technology, pedagogy, and content knowledge. journal of educational computing research, 53(1), 55e74. https://doi.org/10.1177/0735633115585929. louws, m. l., meirink, j. a., veen, k. v., & driel. j.h.v. (2017). teachers' self-directed learning and teaching experience: what, how, and why teachers want to learn. teaching and teacher education, 66, 171-183. https://doi.org/10.1016/j.tate.2017.04.004. mishra, p., & koehler, m. j. (2006). technological pedagogical content knowledge: a framework for teacher knowledge. teachers college record, 108(6), 1017-1054. https://doi.org/10.1111/j.14679620.2006.00684.x. mulyadi, d., wijayatingsih, t, d., budiastuti, r. e., ifadah, m., & aimah, s. (2020). technological pedagogical and content knowledge of esp teachers in blended learning format. international journal of emerging technologies in learning (ijet), 15(06), 124-139. https://doi.org/10.3991/ijet.v15i06.11490 richter, d., kunter, m., klusmann, u., lüdtke, o., & baumert, j. (2011). professional development across the teaching career: teachers' uptake of formal and informal learning opportunities. teaching and teacher education, 27(1), 116e126. http://dx.doi.org/10.1016/j.tate.2010.07.008. sahin, i. (2011). development of survey of technological pedagogical and content knowledge (tpack). turkish online journal of educational technology, 10(1), 97–105. http://www.tojet.net/articles/v10i1/10110.pdf. sang, g., liang, j. c., chai, c. s., dong, y., & tsai, c. c. (2018). teachers' actual and preferred perceptions of twenty-first century learning competencies: a chinese perspective. asia pasific education review, 19(3), 307-317. https://doi.org/10.1007/s12564-018-95220. schmidt, d. a, baran e., thompson a.,koehler m., punya m.& shin t.s. (2009a) examining preservice teachers’ development of technological pedagogical content knowledge in an introductory instructional technology course. paper presented at the society for informationtechnology & teacher education international conference, chesapeake, va, march 2–6. schmidt, d. a., baran e., thompson a. d., mishra p., koehler m.j. & shin t.s. (2009b). technological pedagogical content knowledge (tpack): the development and validation of an assessment instrument for preservice teachers. journal of research on technology in education 42, 123–150. https://files.eric.ed.gov/fulltext/ej868626.pdf. schmidt d. a., sahin e. b., thompson a., & seymour j. (2008). developing effective technological pedagogical and content knowledge (tpack) in prek-6 teachers. paper presented at the society for information technology and teacher education international conference, chesapeake, va, march 3–7. shulman, l. s. (1986). those who understand: knowledge growth in teaching. educational researcher, 15(2), 4–14. https://doi.org/10.3102/0013189x015002004. silva, e. (2009). measuring skills for 21st-century learning. phi delta kappan, 90(9), 630–634. https://doi.org/10.1177/003172170909000905 sladek, r.m., bond, m.j., & phillips, p.a. (2010). age and gender differences in preferences for rational and experiential thinking. personality and individual differences, 49(8), 907-911. https://doi.org/10.1016/j.paid.2010.07.028. suhr, d. d. (2018). exploratory or confirmatory factor analysis? http://www2.sas.com/proceedings/sugi31/200-31.pdf. taber, k. s. (2017). the use of cronbach’s alpha when developing and reporting research instrumen in science education. res. sci. educ. https://doi.org/10.1007/s11165-016-9602-2. teo, p. (2019). teaching for the 21st century: a case for dialogic pedagogy. learning, culture and social interaction, 21, 170-178. https://doi.org/10.1016/j.lcsi.2019.03.009. thompson, a. d., & punya mishra, p. (2007) editors' remarks. journal of computing in teacher education, 24(2), 38-64. https://doi.org/10.1080/10402454.2007.10784583. tokmak, h.s., incikabi, l. & ozgelen, s. (2012). an investigation of change in mathematics, science, and literacy education preservice teachers’ tpack. the asia-pacific education researcher, 22(4), 407– 415. https://doi.org/10.1007/s40299-012-0040-2. tondeur, j., scherer, r., siddiq, f., & baran, e. (2020). enhancing preservice teachers’ technological pedagogical content knowledge (tpack): a mixed-method study. education tech research dev 68(1), 319–343. https://doi.org/10.1007/s11423-019-09692-1. warren, f., apps, e. m., hoskins, s., azmi, z., & boyce, j. (2018). the role of implicit theories, age, and gender in the creative performance of children and adults. thinking skills and creativity, 28, 98-109. https://doi.org/10.1016/j.tsc.2018.03.010. yigit, m. (2014). a review of the literature: how preservice mathematics teachers develop their technological, pedagogical, and content knowledge. international journal of education in mathematics, science and technology, 2(1), 26-35. https://doi.org/10.1007/s11251-008-9075-4 https://doi.org/10.14742/ajet.299 https://doi.org/10.1177/0735633115585929 https://doi.org/10.1016/j.tate.2017.04.004 https://doi.org/10.1111/j.1467-9620.2006.00684.x https://doi.org/10.1111/j.1467-9620.2006.00684.x https://doi.org/10.3991/ijet.v15i06.11490 http://dx.doi.org/10.1016/j.tate.2010.07.008 http://www.tojet.net/articles/v10i1/10110.pdf https://doi.org/10.1007/s12564-018-9522-0 https://doi.org/10.1007/s12564-018-9522-0 https://files.eric.ed.gov/fulltext/ej868626.pdf https://doi.org/10.3102%2f0013189x015002004 https://doi.org/10.1177/003172170909000905 https://doi.org/10.1016/j.paid.2010.07.028 https://doi.org/10.1007/s11165-016-9602-2 https://doi.org/10.1016/j.lcsi.2019.03.009 https://doi.org/10.1080/10402454.2007.10784583 https://doi.org/10.1007/s40299-012-0040-2 https://doi.org/10.1007/s11423-019-09692-1 https://doi.org/10.1016/j.tsc.2018.03.010 microsoft word 60-70 learning solar system using phet simulation to improve students’ understanding and motivation.docx a © 2018 indonesian society for science educator 60 j.sci.learn.2018.1(2).60-70 received: 1 march 2018 revised: 26 march 2018 published: 31 march 2018 learning solar system using phet simulation to improve students’ understanding and motivation eka cahya prima1*, aldia ridwani putri1, nuryani rustaman2 1department of science education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia 2department of biology education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia *corresponding author. ekacahyaprima@upi.edu abstract this research investigated the implementation of phet simulation on students’ understanding and motivation in learning the solar system. the method used in this research was quasi-experimental with matching pretest-posttest group design. the research conducted in one of private junior high school located at padalarang, indonesia with a number of students (n=42). the sampling technique used was the purposive sampling to determine experimental group with phet simulation in the learning science and control group without phet simulation. the quantitative data of this research was obtained through the objective test on the mastery concept solar system, while the qualitative data was detected through motivation rubric and questionnaire. the result shows that the students who learn the solar system with phet simulation have higher both improvement in conceptual understanding and motivation than without phet simulation as teaching media. according to the analysis result, there is a moderate correlation between conceptual understanding and motivation with the learning solar system in the use phet simulation. keywords phet simulation, solar system, students’ motivation, students’ understanding 1. introduction curriculum in indonesia has progressed. during changing of curriculum, there is no other purpose to improve the quality of the learning process and the existing of learning design in the school. therefore, the curriculum in indonesia will continue to evolve and change to need of society and demands of the time. the curriculum 2013 is expected to improve the quality of education in indonesia. this is in line with the undang-undang nomor 20, 2016 that the curriculum is an arrangement of objectives, content, and lesson materials and ways used as guidelines for organizing learning activities to achieve specific educational. in the development of 2013 curriculum, there are various problems and challenges among others related to the globalization and free market, environmental issues, the rapid advancement of information and technology, and transformation in the education sector as well as timss and pisa materials that must be owned by the learner. based on the result of program for international student assessment (pisa) showed that the performance of indonesia students is still relatively low. the achievements of indonesia evaluated by pisa for science, reading, and math are respectively is ranked 62, 61, and 63 of 69 countries (kemendikbud, 2016). the rating and average score of indonesia do not differ from previous pisa 2012 test result and surveys in 2012, which is also in the low material mastery group. however, the oecd not only conducts a test of those three areas, but also they measure a variety of indicators from surveying many matters related to educational activities. the result showed that the index of enjoyment of learning science of indonesia is quite higher than index obtained by countries that scored high such as singapore and japan. many works are attempted to improve indonesia pisa score. one of the efforts is conducted by introducing the use of technology in order to help students’ science literacy. there are several aspects affecting the pisa result such as the role of parents, socioeconomic background, students’ attitude and the quality of learning, and school condition and infrastructure facilities (nizam, 2016). the progress science and technology engages the educational world improving the quality of teaching media. for example, almost all levels of educational institutions in journal of science learning article 61 j.sci.learn.2018.1(2).60-70 indonesia have been using computer-based media education (fitrianingrum, 2014). in educational process, using technology will make students interested in learning science. the computer simulation is one kind of tools that are always used in teaching process to improve students understanding of difficult concept effectively instead of traditional method (alsultanny, et al., 2014). according to bell & smetana (2015), the computer simulation is computer-generated dynamic models, which can explain the concept or simplified model of real-world component, phenomena, or process of concept consisting of animation, visualization, and interactive laboratory experiences. technological advance will bring the instructional digital technologies into science classroom to make these types being interactive, authentic, and meaningful learning opportunities. learning physics is one of the many lessons requiring the media to convey or explain the material (sarofi, 2014). physics also requires experience directly students. one of the roles instructional media is giving students experiences that cannot be obtained directly but it can be presented well using in the teaching media. the main problem of teaching and learning of science especially physics is generally unattractive. consequently, many students who lack understanding the concept of physics become passive (haya, waskito, & fauzi, 2014). to handle this problem, an interactive media makes students feel happy and motivated is needed to enhance students’ science motivation. therefore, the lesson will not just transfer of verbal knowledge. motivation can encourage the student to learn or independently (glynn & kobala, 2006; o.lee, 1989). without motivation, students will not be interested and serious in the learning science. physics education technology (phet) is one of interactive computer simulation is being used in teaching physics and chemistry. phet interactive simulations project was developed by university of colorado boulder (perkins et al., 2012). phet is aimed to be implemented at the college level, but many researched were used in middle school classrooms that focused on studying the design and use of interactive simulations (wieman et al., 2008). the phet project (http://phet.colorado.edu) has developed more than 80 interactive simulations. phet simulations set through standard web browsers and they can be media teaching as lecturing, used as laboratories virtual or as homework assignments, or used as informal resources (wieman et. al., 2008). these cover various topics in physics and real-world applications, such as the greenhouse effect and lasers. there are simulations on chemistry topics, as well as several simulations for math, biology, and earth science. physics education technology (phet) will be promoted as a technological media in order to help learning science. phet is a site that provides free physics and chemistry learning simulation for download for the benefit of classroom teaching or can be used for individual learning purposes (wieman et al., 2010). the simulations provided by phet are highly interactive which invites students to learn by exploring directly. phet simulation creates an animated for abstract fact or invisible phenomena to be modeled by students, such as atoms, electrons, photons, and magnetic fields. in order to encourage a quantitative exploration, simulation also offers measurement instruments including rulers, stopwatches, voltmeter, and thermometers. users can manipulate an interactive tool; immediate animated responses visualize some related science representations such as the motion of objects, graphics, etc. based on these problems, the use of phet simulation is important to improve students’ understanding and motivation in learning the solar system. the use of this simulation can visualize the abstract phenomenon of the solar system. the correlation between students understanding and motivation towards implementation of phet simulation will be further investigated. our previous work has implemented the stellarium as a virtual observatory to improve student’s understanding of learning the solar system (prima, putri, & sudargo, 2017). other work reported the use of digital planetarium visualization to learn the solar system (yu, k. c., sahami, k., & dove, j., 2017). phet simulation was also used to conduct the lesson on quantum mechanics (mckagan, et al., 2008). phet is used to engage and empower students in science learning. the simulations are generally usable, engaging, and effective learning tools for students (perkins et al., 2006). moreover, the phet has successfully improved students’ achievement in the concept of light refraction. students feel enjoy using phet in class (cahyani, et. al., 2013). phet is applied to facilitate learning on waves and sound (maulidah, 2015). in this research, we will implement phet simulation to improve students’ understanding and motivation on learning the solar system. to best of our knowledge, this work shows the first report on the use of phet to learn the solar system. 2. method this research used quasi-experimental, where the researcher gave an assignment, but not random assignment to subject (creswell, 2012). the assignment was given includes pretest and post-test as administering a treatment (mcmillan & schumacher, 2001) for students’ understanding of the content solar system topic. the researcher chose the quantitative approach because of this approach based on the research objectives of this study what will be achieved. the main objective of this study is to determine the differences in students’ understanding when they are learning about solar system topic using and without phet simulation as teaching media in the learning process. in conducted research, it used two classes as a participant to this research, where one class as a control journal of science learning article 62 j.sci.learn.2018.1(2).60-70 group by using powerpoint presentation of solar system as the teaching media, and another class as an experiment that used phet simulation which entitled “gravity and orbits” as the teaching media. this research used the correlation analysis, which considers the relationship between two variables as defined association to share a common relationship with two variables, which are one called as “x”, another one called as “y”. the method used is describing the relation of two variables based on the value of combining quantitative and qualitative method (norušis, 2006). the method is chosen because it is the appropriate way to interpret the use phet simulation as teaching media to improve students understanding and motivation in learning solar system topic. the research design used was matching only pre-test and post-test control group for both two classes (fraenkel, wallen, & hyun, 2012). the implementer of the research design for two classes was student given pre-test before they used media learning as treatment and learned about the topic. the experimental group used phet simulation entitled “gravity and orbits” developed by the university of colorado to learn about the rotation and revolution of earth and moon and the impacts of them. while control group used powerpoint presentation as teaching media when they learned on the same topic with the experimental group. afterward, post-test distributed to two classes is the same questions with pre-test form. the form pre-test and post-test were used is a paper test of the multiple choice to measure the students’ understanding of the topic that student which have been learned during given the treatment. this study was conducted one of a private school at junior high school in padalarang, west bandung which period of 2016/2017. the school implemented kurikulum 2013 of national curriculum. the class activities in the school conducted indonesia and english as the instructional language. the population in this study was 8th-grade students who learn about solar system topic in the national curriculum. while for a sample of this study was two classes, which consist twenty-five students for each, class that chosen as a participant in this study. because the school has three classes for each grade, so researcher used a purposive sampling technique to determine the sample that will be used in this study. to decide which classes involved in this stage, researcher two pieces picked randomly of paper that consisted of three classes name. finally, two classes which have been chosen are viii a and viii b as a participant in this research. viii a determined as the experiment group, while viii b as the control group. objective test in a formed of multiple-choice question consists of twenty question to measure students’ understanding of the learning solar system topic especially the movement of the planet. the question distributed before and after learning activity for both control group and experiment group. the pre-test distributed before students learn about the topic to see their prior knowledge and post-test distributed after all content of the topic has delivered by the research in the class to see the improvement student’ knowledge after they used phet simulation for experiment group and powerpoint presentation for the control group. the cognitive domain was used, there are c1 (remembering) until c4 (analyzing). table 1 cognitive domain classification in pre-test and post-test question based on revised bloom taxonomy indicators cognitive domains the percentage of question number question c1 c2 c3 c4 the cause of rotation and revolution of earth 13, 16, 20 15% 3 gravity force of planet 1, 2, 3, 4, 6, 7, 8, 9, 11 45% 9 rotation and revolution of the earth 14, 15 10% 2 the cause moon rotation and revolution 5, 10, 12, 17, 18, 19 30% 6 table 2 analysis of core competence and basic competence on solar system topic (source: permendikbud, 2016) grade core competence basic competence topic sub topic viii 3. memahami pengetahuan (faktual, konseptual, dan prosedural) berdasarkan rasa ingin tahunya tentang ilmu pengetahuan, teknologi, seni, budaya terkait fenomena dan kejadian tampak mata. 3.1 menganalisis sistem tata surya, rotasi dan revolusi bumi, rotasi dan revolusi bulan, serta dampaknya bagi kehidupan di bumi solar system 1. component of solar system 2. earth rounds to the sun 3. rotation and revolution of moon 4. planets’ motion law 4. encoba, mengolah, dan menyaji dalam ranah konkret (menggunakan, mengurai, merangkai, memodifikasi, dan membuat) dan ranah abstrak (menulis, membaca, menghitung, menggambar, dan mengarang) sesuai dengan yang dipelajari di sekolah dan sumber lain yang sama dalam sudut pandang/teori 4.1 menyajikan karya tentang dampak rotasi dan revolusi bumi dan bulan bagi kehidupan di bumi, berdasarkan hasil pengamatan atau penelusuran berbagai sumber informasi journal of science learning article 63 j.sci.learn.2018.1(2).60-70 the classification of the cognitive domain in the objective test will present in table 1. according to peraturan menteri pendidikan dan kebudayaan republik indonesia nomor 24 tahun, 2016 is about core competence (kompetensi inti) and basic competence (kompetensi dasar) in the lessons curriculum 2013. for the topic, solar system in the seven grade/vii, which is, stated in the basic competence 3.11 and 4.11 as the achievement for a student in a learning activity. the description of core competence and basic competence in kurikulum 2013, as follows in table 2. to investigate the improvement of student’ motivation, this research used a motivational questionnaire that adopted by three questionnaires. they are computer attitude questionnaire (caq) by (knezek & christensen, 1996), science motivation questionnaire by (glynn & kobala, 2006), and determination theory perspective questionnaire (deci, eghrari, patrick,, & leone, 1994). students filled the questionnaire before and after learning activity as a same objective test. there are twenty statements. those are divided into three aspects questionnaire. the data that obtained from students informed four statements idea, which is “strongly agree”, “agree”, “disagree”, and “strongly disagree”. the classification of the statement will be presented the table 3. for all of the instruments that will be used has been analyzed and judged by expert judgment and using the software. for the objective test, which in form of multiplechoice questions that tested by anates 4.9.0 and judged by two experts judgment who professional in earth science subject. moreover, the objective test distributed to another school which students have been learned about solar system topic to test the question before objective test used in the treatment. the result of the test items was analyzed and revised to be used for the participant. 3. result and discussion there are some findings after conducted the research by comparing two groups of the sample, which used different teaching media in a learning activity. teaching media used for two group are control group by using power point, while experiment group by using phet simulation. both of two groups used different teaching media is to find 1) the improvement student’ understanding in learning the concept of solar system, 2) students’ cognitive level which is most influenced by using phet simulation in learning the concept of solar system, 3) students’ motivation in a learning activity. 3.1 implementation of phet simulation as teaching media in learning learning activities using phet simulation  the implementation of phet simulation conducted in the experimental group in school. in the class, students learned about solar system topic using phet simulation is shown in table 4. the table shows the implementation of phet simulation was conducted in six meetings. the result of implementation phet simulation can be seen from the activity in the classroom by recorded in video. in the classroom, students can use phet simulation was provided by a researcher in the computers lab and classroom. table 3 statement classification in motivation questionnaire based on aspect and condition no aspect motivation questionnaire statement total number number of statement 1 computer attitude questionnaire positive 4 1, 2, 3, and 4 negative 3 5,6, and 7 2 science attitude questionnaire positive 4 8, 9, 10, and 13 negative 3 11, 12, and 14 3 determination theory, perspective questionnaire. positive 3 15, 17, and 18 negative 3 16, 19, and 20 table 4 learning activities date activities april, 8th 2017  administering the pre-test for pre-test and questionnaire  distribute the worksheet was about components solar system april, 15th 2017  conducting learning process in the lab computer: introducing phet simulation.  filling the worksheet was about “orbit” april. 9th 2017  conducting learning process in the lab computer “gravity”  discussion of student activities 28th april 2017  learning activity for rotation and revolution and effect of the movement in earth and moon in the earth 13 may 2017  reviewing about gravity and the effect of rotation and revolution of earth and moon using phet simulation may, 20th 2017  administered post-test and questionnaire journal of science learning article 64 j.sci.learn.2018.1(2).60-70 figure 1 shows the implementation of phet simulation in the classroom. students have two meetings for laboratory activity and three meetings for classroom lecturing. in the lab activity, students operated phet simulation based on the worksheet. students worksheet for guiding students during activity and solve the problem question. students solved the questions in the worksheet in-group discussion during performing laboratory activity. some students can access phet simulation to see the materials and doing exploration or games. in the worksheet that has to solve student is how planet move in orbital and the force gravity effect to movement of the planet. students can change the variable the force gravity to see the movement of the planet. students concluded the discussion group based on the result of the laboratory activity. while in the classroom students reviewed the material about the solar system based on students activities on the phet simulation. the researcher explained the material by using phet simulation, which is same as students’ phet simulation. students can apply the interactive simulation and explore the simulation. students’ worksheet  the researcher conducted the learning solar system with phet simulation in accordance with the time allocation based on the lesson plan. to make the lesson efficient, the worksheet for the student was prepared. the students can discuss with their group to solve the problem in the student worksheet. in the experiment group, students are categorized into five groups which are consists four or five students. the student worksheet was given to experiment group because the problem in the worksheet was designed based on analyzing with phet simulation. figure 2 shows the student worksheet that has been filled by students during the learning. the figure indicates that students must answer the worksheet about solar system. students attempted to solve the problem question in the worksheet. the teacher gave this worksheet. in the figure 1 the implementation of phet simulation figure 2 example of the worksheet answered by a student journal of science learning article 65 j.sci.learn.2018.1(2).60-70 worksheet, there are three parts of materials and exercises to be discussed by the students. part a discussed the characteristics of eight planets. in the activity, the researcher gave a card with a description of planets. each group took two cards of planets and they will tell the card to another group, so the other group will write what the other group giving the description of characteristics of the planet. this activity conducted in the first meeting after they took the pretest. the second meeting, the researcher conducted the activity in the lab computer, so students will implement phet simulation. a general description of phet simulation was introduced to students. the instruction was given to use phet simulation especially in the simulation “gravity and orbits” was also given. to make timely efficient, the problem about the orbits of the planet is distributed to all of the student’s groups. they worked in the group to operate phet simulation. this activity placed in part b consists of four questions that should be discussed by the students. for part c, students discussed solar system at the next meeting. they also worked in the group to solve the problem question in their group. for part of exercises, the students worked in the class without using phet simulation. 3.2 students’ understanding the result of students’ understanding  the comparison between pretest score and posttest score of the sample in learning solar system has been analyzed using the average-normalized gain analysis. the result can be found that the average pre-test and post-test between two groups have a different score. the pretest score of control group has achieved an average score of 44.76, while experiment group has achieved an average score of 54.04. for posttest, the score for control group has shown the average score at 68.09, while for the experimental group has shown the average score at 74.76. from the score of average between pretest and posttest from two groups will obtain the average normalized gain value. it is found that the normalized gain between the control group and the experimental group has improvement of students’ understanding in learning the solar system. in control group, the average n-gain is 0.42, while average n-gain for experimental group is 0.60. the result from both groups is found that is a significant improvement of students’ understanding in learning the concept of solar system topic. average n-gain from the experimental group has higher than the control group in a learning activity. testing the normality data used spss 17. normality test to find n-gain data obtained has the distribution is normal or not in both two group (control and experimental) as seen in table 5. in this study, to measure the distribution of data, the method used is saphiro wilk. the data is called normal distribution if the value of shapiro wilk the significant value (0.05) (minium, king, & bear, 1970). in this study, the normalized data used average n-gain to determine data of sample has normalized distribution or not. table 5 shows that the value of the experimental group is 0.037 while the value of control group is 0.262. therefore, based on the criteria of normality test from the value of shapiro-wilk, the experimental group has not normalized distribution because the value of shapiro-wilk is 0.037 value significant 0.05. in control group has normalized distribution because the value of shapiro-wilk is 0.262 value significant 0.05. because one of the group has not normally distribution population, then the next method will be used is mann-whitney. the mann-whitney is the absence of a requirement that the research data should normally distribution and homogeneous. the result from mann-whitney test shown in table 6. for mann-whitney test, there are criteria used as a reference for decision hypothesis. the criteria for mann-whitney test as follow: if asymp.sig. (2-tailed ) probability 0.05, so ha accepted. if asymp.sig. (2-tailed) probability 0.05, so ha rejected. based on to table 6. the result showed that asymp.sig. (2-tailed ) is .038. it can be concluded ha accepted for mann-whitney test because of asymp.sig. (2-tailed ) is 0.038 probability 0.05. therefore, it can be said that it is significantly different from the result students’ understanding between the control group and experimental group. as a result, it is concluded that experimental group has highly significant improvement in student understanding where they learned solar system by phet simulation. the  improvement  of  students’  understanding  cognitive domain  the improvement of students understanding in cognitive domain divided into six cognitive domains based on blooms’ taxonomy revised. in this study, the cognitive domain measured is limited to four cognitive domains spread out in test item. the four cognitive domains that implement into test item are remembering (c1), understanding (c2), applying (c3), and analyzing (c4). the test item was tested to students in the form pre-tests and post-test. the result of test item in two groups are table 5 result n-gain normality from normality test between control group and experiment group group shapiro-wilk statistic df sig. average n-gain control group .944 21 .262 experiment group .901 21 .037 table 6 result n-gain normality from mann-whitney test analysis n-gain mann-whitney u 138.000 asymp. sig. (2-tailed) .038 journal of science learning article 66 j.sci.learn.2018.1(2).60-70 shown in table 7. based on the table, the data shown that post-test score is higher than pretest score in each level cognitive from both groups. for remembering domain (c1), in protest, the experimental group obtains the score 44.44, while control group has score 33.33. after given treatment and taken data for posttest, the experimental group obtains the score 84.12, while control group has the score 76.19. it can be concluded that in remembering domain (c1), experimental group is a higher score than the control group. in an understanding domain (c2), the data showed that post-test score has a higher score than protest in both of groups. a pre-test in control group obtains score 38.62, and experimental group obtains score 51.85. after treatment implemented in both of group, so the posttest score in the control group is 57.67, and experimental group is 75.66. it can be concluded that the experimental group is slightly better than the control group after the experimental group given treatment by phet simulation in solar system topic. it means that phet simulation used in the experimental group is better than using powerpoint slide in control group. in applying domain (c3), a pretest of control group shows the score 54.76, while pretest of experiment group shows the score 69.04. for the posttest also is same with a pretest that experimentally is higher than the control group. where the experimental group shows score 92.85 and control group obtains score 80.95. for analyzing domain (c4), same with cognitive domain has explained before that the posttest shows higher score than pretest score. it the same with a domain where pretest score in control group is 56.34, while for experiment group shows the score 57.14. after both of groups given treatment, there is an improvement in the post-test score. in control group achieves the score 75.39 and the experiment group achieves the score 80.15. the post-test experiment group also shows a higher score than the control group. from the explanation about the comparison in average pretest and post-test scoring from both of two groups, we found that the pretest score of the experimental group has a higher score than the control group. the result of pretest score where the experimental group as not given treatment yet is better than the control group. however, the control group also is better because they score are improved after they have learned solar system using powerpoint. the same the control group, the experimental group also achieves a higher score in post-test than pretest. they improve the score after they used phet simulation in learning solar system topic. based on the result, it is found that four cognitive domains are significant between control and experimental group. all of the cognitive domains are an influence on students’ achievement. however, the line graph which slightly significant is an understanding domain (c2). it shows that the value of average n-gain in experimental is greater than the control group. in experimental group has a value of average n-gain is 0.49 while control group has 0.31. it means that phet simulation was used in the experimental group is influence than using powerpoint slide in control group. three cognitive domains used in evaluating students’ understanding have significant value although they are a weaker understanding domain (c2). even though three domains (remembering, applying, and analyzing) influence in the improvement students’ understanding. because the value of average n-gain of the experimental group is greater than the control group. therefore, the phet simulation is influenced by teaching media to deliver the concept solar system than powerpoint slide was used in control group. the  discussion  of  students’  understanding  and  implementation of phet simulation  according to the result of student understanding, the researcher found that there is an improvement in students’ understanding of the concept of solar system. based on the result mann whitney test, there is a significant difference in pretest and post-test score between the control group and experimental group. the pretest and post-test score is measured from normalized gain. in experimental group (0.60) is greater than the control group (0.42). the finding shows that the quality of experimental group is high, while for the control group is categorized as a medium from the experimental group. thus, phet simulation can be used as a teaching media in the learning process because phet simulation can improve the posttest score of the experimental group. because of the experimental group have four meetings by using phet simulation in a learning activity. in the activity class, a student in control group was operated the simulation table 7 comparison the average of pretest and posttest score between control group and experimental group based on cognitive domain cognitive level control group experiment group pretest posttest n-gain pretest posttest n-gain remembering (c1) 33.33 76.19 0.64 44.44 84.12 0.71 understanding (c2) 38.62 57.67 0.31 51.85 75.66 0.49 applying (c3) 54.76 80.95 0.57 69.04 92.85 0.76 analyzing (c4) 56.34 75.39 0.43 57.14 80.15 0.53 journal of science learning article 67 j.sci.learn.2018.1(2).60-70 process for the topic solar system which has entitled “gravity and orbits”. the teacher gave the problem to students in the worksheet. they analyzed the problem and solve it the worksheet by used phet simulation. in the worksheet paper, students understand the law gravity. because of phet simulation is a simulated animation to prove the gravity law. they can change the variable such as mass, velocity, and gravity force. one by one, the object has changed and students analyzed the object, which has changed by them. they did it seriously and has curious to the variable object. they always answer the question asked by the teacher for giving an explanation if they did not understand it during the learning process. the teacher always guided students when students did in the worksheet. nevertheless, for the simulation that used is not an accurate measurement. because there is no quantity number of the variable object. students only change variable object by a ruler provided in the simulation. a ruler changes the variable objects are only more or less of mass, velocity, or it can be on or off the gravity force. even though phet simulations are interested in experimental students, because it is an interactive multimedia. they can imagine how the planet performs an orbit, how the mass effects of the rotation of the planet, and others which have abstractly idea for a human. because the planetary motion cannot be seen with naked eyes directly. students only receive the information from the news media. phet simulation is actually familiar for both groups because the phet simulation has used by another science teacher in the topic “wave”. they have learned the concept of the wave using phet simulation previously. however, the teacher only gave an introduction the concept without deeper analyze to wave concept. it is in line with the result level cognitive for both groups. from the result, remembering domain (c2) shows higher n-gain score than another cognitive level for both two groups. it shows that the phet simulation that was used will help to improve the students cognitive in the remembering domain. although another cognitive domain such as remembering (c1), applying (c3), and analyzing (c4) is significantly different between the control group and experiment group, phet simulation is influenced in teaching-learning process to students’ understanding. in learning of effect rotation and revolution for the experimental group, there was a problem in a learning activity. there was a meeting among students to vote selection for osis 2017-2018. most students in the experimental group, they were to be committee or candidate. therefore, students did not follow class activity effectively. they do not understand what the content that delivered by the researcher. for the posttest also, they have another subject test at the same time with the posttest of the researcher. they learned about the topic that they did not follow at that time. they also ask the researcher to review again them before they take posttest. the posttest is postponed to next meeting on a science subject. 3.3 students’ motivation the second variable analyzed is student motivation. the data obtained by questionnaire of motivation. the questionnaire developed by the researcher is adapted from three design questionnaire, they are computer attitude questionnaire (caq) by (knezek & christensen, 1996), science motivation questionnaire by (glynn & kobala, 2006), and determination theory perspective questionnaire (deci, eghrari, patrick,, & leone, 1994). the result is shown in table 8. based on table 8, we found that the scoring average between experiment group and control group has a different score. the higher score is experiment group. for computer attitude aspect, it describes the using computer-based learning in the delivering material or utilizing the computer for student activities. in experiment group, the score is 71.62% and control group is 63.25%. for science content, is describes how students learn in the solar system topic. in the score, the control group is higher than experiment group. the score control group is 73% and experiment group is 79.12. the determination self-aspect describes using phet as teaching media in learning solar system topic. the experiment group is higher score about 69.87% than control group is about 62%. generally, in each aspect condition students motivated enough. nevertheless, there are slightly different between with phet simulation and without using simulation. for control group who did not use phet simulation in learning solar system topic, they are the same condition between pre-test and post-test. for computer attitude aspect table 8 comparison average between experiment and control group in each aspect questionnaire control group experiment group aspect of questionnaire statement number of statement average percentage (%) average percentage (%) computer attitude questionnaire positive 1, 2, 3, and 4 2.69 63.25 2.69 63.25 negative 5, 6, and 7 2.37 2.37 science motivation questionnaire positive 8, 9, 10, and 13 2.94 73.00 2.94 73.00 negative 11, 12, and 14 2.90 2.90 determination theory perspective questionnaire positive 15, 17, and 18 2.39 62.00 2.39 62.00 negative 16, 19, and 20 2.57 2.57 journal of science learning article 68 j.sci.learn.2018.1(2).60-70 between pre-test and post-test is a similar condition. in the experiment class, after using the phet simulation in learning the solar system, they slightly increase motivated, so the score is higher than the control group. they have a different score, where the experiment group is higher than the control group. the experiment group, they used a computer to operate the phet simulation, and they are interested in computer-based learning. for several meetings, they used computer-based learning as media tool in a learning activity, such as power point presentation. the researcher used power point presentation as media learning in delivering the content. for science motivation aspect, the control group has same the condition before and after learning the solar system without using phet simulation. while experiment group, they are slightly increasing condition before and after using phet simulation as media learning for solar system topic. the last aspect is the self-perspective determination, where the student has an opinion about using phet simulation as media teaching media in learning solar system topic. based on the data result above, for the control group, they did not use phet simulation as media learning. the researcher uses powerpoint presentation or another media teaching to deliver the materials. phet simulation is only treated in the experiment group. therefore, it will result in the different perspective between the control group and the experiment group about phet simulation. as the result, they are enough motivated in learning solar system topic. nevertheless, if we compare to the experimental group, the control group still showed low motivation than experiment group. because they were treated in the same condition of pretest as a posttest. while the experimental group, they slightly perform different condition between pretest and posttest. they are motivated when they are learning solar system using phet simulation. 3.4 correlation of students understanding and motivation in using phet simulation as teaching media for solar system topic. teaching material comprehension is very important in the learning process with the support good learning motivation. to investigate the correlation between students’ understanding and motivation, table 9 was analyzed using pearson test. based on table 9, it is found that there are some students who attain a high score in objective test and questionnaire. from the data, the result is obtained during did research, the researcher correlated between the average of objective test and questionnaire. the objective test is used to measure the student understanding after students have learned all material. the test has been delivered by the researcher using phet simulation, while questionnaire to measure their perspective about the aspect in learning the content of the topic, as utilizing the computer in the learning process, a science lesson, and utilizing phet simulation as a teaching media in the learning process. the result shows that the correlation between students’ understanding and motivation is correlated each other. the score correlation is 0.46; the category including 0.25-0.5 is moderate based on (sarwono, 2006). it can be concluded that there is a correlation between students understanding and motivation using phet simulation in the learning solar system. to find a student who achieves a higher understanding of higher motivation, it will show in figure 3. figure 3 shows that there is a student who achieves a high score in learning the solar system, they have a higher motivation to use phet simulation. the result of highest achievement will correlate to motivation score. for example, the students are f06, she has a high average of students understanding score, but they also correlate with highest of motivation score. for m07 also has correlated with score motivation and understanding. the researcher found there are six students who correlate with score motivation and understanding. all of the students have not higher score in cognitive achievement has higher motivation. there some students has moderate achievement but they have high motivation, for example, table 9 correlation of students understanding and motivation in learning solar system using phet simulation aspects analysis understanding motivation understanding pearson correlation 1 .440* sig. (2-tailed) .046 n 21 21 motivation pearson correlation .440* 1 sig. (2-tailed) .046 n 21 21 figure 3 correlation of students understanding and motivation in learning solar system using phet simulation journal of science learning article 69 j.sci.learn.2018.1(2).60-70 f15 and m19. they have interest in science learning so they motivation will be high. according to (mutoharo et al., 2015), there is a positive correlation if the motivation in science learning is increasingly better than the understanding concept of science also increases. some students increase their score in posttest compared to pretest, they also increase the score motivation after using phet simulation. this finding also supports the statement by bell & smetana (2015), as with any other educational tool, computer simulations are tools to support learning. computer simulation as media learning which has a role in increasing student motivation and aspect in learning activity becomes more interference and quality of learning (nurohmah, 2015). 4. conclusion based on the result and discussion of the data that has been explained it before about phet simulation as teaching media in learning solar system topic was placed in 8th grade is one of a private school in padalarang, it can be concluded that the implementation phet simulation conducted in seven meetings, which are pretest-posttest and learning activity. the use phet in the learning activities was done well and it can be an improvement for the experimental group. students have utilized phet simulation to learn the solar system. according to the students’ understanding on the concept solar system, especially in orbits and gravity of the planet, it is showed that experimental group has a higher score (<g> = 0.45) than the control group (<g>=0.42). based on the data obtained in this study, there is a significant improvement in students understanding between experimental group and control group. therefore, the hypothesis ha is accepted in this study. the result shows the significant value. it can be concluded ha accepted because asymp. sig. (2-tailed ) is 0.038 probability 0.05. student responses about phet simulation as a teaching media in the learning process show a good response. they are very enthusiastic when they used phet simulation about the solar system. because they can more explore the abstract concept and they can solve the problem from the worksheet that given to them. phetsimulation is very interactive media. moreover, there is a significant correlation between students’ understanding and motivation. the score correlation is 0.46 categorized as moderate. thus, the higher the students’ motivation, the higher the student's achievement on learning the solar system. acknowledgment the authors acknowledge the principal, teachers, and students of al-azhar syifa budi parahyangan school. references alsultanny, y. a., nouby, a. m., & al-enazi, t. t. (2014). effects of using simulation in e-learning programs on misconceptions and motivations towards learning. international journal of science of technology educational research, 5(3), 41-51. bell, r. b., & smetana, l. k. (2008). using computer simulations to enhance science teaching and learning. national science teachers association, 3, 23-32. bloom, b. s. (1956). taxonomy of educational objectives. new york: mckay, 20-24. cahyani, m. d., firman, h., & rochintaniawati, d. (2013). the effect of phet simulation media on improvement students' achievement in the concept of light refraction. bandung: universitas pendidikan indonesia. creswell, j. w. (2012). educational research: planning, conducting, and evaluating quantitative and qualitative research. boston: pearson education, inc. deci, e. l., eghrari, h., patrick, b. c., & leone, d. (1994). facilitating internalization: the self-determination theory perspective. journal of personality, 62(1), 119-142. fitrianingrum, n. (2014). penggunaan media komputer dalam pembelajaran pendidikan agama islam. jakarta: universitas islam negeri syarif hidayatullah. fraenkel, j. r., wallen, n. e., & hyun, h. h. (2012). how to design and evaluate research in education eight edition. new york: the mcgraw-hill companies, inc. glynn, s. m., & kobala, t. r. (2006). motivation to learn college science. national scienceteacher association press (pp. 25-32). arlington: handbook of college science teaching. haya, f. d., waskito, s., & fauzi, a. (2014). pengembangan media pembelajaran gasik (game fisika asik) untuk siswa kelas viii sekolah menengah pertama. journal pendidikan fisika, 1, 11-14. kemendikbud. (2016, december 2016). peringkat dan pencapaian pisa indonesia mengalami peningkatan. jakarta: kemendikbud. retrieved from kementerian pendidikan dan budaya indonesia. knezek, g., & christensen, r. (1996). validating the computer attitude questionnaire. southwest educational research assosiation conference (pp. 1-16). lousiana: educational resources information center (eric). mckagan, s., perkins, k., dubson, m., malley, m., reid, s., lemaster, r., & wieman, c. (2008). developing and researching phet simulations for teaching quantum mechanics. american journal of physics, 76(4), 406-417. mcmillan, j. h., & schumacher, s. (2001). research in education: a conceptual introduction, fifth edition. united states: addison wesley longman, inc. minium, e. w., king, b. m., & bear, g. (1970). statistical reasoning in psychology and education third edition. new york: john wiley&sons, inc. mutoharo, s. z., sudibyo, e., & mitarlis. (2015). hubungan motivasi belajar terhadap pemehaman konsep ipa pada siswa kelas viii smp negeri 2 surabaya. pendidikan sains:e-journal unesa, 3(2). nizam. (2016). ringakasan hasil-hasil asesmen belajar dari hasil un, pisa,timss, inap. retrieved from http://puspendik.kemdikbud.go.id/seminar/upload/nizamringakasan-hasil-hasil-asesmen-belajar-dari-hasil-un, pisa,timss, inap norušis, m. j. (2006, august 2). spss 14.0 guide to data analysis. upper saddle river, nj: prentice hall. retrieved from understanding correlation: a how-to guide. nurohmah, y. a. (2015). peranan media pembelajaran dalam meningkatkan motivasi belajar siswa pada mata pelajaran tarikh kelas viii di smp muhammadiyah 5 surakarta. surakarta: universitas muhammadiyah surakarta. perkins, k., adams, w., dubson, m., finkelstein, n., reid, s., & wieman, c. (2006). phet: interactive simulations for teaching and learning physics. the physics teacher, 44(1), 18-23. perkins, k., moore, e., podolefsky, n., lancaster, k., & denison, c. (2012). towards research-based strategies for using phet simulations in middle school physical science classes. aip conference proceedings, 1413, 295-298. journal of science learning article 70 j.sci.learn.2018.1(2).60-70 prima, e. c., putri, c. l., & sudargo, f. (2017). applying pre and post role-plays supported by stellarium virtual observatory to improve students’ understanding on learning solar system. journal of science learning, 1(1), 1-7. sarofi. (2014). pengembangan media pembelajaran fisika interaktif berbasis macromedia flash pada materi listrik dinamis. 2014: univesitas islam negeri sunan kalijaga. sarwono, j. (2006). metode penelitian kuantitatif dan kualitatif. yogyakarta: graha ilmu. wieman, c. e., adams, w. k., & perkins, k. k. (2008). phet: simulations that enhance learning. education forum, 322(5902), 682-683. wieman, c. e., adams, w. k., loeblein, p., & perkins, k. k. (2010). teaching physics using phet simulations. the physics teacher, 48(4), 225-227. yu, k. c., sahami, k., & dove, j. (2017). learning about the scale of the solar system using digital planetarium visualizations. american journal of physics, 85(7), 550-556. a © 2020 indonesian society for science educator 61 j.sci.learn.2020.4(1).61-68 received: 23 august 2020 revised: 21 november 2020 published: 28 november 2020 determining secondary school students’ knowledge and awareness about antarctica nazihan ursavaş1*, raif kandemir2 1department of mathematics and science education, faculty of education, recep tayyip erdogan university, rize, turkey 2department of geological engineering, faculty of engineering, recep tayyip erdogan university, rize, turkey *corresponding author nazihan.ursavas@erdogan.edu.tr abstract antarctica has been an object of interest for a long. it is considered important to increase individuals awareness and knowledge about antarctica. the accuracy of individuals’ scientific knowledge plays an important role in creating the awareness. in this study, the 349 secondary school students’ knowledge and awareness about antarctica was determined quantitatively with a questionnaire. nineteen questions, including current topics were prepared. that addressed from biological geographic and climatic perspectives. according to the findings, 40% of the students thought antarctica is in the north pole, 42.6% thought polar bears and penguins live together; 46.6% thought antarctica is best visited during summer months, 70.5% thought most glaciers are located in the northern hemisphere. based on these findings, it can be said that students have inadequate and alternative conceptions about antarctica. studies can be conducted to determine how best to increase students’ awareness. keywords antarctica, knowledge, awareness, secondary school students 1. introduction the earth’s polar regions where can provide an insight into fundamental earth system processes are crucial for regulating the world climate (ricci and egerton, 2010). besides, they are the primary resources of freshwater, which is the main source of life and the most critical molecule that distinguishes earth from other planets. still, earth’s water's origin is a mystery (drake, 2015; elkinstanton, 2011; genda, 2016; marty & yokochi, 2006; robert, 2001). a possible solution to this mystery plays a crucial role in reading the world's evolutionary history and modern structure, exploring water, and measuring its quantity on different planets today (wu et al., 2018). in understanding the evolutionary history of the world, the information that water itself contains is also crucial. this critical information is hidden mostly in glaciers, namely the solid state of water since glaciers are rare sources that can hold information about climate changes dating back thousands of years in air bubbles trapped in them (benn & evans, 2010; dahl-jensen, 2018). thanks to such information obtained from the glaciers, the causes of the glacial ages' changes can be better highlighted. future climatic changes can be predicted to take necessary precautions in advance (azzolini, campus & weber, 2012; baydemir, 2019). as it is known, the regions with intense glacial layers are located around the north and south poles. although glaciers constitute only approximately 2.1% of our planet's total water, they inhold an essential volume of usable water resources. 90% of these glaciers are located in antarctica and 9% in greenland, the two poles of earth (lucchitta, & ferguson, 1986). however, the glaciers at each of the poles are unique on account of a significant difference. the glaciers in greenland consist of sea glaciers that contain saltwater, whereas those in antarctica are made up of freshwater. although antarctica's icy landscape looks distant, scientific research in this region can give insight into the changes that matter to the entire planet (the national academy of sciences, 2011). today, a wide array of scientific studies are being conducted on antarctica by many countries searching fossils, lake ecosystems, geological, rock layers and layering, how the earth surface processes, archaeological, the structure and properties of glacier ice, its formation and the interactions journal of science learning article doi: 10.17509/jsl.v4i1.27537 62 j.sci.learn.2020.4(1).61-68 of ice accumulation with climate, meteorological and climatological aspects along with impact monitoring and anthropogenic activities on the continent (benayas et al., 2013). recently, turkey has also contributed to such studies. istanbul technical university polar research application and research and centre (polrec) was established as the first turkish organization specialized in the polar regions. with the coordination support of this center, the turkish antarctic science expedition i (taei) was held in 2017. it had its sequels as tae-ii and taeiii in the years 2018 and 2019, respectively, within the "turkish antarctic science base project" (in karahalil, özsoy, oktar, yirmibeşoğlu, & vural, 2018). even another expedition to antarctica was realized in 2020 (tae-iv). antarctica has been an object of interest for a long. on top of everything, the existing descriptive scientific studies respecting antarctica attract scientists’ attention. issues such as stratospheric ozone depletion and enhanced ultraviolet effects, environmental and climatic archive from ice cores, detection of anthropogenic pollution, the study of global climate change (such as the carbon cycle and sea level), and analysis of unique collections of meteorites have attracted and focused unprecedented international attention on antarctica (drewry, 2009). besides, most of the clean water needed for the continuation of life is frozen in antarctica. these waters are in danger due to environmental issues (gröndahl, sidenmark, & thomsen, 2009). it is thought that more information should be in hand about antarctica, and awareness should be raised in individuals about this continent, which does not belong to any particular person or state. in addition to supporting antarctica's expeditions, polrec also carries out studies to raise awareness of individuals. these studies include sharing of the knowledge and experiences acquired by the scientists who visited the continent with students, various scientific club studies, interviews, seminars, or painting contests (karahalil, özsoy, oktar, yirmibeşoğlu, & vural, 2018; vural, yirmibeşoğlu, oktar, & özsoy, 2018). other countries are engaged in awareness-raising works about antarctica. in particular, the ministry of education of new zealand encourages studies in this area and integrates antarctica content into its formal education curriculum (mcfadyen, 2011). australia is developing activity-based content for teachers to increase knowledge and awareness about antarctica and is distributing the books compiled of such contents to all schools with the support of the ministry of education (australian science teacher association, 2007). what is more, the american national science foundation (nsf) implemented a project for the international polar year to increase societal awareness. they chose 27 students and teachers from secondary and tertiary educational institutions. the group was first trained accordingly, and then they toured antarctica on a field trip (garcia, robertson, lougheed, tweedie, & velasco, 2013). one influential factor for increasing the awareness of individuals about antarctica is undoubtedly the storage of 69% of waters that can be used by living things on that continent (usgs, 2020). these glaciers will melt mainly due to climate change, and all that freshwater will be mixed with saltwater leading to the improbability of using such water or its contamination otherwise. this result in itself poses a significant threat to humanity about access to water. besides, the melting of the glaciers on this continent may endanger life due to severe rises at sea level in the future (benn, & evans, 2010). today, water can be used and easily accessible, but it is at risk because of the misuse of water and wrong policies; this ultimately places more pressure on the glaciers, like the world's carboy (bastmeijer, & roura, 2008). the accuracy of individuals’ scientific knowledge plays an essential role in creating awareness (salazar, 2012; shabudin, rahim & ng, 2016 ). however, individuals do not possess sufficient information about the environment (özbebek-tunç, akdemir & düren, 2012), and it is stated that scientific information from the polar regions boosts people’s knowledge of the environmental issues facing the world. increasing the knowledge and awareness about the continent will both promote the continent and acknowledge its importance. starting to raise awareness at an early age is useful for training conscious individuals (ursavaş, & aytar, 2018). at the following stage, revealing the individuals’ perceptions and levels of understanding of the subject can guide such works' context, manner, and direction (karataş, köse, & coştu, 2003). this particular study aimed to identify the knowledge and awareness of secondary school students about antarctica, and answers were sought for the following research questions for this purpose. what knowledge do secondary school students have about various geographical issues related to antarctica?. what knowledge do secondary school students have about various biological issues related to antarctica?. what knowledge do secondary school students have about various climatic issues related to antarctica? 2. method 2.1. study design and participants this study was conducted as a descriptive survey research. there are several types of descriptive research. to reveal students’ knowledge about antarctica before the speech on antarctica and due to produce a “snapshot”, cross-sectional survey research was run for this study (cohen, manion, & morrison, 2007). the study's universe consists of secondary school students, and purposive sampling was used to choose the sample. it is purposive because students were chosen among from who attend the scientific speech. students took part in the study journal of science learning article doi: 10.17509/jsl.v4i1.27537 63 j.sci.learn.2020.4(1).61-68 voluntarily. as a procedural requirement, necessary permissions were obtained from the provincial directorate of national education. once the implementation stage was launched, data were collected during the 2019-2020 academic year, fall semester, outside the classroom hours, and without interrupting regular educational and instructional activities. 349 students (188 females and 161 males) took part in the study and were aged 9 to 14 with an average age of 11.53 (sd=2.48). 2.2. data collection tool and process a self-administered questionnaire was used as a data collection tool to describe data on variables of interest. the researchers developed the tool. one of the researchers is a pedagogical expert of biology, and the other is a geological engineer who was a team member for the tae-ii mission to antarctica in 2018. the data collection tool was named “questionnaire on knowledge and awareness about antarctica” and consisted of two parts. part one was structured to collect demographic information about participants, including their age, class, and gender. the second part contained a blend of 19 dichotomous, multiple-choice, contingency, and open-ended questions. the questionnaire questions were obtained from the literature and the students' questions after the informative speeches one of the authors gave at schools under awareness studies. the feedback provided by students like “i did not know the polar bears and penguins live in separate polars.”to learn that having a science center in antarctica belongs to turkey is proudful.” quotes are also used as questions. the questionnaire items covered a wide range of up-todate information questions about the poles, the pole in which antarctica is located, where polar bears and penguins live, and so on. students were asked to complete the forms in an adequate period according to their own pace. they were instructed to do so after being informed about the scope of the study. the administering of the questionnaire was followed by visits to schools to give a speech on antarctica's awareness. in this way, it was made sure that the students could make more sense of the information they learned by listening to the speeches; they could pick up answers to the questions asked beforehand, focusing on the task better. for the sake of research ethics, it was explained to the participants that their answers would be held to be used for research purposes only. 2.3. data analysis the data obtained with the questionnaire were analyzed using descriptive statistical methods in spss package programs (ibm statistical package for the social sciences statistics; armonk, ny, usa). the descriptive data were shown with numbers (n), percentile (%), and mean ± standard deviations. differences were tested using the chisquare test. the statistical significance level was set to p <.05. the results were handled in line with the answers given for each research question. for this reason, the number of respondents did not prove the standard among research questions. in other words, the respondents and the sample of this research were divergent in some instances. apart from these, missing data analysis was not performed since participation relied on voluntariness. 3. result and discussion the students’ replies to the questionnaire items were presented under separate headings regarding their geographical, biological, and climatic facts. the distribution of the items investigating respondents’ geographical knowledge, biological knowledge, and climatic knowledge was as ten questions, five questions, and four questions, respectively. 3.1. students’ geographical knowledge about antarctica the participants’ responses to the questions related to the poles' geography (questions no 2, 3, 7, 8, 9, 17, 21, 22, 23, 25) are displayed in the tables below. the tables were arranged by taking into account the pattern or format of the questions concerned. the respondents’ judgments about the poles are shown in table 1. it is seen in table 1 that a total of 336 students answered question 2, 237 of whom (70.5%) stated that (38.3% females, 32.2%males) the north pole hosts more ice than the south pole. question 8 answered that 340 students, 136 of whom (40%) considered the antarctic continent to be in the northern hemisphere, while 204 (60%) suggested the opposite. to check if there was a significant difference in these responses based on gender variable, chi-square analysis was performed. such a difference was noticed in question 2 at the significance level of p = 0.001. another question concerning antarctica's geographical features was, "which continent does not contain a country/state?" the answers to this question are given in table 2. the highest proportion of the respondents (n=209; 66.1%) provided the correct answer for the question inquiring about the uninhabited continent, which is antarctica. this response was followed by “america” as pointed out by 27 respondents (8.5% ), “other” by 26 respondents (8.2%), “africa” by 20 (6.3%), “australia” by table 1 questions and answers about the north pole and the south pole north pole south pole questionnaire item female male female male total q2which pole of the world harbors more glaciers? 129(38.3) 108(32.2) 53(15.7) 46(13.8) 336 s8in which hemisphere of the world is antarctica located? 79(23.2) 57(16.8) 104(30.6) 100(29.4) 340 journal of science learning article doi: 10.17509/jsl.v4i1.27537 64 j.sci.learn.2020.4(1).61-68 19 (6.0%), and “asia” by another 15 respondents (4.7%), respectively. one question was significantly related to the continent which encapsulates the south pole. table 3 shows that 325 students answered this question, but 24 did not do so. of the former, 211 (64.9%) people could answer it correctly by explaining that the south pole is situated within the borders of the continent of antarctica. the second-largest group of respondents (n=42; 12.9%) uttered “alaska”, which is at the north pole. other wrong answers included “africa” indicated by 29 students (8.9%), “asia” by 20 (6.2%), “america” by 15 (4.6%), and “other” by eight students (2.5%). four of the questions about antarctica's geographical features were dichotomous, requiring the answer of “yes” or “no”. the answers in this regard are shown in table 4. table 4 shows that 84.2% of the students believe that it is possible to go to antarctica. besides, 86.2% think that antarctica is glaciated, 78.1% believe that there is land over there, and 92.5% believe water on that continent. apart from these, 41.2% of the participants pointed out a science base in antarctica, which belongs to turkey, while 59% think the opposite. antarctica's noteworthy geographical features are that it holds approximately 69% of all potable water resources. to see the students’ level of awareness on this matter, a question was included to put forward the most extensive freshwater supply location from the students’ point of view. the answers are demonstrated in table 5. as table 5 reveals, 38.8% of the students stated that the highest freshwater is in underground resources. it was followed by other suggestions, including rivers (29.5%), lakes (11.2%) and, lastly, glaciers (10.9% ). as a final point, it was found out that a smaller portion of the respondents (4.8%) see oceans as a source of fresh water. 3.2. students’ biological knowledge about antarctica in the questionnaire, some of the items were designed to address antarctica’s biological facts in the students' eyes (question no 5, 6, 12, 14, 16). the answers obtained from these questionnaire items are presented in the tables below. the contents of the tables were arranged according to their relevance to biology and the consistency of format. three of the biology questions were close-ended, requiring a positive or negative one-word answer. answers to these questions can be seen in table 6. it is seen that 88.6% of the students think that antarctica hosts some table 2 responses about the continent without a country/state on it distribution continent (s3) n % antarctica 209 66.1 america 27 8.5 other 26 8.2 africa 20 6.3 australia 19 6.0 asia 15 4.7 total 316 100 table 3 responses about the continent enclosing the south pole distribution continent (s7) n % antarctica 211 64.9 alaska 42 12.9 africa 29 8.9 asia 20 6.2 america 15 4.6 other 8 2.5 total 325 100 table 4 yes/no questions and answers about antarctica questionnaire item yes n (%) no n (%) answers total no answer total q9 is it possible to travel to antarctica? 287 (84.2%) 54 (15.8%) 341 8 q17 are there any glaciers in antarctica? 288 (86.2%) 46 (13.8%) 334 15 q21 does turkey have a science base in antarctica? 132 (41.8%) 190 (59.0%) 322 27 q22 is there land in antarctica? 257 (78.1%) 72 (21.9%) 329 20 q23 is there water in antarctica? 310 (92.5%) 25 (7.5%) 335 14 table 5 responses about the place where the most freshwater is available distribution place (q25) n % groundwater 121 38,8 rivers 92 29,5 lakes 35 11,2 glaciers 34 10,9 oceans 15 4,8 other 15 4,8 total 312 100 journal of science learning article doi: 10.17509/jsl.v4i1.27537 65 j.sci.learn.2020.4(1).61-68 animal species, 88.6% think that the plant species in their environment cannot survive in antarctica, and 57.6% think that there are habitats in antarctica. thanks to the printed and visual media, today, it is wellknown that penguins and polar bears, two of the animals that first come to mind when earth’s poles are referred to, do not live together. to find out the students’ knowledge, the questions in table 7 were addressed, and their answers were noted. the north pole was mentioned at the highest rate under both of the questions. in other words, the students think that polar bears and penguins mostly live at the north pole. still, the conception that polar bears live at the north pole was more weighted than penguins' misconception. the cross-tabulation analysis further found that 36.8% of students think that both polar bears and penguins live at the north pole, and another 5.8% think that both of the species live at the south pole. in total, 42.6% of surveyed participants believe that penguins and polar bears live in the same region. from the perspective of gender, these questions' answers did not differ significantly between girls and boys. 3.3. students’ climatic knowledge about antarctica a number of the questionnaire items (question no 1, 4, 10, 20) were planned to unearth students’ knowledge and awareness about climate conditions. the participants’ answers are listed below by showing due diligence to the consistency of both content and style of the questions. the views of the students regarding the poles are summarized in table 8 according to table 8, 245 (71.3%) of the respondents stated that the coldest place in the north pole, and another 213 (65.9%) pointed out that the world's coldest and windiest part is, again, the north pole. however, the correct answer to both questions should be “the south pole”. to test whether the questions' responses have a statistically significant gender difference, chi-square analysis was performed. as a result, a difference was found in answers to questionnaire item 1 at the significance level of p=0.001. another item regarding the climatic facts was designed to determine whether global warming could affect antarctica in students’ perceptions. 310 persons (91.7%) answered in the affirmative, whereas 28 persons (8.3%) responded negatively. these are shown in table 9. 310 persons (91.7%) answered in the affirmative, whereas 28 persons (8.3%) responded negatively. in this group of questions, it was also surveyed whether students could travel to antarctica. those who replied positively were asked to mark the best choice symbolizing the three months as the ideal season. the distribution of the answers is given in table 10. the best time to visit antarctica is from december to february. in reply to the question in table 10, 37.1% of the respondents chose this december-january-february option. contrarily, the june-july-august period was preferred by 46.6% of all students. the sum of the remaining options was equal to 16.4%. this study was carried out to get the picture of antarctica's knowledge by giving students a questionnaire consisting of items concerning general and current facts about the continent. the participants’ answers were processed using simple statistical methods, and the results were divided into three categories. these categories were titled with relevance to geographic, biological, and climatic attributes of antarctica. in our findings, 60% of the students could accurately tell in which hemisphere antarctica is located. also, 64% are seen to know which continent accommodates the south pole. it can thus be said that the students can associate antarctica and the south pole well. still, the rate of wrong answers seems too striking, considering this particular piece of knowledge. to clarify, 35,1% of our participants hold false conceptions about the continent, which shelters the south pole, and another 40% deem that antarctica lies in the northern hemisphere. additionally, table 6 yes/no questions and answers about antarctica questionnaire item yes n (%) no n (%) answers total no answer total q12 are there animal species living in antarctica? 302 (88.6%) 39 (11.4%) 341 8 q14 can all of the plants you see around live in antarctica? 39 (11.4%) 302 (88.6%) 341 8 q16 do you think some people settled and live in antarctica? 173 (57.6%) 131 (42.4%) 309 40 table 7 questions and answers about the habitat of penguins and polar bears questionnaire item north pole south pole africa south america europe north america asia other total q5 where do polar bears live? 223 (75.6%) 56 (9.0%) 7 (2.4%) 1 (0.3%) 8 (2.7%) 295 (100%) q6 where do penguins live? 173 (57.1%) 108 (35.6%) 3 (1.0%) 11 (3.6%) 2 (0.7%) 6 (2.0%) 303 (100%) journal of science learning article doi: 10.17509/jsl.v4i1.27537 66 j.sci.learn.2020.4(1).61-68 the percentage as high as 70,5% of students who feel that the north pole is home to glaciers suggests a lack of geographical apprehension about antarctica. a study held in turkey revealed that 50% of people involved in the study have difficulties in finding the directions and the placement of countries on the map (kaya, 2012; tuna, demirci & gültekin, 2012). another question probing geographical information was about the location where the highest quantity of potable water is available. although the north pole is covered in sea ice, those sea ice trapped some salt in the small pockets between ice crystals; consequently, that type of water is not drinkable at first for many creatures. on the other hand, 90% of the glaciers are deposited in antarctica, and these terrestrial glaciers are made up of frozen fresh waters that can be used by living things. therefore, it would be quite sensible to call antarctica the world’s big water bottle. however, 38.8% of the participants here accept underground waters as the primary freshwater resource on earth. what is more, “glaciers”, which is honorable among all six options under the question, ranked fourth based on the corresponding percentage of 10.9%. these figures imply that the students are not knowledgeable about antarctic freshwater reservoirs. people associate water and antarctica only with sea-level rise as the glaciers melt (shabudin, rahim, & ng, 2016). after the expedition to antarctica in 2019, turkey completed a third explorative visit to the continent, followed by the fourth in 2020. as a result of these missions, a temporary science base was established on the continent during the 2019 visit. moreover, at the time of our survey, turkey’s science base was not in its place in antarctica. for this reason, regarding this question, negative responses were taken as the correct answer, but the opposite was deemed wrong. in summary, 41.8% of the students provided a wrong answer assuming a science base while filling in the questionnaire. when it comes to knowledge of antarctica's biological dimensions, the responses concerning the habitat of polar bears and penguins revealed that they have only a naive understanding of this topic. recently, polar bears have been significantly related to global warming, and they are being given a large place in both the printed and visual media. that is why they are one of the animal species the most familiar to children in general. in a longitudinal study conducted on children aged 4 to 10 in the uk, polar bears and penguins were put in the list of the most known animals rather than creatures living in the rain forests even though the natural habitat of the latter is similar to that of human beings (palmer, & suggate, 2004). this finding was explained with the background knowledge children learned from the tv show “life in the freezer ” (palmer, & suggate, 2004). on the contrary, our research students signaled only a modest intellectual level about the two prominent species peculiar to the polar regions, penguins and polar bears. even though most of our respondents (75.6%) could spot the natural habitat of polar bears accurately, which is the north pole, they marked the same option as penguins’ habitat as well, producing a wrong answer (57.1%). moreover, 42.6% of the research participants think that penguins and polar bears share the same living area in the north or south edge of the globe. this result can be accounted for by students’ inadequate information about exactly where these two different animals live, although these species are associated with earth's poles. the reason for this may be the source of information where they have. salazar (2012) found that 29.1% of santiago residents and 42,5% of punta arenas residents have information about antarctica from tv. about the last component of the biological notions, 57.6% of our participants believe that humans live in antarctica, which is uninhabited in practice. the truth is table 8 questions and answers about the north and south pole the north pole the south pole questionnaire item female male female male total q1which pole of the world is colder? 136 (39.6) 109 (31.7) 47 (13.7) 51 (14.86) 343 q4what is the coldest and windiest place in the world? 118 (36.5) 95 (29.4) 58 (18.0) 52 (16.1) 323 table 9 responses to “does global warming have an impact on antarctica?” questionnaire item yes n (%) no n (%) answers total no answer total q20 does global warming have an impact on antarctica? 310 (91,7%) 28 (8,3%) 338 11 table 10 students’ suggestions for best months for travel to antarctica distribution season (q10) n % june-july-august 117 46,6 december-january-february 93 37,1 september-october-november 24 9,6 march-april-may 17 6,8 total 251 100 journal of science learning article doi: 10.17509/jsl.v4i1.27537 67 j.sci.learn.2020.4(1).61-68 that there is life at the north pole, not at the south pole. apart from that, the students stated that some animal species could live there, but it is not the case for plant species. however, two types of flowering plants are already known to exist on the continent alongside various mosses, lichens, liverworts, and fungi (mcfadyen, 2011). as far as this finding is concerning, it can be said that the students are deficient in some biological matters related to antarctica. lastly, the findings reached from four items about antarctica's climatic conditions reveal that the students have inadequate information about this continent. they suppose that the north pole is colder than the other, and the north pole is the coldest and windiest part of the world. however, the correct answer for both of the questions must be “the south pole”. as another finding, students who think that one can go to antarctica believe that the best season is from june to august (46.6%). however, the fact is that the right time to visit that continent covers december, january, and february because antarctica is located in the south hemisphere. the students cited the summer months because they tend to expect high temperatures in the other half of earth when the temperatures are highest across the northern hemisphere. besides these, the rate of students who think that global warming impacts antarctica was found to be 91.7%. 4. conclusion when all of the findings as to students’ geographical, biological, and climatic knowledge are evaluated all together, it becomes clear that the students predominantly associate antarctica with the north pole. the students might have connected the north with cold weather and the south with hot temperatures simply because of their hemisphere. this inference is based on several findings. for example, they said that polar bears and penguins both live at the north pole, and they referred to summer months for any potential visit to antarctica. further evidence includes their arguments that the most massive glaciers are located at the north pole, the coldest pole of earth is the north pole, and the coldest and windiest place is the north pole again. students anticipate that the weather gets colder as one moves further north and gets warmer towards the south. hence, it can be suggested that the north's direction brings cold weather to mind mainly because we live in the north hemisphere. consequently, this might have led to the formation of geographical, biological, and climatic misconceptions about antarctica on students’ minds. from the quality of the answers reported here, it can be inferred that the knowledge of students about antarctica is far from being sufficient. in light of the preceding, the recommendation is that students' knowledge and awareness must be increased since antarctica bears particular importance as the world’s most significant freshwater resource. secondly, they should be taught the steps taken in our country about antarctica. thirdly, this topic should be inserted into turkey's formal curriculum by modeling practices in different countries. specifically, it needs to be made sure that teaching of antarctica's location and general characteristics should be realized in a more meaningful way as a part of geography or social studies lessons. besides this, topics such as the importance of water for living things can be taught with special consideration of antarctica's aqueous features within the scope of biology and science lessons. also, life forms on this continent can be elaborated in the abovementioned school lessons. finally, the results obtained in this research apply to a relatively limited universe. thus, it is recommended to replicate or adapt this study to different samples for increased generalizability. acknowledgment the data used in this research were obtained on the tae-ii, which was carried out under the auspices of the presidency of the republic of turkey, under the responsibility of the ministry of science, industry, and technology of the republic of turkey, in coordination with the istanbul technical university polar research and application and research centre. the authors are grateful to these institutions for their respective support. references australian science teachers association (asta). (2007). antarctic science. a resource book of ideas for teachers for national science week. https://asta.edu.au/generic/file-widget/download/id/1165. azzolini, r., campus, p., & weber, j. (2012). the polar ocean in a warming planet: understanding for managing a unique resource of the humankind. egu general assembly conference abstracts, 14, 13660. bastmeijer, k. and roura, r. (2008). environmental impact assessment in antarctica. in:bastemejer, k. & roura, r. theory and practice of transboundary environmental impact assessment. first edition. brill/nijhoff, 175-219. doi: doi.org/10.1163/ej.9789004164796.i-400.57. baydemir, t. (2019). buzullardan öğreneceklerimiz var. bilim ve teknik dergisi, 618. benayas, j., pertierra, l., tejedo, p., lara, f., bermudez, o., hughes, k., and quesada, a. (2013). a review of scientific research trends within aspa no. 126 byers peninsula, south shetland islands, antarctica. antarctic science, 25(2), 128-145. doi:10.1017/s0954102012001058. benn, d. i. & evans, d. j. a. (2010). glaciers and glaciation. second edition, routledge, new york, usa. cohen, l., manion l. & morrison, k. (2007). research methods in education. new york, ny: routledge. dahl-jensen, d. (2018). drilling for the oldest ice. nature geoscience, 11, 702-706. drake, m. j. (2015). origin of water in the terrestrial planets. meteoritics and planetary science, 40(4), 1–9. drewry, d.j. (2009). the future of antarctic scientific research. polar record, 29(168), 37-44. elkins-tanton, l. t. (2011). formation of early water oceans on rocky planets.astrophysics and space science, 332(2), 359–364. doi: 10.1007/s10509-010-0535-3. garcia, c. v., robertson, w.h., lougheed, v., tweedie, c. & velasco, a. (2013). journey to the end of the earth: academic and professional benefits for students participating in a field-based https://asta.edu.au/generic/file-widget/download/id/1165 journal of science learning article doi: 10.17509/jsl.v4i1.27537 68 j.sci.learn.2020.4(1).61-68 research program in antarctica. journal of college science teaching, 42(4), 72-81. genda, h. (2016). origin of earth’s oceans: an assessment of the total amount, history and supply of water. geochemical journal, 50(1), 27– 42. doi: doi.org/10.2343/geochemj.2.0398. gröndahl, f., sidenmark, j. & thomsen1, a. (2009). survey of waste water disposal practices at antarctic research stations, polar research, 28(2), 298-306. doi: 10.1111/j.1751-8369.2008.00056.x karahalil, m., özsoy, b., oktar, ö., yirmibeşoğlu, s. & vural, d. (2018). türkiye’de kutup bölgeleri hakkında farkındalığı arttırmak üzere yapılan çalışmalar. ii. kutup bilimleri çalıştayı özet kitabı, 86, 12-13 eylül, i̇sanbul-türkiye. karataş, f. ö., köse, a. g. s. & coştu, a. g. b. (2003). öğrenci yanılgılarını ve anlama düzeylerini belirlemede kullanılan i̇ki aşamalı testler. pamukkale üniversitesi eğitim fakültesi dergisi, 13(13), 54-69. kaya, e. (2012). i̇lköğretim öğrencilerinin küresel konumlandırma becerileriyle medyadan yararlanma durumları ve sosyal bilgiler dersindeki başarıları arasındaki ilişki. türkiye sosyal araştırmalar dergisi, 16(2). lucchitta, b.k. & fergusson, h.m. (1986). antarctica: measuring glacier velocity from satellite images. science, 234(4780), 1105-1108. doi: 10.1126/science.234.4780.1105. marty, b. & yokochi, r. (v). water in the early earth. reviews in mineralogy and geochemistry, 62(1), 421–450. doi: doi.org/10.2138/rmg.2006.62.18 mcfadyen, e. (2011). antarctic future – education learning about antarctica in the 21st century. a review of selected education resources available to primary teachers. university of canterbury, postgraduate certificate, postgraduate certificate in antarctic studies. özbebek tunç, a., akdemir ömür, g. & düren, a. z. (2013). çevresel farkındalık. i̇.ü. journal of faculty of political science, 0(47), 227-246. palmer, j.a. & suggate, j. (2004) the development of children’s understanding of distant places and environmental issues: report of a uk longitudinal study of the development of ideas between the ages of 4 and 10 years. research papers in education, 19(2), 205-237. ricci, c. a., & egerton, p. (eds.). (2010). european research in the polar regions: relevance, strategic context and setting future directions in the european research area (epb strategic position paper). european science foundation. robert, f. (2001). the origin of water on earth. science, 293(5532), 1056– 1058. doi:doi.org/10.1126/science.1064051. salazar, j.f. (2012). perceptions and opinion of antarctic values in chile. in liggett, d. & hemmings, a.d (eds.) exploring antarctic values. christchurch: university of canterbury, gateway antarctica special publication series. shabudin, a. f. a, rahim, r. a. & ng, t. f. (2016). strengthening scientific literacy on polar regions through education, outreach and communication (eoc). international journal of environmental & science education, 11(12), 5498-5515 the national academy of sciences (nas). (2011). future science opportunities in antarctica and the southern ocean, the national academies press: washington dc. tuna, f., demirci, a. & gültekin, n. (2012). temel coğrafi bilgi ve beceriler toplumda ne ölçüde kullanılıyor? yön, konum ve harita becerilerinde mevcut durum analizi. milli eğitim, 195, 211-227. ursavaş, n., & aytar, a. (2018). okul öncesi öğrencilerinin su farkındalığı ve su okuryazarlıklarındaki gelişimin incelenmesi. journal of research in informal environments, 3(1), 19-45. usgs (2020). how much of the earth's water is stored in glaciers? https://www.usgs.gov/faqs/how-much-earths-water-storedglaciers?qt-news_science_products=0#qtnews_science_products. vural, d., yirmibeşoğlu, s., oktar, ö. & özsoy, b. (2018). kutup bölgelerindeki bilimsel araştırmaların eğitim ve farkındalık çalışmalarına etkisi, ii. kutup bilimleri çalıştayı özet kitabı, 76, 12-13 eylül, i̇sanbul-türkiye. wu, j., desch, s. j., schaefer, l.,elkins-tanton, l. t., pahlevan, k. & buseck, p. r. (2018). origin of earth’swater: chondritic inheritance plusnebular ingassing and storage of hydrogen in the core. journal of geophysical research: planets, 123(10), 2691–2712. https://www.usgs.gov/faqs/how-much-earths-water-stored-glaciers?qt-news_science_products=0#qt-news_science_products https://www.usgs.gov/faqs/how-much-earths-water-stored-glaciers?qt-news_science_products=0#qt-news_science_products https://www.usgs.gov/faqs/how-much-earths-water-stored-glaciers?qt-news_science_products=0#qt-news_science_products a © 2020 indonesian society for science educator 91 j.sci.learn.2020.4(1).91-100 received: 24 august 2020 revised: 19 november 2020 published: 28 november 2020 an enrichment workshop using argumentation-based forensic chemistry activities to improve the critical thinking of gifted students mustafa tuysuz1*, ummuye nur tuzun2 1faculty of education, van yuzuncu yil university, van, turkey 2ankara yenimahalle science and art centre, ministry of national education, ankara, turkey *corresponding author: mustafatuysuz@yyu.edu.tr abstract this research aims to evaluate a workshop using argumentation-based forensic chemistry activities to enhance gifted students' critical thinking. a workshop program, consisting of seven argumentation-based forensic chemistry activities, was conducted with 20 students at a gifted school in turkey. a qualitative experimental design was used. an experiment or drawing activity was first carried out. following this step, the gifted students reconstructed the activity as an argument after an extensive group discussion. the data collected in the student-constructed arguments and evaluation were analyzed for content. the study's findings show that argumentation-based forensic chemistry activities contributed positively to these gifted students' critical thinking development. keywords enrichment workshop, forensic chemistry education, argumentation, critical thinking, gifted students 1. introduction there always have different individuals within the community who have different learning speeds and inspire dazzling products by producing innovative ideas. the primary purpose of gifted education is to identify, develop, and support these individuals' special features (subotnik, olszewski-kubilius, & worrell, 2011). renzulli (2012) emphasized two essential characteristics of effective gifted education. first, "the approaches should exhibit a logical relationship between the theory-guided services provided to students and the concept of giftedness that serves as a rationale for the development of that theory." for example, an accelerated theory that suggests the utilization of advanced mathematics lessons should relate to an uppercognitive understanding of mathematics aimed at students with high abilities. second, and in particular concerning the enrichment-based theory, "services should be provided for both advanced cognitive development and what are referred to below to as 'intelligence outside the normal curve'" (pp. 150-151). this situation's emergence has been conceived as a guideline for integrating and applying gifted young students to special teaching programs to improve their intelligence levels in the last 30 years. according to acceleration-based and enrichment-based theories, different programming options for the gifted were searched in the literature. for example, ng and nicholas (2010) conducted a case study regarding the exploration of the nature of interactions in an online learning environment. the investigation sample includes a group of high-ability 14-year-old students who participated in an extended learning project online as an extracurricular activity for approximately six months. the study results indicated that the students interacted differently online depending on the task at hand, and seven of them finished the final task of creating a learning product as a. in another example, yoon (2009) investigated the effects of selfregulated learning on scientific inquiry performance in a sample of scientifically gifted middle school students. the participants were involved with educating after-school enrichment programs and responded to the self-report. the findings showed that open inquiry learning influenced the use of self-regulatory strategies. vural (2010) conducted a study based on the learning cycle model as the enrichment for gifted students' education. in the research, gifted students' mental models and pre and post-misconceptions were analyzed. at the end of the study, it was found that the scientifically correct knowledge was constructed instead of gifted students' scientifically incorrect preknowledge after the learning cycle model-based mailto:mustafatuysuz@yyu.edu.tr journal of science learning article doi: 10.17509/jsl.v4i1.27570 92 j.sci.learn.2020.4(1).91-100 enrichment. just as in the previous example, materials based on the standard knowledge construction model were utilized to enrich the education of the gifted that resulted in gifted students' use of the scientific knowledge for daily life (demircioglu & vural, 2016). as the last example of the enrichment for gifted education, cakir's study (2011) could be presented in which materials based on the predictobserve-explain model were applied. at the end of the research, it was determined that gifted students could make proper choices. after backing their decisions with observations, they could form better mental models, resulting in better knowledge construction. in addition to these studies, it is emphasized in the literature that studies on gifted students should be at a higher level (ziegler, stoeger, & vialle, 2012). the gifted education literature describes several skills in which gifted students perform at higher levels than similar students in the same class or school. these advanced skill differences manifest themselves in various school curriculum levels, particularly in mathematics and reading, in education. this differentiation must extend to the teaching of critical thinking skills as part of the core curriculum. because developing critical thinking skills is the 21stcentury main goal for all students (kettler, 2014). by being aware of the lack of critical thinking education programs for the gifted, argumentation can be offered as programming options for enhancing gifted students’ critical thinking skills (tuzun, eyceyurt-turk, harmanci, & ertem, 2017). critical thinking means making plausible decisions (freeley & steinberg, 2005). in other words, critical thinking implies an argumentation process consisted of constructing an argument, presenting evidence and backings to this argument, or building up counterarguments (joung, 2003). in contrast, argumentation means coordination of evidence and theory to support or refute a conclusion (osborne, erduran, & simon, 2004). individuals make decisions about their lives, such as career planning, friendship preferences, and where they will live. their choices to be well and responsible citizens are based on their ability to think critically. critical thinking allows them to reduce their argument to their parts to assess their relative validity and strength. critical thinkers are better informants, as well as better advocates (freeley & steinberg, 2005). hence, critical argumentation is a practical skill that should be developed through the integration into actual or realistic daily life examples of arguments not just for gifted individuals but also for all students from the very beginning of their education levels. critical argumentation is not only a skill, but it is also related to an attitude within a decision-making process or a solution to a problem. however, they are most often used in both sides' logical thinking processes and balanced issues (walton, 2006). in terms of a logical, sequential rational argument, bowell and kemp (2005) stated that it is necessary to define the controversial subject and determine whether the other side of the subject's speaker is trying to persuade. then, the argument needs to be restructured, and the debate is evaluated at the final stage to determine what is right and what is wrong. in the literature, nussbaum and edwards (2011) explored the concepts of critical questions and argument stratagems as an approach for teaching argumentation and critical thinking. for six months, the study was conducted in social classes, in which 30 seventh-grade students discussed and wrote about current events. participants were divided into two groups; experimental and control. the study's findings showed that the experimental group made more arguments combining both sides of each topic over time. the experiment group collectively created critical questions, mostly about weight values and the design of practical, creative solutions. west (1994) conducted a study to investigate at a university for determining the effects of argumentation on critical thinking skills. during the implementation, a micro-unit in argumentation theory was tested to impact critical thinking through a quasi-experimental design. the investigation findings showed that there were statistically significant results for both speech core treatment on the 'interpretation of data' subtest and general education public speaking students on the 'argument' subtest. another research was examined the impact of a unit that integrates the explicit teaching of general reasoning patterns into the teaching of specific science content. it mainly investigated the teaching of argumentation skills in the context of dilemmas in human genetics to seventh-ninth grade students. experimental group participants' mean was significantly higher than the other group participants for testing of genetic knowledge. there was also an increase in the quality of students' argumentation. moreover, students in the experimental group could transfer the reasoning skills taught in genetic to the context of dilemmas given from daily life (zohar & nemet, 2002). kabataş-memiş and çakan-akkaş (2020) researched the efficiency of the argumentation-based inquiry on the critical thinking skills of fifth-grade students. a semiexperimental design was used. moreover, argumentationbased inquiry activities were utilized for the experimental group. it was seen that argumentation-based inquiry courses made experimental group students' critical skills improved. in another study, think-read-group-sharereflect (trgsr) scientific argumentation strategy was used for determining the efficiency of the toulmin argument pattern (tap) on high school students' critical thinking. a semi-experimental design was deployed. the participants were 50 twelfth grade students. the courses in the experimental group were with tap within trgsr. watson-glaser critical thinking appraisal form–s was administered as a pre and post-test to both groups for determining critical thinking. covariance findings showed a significant difference between the groups' critical thinking journal of science learning article doi: 10.17509/jsl.v4i1.27570 93 j.sci.learn.2020.4(1).91-100 after nine weeks (giri & paily, 2020). however, in the literature, fewer studies were exploring critical argumentation for the education of the gifted. a study investigating the effect of thought experiments-based argumentation for enhancing critical thinking skills on the education of the gifted suggested the thought experiment argumentation critical thinking triangulation as a programming option enrichment for alignment with the gifted students' acceleration. in the study, single-subject research was used, and interventions were utilized based on single-subject research's theory. at the end of the study, it was found that thought experiment argumentation critical thinking triangulation was an effective programming option enrichment for alignment with the gifted students' acceleration (tuzun, eyceyurt-turk, harmanci, & ertem, 2017). enrichment offers gifted students a deeper interaction with a particular subject than they could do in a regular class to align with their acceleration, their more rapid cognitive processing (subotnik, olszewski-kubilius, & worrell, 2011). for example, specific forensic chemistry activities could be constructed as a subject since the gifted students do not encounter such a subject in their regular classes. there is only research about forensic chemistry activities for the gifted in the literature (tuysuz & tuzun, 2019). the determined subject ‘forensic chemistry’ is the chemistry that conducts the chemical analysis for proof to be used in the legal system (gercek, 2012). moreover, the development of the sensitivity and selectivity for detection of analytical methods in recent years has made progress in the forensic analysis for various evidence, including narcotics and explosives, and material conveyed to the victim or crime scene (almirall, 2005). an example of the forensic analysis from the literature could be seen in figure 1. forensic chemistry education is needed since the students choose it as a career (gercek, 2012). thus, training programs should provide better-prepared graduates and future leaders in the forensic chemistry profession (almirall, 2005). this research aims to evaluate the programming options (enrichment) for the workshop of gifted students. forensic chemistry activities based on argumentation were applied for improving gifted students' critical thinking. in light of the preceding, the research's importance is to fill the literature gap about specific forensic chemistry education enrichment studies on gifted education education. the main question (problem) of the research was determined as "how could the gifted students' critical thinking skills be improved by the programming options, the enriched workshop based on argumentation activities on the topic of forensic chemistry?" 2. method 2.1.research design the current investigation was employed case study methodology as a type of qualitative research. the cases of interest in education are the people and the programs. the researchers are interested in both of them for their uniqueness and commonality. they seek to understand the factors and variables that impact them. they would like to hear their stories and their feelings, emotions, and affective states (stake, 1995). for this research, the case of interest for a more in-depth understanding is the enrichments programming options for the gifted education. 2.2. sample of the study being different from the quantitative methods, qualitative methods it was enough to work with a smaller participant number since the qualitative findings would provide much more knowledge in depth (yildirim & simsek, 2008). thus, this research was conducted on 20 gifted students at a school for gifted in turkey in the 20172018 academic year. furthermore, there was no need to generalize the findings to a population in qualitative methods (yildirim & simsek, 2008). programming options for the workshop for gifted students were the focus of this research. the sample selection method was a purposive sample. according to the purposive sample's criteria, the sample's students had to be intended as gifted, educated at a science and art center, and willing. the gifted students' age average was nine. 13 of the participants were female, and the others were male. 2.3. research instruments firstly, as being the research instrument, the worksheets were used. argumentation focused forensic chemistry activities teaching guide worksheets for the alignment with gifted students' acceleration and improving their critical thinking were constructed. the teaching guide consisted of seven forensic chemistry activities that made students construct their arguments, according to walton (2006) argument pattern components after arguing the experiment/drawing part of the activity. walton's (2006) argument pattern components were a conclusion (a constructive claim) based on three premises (warrants of the conclusion). two different figure 1 the footprint in the aluminum dust (royds, lewis & taylor, 2005, p. 267) journal of science learning article doi: 10.17509/jsl.v4i1.27570 94 j.sci.learn.2020.4(1).91-100 science educators checked the content validity of the teaching guide. an example from the teaching guide's worksheets could be seen in figure 2. secondly, participant observation notes were employed as research instruments. in other words, the whole process' evaluation was made by using gifted students' observation notes. the same two science educators checked the observation notes' content validity. the instruments' reliability was checked by two independent science educators' coding and categorizing consistency, which was determined as 95 percentages. the correlation to the formula was given as the coding/categorizing consistency percentage = (parallel codes / all codes) x100. the independent researchers reached a consensus when there was incompatible coding and categorizing them by redefining codes and categorizes them for these pieces of knowledge. data triangulation, investigator triangulation, theory triangulation, methodological triangulation, and environmental triangulation were offered in the literature (guion, diehl, & mcdonald, 2002). in this research, to establish the study's validity, data triangulation was made using two different data collecting tools, and investigator triangulation was done using two different researchers in the analysis process. 2.4. procedure before the workshop, forensic chemistry, argumentation, argument, and critical thinking were introduced to the gifted students. during the workshop, firstly, the experiment/drawing part of the activity was conducted, and then the gifted students reconstructed the activity as an argument according to walton's (2006) argument pattern components (conclusion, premise, premise, premise) after extensive group discussions. after carrying out the workshop, the gifted students evaluated the whole process. 2.5. data analysis the content analysis was utilized for the collected data. the codes and categories were constructed, and then frequencies and percentages were calculated. the crosscontent analysis was utilized for data reliability to ascertain that each of the codes was under a category (erickson, 2004). thus, there would be no codes that eluded observation during categorizing. in other words, each code belonged to a category. the content analysis would be beneficial to denote how gifted students' critical thinking was measured. based on cambridge assessment international education, the gifted students were thought to improve their critical thinking skills if they could construct scientifically correct arguments while reconstructing an activity as an argument. also, the arguments must contain walton's (2006) argument pattern components. on the other hand, the students' argument construction skills were evaluated as sufficient if only their scientifically accurate constructed and all walton argument pattern components consisted arguments percentages were higher than %33 of all the participants for each of the forensic chemistry experiments/drawings. 3. result and discussion after utilizing the content analysis to gathered data, the results accessed were given in two subtitles: students’ figure 2 an example from the worksheets journal of science learning article doi: 10.17509/jsl.v4i1.27570 95 j.sci.learn.2020.4(1).91-100 argument constructing skills and students’ process evaluation. 3.1. students’ argument construction skills students constructed arguments for each of the seven forensic chemistry experiments/drawings of the workshop. the indicators to measure the ability to argue were first scientifically correct constructed arguments and second walton (2006) argument patterns consisted of arguments. therefore, student-constructed arguments were analyzed if only they were scientifically correct constructed. students' scientifically accurate constructed arguments were then coded according to walton's (2006) argument pattern components, and then categories were formed. in the end, frequencies and percentages were computed for each of the categories. the cross-content analysis was utilized, too. the participant constructed arguments' codes, categories, frequencies, and percentages could be seen in table 1. in table 1, the conclusion was shown by c, premise by p, frequency by f, and the percentage by (%). for example, a 'pp' category means a scientifically valid constructed argument consisting of the walton argument pattern premise and premise components as codes without a conclusion component code. a 'cppp' category means a scientifically valid constructed argument consisted of all walton argument pattern components, conclusion, premise, premise, premise as codes. another example could be the 'cppppp' category, which means a scientifically correct constructed argument consisted of all walton argument pattern components, with an extra two premises components as codes. as seen in table 1, students' scientifically accurate constructed arguments total frequencies for each of the forensic chemistry experiments/drawings were equal to the total number of participants. students' scientifically correct constructed arguments consisting of all walton (2006) argument pattern components were 60% for the first experiment, 65% for the second experiment, 45% for the third experiment, 90% for the fourth experiment, 60% for the fifth experiment, 95% for the sixth drawing, and 70% for the last experiment. the total percentages of conclusion-premise-premise-premise 'cppp' category, conclusion-premise-premise-premise-premise 'cpppp' category, and conclusion-premise-premise-premisepremise-premise 'cppppp' category were taken as categories consisting of all walton argument pattern components for each of the forensic chemistry experiments/drawings. the students' argument construction skills were evaluated as sufficient by the two science educators if only their scientifically correct constructed and all walton argument pattern components consisted arguments percentages were higher than 33% of all the participants for each of the forensic chemistry experiments/drawings. in other words, the students' ability to argue was thought enough if only they constructed scientifically valid arguments based on walton's (2006) argument pattern components. it could be seen in table 1 that all the percentages are over 33% for each of the forensic table 1 student constructed arguments’ analysis* forensic chemistry experiments/ drawings f % pp ppp pppp ppppp cp cpp cppp cpp pp cpp ppp total 1.taking fingerprint 1 % 5 2 % 10 1 % 5 1 % 5 1 % 5 2 % 10 7 % 35 2 % 10 3 % 15 20 % 100 2.isolation of dna 3 % 15 4 % 20 8 % 40 5 % 25 20 % 100 3.taking footprint and calculation of height from footprint 1 % 5 4 % 20 6 % 30 7 % 35 1 % 5 1 % 5 20 % 100 4.taking teeth print 1 % 5 1 % 5 16 % 80 1 % 5 1 % 5 20 % 100 5.the mystery at the note 2 % 10 6 % 30 12 % 60 20 % 100 6.forensic chemistry paintings 1 % 5 15 % 75 3 % 15 1 % 5 20 % 100 7.soil analysis 2 % 10 4 % 20 14 % 70 20 % 100 * in table 1, conclusion was shown by c, premise by p, frequency by f, and percentage by (%). journal of science learning article doi: 10.17509/jsl.v4i1.27570 96 j.sci.learn.2020.4(1).91-100 chemistry experiments/drawings; therefore, it could be stated that students' argument construction skills were sufficient, so their critical thinking skills were too. in other words, the gifted students' argument construction ability was enhanced since students' percentages were increased throughout the argument construction processes. according to literature, it could be stated that scientifically accurate argument construction means being able to think critically (cambridge assessment international education, 2017). thus, it could be said that the gifted students’ argument construction skills were sufficient and that this contributed positively to their critical thinking skills. to strengthen these qualitative findings, studentconstructed arguments from the teaching guide’s worksheets were given below. g showed ‘gifted student’. as seen in figure 3, (1) taking fingerprint: take your fingerprint by using a graphite pen, a sheet of white paper, adhesive tape. compare your fingerprints with your peers. (students’ fingerprints were used by only themselves according to ethics.) the argument constructed by g7 (g for gifted): we took our fingerprints by using graphite (conclusion). we can take fingerprints by using iodine (premise). we can take fingerprints by using silver nitrate, too (premise). if one's finger was bloody, then we could take fingerprint by spraying luminol (premise). as seen in figure 4, (2) isolation of dna: isolate your dna from your saliva by using saturated salty water solution, detergent, and alcohol and examine the macroscopic shape of your dna. (only the teacher's bio-matrix was used because of ethics.) the argument constructed by g3: we isolated dna from saliva by using saturated saline water solution, detergent, and alcohol (conclusion). one could isolate dna from hair also (premise). one could also isolate dna from blood (premise). we can find relations with the help of dna (premise). as seen in figure 5, take the footprint and calculation of height from the footprint. moreover, take your teacher's footprint by using starch and a glass-sheet. calculate your teacher's height by using her footprint. you can also calculate her average-weight by using her footprint too. the argument constructed by g11: we took footprint by using starch (conclusion). we can calculate the adult's height from her footprint length. (24.5 centimeters / the adult's height) x 100 = 15) (premise). we can compute the adult's average weight from her footprint. (after calculating the adult's height from her footprint length, the average weight would be on the minus five plus five scales of the adult's height's last two numbers.) (premise). we can analyze the footprint by ultraviolet radiation, too (premise). as seen in figure 6, (4) taking teeth print: take your teeth to print using a plaque made from a candle and use a heater and forceps. heat the plaque by using forceps; fold it at the shape of a horseshoe by using forceps. and then bite it but be sure the plaque is not so hot for your mouth's safety. wash your mouth with water after biting the table 2 students’ process evaluation categories codes f % learning outputs learning new information 20 100% enjoyable learning 7 35% learning through experiment 8 40% learning through drawing 3 15% critical thinking outputs criticizing their own thinking strategies 11 55% criticizing others’ thinking strategies 11 55% total 20 100% (1) taking fingerprint; take your fingerprint by using a graphite pen, a sheet of white paper, adhesive tape. compare your fingerprints with your peers. (students’ fingerprints were used by only themselves according to ethics.) figure 3 a fingerprint (bbc.com) (2) isolation of dna: isolate your dna from your saliva by using saturated salty water solution, detergent, and alcohol and examine the macroscopic shape of your dna. (only teacher’s bio-matrix was used because of ethics.) figure 4 isolated dna(photo taken through workshop) (3) take the footprint and calculation of height from the footprint take your teacher’s footprint by using starch and a glass-sheet. calculate your teacher’s height by using her footprint. you can also calculate her average-weight by using her footprint too. figure 5 a footprint (photo taken through workshop) journal of science learning article doi: 10.17509/jsl.v4i1.27570 97 j.sci.learn.2020.4(1).91-100 plaque. compare your print with your peers'. (students' teeth prints were used by only themselves according to ethics.) the argument constructed by g8: we took teeth print by using candle plaque (conclusion). we calculated age with the help of teeth print. (as an example, if the candle plaque contained a lack of two/three teeth prints and the other existing ones were under average size, it would back up that the teeth print was eight/nine years old.) (premise). teeth print could be made observable by using carbon paper (too). (two sheets of carbon papers standing back to back must be put in the two layers of white paper then the paper must be bitten.) (premise). footprint could be taken from the feet (by using warm candle plaque. it could give hints the person's weight with the pressure made.) (premise). as seen in figure 7, (5) the mystery at the note: suppose you found a note written with a blue color pen at a crime scene. also, suppose you had two suspects, and you found two different blue color pens at their homes. find whose pen was used for writing the note. use chromatography. the argument constructed by g1: we found which of the pens was used for writing the note by using chromatography (conclusion). we used adsorbent layers and alcohol solution (premise). the link found at the representative crime scene and one of the typical pens’ ink walked simultaneously, and the other pen’s ink walked much more rapidly on the adsorbent layer (premise). we could do the same experiment by using paper-towel as the adsorbent layer and water as adsorbent (premise). as seen in figure 8, (6) forensic chemistry paintings: draw a painting about forensic chemistry. compare your drawing with your peers’. the argument constructed by g14: i drew a forensic chemistry painting (conclusion). i drew luminol's chemical structure (premise). i drew dna's chemical structure (premise). i drew my handprint (premise). hence, i made science and art integration (premise). (the painting could be seen in figure 9.) the argument constructed by g19: we drew forensic chemistry paintings (conclusion). i drew dna's chemical structure (premise). i drew luminol's chemical structure (premise). i drew graphite is one layer's structure (premise). (the painting could be seen in figure 10.). (7) soil analysis: suppose you found soil at the crime scene. find where the soil came from. consider alkali concentration for representative soil samples, which means ph would be specific for soils from different regions. the argument constructed by g19: we analyzed the representative soil samples by determining their ph (conclusion). thus, the representative crime scene’s soil and the representative soil samples obtained from the suspects could be compared (premise). because ph is specific for soil from different regions (premise). and there are other instrumental techniques for advanced soil analysis (premise). 3.2. students’ process evaluation participant observation notes were used as a research instrument in this section. there were not any questions (4) taking teeth print: take your teeth print by using a plaque made from candle, also use heater and forceps. heat the plaque by using forceps; fold it at the shape of a horseshoe by using forceps. and then bite it but be sure the plaque is not so hot for your mouth’s safety. wash your mouth with water after biting the plaque. compare your print with your peers’. (students’ teeth prints were used by only themselves according to ethics.) figure 6 teeth print (photo taken through workshop) (5) the mystery at the note: suppose you found a note written by a blue color pen at a crime scene. also suppose you had two suspects and you found two different blue color pens at their homes. find whose pen was used for writing the note. use chromatography. figure 7 chromatograms (photo taken through workshop) (6) forensic chemistry paintings: draw a painting about forensic chemistry. compare your drawing with your peers’. figure 8 a painting (photo taken through workshop) figure 9 a forensic chemistry painting (by g14) figure 10 a forensic chemistry painting (by g19) journal of science learning article doi: 10.17509/jsl.v4i1.27570 98 j.sci.learn.2020.4(1).91-100 asked to participants; instead, they took some notes evaluating the process as learning and critical thinking outputs. data gathered from students' observation notes for the whole process evaluation was coded and categorized. then frequencies and percentages were calculated. the cross-content analysis was utilized too. codes, categories, frequencies, and percentages for students' process evaluation could be seen in table 2. in table 2, the frequency was shown by f and percentage by %. the two science educators considered the percentages if only the percentages were over 33% of all the participants. it could be seen in table 2 that students' process evaluation was about new information learning (100%), enjoyable learning (35%), learning through experiment (40%), criticizing their thinking strategies (55%), and criticizing others' thinking strategies (55%). to strengthen these qualitative findings, students’ process evaluation from the students’ observation notes were given below: g1: … i took my fingerprint. we took teeth prints. then, we solved the mystery at the note. i analyzed the saliva and soil. i calculated the person’s height from her footprint … (learning new information code). g3: … i took my fingerprint. i solved the mystery at the note. i analyzed the saliva. i took a footprint. i analyzed soil. all was so amusing … (learning new information code, enjoyable learning code). g9: … we did so many experiments about forensic chemistry. all was so amusing. i liked forensic chemistry so much (enjoyable learning code). we took footprint. we took fingerprint (learning new information code). everybody's thinking differed from each other (criticizing others' thinking strategies code). my thought became different through the experiments (learning through experiment code, criticizing my own thinking strategies code). g20: … i did experiments (and got the information). if i could use advanced instrumental analysis techniques, i could get much more information (learning new information code, criticizing my own thinking strategies code) 3.3 discussion the main goal of gifted education is to help gifted students develop their talents. gifted researchers and educators have designed specific services (e.g., programs, interventions, curricula) based on the gifted learners’ characteristics to effectively address their learning needs (jen, moon, & samarapungavan 2015). in this research, firstly, it could be said that gifted students' critical thinking skills improved by making them construct forensic chemistry activities as arguments during the workshop. according to the literature, scientifically proper constructed arguments based on an argument pattern could improve students' critical thinking skills (cambridge assessment international education, 2017). as a reason for this improvement, gifted students encountered a differentiated-enriched learning environment that they have never encountered before. this result may have led them to increase their interest and to think more deeply about the situations they face. moreover, the exchange of views in the process might have contributed to their ideas regarding the situations they face. like this research's findings, the literature offers accelerated and enriched programming options for gifted education (stott & hobden, 2016). in other words, carefully planned acceleration offers educational benefits to high-ability learners (dare & nowicki, 2015). it was also suggested to look for ways to further the sequence of critical thinking skills in specific learning domains (kettler, 2014). hence, in this research, specific programming options were studied to further gifted students' critical thinking strategies in a particular argumentation-based forensic chemistry learning domain with the awareness of the gifted students' acceleration. there is only one research about forensic chemistry activities for the gifted (tuysuz & tuzun, 2019). this research illustrated an argumentation-based forensic chemistry activities application in detail for workshop enrichments for the gifted education. thus, it was thought that this study might contribute to the gap in the literature about specific enrichment studies. moreover, the teachers of the gifted could conduct a similar application process in their workshops. secondly, the students evaluating the process showed that the process made them able to learn new information and made them able to criticize their own and others' thinking strategies. kaya and kilic (2008) said that students educating in argumentation-based lessons could learn the scientific concepts in detail, and through criticizing their own and others' thinking strategies, they could also know the nature of science and how science works. another study from the literature underlined the importance of small-group discussions for high-ability students (jen, gentry, & moon, 2017). a microanalysis of student initiations in gifted classes and their effect on the emergence of dialogic discourse showed that student initiations trigger teacher responses and student responses that often lead to additional student initiations. all these options give rise to a more balanced classroom setting, in which both teacher and student create meaning, amounting to dialogic discourse as defined by lotman (cf., netz, 2014). in this research, students' evaluation of the usefulness of criticizing their own and others' thinking strategies justifies the previous research. in the literature, forensic chemistry applications were not so familiar. hence, the education of forensic chemistry applications, in other words, the teaching of chemical analysis of evidence was crucial for gifted students and all students for training future leaders in the forensic chemistry profession (almirall, 2005; gercek, 2012). this research's application process, its forensic chemistry experiments, and its argument evaluation process were modeled on how to make forensic chemistry education by journal of science learning article doi: 10.17509/jsl.v4i1.27570 99 j.sci.learn.2020.4(1).91-100 a basic logic for educators who would want to make such an instruction. the limitations of this research were theory triangulation (use of outside professionals' views) and methodological triangulation (use of different methods), and environmental triangulation (various settings), which could not be utilized. for further studies, different programming options to align with gifted students' acceleration and improve their critical thinking skills could be constructed on specific subjects based on more triangulations. another limitation of this research was its multidisciplinary was limited by only arts integration to forensic chemistry. future giftedness research needs more multidisciplinary for several reasons. first, to prove the credentials of gifted education, researchers specializing in the economics of education are required. additionally, many other sciences might be valuable allies, such as political sciences, arts, sports science, etc. second, when we adopt an ecological or a systemic approach, a single disciplinary approach will rarely suffice (ziegler, stoeger, & vialle, 2012). as students evaluated the process, one of the gifted students said that "… i did experiments and got information. if i could use advanced instrumental analysis techniques, i could get much more information …" the participants' age was nine in this study. it could be against the ethics of performing advanced forensic chemistry experiments with these students because, in this research, basic experiments that would not make students apprehensive were constructed. however, if the participants were gifted students from high school level or were university students, advanced crime scene scenarios and advanced instrumental analysis techniques could be used for further studies. 4. conclusion the findings of the study showed that argumentationbased forensic chemistry activities contributed positively to the participants' critical thinking development as getting 60% success for the first activity, 65% for the second, 45% for the third, 90% for the fourth, 60% for the fifth, 95% for the sixth, and 70% for the last. in light of these findings, it might be stated that this study contributed to developing the argumentation structuring levels of gifted students since students' percentages were enhanced throughout the argument construction processes. according to literature, it could be stated that scientifically accurate argument construction means being able to think critically (cambridge assessment international education, 2017). therefore, it could be said that the gifted students’ argument construction skills were sufficient, and this situation contributed positively to the development of their critical thinking skills. moreover, when the students’ evaluation of the process was examined, they stated that the process made positive contributions for them, such as new information learning, enjoyable learning, learning through experiments, criticizing their thinking strategies, and criticizing others’ thinking strategies. references almirall, j. r. (2005). forensic chemistry education. analytical chemistry, 1, 69-72. bowell, t., & kemp, g. (2005). critical thinking: a concise guide. taylor and francis group. cambridge assessment international education. (2017). cambridge international as & a level thinking skills 9694, syllabus (2020-2022). retrieved from https://www.cambridgeinternational.org/images/415052-20202022-syllabus.pdf cakir, m. (2011). üstün yetenekli öğrencilerin iletkenlik ve yalıtkanlık kavramları hakkındaki zihinsel modellerinin incelenmesi [examining the mental models of gifted students on the concepts of conductivity and insulativity] (master thesis, yüzüncü yıl university, institute of science, van). retrieved from https://www.ulusaltezmerkezi.net/ustun-yetenekli-ogrencileriniletkenlik-ve-yalitkanlik-kavramlari-hakkindaki-zihinselmodellerinin-incelenmesi/ dare, l., & nowicki, e. (2015). conceptualizing concurrent enrollment: why high-achieving students go for it. gifted child quarterly, 59(4), 249-264. https://doi.org/10.1177/0016986215597749 demircioglu, h., & vural, s. (2016). ortak bilgi yapılandırma modelinin sekizinci sınıf düzeyindeki üstün yetenekli öğrencilerin kimya dersine yönelik tutumları üzerine etkisi [the effect of the common knowledge structuring model on the attitudes of gifted students at the eighth grade towards chemistry course]. hasan ali yücel eğitim fakültesi dergisi, 13(1), 49-60. erickson, e. (2004). demystifying data construction and analysis. anthropology and education, 35(4), 486-493. freeley, a. j., & steinberg, d. l. (2005). argumentation and debate: critical thinking for reasoned decision making. thomson wadsworth. gercek, z. (2012). adli kimya eğitimi [forensic chemistry training]. yükseköğretim ve bilim dergisi, 2(3), 201-204. https://doi.org/10.5961/jhes.2012.051 giri, v., & paily, m. u. (2020). effect of scientific argumentation on the development of critical thinking. science & education, 29, 673-690. guion, l. a., diehl, d. c., & mcdonald, d. (2011). triangulation: establishing the validity of qualitative studies. edis, (8), 3-3. jen, e., gentry, m., & moon, s. m. (2017). high-ability students’ perspectives on affective curriculum in a diverse, university-based summer residential enrichment program. gifted child quarterly, 61(4), 328-342. https://doi.org/10.1177/0016986217722839 jen, e., moon, s., & samarapungavan, a. (2015). using design-based research in gifted education. gifted child quarterly, 59(3), 190-200. https://doi.org/10.1177/0016986215583871 joung, s. (2003). the effects of high-structure cooperative versus low-structure collaborative design on online debate in terms of decision making, critical thinking, and interaction pattern (doctoral dissertation, the florida state university, florida). retrived from: https://fsu.digital.flvc.org/islandora/object/fsu%3a181961 kabataş-memiş, e., & çakan-akkaş, b. n. (2020). abi approach used in science lessons had improved the critical thinking skills of the experimental group students. asia pacific education review, 21, 441453. kaya, o. n., & kilic, z. (2008). etkin bir fen öğretimi için tartışmacı söylev [argumentative discourse for effective science teaching]. ahi evran üniversitesi kırşehir eğitim fakültesi dergisi, 9(3), 89-100. kettler, t. (2014). critical thinking skills among elementary school students: comparing identified gifted and general education student performance. gifted child quarterly, 58(2), 127-136. https://doi.org/10.1177/0016986214522508 netz, h. (2014). gifted conversations: discursive patterns in gifted classes. gifted child quarterly, 58(2), 149-163. https://doi.org/10.1177/0016986214523312 journal of science learning article doi: 10.17509/jsl.v4i1.27570 100 j.sci.learn.2020.4(1).91-100 ng, w., & nicholas, h. (2010). a progressive pedagogy for online learning with high-ability secondary school students: a case study. gifted child quarterly, 54(3), 239-251. https://doi.org/10.1177/0016986209355973 nussbaum, e. m., & edwards, o. v. (2011). critical questions and argument stratagems: a framework for enhancing and analyzing students’ reasoning practices. the journal of the learning sciences, 146. https://doi.org/10.1080/10508406.2011.564567 osborne, j., erduran, s., & simon, s. (2004). enhancing the quality of argumentation in school science. journal of research in science teaching, 41(10), 994-1020. https://doi.org/10.1002/tea.20035 renzulli, j. s. (2012). reexamining the role of gifted education and talent development for the 21st century: a four-part theoretical approach. gifted child quarterly, 56(3), 150-159. https://doi.org/10.1177/0016986212444901 stake r. e. (1995). the art of case study research. sage. stott, a., & hobden, p. a. (2016). effective learning: a case study of the learning strategies used by a gifted high achiever in learning science. gifted child quarterly, 60(1), 63-74. https://doi.org/10.1177/0016986215611961 subotnik, r. f., olszewski-kubilius, p., & worrell, f. c. (2011). rethinking giftedness and gifted education: a proposed direction forward based on psychological science. psychological science, 12(1), 3-54. https://doi.org/10.1177/1529100611418056 tuysuz, m., & tuzun, u. n. (2019). özel yetenekli öğrenciler için adli kimya eğitimi [forensic chemistry training for special talented students]. başkent university journal of education, 6(2), 213-224. tuzun, u. n., eyceyurt-turk, g., harmanci, a. b., & ertem, n. (2017). bilim eğitiminde üstün zekâlı bireylerin düşünce deneyleriyle eleştirel düşünme becerilerinin geliştirilmesine yönelik bir öğretim dizini yapılandırma [building a teaching directory for the development of critical thinking skills of gifted individuals in science education with thought experiments]. paper presented at 8th international education management forum, tobb university, ankara. vural, s. (2010). yapılandırmacı yaklaşıma uygun geliştirilen etkinliklerin üstün yetenekli öğrencilerin kavramları anlamalarına etkisi: ‘erime, donma, buharlaşma, kaynama ve yoğuşma’ [the effect of activities developed in accordance with the constructivist approach on gifted students' understanding of the concepts: 'melting, freezing, evaporation, boiling and condensation'] (master thesis, karadeniz technical university institute of science, trabzon). retrieved from: https://tez.yok.gov.tr/ulusaltezmerkezi/tezdetay.jsp?id=msap f1f-yfnge4aovdtj8a&no=ks9d8asrfdlnrles6mqtva walton, d. (2006). fundamentals of critical argumentation. cambridge university press. west, t. l. (1994). the effect of argumentation instruction on critical thinking skills (doctoral dissertation, southern illinois university, chicago). yıldırım, a., & şimşek, h. (2008). sosyal bilimlerde nitel araştırma yöntemleri [qualitative research methods in the social sciences]. seçkin. yoon, c-h. (2009). self-regulated learning and instructional factors in the scientific inquiry of scientifically gifted korean middle school students. gifted child quarterly, 53(3), 203-216. https://doi.org/10.1177/0016986209334961 ziegler, a., stoeger, h., & vialle, w. (2012). giftedness and gifted education: the need for a paradigm change. gifted child quarterly, 56(4), 194-197. https://doi.org/10.1177/0016986212456070 zohar, a., & nemet, f. (2002). fostering students’ knowledge and argumentation skills through dilemmas in human genetics. journal of research in science teaching, 39(1), 35-62. https://doi.org/10.1002/tea.10008 a © 2020 indonesian society for science educator 41 j.sci.learn.2020.4(1).41-49 received: 14 may 2020 revised: 23 november 2020 published: 28 november 2020 effect on academic achievement and misconceptions of pre-service teachers through combining different teaching methods in a preschool science course nur akcanca1*, lale cerrah özsevgeç2 1early childhood education, çanakkale onsekiz mart üniversitesi, turkey 2primary education, karadeniz teknik üniversitesi, turkey *corresponding author: nurakcanca@comu.edu.tr abstract this study aimed to investigate the effects of different active teaching techniques on pre-school student teachers' concept learning and academic achievement. the study group consisted of 46 third year pre-school student teachers in a public university. different active teaching methods were used during the single term science course, a compulsory course in the preschool programme. the treatment process took ten weeks in total (4 hours per week). the study had both qualitative and quantitative data. the quantitative data were collected using a three-tiered science concepts test, and qualitative data were collected through observation by the researcher. the alpha cronbach values calculated for the test's reliability were 0.642 for success-1 (s1) and 0.52 for misconception-3 (mc3) scores. a dependent t-test was used to compare the pre and post-test scores of the pre-school student teachers. the researcher took observation notes during the in-class teaching exercises. the findings revealed that there were significant differences between the student teachers' pre and post-test scores. they understood science concepts significantly better by the end of the course. it is concluded that combining different teaching methods enhances science concept understanding among student teachers. in addition, misconceptions decreased after instruction. keywords active teaching methods, success, conceptual understanding, science education, pre-school student teachers 1. introduction preschool constitutes the first stage of the education system and uses a different education process from the other education stages. preschool education program is planned by considering the characteristics of children's physical, mental, and emotional development. pre-school education aims to prepare children for elementary education by developing their language, motor, and cognitive skills (ministry of national education [mone], 2013). at this stage, children are expected to develop their fine motor skills, express themselves well, and acquire selfcare skills. furthermore, they are expected to recognize, categorize, and order the scientific phenomena occurring around them at a basic level. in this context, turkish, science, mathematics, and art activities are used in education not as an aim but as a means for children's development in various aspects. in the pre-school period, science is a part of children's games, investigations, inquiries, and experiences (tahta & i̇vrendi, 2010). children at these ages are directly involved in relationships with living creatures and each other. many scientific concepts include the sky, sun, moon, heat, temperature, sound, weight, environment, and environmental problems. accordingly, many scientific concepts are constructed in minds during this life period (demir & şahin, 2015). meaningful construction of scientific concepts in the early stages is essential to facilitate future learning. while teaching new concepts, the teacher uses the students' previous knowledge. there are science activities in the preschool curriculum to teach some science concepts to children. it is crucial to teach these concepts correctly to ease the students' learning and teachers' teaching during elementary education (gemici, 2008). in this context, it is essential that pre-school teachers, who will later be responsible for teaching these concepts, have also learned the science concepts correctly. the related literature revealed that pre-school student teachers lacked competence in teaching scientific concepts (brenneman, 2011). they had problems in knowledge mailto:nurakcanca@comu.edu.tr journal of science learning article doi: 10.17509/jsl.v4i1.24672 42 j.sci.learn.2020.4(1).41-49 levels regarding science concepts (cho, kim, & choi, 2003; kallery, 2004). çamlıbel çakmak (2012) revealed that preservice pre-school teachers had misconceptions about the concepts related to heat and temperature, space, floating and sinking, and living things. moreover, timur (2012) determined that pre-school student teachers possessed many misconceptions about force and motion. ültay & can (2015) concluded that pre-service pre-school teachers had knowledge problems related to heat and temperature. it is known that when pre-service teachers whom themselves possess this incorrect or inadequate information, or indeed these misconceptions, are appointed to their profession, there is a strong possibility that they will transfer these to their students (çamlıbel çakmak, 2012). based on the necessity to solve the problem at its source, it is crucial that candidates' conceptual understanding and knowledge levels are improved at the undergraduate level (özbek, 2009). in the pre-school teaching curriculum, there is a course named "teaching science in early childhood". this course expects candidates to learn science concepts and acquire skills to teach and plan science activities according to their children's ages and developmental levels (alisinanoğlu, özbey, & kahveci, 2011; tahta & i̇vrendi, 2010). furthermore, pre-school teachers are required to comprehend the aim of science education in this course. they wanted to be equipped with teaching methods to make their children active and excited and make their learning meaningful (demir & şahin, 2014). in studies conducted with pre-school teachers, it was determined that teachers experienced difficulties related to planning and implementing science activities and to use different techniques for teaching scientific concepts (bilaloğlu, aslan, & arnas, 2008). the methods such as an educational game, semantic network, drama, projects, brainstorming, problem-solving should be used to teach science (sığırtmaç & özbek, 2011). it is stated that teachers who cannot acquire skills at the desired level may be the root of their undergraduate education (orçan, 2013). the study (ayvacı, devecioğlu, & yiğit, 2002) revealed that most pre-school teachers have limited knowledge about student-based teaching methods, and they prefer using traditional methods such as lecturing, questioning, and answering, demonstration and observation. it has been revealed that the science courses at the undergraduate level need revision in terms of their content as well as the methods and techniques used for the activities and experiments (kallery, 2004; bilaloğlu et al., 2008; çamlıbel çakmak, 2012). it is considered that a lesson procedure in which the active participation of pre-service teachers is enabled and science concepts are taught with applications can be helpful about increasing the quality of learning and reducing problems that may occur in the future (okur akçay, 2014; demir & şahin, 2015). it is also believed that science instruction delivered within such a framework will eliminate existing misconceptions (ültay & can, 2015). in line with all this information, it was decided to combine some techniques which were stated as useful for science teaching education in the literature. so the main problem of this research consists of the question, "to what extent will a science instruction process, conducted by combining different active teaching methods that place the student at the center, be effective for pre-school student teachers' learning of science concepts?" using more than one teaching technique serves different learning preferences of the other learners in the class. thus it strengthens understanding more effectively. it is thought that the findings obtained in this study will make positive contributions to the planning of science courses at the undergraduate level and serve as a guide to faculty members teaching these courses. this study aims to determine pre-school student teachers' learning and evaluate the effect of science education courses conducted with different teaching methods on success and conceptual understandings of preschool student teachers (psts). 2. method 2.1 research model in this study, one type of pre-experimental design, single-group pretest-posttest design, was used. during the study, one group is exposed to a treatment or condition, and then the results of the process are measured. a different control group was not used to evaluate this external intervention's effectiveness better, and a single group was worked. since the aim was to examine in detail the student teachers' conceptual changes and achievement levels before and following the intervention, a single-group pre, and post-test practical research design model was used (frankel & wallen, 2003). 2.2 research group the sample of the study was in a state university located in kars, turkey. for the selection of the participants, a purposive sampling approach was used. use of this sampling technique is preferred in exceptional cases that have specific criteria and characteristics (büyüköztürk, kılıç çakmak, akgün, karadeniz, & demirel, 2014). in this study, 46 (35 females, 11 males) third-year pre-service students were selected for the study because they took a course titled "science education" and were learning science topics in this course. at the beginning of the study, a pre-test was taken from all student teachers, and then an experimental procedure was applied, which was prepared by the researchers. the same test was used as a post-test after the experimental procedure. 2.3 data collection tool to determine the student teachers' conceptual understanding levels and misconceptions related to the science concepts specified, a "three-tiered science journal of science learning article doi: 10.17509/jsl.v4i1.24672 43 j.sci.learn.2020.4(1).41-49 concepts test" (ttsct) was developed. at the first tier of the test, there was a question searching student teachers' knowledge. in the second tier, the reasons for the answer given to the first question. in addition to the four reasons, an 'e' option was given to elicit the psts' misconceptions, and an adequate space was given for the question "what do you think the reason should be?". the third tier was the confidence tier to examine whether the psts were sure or not their previous answers. basic science concepts included in the science education course comprise three main science fields: life sciences, physical sciences, and earth and space sciences (gonzalez et al., 2011; martin, 2012). the questions included in the ttsct were prepared for the subjects of (i) sound, mass, weight, and gravity for physical sciences, (ii) living organisms and their categories, global warming, and the greenhouse effect for life sciences, and (iii) solar system, movements of the earth, sun, and moon for earth and space sciences. while preparing the ttsct, firstly, the concepts for the relevant science subjects were determined, and the misconceptions related to these concepts mentioned in the literature were examined. some of the table 1 educational activities conducted with psts themes/subjects activities p h y si c al s c ie n c e s theme: matter gravitational force mass and weight popping balloons (educational game) who spokes the truth? (concept cartoon) wet towel and popped balloon (experiment) an apple fell from the sky (story writing) what does a dynamometer look like? (synthetic) e ar th a n d s p ac e s c ie n c e s theme: space solar system planets the magic school bus in space (cartoon film) look at the first letters (acrostic activity) examining materials in the solar system spin around me (creative drama) space journey (product design) what are they talking about-1? (cartoon completion) e ar th an d s p ac e s c ie n c e s theme: space movement of the earth and moon high/low tides giant-dwarf following activity (educational game) what happened to the sun? (thought-provoking questions) solar and lunar eclipses (experiment) tidal festival in china (creative drama) l if e s c ie n c e s theme: environment -living organisms -categories of living organisms freeze! (still image) categorization of living organisms (presentation) let me categorize living organisms (semantic network) what if it were like this? (scamper) let’s combine ideas (bread and butter technique) l if e s c ie n c e s theme: environment greenhouse effect global warming asya’s letter (case study-creative problem solving) let’s form a committee (six action shoes) let’s read memos (conceptual change text) let’s find keywords (puzzle) migrating birds (story completion) p h y si c al s c ie n c e s theme: matter sound who am i? (puzzles) ali’s voice (case study-creative problem solving) what kind of tool is a telephone? (ordering qualities) what are they talking about-2? (cartoon completion) can you hear me? (roleplay) p h y si c al , l if e , an d e ar th a n d s p ac e s c ie n c e s theme: matter environment space let’s learn while traveling (station technique) what are they talking about-3? (cartoon completion) let’s write a story (familiar story adaptation) let’s sing (song composition) global warming! (poster preparation) environment friendly (product development) pitchfork, glass flute, and making a duck sound with vibrations (experiment) lifesaving lunar eclipse (story completion) design your own space (product design) grandpa doesn’t sleep (experiment) p h y si c al , l if e , an d e ar th an d s p ac e s c ie n c e s theme: matter-environment space let’s make our own mental map (mental map) journal of science learning article doi: 10.17509/jsl.v4i1.24672 44 j.sci.learn.2020.4(1).41-49 questions were obtained directly from the sources examined, while the lecturers prepared others. furthermore, the changes in the conceptual understanding levels of psts were determined based on the researcher's observation records during the in-class teaching practices. the observation data, which included video recordings, were systematically evaluated, arranged, and reported at each lesson's end. 2.4 teaching process the research's implementation stage was completed in 40 hours (4 hours per week). lessons in which pre and post-test were implemented were excluded from this period. during the treatment process, which lasted for one term, several different techniques were used. before the instruction, other active teaching techniques were planned based on the characteristics of the topics. these techniques are concept map, semantic network, brainstorming, case study, mental map, scamper, role play, creative drama, still image, story writing, synthetic, cartoon completion, six action shoes, station technique, creative problem solving, thought-provoking questions. some activities were planned as individual projects, whereas others were designed for team works to provide peer interactions. members of the groups were determined according to classmates with whom they had less interaction. the first two weeks of instruction were carried out in pre-briefing and introduction to the teaching techniques. the activities used for subjects of the science education course were given in table 1. 2.5 data analysis for scoring the ttsct, the responses that psts gave to each of the three tiers for each question were coded on an excel sheet. the coding was made according to the scoring system created by göksu (2011), and an example has been provided to explain how questions were scored. psts questions in the first three tiers were recorded in three categories: correct, incorrect, or misconceptions. if a pst chose answer c for item 7, c for 7.1, and a for 7.2. all these three answers are correct, and the pst's point was recorded as 1 in the correct row, and a score of 0 was recorded in both the incorrect and misconceptions rows. coding questions calculate the success-1 (s1) score as 1 point if students' teachers' answers to first-tier questions are correct. the success-2 (s2) score is obtained by evaluating the students' responses to the items' first and second tiers. when a student-teacher selects the correct statement at the first tier and chooses the valid reason, his/her s2 score was coded as 1, while incorrect and misconception lines were coded as 0. finally, the success3 (s3) score was calculated by taking all three stages into account, and coding was made as 1 point in case of certainty for the answer. they were assuming that pst chose answer d for question 7, d for 7.1, and 7.2. since a response of d indicates that the respondent has misconceptions regarding this question's content, a score of 1 was recorded in the misconceptions row. a response of d in tier 2 supports the same misconception. a score of 1 was recorded in the misconceptions row. the respondent indicated that he was confident of the response he gave in tier 1 by selecting a response in tier 3. all these scores were taken into consideration when calculating the pst's misconception-1 (mc1), misconception-2 (mc2), and misconception-3 (mc3) points. an independent t-test was performed for the scores obtained by the psts' pre, and post-test. observational data consisted of both in-class observational notes, and video recordings were analyzed together with personal documents. the data obtained from these documents were used to support or refute the data collected during observations or offer an alternative interpretation of the study's findings. table 2 dependent t-test analysis of psts’ pre and post-test success-1, 2, and 3 scores on the ttsct class test n �̅� s sd t p ttsct s1 pre-test 46 6.98 3.84 45 5.96 .000* post-test 46 10.20 2.17 ttsct s2 pre-test 46 6.87 3.84 45 6.08 .000* post-test 46 10.17 2.24 ttsct s3 pre-test 46 5,17 3,47 45 8.05 .000* post-test 46 8,89 2,47 *p<.05 table 3 dependent t-test analysis of psts’ pre and post-test misconception-1, 2, and 3 scores on the ttsct class test n �̅� s sd t p ttsct mc1 pre-test 46 6.98 3.75 45 5.99 .000* post-test 46 3.78 1.87 ttsct mc2 pre-test 46 6.89 3.79 45 5.81 .000* post-test 46 3.74 1.93 ttsct mc3 pre-test 46 5.61 4.18 45 4.94 .000* post-test 46 2.72 1.98 *p<.05 journal of science learning article doi: 10.17509/jsl.v4i1.24672 45 j.sci.learn.2020.4(1).41-49 2.6. validity and reliability of data collection tools content validity of the ttsct was examined by calculating incorrect negative and incorrect positive numbers. a false negative case is selecting the wrong answer for the first tier of the question and choosing the second tier's right reason. an incorrect positive case is selecting correct answers and a false explanation of the reason (hestenes & halloun, 1995). furthermore, choosing the correct answer and the correct explanation of the reason is defined as scientific knowledge; however, giving an incorrect answer and a false explanation is defined as knowledge deficiency (peşman, 2005; göksu, 2011). the incorrect positive rate for the ttsct was calculated as 6.52% (n=3), and the incorrect negative rate was calculated as 8.69% (n=4). in comparison, the incorrect positive and incorrect negative rates for the post-test were calculated as 4.35% (n=2). the fact that the false negative, incorrect positive, and knowledge deficiency percentages of the ttsct were estimated to be below the limit value of 10% indicates that the test is reliable (hesteness & halloun, 1995). the cronbach's alpha values calculated for the reliability of the ttsct were 0.642 for s1 scores and 0.52 for m3 scores. moreover, item-total correlations were examined, questions with negative correlations were removed from the test, and 15 items were obtained. the questions of the test were viewed by two academicians, experts in science education. for the test's structure validity, the correlation between the correct answers of the first two tiers, and the case of table 4 misconceptions of psts based on their pre and post-test misconception-3 scores about the concept of matter misconceptions pre post matter m1. mass is the name given to the weight of matter. 17% 9% m22. weight is the name given to an object’s mass. 17% 9% m3. an object’s mass decreases on the moon in comparison to its mass on the earth. 28% 20% m4. weight is measured using balance scales. 74% 52% s1. we hear thunder after lightning strikes as a result of the sound’s reverberating. 24% 9% s2. we cause the sound to be thinner when we decrease a radio’s sound volume. 4% 0% s3. a portion of the sound is lost when sound volume is decreased. 39% 4% table 5 misconceptions of psts based on their pre and post-test misconception-3 scores about the concept of environment environment g1. global warming causes the ozone layer to decrease in thickness. 33% 24% g2. global warming means that the earth’s temperature is the same in every part of the world. 0% 2% g3. reducing co2 amounts will not impact global warming. 26% 4% l1. whales are members of the fish family. 24% 13% l2. seals are members of the fish family. 17% 4% l3. plants feed on their roots. 67% 57% l4. plants obtain their substance by breathing. 7% 4% l5. penguins are mammals. 22% 9% l6. penguins are members of the frog family. 2% 2% l7. penguins are members of the fish family. 13% 4% table 6 misconceptions of psts based on their pre and post-test misconception-3 scores about the concept of space space sp1. the moon is the largest object in space. 24% 2% sp2. the earth is larger than the sun and the moon. 13% 2% sp3. the milky way galaxy is bound to the sun. 24% 2% sp4. jupiter is a large star. 0% 2% sp5. there is air in space. 41% 24% sp6. the moon’s size changes depending on its phase. 13% 0% sp7. the moon does not revolve around the sun. 13% 11% sp8. the moon does not revolve around itself. 24% 15% sp9. earth is the largest planet. 17% 0% sp10. the sun’s gravitation pull is stronger during high tide. 30% 9% journal of science learning article doi: 10.17509/jsl.v4i1.24672 46 j.sci.learn.2020.4(1).41-49 certainty for the third tier (reliability level) was examined. the correlation coefficient was calculated as 0.222 for s2, at a 0.01 significance level. when the psts' scores increased at the second tier, an increase in reliability levels was also observed. it means that the student teachers gave correct answers, and they were also sure of their responses. a low relationship between the misconceptions and reliability level means that student teachers had misconceptions, and they were not entirely sure of their answers. furthermore, the observation notes, which were another measurement tool, were supported with the statements obtained in the data collected from the written documents. while these statements were being made, codings in pst1, pst13, and pst16 were used. 3. result table 2 represents the dependent t-test for the differences between psts' pre and post-test success-1, 2, and 3 scores. in table 2, significant differences were found between pre-and post-test scores at all three stages of the three-tier test (success-1, 2, and 3). it was revealed that all scores related to success were increased after treatment. it was also determined that the rate of correct answers given to the question in the first stage of the three-tier test was higher than the appropriate steps regarding the explanation of the answer and whether or not to be sure about the answer. although tiers scores increased, average pre-and post-test success scores decreased. it is thought that the practices carried out throughout the research process positively affect the success of pre-service teachers in related science subjects. in table 3, when the pre-service teachers' misconception scores obtained from all three stages of the test are examined (misconceptions-1, 2, and 3 scores), significant differences were found between pre and posttest scores. average misconception scores decreased for all three-tier of the test after the experimental process. it is thought that the practices carried out throughout the research process are useful in overcoming the pre-service teachers' misconceptions in related science subjects. in the following tables, misconceptions were presented coding as mass (m), sound (s), living organisms (l), global warming (g), space (sp). although some misconceptions (m3, m4, g1, l3, sp5, sp8) were decreased in the posttest, some misconceptions were resistant to change and not eliminated. this finding obtained from the research will be explained with the observation data in the next title. as seen in table 4, although the ratios of the misconceptions of pst decreased in the post-test, the misunderstanding about the matter was still higher. the effectiveness of the synectic technique is also tricky for application in the class because of this result. pre: psts' misconceptions on the pre-test post: psts' misconceptions on the post-test. as shown in table 5, the percentages of the misconceptions were low in the posttest, but the statement's ratio as "plants obtain their substance from the soil via their roots" was still high. this belief is quite common among the students from primary school to university, so it may be resistant to change. their thoughts may back to old after the instruction. as presented in table 6, the ratios of the misconceptions about space were low in the post-test. 3.1. observational results about the teaching process psts' misconceptions were observed during class activities. during concept cartoon activities, psts were observed to confuse mass and weight concepts and had problems with measurement units. they believed that an object's mass on earth and the moon was the same but had trouble describing weight. some psts stated that an object's weight in space either changed as it did on the moon or decreased. a few psts said, "weight was zero in space because there was no gravity in space." pst6 stated that "i do not think that matter's mass or weight changes. however, there may be gravity, but they still do not change." after the cartoon activity concept, pst6 changed his idea as "weight was matter's unchanging identity that does not change on the earth or the moon". pst44 stated that "weight changes depending on gravity. it can be different in various places of the earth, and the source of gravity is the core. weight is less at the equator and more at the poles. an object's mass does not change; it never changes anywhere" after the activity. psts wrote stories regarding mass and weight from the perspective of a tree or an apple. it is a fact that understanding mass, sound, figure 2 an example from cartoon completion activity (özdemir, 2006; pp. 19) figure 1 example question from the ttsct journal of science learning article doi: 10.17509/jsl.v4i1.24672 47 j.sci.learn.2020.4(1).41-49 global warming, and living organisms poses challenges for students at different educational levels. because of reasons such as complexity of the concept, teaching method, and textbook, students memorize the science concepts without deeper understanding at the primary level. this limited understanding of students causes negative feelings to science learning, and students carry these learning problems to the next school level. psts were also observed to confuse reverberation and speed of sound. one group designed creative drama for teaching the statement that “thunder is heard after lighting is seen”. figure 2 shows an example from the cartoon completion activities of groups. it revealed their beliefs that sound could be transmitted in space and that the characters needed to speak louder. for eliciting misconceptions of psts about living organisms, semantic network activity was planned. they were given pictures of living organisms to prepare their semantic networks. before the activity, they were observed that they had learning problems of classification of living organisms. figure 3 shows examples of semantic networks prepared by two psts before the application. the figure divided living organisms into plants and animals or organisms that live in water, organisms that live on land, and reptiles. figure 3 reveals that although they still exhibited incorrect understandings, psts could categorize living organisms much more accurately than previously. psts also mentioned microscopic organisms and mushrooms and gave correct examples of flowering and non-flowering plants and vertebrates and invertebrates. psts were observed to have problems conceptualizing the solar system, the essential characteristics of planets, the sun, earth, and moon's size and the relationship between space and the atmosphere under the theme space. psts were shown a cartoon film about planets. following the film, faulty parts of the film were asked and discussed. psts were also asked to write an acrostic poem. one pst's acrostic poem for venus, for example, highlighted that it was known as the earth's twin and mentioned that it was the hottest of all planets. they also completed the cartoons. figure 4 represents an example of the cartoons that psts completed. the study's main conclusions may be presented in a short conclusions section, which may stand alone or form a subsection of a discussion or results and discussion section. figure 4 reveals that pst39 was aware that mercury was the closest planet to the sun and that both day and night on this planet were scorching. in other cartoons, psts were found to have learned that venus had dense clouds of sulfuric acid, that jupiter could be seen with the naked eye, and that the surface of mars was red because of iron oxide. psts also learned the correct order of planets by size. the observation records showed that psts were made aware of their misconceptions and adopted correct thinking frames by using various active teaching techniques. after the treatment process, it was observed that psts had fewer misunderstandings and to be able to adapt newly acquired scientific information to new situations they encountered. 4. discussion the aim of using different active teaching methods is to affect academic achievement and conceptual understanding of pre-school. post-tests' scores revealed that psts' success scores increased for all three tiers after the treatment process. various teaching techniques and approaches (ayverdi, asker, öz aydın, & sarıtaş, 2012; buntod, suksringam, & singseevo, 2010) were found to have a positive impact on students' academic performance. these findings from other studies are consistent with the findings of this study. the psts' post-test average misconception scores also showed that their misconception scores decreased for all three tiers after the experimental process. the test results also revealed that psts were continuing to have some misconceptions such as mass and weight, but their ratios were lower than the beginnings. this result supports the idea that some misconceptions are persistent to change. the result of this research agrees with koray, özdemir, and tatar (2005). the concept cartoons that psts completed in this study were thought to reduce their mass and weight misconceptions. in contrast, educational games, story writing, and synthetic techniques were not sufficiently significant to overcome psts' misconceptions. yağbasan and gülçiçek (2003) stated that misunderstandings are not structured differently from students' other knowledge, and they are included in newly acquired information. therefore, they mentioned the difficulty of eliminating misconceptions. figure 3 examples of psts’ first semantic networks figure 4 cartoon completion by pst39 journal of science learning article doi: 10.17509/jsl.v4i1.24672 48 j.sci.learn.2020.4(1).41-49 considering that pre-service teachers develop their misconceptions due to their own experiences and observations over a long time, it is an expected result that some misconceptions will not be eliminated. in this study, the related gains' time may not have been sufficient to eliminate the pre-service teachers' misconceptions. psts had misconceptions regarding sound before the implementation. many other researchers also determined similar misconceptions of the students from different educational grade levels (hrepic, 2002; küçüközer, 2009). the post-related gains' times of the psts decreased. using concept cartoons, creative problem solving, and role play may help overcome the psts' misconceptions. in the literature, studies are supporting this idea (atasoy, tekbıyık, & gülay, 2013; çalık, okur, & taylor, 2011). psts also had learning problems about the classification of living organisms. türkmen, çardak, & dikmenli (2005) found that students had incorrect knowledge about the diversity and classification of living organisms and semantic networks were highly effective in eliminating these misconceptions. kaya (2010) concluded that conceptual change texts effectively reduced university students' misconceptions about photosynthesis and respiration. techniques like scampers and semantic networks were effective in diminishing misconceptions of psts in this study. psts also had incorrect knowledge about global warming and the greenhouse effect like other students at different grade levels (bal, 2004; ünlü, sever, & akpınar, 2011). it was observed that they had even corrected several of these misconceptions after usage of techniques as cooperative learning, conceptual change texts, creative problem solving, and six action shoes. data collected on the theme of space revealed that psts had several misconceptions about the solar system, planets, sun, earth, and moon, and the relationship between space and the atmosphere. specifically, different grade levels of students were found to have gaps in their knowledge of astronomical concepts (caballero, moreira, & rodriguez, 2008; taşcan & ünal, 2016). the present study found that the psts' insufficient knowledge was partially filled using animation and cartoon films and drama during the treatment process. demirel and aslan (2014) found that concept cartoons effectively eliminated students' misconceptions about the solar system held by students. türk, alemdar, and kalkan (2012) found that students' participation in hands-on learning activities was beneficial for meaningful learning of astronomy subjects. conduction of relation between science content and daily life events was also useful for students' meaningful learning. on the other hand, it was observed that experiment-based activities and product design techniques were not significant enough to overcome the misconceptions of the psts. 5. conclusion in summary, this study's activities facilitated psts' learning by giving them the responsibility for their education. they also enjoyed the learning process, and this increased their learning motivation. thus, meaningful learning may have been achieved. the findings of the research of bulunuz (2012) also support these studies' results. group working may help get away psts' boringness in the classroom and reduce their learning stress. representing active teaching techniques and a democratic instructional atmosphere, cooperative learning may help facilitate instruction and elicit misconceptions. shortly, psts constructed their meaning by discussing, watching, listening, writing, reading, and making. based on all conclusions, it may be suggested that: preschool student teachers had problems learning science concepts, and they had misconceptions about science concepts. the lecturer should be aware of these misconceptions. s/he should be planned lessons by using different active teaching methods. both group and individual activities should be used. a democratic learning environment should be created. it is remembered that misconceptions may be persistent to change. when designing similar studies in the future, it is recommended to determine these persistent misconceptions and develop a treatment process based on these conceptions with different activities. acknowledgment the authors would like to acknowledge and thank the student teachers for their participation in the research. references alisinanoğlu, f., özbey, s., & kahveci, g. (2011). erken çocukluk döneminde fen eğitimi [science education in early childhood]. maya academy. atasoy, ş., tekbıyık, a., & gülay, a. (2013). the effect of concept cartoons on understanding of sound concept of fifth grade students. journal of turkish science education, 10(1), 176-196. ayvacı, h. ş., devecioğlu, y., & yiğit, n. (2002). okulöncesi öğretmenlerinin fen ve doğa etkinliklerindeki yeterliliklerinin belirlenmesi [determining efficacy of preschool teachers in science activities]. paper presented at the national science and mathematics congress, ankara. ayverdi, k., asker, e., öz aydın, s., & sarıtaş, t. (2012). determination of the relationship between elementary students’ scientific creativity and academic achievement in science and technology courses. elementary education online, 11(3), 646-659. bal, ş. (2004). determining misconceptions about science teacher candidates about greenhouse effect. eurasian journal of educational research, 17, 102-111. bilaloğlu, r. g., aslan, d., & arnas, y. a. (2008). the analyzing of preschool teachers' levels of knowledge about science activities. national educational journal, 178, 88-104. brenneman, k. (2011). assessment for pre-school science learning and learning environments. early childhood research & practice, 13(1), 19. bulunuz, m. (2012). developing turkish pre-service pre-school teachers' attitudes and understanding about teaching science through play. international journal of environmental & science education, 7(2), 141-166. journal of science learning article doi: 10.17509/jsl.v4i1.24672 49 j.sci.learn.2020.4(1).41-49 buntod, c. p., suksringam, p., & singseevo, a. (2010). effects of learning environmental education on science process skills and critical thinking of mathayomsuksa 3 students with different learning achievements. journal of social sciences, 6(1), 60-63. http://doi:10.3844/jssp.2010.60.63 büyüköztürk, ş., kılıç çakmak, e., akgün, ö. e., karadeniz, ş., & demirel, f. (2014). bilimsel araştırma yöntemi (16. baskı) [scientific research method (16th edition)]. ankara: pegem academy. caballero, c., moreira, m. a., & rodriguez, b. l. (2008). concept mapping as a strategy to explore teachers’ mental representations about the universe, concept mapping: connecting educators. in proceedings of 3th international conference on concept mapping, finland. tallinn, estonia and helsinki. cho, h. s., kim, j., & choi, d. h (2003). early childhood teachers’ attitudes toward science teaching: a scale validation study. educational research quarterly, 27(2), 33-42. çalık, m, okur, m., & taylor, n. (2011). a comparison of different conceptual change pedagogies employed within the topic of “sound propagation”. journal of science education and technology, 20, 729-742. http://doi:10.1007/s10956-010-9266-z çamlıbel çakmak, ö. (2012). examining the relationship between preschool teacher candidates' attitudes towards science education and understanding levels of science concepts. journal of turkish science education, 9(3), 40-51. demir, s., & şahin, f. (2015). pre-school teacher candidates’ beliefs about to make experiments by using 5e method. the journal of academic social science studies, 35, 385-397. http://dx.doi.org/10.9761/jasss2808 demir, s., & şahin, f. (2014). assessment of open-ended questions directed to prospective science teachers in terms of scientific creativity. procedia-social and behavioral sciences, 152, 692-697. https://doi.org/10.1016/j.sbspro.2014.09.264 demirel, r., & aslan, o. (2014). the effect of science and technology teaching promoted with concept cartoons on students' academic achievement and conceptual understanding. journal of theory and practice in education, 10(2), 368-392. frankel, j. r., & wallen n. e. (2003). how to design and evaluate research in education. mcgraw hill. gemici, ö. (2008). fen ve teknoloji eğitiminde kavram öğretimi [concept teaching in science and technology education]. in ö. taşkın (ed.). fen ve teknoloji öğretiminde yeni yaklaşımlar [new approaches in science and technology teaching] (pp. 125-148). pegem academy. gonzalez, j. e., pollard-durodola, s., simmons, d. c., taylor, a. b., davis, m. j., kim, m., & simmons, l. (2010). developing lowincome preschoolers’ social studies and science vocabulary knowledge through content-focused shared book reading. journal of research on educational effectiveness, 4(1), 25-52. göksu, v. (2011). sorgulamaya dayalı ve doğrulama laboratuvarı öğretiminin fen bilgisi öğretmen adaylarının başarıları, kuvvet ve hareket hakkındaki kavram yanılgıları ve epistemolojik inançları üzerindeki etkisi [the effects of inquiry based and verification laboratory instruction on preservice science teachers’ achievement, misconceptions about force and motion and their epistemological beliefs] (unpublished doctoral thesis). gazi university, ankara. hestenes, d., & halloun, i. (1995). interpreting the force concept inventory. the physics teacher, 33, 502-506. https://doi.org/10.1119/1.2344278 hrepic, z. (2002). identifying students’ mental models of sound propagation (unpublished master’s thesis). kansas state university, manhattan. kallery, m. (2004). early years teachers' late concerns and perceived needs in science: an exploratory study. european journal of teacher education, 27(2), 147-165. https://doi.org/10.1080/026197604200023024 kaya, f. (2010). fen bilgisi öğretmen adaylarının bitkilerde fotosentez ve solunum konusundaki yanılgılarının üstesinden gelmek için bilgisayar destekli kavramsal değişim metin talimatlarının etkisi [the effects of computer assisted conceptual change texts instructions on overcoming preservice science teachers’ misconceptions of photosynthesis and respiration in plants] (unpublished master’s thesis). pamukkale university, denizli. koray, ö., özdemir, m., & tatar, n. (2014). primary school student's misconceptions about "units": example of mass and weight. primary education online, 4(2), 24-31. küçüközer, a. (2009). investigating prospective science teachers' misconceptions of sound. primary education online, 8(2), 313-321. martin, d. j. (2012). elementary science methods: a constructivist approach (6th edition). wadsworth. ministry of national education [mone] (2013). pre-school educational programme. general directorate of basic education. okur akçay, n. (2014). determining the views and adequacy of the preschool teachers related to science activities. the journal of academic social science studies, 30(1), 325-336. https://doi.org/10.13189/ujer.2016.040419 orçan, a. (2013). çizgi roman tekniği ile geliştirilen bilim kurgu hikayelerinin öğrencilerin yaratıcı düşünme becerileri ve fizik tutumlarına etkisi [the effect of science-fiction stories developed by comics technique, on creative thinking skills and physics attitudes of students] (unpublished master’s thesis). gazi university, ankara. özbek, s. (2009). the analyze of pre-school teachers' sights of science education and its application (unpublished master’s thesis). çukurova university, adana. peşman, h. (2005). development of a three-tier to assess ninth grade students' misconceptions about simple electric circuits (unpublished master’s thesis). middle east technical university, ankara. sığırtmaç, a., & özbek, s. (2011). the analyze of pre-school teachers' opinions of science education and its implementation. e-journal of new world sciences academy education sciences, 6(1), 1039-1056. tahta, f., & i̇vrendi, a. (2010). okul öncesi eğitimde fen öğrenimi ve öğretimi (2. baskı) [science learning and teaching in preschool education (2nd edition)]. kök publishing. taşcan, m., & ünal, i̇. (2016). an investigation of science teachers’ knowledge levels of basic astronomy contents in terms of demographic variable. necatibey faculty of education, electronic journal of science and mathematics education, 10(1), 60-84. https://doi.org/10.17522/nefefmed.10897 timur, s. (2012). examining cognitive structures of prospective preschool teachers concerning the subject "force and motion". educational sciences: theory and practice, 12(4), 3039-3049. türk, c., alemdar, m., & kalkan, h. (2012). primary school students' seasons determination of understanding levels. journal of educational and instructional studies in the world, 2(1), 62-67. türkmen, l., çardak, o., & dikmenli, m. (2005). using concept maps changing the misconceptions of the first-year high school students in biology courses in classification of living things and their diversity. gazi university journal of gazi educational faculty, 25(1), 155-168. ünlü, i̇., sever, r., & akpınar, e. (2011). analysis of the academic studies’ results in environmental education in turkey about global warming and greenhouse effect. erzincan university journal of education faculty, 13(1), 39-54. ültay, e., & can, m. (2015). determination of pre-school student teachers' conceptual knowledge about heat and temperature. karadeniz journal of social sciences, 7(2), 1-23. yağbasan r., & gülçiçek ç. (2003). describing the characteristics of misconceptions in science teaching. pamukkale university journal of education, 13, 110-128. http://dx.doi.org/10.9761/jasss2808 a © 2020 indonesian society for science educator 156 j.sci.learn.2020.3(3).156-164 received: 17 march 2020 revised: 20 june 2020 published: 28 july 2020 stem-focused activities to support student learning in primary school science neslihan ültay1*, arzu zıvalı1, hilal yılmaz1, hilal k. bak1, kezban yılmaz1, melek topatan1, pelin gül kara1 1department of elementary education, faculty of education, giresun university, giresun, turkey *corresponding author neslihan.ultay@giresun.edu.tr abstract this study examines the effect on student academic success through teaching the "let's know the matter" grade 3 science unit accompanied by stem (science, technology, engineering, and mathematics)-focused activities. the study group consisted of 24 third grade students studying in the 20192020 academic year. teaching was done with a 5e (enter, explore, explain, elaborate, evaluate) instructional model, and student opinions about the stem activities were collected. a group pretest-posttest research design was used in which the "let's know the matter test (mt)" was administered at the beginning to measure the students' prior knowledge, and again at the end of the unit to determine what students had learned. their opinions of the lessons taught with stem-focused activities were collected using a semistructured interview technique. qualitative data were divided into appropriate categories under common themes. it was observed that the 5e instructional model contributed to their academic success. also, it was found that their opinions about the lessons taught with stem activities were quite positive. based on the results obtained from the research, it is suggested that stem activities be used in the teaching of other topics. keywords matter, primary school, stem, third graders 1. introduction in the 21st century, the field of technology and engineering is developing and changing rapidly. as a result of the reflection of these changes to education, technology, and engineering fields have become an integral part of education. this situation increases the need for renewal studies in education. one of the new approaches of the 21st century is stem education. stem acronym is a teaching and learning approach consisting of the initials of the words science, technology, engineering, mathematics. stem helps to teach students science, mathematics, engineering, and technology together, and it enables students to transfer what has been learned to live. the concept of stem was first expressed by the national science foundation education director judith ramalev in 2001 (sanders, 2009; yıldırım & altun, 2015). stem education can be described as the change of integrated education, which does not require the division of science and mathematics courses, including more than one discipline, into different branches (riechert & post, 2010). stem education aims to teach science, mathematics, engineering, and technology, which includes lifelong education activities suitable for all grade levels starting from pre-school to post-doctorate. however, it seems that the renewed curriculum in turkey mentions stem education under the name of science and engineering applications. it takes place from 4th grade to 8th grades (hiğde, 2018), whereas stem education can be applicable for all grades. also, in the literature, not only in turkey, a few studies are focusing on third graders. because this study focuses on third graders, it makes this study significant. in order for countries to keep up with technological and scientific developments in the world and to be among the mailto:neslihan.ultay@giresun.edu.tr journal of science learning article doi: 10.17509/jsl.v3i3.23705 157 j.sci.learn.2020.3(3).156-164 developed countries, it needs individuals with an entrepreneurial spirit who are interested in stem fields and can think innovatively. meeting this need is possible with an education system that enables students to take responsibility, allow them to make mistakes, enable them to produce everyday products, enable them to think and produce. it is evident that the developing countries need stem education, and their curricula should be rearranged to include entrepreneurship, art design, and programming (akgündüz et al., 2015, ültay & ültay, 2020). stem education increases competitiveness between countries and supports countries' social and economic development (çakmakçı, 2016). another aim of stem education is to raise individuals who will contribute to the industrialization of countries and to make countries economically and industrially developed countries (çevik & özgünay, 2018). although stem is understood in different ways in the world, the common characteristics of all are bringing together different disciplines and applying them to the problems we may encounter in daily life as a result of using the integration of concepts and skills together. stem education includes the knowledge, skills, and opinions that arise from the point where two or more subject areas are integrated (çorlu, capraro, & capraro, 2014). in stem education, it is aimed that individuals can solve problems, think in many ways, be confident, transfer technology to learning, and develop creative and communication skills (bybee, 2010; morrison, 2006). morrison (2006) listed the skills that individuals should acquire with stem education as follows: innovative, problem solving, inventing, selfconfidence and self-motivation, consistent thinker, and technology literate. stem education should be able to improve students' characteristics, such as understanding how the equipment works and using technology well (bybee, 2010). in stem education, there is a presentation of a problem that students may encounter from daily life within a particular context. students should also design and produce a solution proposal for this problem (ültay & ültay, 2020). these designs may be technological or engineering-based (felix, 2016). stem education is planned to gain skills such as inquiry, research, problemsolving, communication, and teamwork to individuals. stem education should include activities that attract attention and make the individual eager to learn (baran, canbazoğlu bilici, & mesutoğlu, 2015). when looking at some studies on stem, it is seen that yamak, bulut, & dündar (2014) examined the effects of stem-based activities on 5th-grade students' attitudes towards scientific process skills and science education. the study concluded that stem activities positively improved students' attitudes towards scientific process skills and science education. in the study of cotabish, dailey, robinson, & hughes (2013), students and teachers from the 2nd to 5th grade participated in the study, and it was found that the control group was more successful compared to the control group in terms of the science process skills, science concepts, and science content knowledge. also, they found that the students who participated in stem education were also more successful. park & yoo (2013), found that teaching the subject of light with 6th-grade students with a steam (stem+art) approach provided a definite increase in students' interests and motivations, but could not detect a significant difference in their scientific process skills. kong & huo (2014) examined steam education's effect on 4th-grade students' attitudes towards science, self-efficacy, and science self-efficacy. however, there was no significant difference between the groups; the average of the experimental group students was higher than the control group. karahan, canbazoğlu bilici, & ünal (2015) aimed to determine 8th-grade students' attitudes towards science lesson and their views on stem approach with their stem activities. as a result of the research, it was stated that the students understood the science lesson better, the lesson became more enjoyable and contributed to their teamwork skills. yıldırım & altun (2015) examined the effect of stem education and engineering practice on academic success in the science and technology course. in the quasiexperimental model conducted with the pre-service science teachers who constitute the study group of the research, a significant difference was found in favor of the experimental group where stem education and engineering education were applied, and it was concluded that it affects the academic success of the students positively. gülhan & şahin (2016) examined the effect of stem integration on 5th-grade students' perceptions and attitudes in stem fields. in the study in which quasiexperimental design was used, perception and aptitude tests revealed that stem activities improved students' perception and attitudes towards stem. baran, canbazoğlu bilici, mesutoğlu, & ocak (2016) implemented an out-of-school stem education program for 6th-grade students. according to the results of the research, it was suggested that integrating stem activities into the out-ofschool education program could support the development of students' interest in following their stem-related careers. knop et al. (2017) stated that within the scope of a five-day study conducted with 31 secondary school students, designing interactive robots in stem education on students' attitudes was examined, and their opinions were received. as a result of the study, they observed that the students' interest and attitude towards stem increased and emphasized that they attracted the students' attention. in the study, in which azgın & şenler (2019) took 3rdgrade and 4th-grade students as a sample, it was aimed to examine students' stem career interests and attitudes towards stem in terms of some variables. as a result of the study, it was determined that male students' stem career interests were higher than female students, and journal of science learning article doi: 10.17509/jsl.v3i3.23705 158 j.sci.learn.2020.3(3).156-164 stem attitude scores did not differ significantly between male and female students. as can be understood from meta-analysis and metasynthesis studies for stem education, most of the studies focus on teacher education and secondary and postsecondary levels (becker & park, 2011; herdem & ünal, 2018). the studies carried out at the primary school level are very few and generally at the 4th-grade level. in this sense, the study can make useful contributions to the related field. as mentioned above, stem education focuses on 21stcentury skills such as critical thinking, creativity, innovation, etc. according to choe (2006), scientific creativity has recently become an essential educational target and a social issue for nations' welfare. from this point of view, determining the creative thinking of students is essential in terms of education. morrison (2006) describes the students who take stem education as 'inventors' because they become aware of the needs of the world and find and implement creative solutions. in addition, not only creativity but also all 21st-century skills are considered very important for students because they are also signals of people who will improve the countries for further. from this point of view, the study aimed to examine the effect of teaching "let's know the matter" unit with a 5e instructional model accompanied by stemfocused activities on grade 3 students' academic success and their opinions about stem. 2. method as this study aimed to determine the effect of teaching the "let's know the matter" unit based on the 5e instructional model accompanied with stem-focused activities, it does not have only the quantitative dimension. it also has a qualitative study since it aims to determine students' thoughts about stem teaching. in this study, one group pretest-posttest research design was used. onegroup pretest-posttest research design can be used to determine the effect of an implementation (allen, 2017). in this study, the effect of the 5e instructional model accompanied by stem-focused activities was researched, and also, students' opinions about stem was tried to be revealed. 2.1 study group the study group of the research consisted of 24 students studying at the 3rd grade level of a primary school affiliated to the ministry of education in the central district of a province in the eastern black sea region of turkey in the academic year 2019-2020. the students' age was between 8 and 9. 12 of the students were female, and 12 of them were male students. students were coded as s1, s2,…., s24. 2.2 data collection tools the "let's know the matter" unit, which is the 4th unit of the 3rd-grade science course, was taught to students in a total of 17 lesson hours, based on 5e instructional model accompanied with stem activities, following the program. before the application started, "let's know the matter test (mt)" was applied as a pre-test to measure the students' pre-knowledge. multiple choice test was preferred as a measurement tool to get more objective results. also, students' ages range between 8-9, and because of their ages, they were very active in the classroom, and as observed in the activities, they were bored quickly. the researchers decided the way to get the most reliable results without boring them was to perform multiple-choice testing. mt contains 20 multiple choice test items. discrimination and difficulty indices for each item are given in appendix 1. the researchers prepared mt under unit acquisitions. the pre-application for validity and reliability analyzes was carried out with a group of 3rd-grade students (30 students) different from the application group. in the pilot application, the reliability coefficient of the test (cronbach alpha) was 0.737, and in the application of the study group, it was 0.741. for content validity, opinions were received from six primary teaching educators and a primary school teacher, and the opinion that the scope validity was obtained as appropriate. also, opinions on the suitability of the structure and appearance validity were taken from two science education experts. after the application, mt was reapplied as a post-test. two sample questions from mt are presented below: 2. which of the following items has a smooth surface? a) mirror b) carpet c) sofa 4. in which of the following are all substances liquid? a) cologne, water, olive oil b) air, fruit juice, soda c) syrup, milk, fruit after the application ended, the researchers asked students who were volunteers to interview, and five students were volunteers. then, open-ended questions were asked to these five volunteer students to reveal students' opinions on stem activities. the questions were constructed by the researchers and were collected from the students through a semi-structured interview technique. the interviews lasted for approximately 10-15 minutes and were recorded with the voice recorder by obtaining the consent of the students. 2.3 data analysis the data obtained from the mt, which was applied as a pre-test and post-test, were scored "1 (one)" for correct answers, and false/empty answers were scored "0 (zero)". the first condition for using parametric testing in data analysis is to implement the application with at least 30 participants. since the number of students participating in the application was below 30, it was deemed appropriate to perform analysis with non-parametric tests. the total journal of science learning article doi: 10.17509/jsl.v3i3.23705 159 j.sci.learn.2020.3(3).156-164 scores obtained from mt were analyzed with the mann whitney u test, one of the non-parametric tests with the help of the statistical package for the social sciences, and is shown in the findings section with tables. in addition, the data obtained from semi-structured interviews applied to reveal students' opinions on stem activities were transcribed first, and appropriate categories were identified under common themes after necessary reductions were made. 2.4 validity and reliability researchers created the mt used in the application, a pilot study was applied to a group of similar features for reliability analysis, and the reliability coefficient (cronbach alpha) was calculated to be 0.737. apart from this, the scope validity was obtained by the opinions of six classroom education experts and a classroom teacher regarding whether the mt test questions and interview questions asked following the acquisitions cover the subject area. structure and appearance validity of data collection tools were tried to be obtained by taking the opinions of two science education experts. in the classroom where the researchers are conducting the research, many stem applications have been made in the 'science' course before this application by different researchers. from this point of view, the applied group is accustomed to being taught with stem-focused activities. in this application, the teacher of the class participated as an audience. thus, an environment was tried to be created for students to feel safer and more comfortable. the implementation was carried out by the researchers who are also the authors of this manuscript. because the teacher had not information and experience about stem and its implementation, the researchers who were studying for stem in a primary school in the master of education programs, the researchers carried out the implementation under the supervisory of the first author. 2.5 implementation the research aims to teach "let's know the matter" unit with stem-focused activities, and lesson plans are prepared according to the 5e model. the first step of 5e is to enter/engage in which students' attention is directed towards the related topic, and their pre-existing knowledge is stimulated. the second step of 5e is exploring where the students make an observation, use their pre-knowledge, conduct experiments, and discover new knowledge. the third step of 5e is to explain in which students ask the teacher to disconfirm/confirm the students' gained knowledge. the fourth step of 5e is elaborate in which students elaborate their acquired knowledge within interdisciplinary or interrelated concepts. the final step of 5e is evaluating in which students evaluate what they learned. the first lesson plan prepared with the 5e model is in appendix-2. at the beginning of the implementation, students were divided into six groups of 4 and 5 people. later, the students were pre-tested, and their prior knowledge was determined. the activities in the lesson plan were administered to the students, respectively. first of all, various activities were held in the course to explain the "characteristics of the matter." in the first activity of this subject, which lasted for 11 lessons, one student was selected from the groups. the eyes of the selected student were closed with a cloth. the student was asked to take one of the objects in the previously prepared boxes, and the student described the properties of this object and tried to guess the object. this activity continued until there were no objects in the box. after this activity, another activity (let us learn) was made in the exploration phase, and various materials were given to the student groups. after the students examined the distributed materials, they wrote the properties of the objects. then, researchers asked various questions about objects (for example, what do you feel when you touch it? does this matter shine a light?). videos were watched from various websites on the subject (e.g., eba, okulistik, etc.). in the third activity, "what are the characteristics of that matter?" game was played with the students. visuals about the activity are given in appendix3. in the explanation step, the fourth activity was made with "dramatization" worksheets prepared in accordance with the acquisition of "touching, looking at, tasting, tasting, and smelling some substances may harm the living body." worksheets are distributed to groups. the students read these dramas with their groups, answered the questions written under the drama, and expressed their solutions through the group representative. an example of worksheets is given in appendix-4. in the fifth activity, volunteer students played the dramas written. from the characteristics of the matter, the student chose a feature. the feature that the student chose was written on his collar, and the badges were hung. phrases such as 'hard matter, introduce yourself' were established. the student put this feature in his place, introduced the feature of the chosen item with his sentences. in the elaboration stage, in the sixth activity, it was tried to explain whether the items were rough or smooth. in this activity, students told their properties by touching the objects around. subsequently, the items given were asked to smooth the rough ones, and smooth ones to be roughened. for this, sanding paper and rough wood pieces, aluminum foil, etc. were given to the groups. visuals about the activity are given in appendix-5. the stem-focused activity was started in the seventh activity in the elaboration stage. the text and studies for stem activity are given in appendix-6. in the evaluation phase, the last activity of the subject was made. in the activity, students were asked to make a poster study on the properties of matter. poster works are shown in appendix7. after the subject of ' characteristics of the matter ', the subject of 'states of the matter', which will take 6 lesson hours, was started. for this subject area, a lesson plan with journal of science learning article doi: 10.17509/jsl.v3i3.23705 160 j.sci.learn.2020.3(3).156-164 the 5e model was prepared. the prepared lesson plan is given in appendix-8. in this subject area, "do you know my features?" activity was made. in this activity, different items were placed on the table. the students were asked to examine them and talked about their features. in the second activity, different materials were put into three gloves. a glove was filled with water, a glove was filled with stones, and the other was inflated with air. the students were asked about the differences between the gloves and asked to explain their reasons. a section from the cartoon was watched, and students were asked about the items they saw in the cartoon. their attention was drawn by asking riddles. the riddles asked are in appendix-9. in the third activity, the starch-water experiment was done to understand the concepts of solid and liquid. the experiment carried out is in appendix-10. in the fourth activity, the wind was blown to understand the concept of gas, and then they were asked to cool their cartons by shaking them to their friends, and they were made fan. it was provided to the students the balloons to be inflated and had made them feel the gas exit from the balloon by touching their cheeks. in the fifth activity, students did breathing exercises. in the explanation part of the course, students were given information about the states of matter. after the explanation, the students read the story text to be used in the stem activity, and questions were asked. the story text is given in appendix-11. students were asked to design and create a new product according to the given text. in this process, students were helped by asking questions at a level where they could have improved their products. the resulting products are shown in figure 1. at the evaluation stage of the lesson, the game prepared for the students was played. the game is given in appendix-12. the game was as follows: the students were ranked among themselves, and the game started. the student rolled the dice and advanced the box by the number of dice that he threw his pawn. the student said in what form the word, in the box it came from. if he said it wrong, it passed to the other student. if he answered the question correctly, he brought points to his group. in the last stage of the course, after the implementation of the 5e model, open-ended questions were asked to 5 volunteer students for qualitative evaluation. the post-test was administered, and the implementation was finished. the implementation lasted for 17 lesson hours in 6 weeks. 3. result and discussion this study aimed to determine the effect of teaching third-grade students based on the 5e instructional model accompanied with stem-focused activities on the "let's know the matter" unit and to determine the thoughts of the students about stem teaching data was collected for this purpose. mt was applied as a pre and post-test to measure the success of students. the findings obtained as a result of analyzing the data obtained from mt with the mann whitney u test are presented in table 1. according to table 1, in the implementation of "let's know the matter" unit, which is taught based on the 5e instructional model accompanied by stem-focused activities, mt was administered as a pre and post-test on the subject. according to the table, while the pre-test point average of the group was 17.31, the post-test point average table 1 findings from the analysis of the data obtained from mt with the mann-whitney u test (n: 24) mt n x std. dev. u p pre test 24 17,31 3,10 115,50 ,00 post test 31,69 ,51 table 2 students' answers to the interview questions question 1: how did you find the activities done in this lesson? themes students ƒ it was nice, it was fun. s5, s17, s19 3 it was instructive. s16, s20 2 question 2: what was your favorite section in the implementation? themes students ƒ shipbuilding s5, s16, s19, s20 4 all of stem activities s17, s20 2 question 3: was there any part you had difficulty in the activities? if yes, which part? themes students ƒ there was complexity when making ships. s5, s19, s20 3 it was difficult for the planning phase of stem activities. s16, s17, s20 3 question 4: were there any parts you did not like in the activities? explain why you don't like it. themes students ƒ there was no part i did not like. s16, s17, s19, s20 4 the starch experiment is not liked. s5 1 question 5: would you like to learn other subjects in this way in science class? themes students ƒ yes, i would. it was fun. s5, s16, s17, s19, s20 5 journal of science learning article doi: 10.17509/jsl.v3i3.23705 161 j.sci.learn.2020.3(3).156-164 was 31.69. as can be seen from the table, as a result of the analysis, it is seen that the subject taught with a 5e instructional model accompanied by stem-focused activities creates a significant difference in the success of students (115.50, p <.05). after the post-test application, semi-structured interviews were done with five volunteer students. the data obtained through the semi-structured interview form was edited and presented in table 2. the names of the students were kept confidential and given with their code. question 1 was asked about how students found the activities and the findings were presented in table 2. as can be seen from the answers given to the question, the students stated that they liked the lesson's activities. student coded s5 said, "i like it because it was fun and nice." question 2 was asked to find out what sections they liked most in the activities held for students, and s17 coded student said, "i like the activity sections, we build boats, i like to do different things. we were trying to come in front of the board, touch something, and guess them. we were doing different experiments. stem events were very nice". s20 coded student said, "i liked the sections where we designed something for stem activities the most. i loved making ships," she stated her opinion. the ship samples that the students made within the scope of stem activities are shown in figure 1. examples of products developed by students to protect from rain are also shown in figure 2. the third question was asked to students to find out if there were any difficulties in their activities. the findings are presented in table 2. as can be seen from their answers to the question, there have been problems in shipbuilding and planning parts of stem activities. an s5-coded student answered as "it was a little complicated when we were making a ship." the student coded s16 said, "there was confusion because we had no idea." similarly, the student coded s20, similar to the student with the code of s16, said, "there were places where i had difficulty. we had difficulties most during the planning phase. for example, it was difficult to plan how to make the ship." question 4 was asked to find out if there were any parts of the activities that students did not like, and s5coded student said that "i did not like the starch experiment because my hands were sticky." apart from this, other students stated that they liked all of the activities. question 5 was asked to students to find out if they would like other subjects to be covered based on stem-focused activities in a science lesson, and s20 coded student said, "yes, i would. it both entertains and keeps it in our minds." similar answers were received from other students. this study was carried out to determine the effect of teaching in the 3rd grade "let's know the matter" unit with a 5e instructional model accompanied with stem-focused activities on students' academic achievement and to get their opinions about stem activities during 17 lesson hours. as a result of the study, a significant increase was observed in students' achievements. similar studies on the impact of stem activities on success have also been shown the contribution to academic success (herdem & ünal, 2018). traditional learning in science education based on direct instruction and books is not effective in students' learning concepts (akçam, dökme, & tunç, 2011; ültay, 2012). thus, some students stated in the interview (in table 2) the lessons were instructive. they were aware that they had learned better with the 5e instructional model accompanied with stem-focused activities. therefore, stem studies are of great importance in education. also, stem activities develop positive attitudes among students (ültay, emeksiz, & durmuş, 2020). it broke the prejudice against science lessons created by students and raised their interest and attitudes towards this area (gülhan & şahin, 2016). in other words, students have a negative attitude towards science lessons, and stem studies broke this prejudice and brought a positive attitude to it (aydın, guzey, & saka, 2017; yamak, bulut, & dündar, 2014). in the interview, the students stated that they wish all courses to be taught with stem. the reason for this, their interest and motivation were always alive during the courses. thus, their attitudes were affected positively. this study also contributed to the field in terms of seeing how much stem activities affect the teaching of "let's know the matter" unit. between the pre and posttest results after the application, it was seen that stem activities increased the students' success in terms of learning the "let's know the matter" unit. in other words, students were able to learn the "let's know the matter" unit effectively with the 5e instructional model figure 1 ship samples designed by students figure 2 examples of raincoats developed by students to protect from rain journal of science learning article doi: 10.17509/jsl.v3i3.23705 162 j.sci.learn.2020.3(3).156-164 accompanied with stem-focused activities. this result can be arisen from using not only stem-focused activities but also the 5e instructional model. 5e instructional model is a well-known model to attract students' interest by associating the content knowledge to the daily life examples (er nas, çoruhlu, & çepni, 2010; ültay & çalık, 2011). furthermore, using the 5e instructional model accompanied with stem-focused activities had probably more motivated students than using stem-focused activities alone. in particular, stem-focused activities were prepared by considering students' interests, such as designing a raincoat. these activities also motivated students during the whole lesson, and their academic successes were also improved. this result is similar to some of the studies' results in the literature. many studies have found that teaching based on stem activities contributes to students' academic success (aydın, guzey, & saka, 2017; ercan & şahin, 2015; wade-shepherd, 2016). 4. conclusion it has been revealed that elementary school students have many misconceptions or misleading for different reasons (gödek & polat, 2017). along with the teachings made with stem activities, students can correct their wrong information by testing their knowledge. also, teachings made with stem activities lead students to learn, research, and question. stem activities are concerned not only with existing problems but also with probable problems (tabaru, 2017). in this respect, this study focused on the problems that umbrellas and ship models made by students can encounter in daily life. the problems that they may encounter in daily life have been narrated, and students have the feeling of developing solutions for these problems. for example, a child who caught a little fish on the shore with his father wanted to sail with his father, and students decided to build a boat to help this child. later, the students were divided into groups, and as a result, different materials emerged to solve the problem. also, students realized that teamwork is essential in stem activities for reasons such as acting together as a group and supporting each other. the gathering of students within the scope of stem activities leads to the development of communication and cooperation between individuals (choi & hong, 2013; kim, ko, han, & hong, 2014). at the end of the study, five open-ended questions were asked to five students, and the students' opinions about stem activities were determined. tables were examined for this situation, and some conclusions were reached. for example, all of the students stated that they liked stem activities. this result is compatible with many studies (damar, durmaz, & önder, 2017; karakaya, yantırı, yılmaz, & yılmaz, 2019; ültay, emeksiz, & durmuş, 2020). also, another striking result obtained in the interview is that students' favorite parts are stem activities. at the same time, the most challenging stage is the planning stages of stem activities such as shipbuilding. the reason for this may be the anxiety that students experience about how to use the materials. in their study, akgündüz & akpınar (2018) stated that the students' most challenging stage is the design phase of the products. eroğlu & bektaş (2018) also stated that the lack of information about the products and target behaviors that the students will design prevent the efficiency of stem activities. another result obtained from the research is that students want to learn other courses in this way. this situation may be the result of the active participation of the students in the lesson, the effectiveness of their learning performance, and the pleasure of the students in producing the products. the contribution of stem activities to the creativity of students and the development of their imagination is also known (hiğde, 2018). it also improves the skills of producing solutions to daily life problems (karakaya, yantırı, yılmaz, & yılmaz, 2019; roberts, 2012). however, it also coincides with other studies in which students love to learn science subjects through experiments or have fun activities (ültay & alev, 2017; ültay, durukan, & ültay, 2014). based on the results obtained from the research, it may be suggested to use stem activities for teaching other subjects due to the positive behavior and thoughts of the students about stem activities. especially considering that the vast majority of stem-focused studies in the literature started from the 4th grade, subject teaching with stem activities can also be recommended for younger age groups. the reason for this can be said that the foundations of the scientific process skills required during the stem activities are formed at a young age. in addition, the concept and academic success can be explored in studies conducted with stem. references akçam, h. k., dökme, i̇., & tunç, t. (2011). üç aşamalı sorularla sınıf öğretmeni adaylarının bazı temel fen kavramları hakkında sahip oldukları kavram yanılgıları [primary school teacher candidates’ misconceptions on some fundamental science concepts with three annotated questions]. gazi university gazi education faculty journal, 31(2), 817-842 (in turkish). akgündüz, d., & akpınar b. b. (2018). okul öncesi eğitimde fen eğitimi temelinde gerçekleştirilen stem uygulamalarını öğrenci, öğretmen ve veli açısından değerlendirilmesi [evaluation of stem applications based on science education in pre-school education in terms of students, teachers, and parents]. journal of education for life, 32(1), 1-26 (in turkish). akgündüz, d., aydeniz, m., çakmakçı, g., çavaş, b., çorlu, m.s., öner, t. & özdemir, s. (2015). stem eğitimi türkiye raporu [stem education turkey report]. scala publishing (in turkish). allen, m. (ed.). (2017). the sage encyclopedia of communication research methods. sage publications. aydın, g., guzey, s., & saka, m. (2017). 4-8. sınıf öğrencileri fen, teknoloji, mühendislik, matematik (stem = fetemm) tutumlarının incelenmesi [science, technology, engineering, mathematics (stem) attitude levels in grades 4th 8 th]. mersin university education faculty journal, 13(2), 787-802 (in turkish). azgın, a. o., & şenler, b. (2019). i̇lkokulda stem: öğrencilerin kariyer ilgileri ve tutumları [stem in primary school: students’ career journal of science learning article doi: 10.17509/jsl.v3i3.23705 163 j.sci.learn.2020.3(3).156-164 interest and attitudes]. journal of computer and education research, 7(13), 213-232 (in turkish). baran, e., canbazoglu bilici, s., mesutoglu, c., & ocak, c. (2016). moving stem beyond schools: students’ perceptions about an out-of-school stem education program. international journal of education in mathematics, science and technology, 4(1), 9-19. baran, e., canbazoğlu bilici, s., & mesutoğlu, c. (2015). fen, teknoloji, mühendislik ve matematik (fetemm) spotu geliştirme etkinliği [science, technology, engineering, and mathematics (stem) public service announcement (psa) development activity]. journal of inquiry based activities, 5(2), 60-69 (in turkish). becker, k., & park, k. (2011). effects of integrative approaches among science, technology, engineering, and mathematics (stem) subjects on students' learning: a preliminary meta-analysis. journal of stem education: innovations & research, 12(5-6), 23-37. bybee, r.w. (2010). what is stem education? science, 329, 996. choe, i. s. (2006). creativity—a sudden rising star in korea. in j. c. k. kaufman & r. j. sternberg (eds.), (pp. 395–420). cambridge university. choi, y., & hong, s.h. (2013). the development and application effects of steam program about 'world of small organisms' unit in elementary science. elementary science education, 32(3), 361-377. cotabish, a. dailey, d., robinson, a., & hughes, g. (2013). the effects of a stem intervention on elementary students’ science knowledge and skills. school science and mathematics, 113(5), 215226. çakmakçı, g. (2016, april 14). dünya ile rekabet için stem şart [stem is essential for competition with the world]. hürriyet. http://www.hurriyet.com.tr/dunya-ile-rekabet-icin-stem-sart40087941 (in turkish) çevik, m., & özgünay, e. (2018). stem education through the perspectives of secondary schools teachers and school administrators in turkey. asian journal of education and training, 4(2), 91-101. çorlu, m. s., capraro, r. m., & capraro, m. m. (2014). introducing stem education: implications for educating our teachers for the age of innovation. education and science, 39(171), 74-85. damar, a., durmaz, c., & önder, i̇. (2017). middle school students’ attitudes towards stem applications and their opinions about these applications. journal of multidisciplinary studies in education, 1(1), 47-65. er nas, s., coruhlu, t. s., & cepni, s. (2010). an assessment on the effectiveness of the material developed for the elaborate stage of the 5e model. journal of ondokuz mayıs university faculty of education, 29(1), 17-36 (in turkish). ercan, s., & şahin, f. (2015). fen eğitiminde mühendislik uygulamalarının kullanımı: tasarım temelli fen eğitiminin öğrencilerin akademik başarıları üzerine etkisi [the usage of engineering practices in science education: effects of design based science learning on students’ academic achievement]. necatibey faculty of education electronic journal of science and mathematics education , 9(1), 128-164 (in turkish). eroğlu, s., & bektaş, o. (2016). stem eğitimi almış fen bilimleri öğretmenlerinin stem temelli ders etkinlikleri hakkındaki görüşleri [ideas of science teachers took stem education about stem based activities]. journal of qualitative research in education, 4(3), 43-67 (in turkish). felix, a. (2016). design based science and higher order thinking [doctoral dissertation, virginia polytechnic institute and state university]. virginiatech. gödek, y., & polat, d. (2017). fen eğitiminde kavram öğretimi [concept teaching in science education]. in d. g. mutlu pınar, fen bilimleri öğretimi [in science teaching] (s. 48-70). pegem publishing (in turkish). gülhan, f., & şahin, f. (2016). fen-teknoloji-mühendislik matematik entegrasyonunun (stem) 5. sınıf öğrencilerinin bu alanlarla ilgili algı ve tutumlarına etkisi [the effects of science-technologyengineering-math (stem) integration on 5th grade students’ perceptions and attitudes towards these areas]. international journal of human sciences, 13(1), 602-620 (in turkish). herdem, k., & ünal, i̇. (2018). stem eğitimi üzerine yapılan çalışmaların analizi: bir meta-sentez çalışması [analysis of studies about stem education: a meta-synthesis study]. marmara university atatürk education faculty journal of educational sciences, 48, 145-163 (in turkish). hiğde, e. (2018). the investigation of the effect of the stem activities prepared for 7th class students in terms of different variables [unpublished doctoral dissertation]. aydın adnan menderes university. karahan, e., canbazoğlu bilici, s., & ünal, a. (2015). integration of media design processes in science, technology, engineering, and mathematics (stem) education. eurasian journal of educational research, 60, 221-240. karakaya, f., yantırı, h., yılmaz, g., & yılmaz, m. (2019). i̇lkokul öğrencilerinin stem etkinlikleri hakkında görüşlerinin belirlenmesi: 4. sınıf örneği [determination of primary school students' views about stem activities: example of 4th grade]. international journal of turkish education sciences, 7(13), 1-14 (in turkish). kim, d. h., ko, d. g., han, m. j., & hong, s. h. (2014). the effects of science lessons applying steam education program on the creativity and interest levels of elementary students. journal of the korean association for science education, 34(1), 43-54. knop, l., ziaeefard, s., ribeiro, g. a., page, b. r., ficanha, e., miller, m. h., & mahmoudian, n. (2017). a human-interactive robotic program for middle school stem education. 2017 ieee frontiers in education conference (fie) (pp. 1-7). ieee. kong, y. t., & huo, s. c. (2014). an effect of steam activity programs on science learning interest. advanced science and technology letters, 59, 41-45. morrison, j. (2006). attributes of stem education: the student, the school, the classroom. ties (teaching institute for excellence in stem), 20, 2-7. park, s. j., & yoo, p. k. (2013). the effects of the learning motive, interest and science process skills using the 'light' unit on sciencebased steam. journal of korean elementary science education, 32(3), 225-238. riechert, s., & post, b. (2010). from skeletons to bridges and other stem enrichment exercises for high school biology. the american biology teacher, 72(1), 20-22. roberts, a. (2012). a justification for stem education. technology and engineering teacher, 71(8), 1-4. sanders, m. (2009). stem, stem education, stemmania. the technology teacher, 68(4), 20-26. tabaru, g. (2017). i̇lkokul 4. sınıf öğrencilerine fen bilimleri dersinde uygulanan stem temelli etkinliklerin çeşitli değişkenlere etkisi [elementary school 4th grade the effects of stem-based activities applied in science lessons to primary education students in terms of various variables] [unpublished master thesis]. niğde ömer halisdemir university (in turkish). ültay, e. (2012). implementing react strategy in a context-based physics class: impulse and momentum example. energy education science and technology part b: social and educational studies, 4(1), 233240. ültay, e., & alev, n. (2017). investigating the effect of the activities based on explanation assisted react strategy on learning impulse, momentum and collisions topics. journal of education and practice, 8(7), 174-186. ültay, n., & çalık, m. (2011). distinguishing 5e model from react strategy: an example of ‘acids and bases’ topic. necatibey faculty of education electronic journal of science and mathematics education, 5(2), 199-220. ültay, n., durukan, ü. g., & ültay, e. (2014). determination of student teachers' views about react strategy. icemst 2014: international conference on education in mathematics, science & technology, the eurasia proceedings of educational & social sciences (epess) 1, 298-302. ültay, n., emeksiz, n., & durmuş, r. (2020). steam yaklaşımına ilişkin örnek bir uygulama ve uygulama hakkında öğrenci görüşleri journal of science learning article doi: 10.17509/jsl.v3i3.23705 164 j.sci.learn.2020.3(3).156-164 [an example steam implementation and student opinions about the implementation]. journal of science teaching, 8(1) 1-17 (in turkish). ültay, n., & ültay, e. (2020). a comparative investigation of the views of pre-school teachers and teacher candidates about stem. journal of science learning, 3(2), 67-78. wade-shepherd, a. a. (2016). the effect of middle school stem curriculum on science and math achievement scores [doctoral dissertation, union university]. proquest dissertations and theses. yamak, h., bulut, n., & dündar, s., (2014). 5. sınıf öğrencilerinin bilimsel süreç becerileri ile fene karşı tutumlarına fetemm etkinliklerinin etkisi [the impact of stem activities on 5th grade students’ scientific process skills and their attitudes towards science]. gazi university gazi education faculty journal , 34(2), 249265 (in turkish). yıldırım, b., & altun, y. (2015). stem eğitim ve mühendislik uygulamalarinin fen bilgisi laboratuar dersindeki etkilerinin incelenmesi [investigating the effect of stem education and engineering applications on science laboratory lectures]. el-cezeri journal of science and engineering, 2(2), 28-40 (in turkish). a © 2020 indonesian society for science educator 80 j.sci.learn.2020.4(1).80-90 received: 23 february 2020 revised: 12 november 2020 published: 28 november 2020 improving pre-service science teachers’ content knowledge and argumentation quality through socio-scientific issues-based modules: an action research study nurcan tekin1*, oktay aslan2, süleyman yılmaz1 1mathematics and science education department, faculty of education, aksaray university, aksaray, turkey 2mathematics and science education department, ahmet keleşoğlu faculty of education, necmettin erbakan university, konya, turkey *corresponding author. tekinnurcann@gmail.com abstract this paper aimed to assess improvements in content knowledge and argumentation quality of pre-service science teachers (pst) through a socio-scientific issues-based (ssi-based) module course. the study was designed as action research with 25 psts. data collection instruments were an energy-related content knowledge questionnaire and video recordings. an energyrelated content knowledge rubric was used to evaluate the content knowledge of psts, while their argumentation quality was analyzed using a video analysis inventory. the module course was completed in eight weeks. findings showed that the ssi-based teaching modules course improved energy-related content knowledge with all psts increasing by a minimum of one criterion on the rubric. although the number of arguments decreased weekly, the argumentation quality also increased. thus, it can be inferred that ssi-based module teaching is a practical tool to teach energy-related content knowledge and argumentation qualities. the study suggests that action research can contribute to developing an effective learning environment. further studies that include the actual practices of psts as they reflect and revise their learning are indicated. keywords argumentation quality; content knowledge; socio-scientific issues-based teaching 1. introduction in recent decades, commentators have opined that science should better match society's requirements and that science is related to the environment and technology (topçu, 2008). socio-scientific issues (ssis) give the idea that there are moral principles in students' physical, social, and personal lives. moreover, these issues provide students with a healthy perspective on how to think about sciencebased issues (topçu, 2015). according to zeidler, sadler, simmons, & howes (2005), ssis impact individuals’ intellectual development in terms of their personal and social context. this result is provided by using controversial issues and dilemmas, which activate how to think about these kinds of issues. thus, ssis can be regarded as a context to learn science and as a pedagogical strategy that has clear objectives. ssis are scientific-related social issues with ethicalmoral, political, and religious contents (borgerding & dagistan, 2018). also, ssis can be regarded as a complement of scientific products or processes that can bring about social debates. sadler & zeidler (2005) indicated ssis' features: they are based on scientific concepts and have controversial nature, and they are discussed in public usually and subject to political and social influences (e.g., nuclear power plants). a few of the most typical ssis are cloning stem cells and genetically modified foods, biotechnological outputs, and implementations (sadler & zeidler, 2005). because ssis are open to argument on the topics held during the learning process, these issues are integral parts of the curriculums in most countries (council of ministers of education, canada [cmec], 1997; western australia curriculum council [wacc], 1998). according to the ministry of national education (mone) (2013), ssis involve scientific and moral discernment for socioscientific problems about science and technology. journal of science learning article doi: 10.17509/jsl.v4i1.23378 81 j.sci.learn.2020.4(1).80-90 1.1 teaching socio-scientific issues ssis, which contribute to students’ science literacy, should not be seen as a magic-bullet solution (sadler, 2011a). several studies have pointed out that ssi-based teaching practices struggle teachers in a science course (hanley, ratcliffe, & osborne, 2007). according to windschitl, thompson, braaten, & stroupe (2012), science teachers struggle to include students in lessons as a social aspect of learning. for instance, simon & amos (2011) reported that the teacher struggled to recognize students participating in the instructional process and argumentation practices. besides, pitiporntapin, yutakom, & sadler (2016) emphasized that pre-service science teachers struggle with a lack of integration of ssis to their lessons, difficulties linking ssis to scientific concepts, and difficulties in eliciting students’ prior knowledge of ssis. a literature review suggests that it is challenging to use ssis in the classroom, but teachers from different backgrounds can succeed with appropriate support (sadler, 2011a). for this purpose, teachers should be supported as professionals to be more experienced in teaching ssis. to successfully integrate ssi-based instruction in the classroom, teachers must provide scaffolding for students to engage in higher-order thinking processes, reflect learning, argumentation processes, reasoning, and decision-making (presley et al., 2013). besides, teachers can use newspapers, interviews, reports (klosterman, sadler, & brown, 2012), and technological materials (evagorou, 2011) to promote ssi-based practices in the classrooms. if they can put this approach into practice, students can deeply understand ssis' place in scientific content (sadler, 2011a). thus, teachers should guide students during lessons and evaluate whether the students can make a proper claim or not (nielsen, 2012). according to ratcliffe & grace (2003), practices including learning strategies are crucial, as they make the learning goals and structure of the practice clearer, make the learning process visible, and define the teacher's role during the discussion. besides, bencze (2000) indicated that students should engage in student-directed, open-ended investigation processes, which lead them to reach their conclusions about sociopolitical issues. moreover, because partnerships between researchers and teachers constitute another significant issue, it needs to be defined whether university-based education helps classroom-based education or not (sadler, 2011a). it is also important to remember that teachers need both subject-matter knowledge and pedagogical-content knowledge (magnusson, krajcik, & borko, 1999) to teach ssis effectively. 1.1.1 socio-scientific issues teaching frameworks different models have been developed for ssi-based teaching include a model for decision-making (keefer, 2003), a model for bioethics (dawson, 2001), and ssibased teaching frameworks (presley et al., 2013; sadler, 2011b). sadler (2011b) has drawn a frame to teach ssis, which includes four aspects: (a) designing elements, (b) learner experiences, (c) classroom environment, and (d) teacher attributes. designing elements and learner experiences as core features of the framework are shown in the figure's center. the other dimensions (classroom environment and teacher attributes) are seen as peripheral contributions to shape the core aspects. in contrast, presley et al. (2013) presented a framework that puts the aspect of teacher attributes in the central region. some modules have been designed to teach ssis by using different models (evagorou, guven, & mugaloglu, 2014). for example, three modules have been developed in preparing elementary and secondary pre-service teachers for everyday science (evagorou, guven, & mugaloglu, 2014). here the main aim was to improve teachers’ content knowledge, pedagogical knowledge, and skills to teach ssis with seven participant countries (topçu, muğaloğlu, & güven, 2014). 1.1.2 argumentation for teaching socio-scientific issues in science education, it is crucial to learn well-accepted scientific practices, including making a claim and providing an argument to make a claim clear (andrews, costello, & clarke 1993). ssis are contradictory by nature (sadler & zeidler, 2005), and this nature makes ssis debatable. therefore, argumentation can be one of the best strategies to meet ssi teaching needs (osborne, erduran, & simon, 2004). multiple studies have been published (dawson & carson, 2017; erduran, simon, & osborne, 2004; zohar & nemet, 2002) related to argumentation, which is seen as a critical factor of ssi teaching (osborne, erduran, & simon, 2004; presley et al., 2013). according to duschl, schweingruber, & shouse (2007), argumentation is a sound practice to teach how to learn about the natural world, how to produce and evaluate scientific evidence and explanations, how to understand the epistemic nature of scientific knowledge, and how to participate in scientific practices and discussions. the most widely used model about argumentation is toulmin’s argument pattern (tap). while using tap to analyze arguments focuses on small-group discussions among students in science education (erduran, simon, & osborne, 2004). tap defines argumentation as a series of claims that are mutually related. according to this pattern, data supports a claim, a warrant connects data to the claim, backing supports the warrant, and rebuttal shows that the claim may not be valid in certain situations. 1.1.3 teaching energy-related socio-scientific issues energy is a concept used to explain scientific phenomena among different disciplines (eisenkraft et al., 2014). students should be encouraged to be engaged citizens to learn energy-related ssis (sakschewski, eggert, schneider, & bögeholz, 2014). thus, energy should be seen journal of science learning article doi: 10.17509/jsl.v4i1.23378 82 j.sci.learn.2020.4(1).80-90 not only as a concept but as an educational opportunity in terms of ssis, citizenship education, and sustainable development (sakschewski, eggert, schneider, & bögeholz, 2014). science education curriculums are also extensively related to energy and students’ misconceptions about this concept. in a similar vein, energy topics are an essential consideration in turkey (mone, 2013). however, it is not sufficient to learn concept-based knowledge to make decisions to evaluate energy technologies critically. although energy is a crucial concept having rich connections among disciplines, the issues that include this concept are difficult for students. for example, students struggled with the long-term process of defining a problem about energy issues (bartosch, 2018). according to sadler (2009), energy storage technologies, the construction of offshore wind power systems, or energyefficient buildings as ill-structured problems encounter students in their daily lives. these problems are not only scientific or engineering-related issues but also ethical or political social-related controversial issues (bartosch, 2018). moreover, people have a significant and quick response to these issues because they have limited time and information to decision energy-related issues (lee, 2015). energy-related ssis provide a multidisciplinary approach and a decision-making process, producing engaged social debates and improved citizenship consciousness, which are reasons to select this issue in this study. besides this, this topic reflects our domestic agenda. furthermore, there are many design model/module studies for ssis in terms of climate change (klosterman & sadler, 2010), biotechnology (presley et al., 2013), ecology (sadler et al., 2015), and natural selection (friedrichsen, sadler, graham, & brown, 2016). hence, we believe that this study contributes to the literature on module usage related to energy-related ssis. other studies related to ssis have investigated argumentation practices (venville & dawson, 2010), teacher experiences (ekborg, ottander, silfver, e., & simon, 2013), decision-making skills (gresch, hasselhorn, & bögeholz, 2013), and learning the nature of science (lederman, antink, & bartos, 2014). in these studies, ssis were used as a context. also, in studies related to designing modules, most have not focused directly on ssis. in this study, ssis are used not as context but as content to learn. participatory action research (par) studies are deal with both empirical research findings that obtained classroom activities and the development of innovative, evidence-based curricula, pedagogies, and teaching materials (eilks, 2018). in par model, it aimed five areas of objectives in practical science education: (1) new concepts and materials for teaching, (2) knowledge about teaching and learning, (3) developed practice, (4) trained teachers, and (5) documentation of teaching practice (eilks, 2018). energy-related ssis-based module used in this study as a guide for teachers in teaching ssis. we intend to both develop teaching practices that pre-service science teachers will use in their courses and encourage them professional development. this study aimed to evaluate what improvements the pre-service science teachers’ (psts) content knowledge and argumentation quality in socio-scientific issues-based module used courses. the research questions are (1) what improvements are in the content knowledge level of psts when implementing socio-scientific issues-based module used courses? moreover, (2) what improvements in the argumentation quality of implementing socio-scientific issues-based module used courses? 2. method 2.1. study design this study was designed based on action research. action research aims to study a social situation to improve the quality of action (elliot, 1991). in this case, an action study is studying a school or class situation to understand and improve the quality of education (mctaggart, 1997). action research is often used to design curriculum, advance professional development, or undertake systematic planning or policymaking in education. eilks and ralle’s (2002) participatory action research (par) project for chemistry education is an example of action research related to science education. par is a collaborative strategy for curriculum and classroom-based studies (eilks & ralle, 2002) because this approach provides collaboration among researchers and teachers. as universities and schools have their advantages and limitations (mcintyre, 2005), par aims to show that both universities and schools can benefit from each other’s strengths and address their missions collaboratively. in the current study, the role of the researcher was a “participatory observer”. the researcher helped the psts during the distribution of materials, giving information about the scenarios and activities and explaining the subjects, so the researcher was a participant during these processes. on the other hand, the researcher was a passive observer during conduct activities and the argumentation process. 2.2 participants in this study, criterion sampling, one of the purposeful sampling methods, was used to create the participant group. criterion sampling provides researchers the opportunity to deeply study a situation by providing a vast amount of related data (creswell, 2009). purposeful sampling is a crucial method to collect open-ended data (creswell, 2009). this study's participants were 18 women and 7 men who were studied in science teaching with grade three pre-service teachers in turkey, individuals who want to become teacher, and they enter the university entrance exams after 12 years compulsory education. after matriculation of education faculty, pre-service teachers whom trainee four years qualify for a teacher. if they wish, graduated teachers can be appointed as a teacher to schools journal of science learning article doi: 10.17509/jsl.v4i1.23378 83 j.sci.learn.2020.4(1).80-90 by taking a new exam. therefore in this study, the expression ‘pre-service science teachers’ refers to students studying at the education faculty. the selection criteria were that the psts were grade three, they knew the basic concepts, and they had not taken a course about energyrelated ssis, and they had not attended any program about teaching ssis. although psts who participated in this study knew fundamental physics and chemistry concepts (e.g., dynamic, electricity, optic, energy conversions) and general science education knowledge (e.g., general educational sciences, science curriculum), most psts did not have prior knowledge on ssis, and they did not learn argumentation process and skills at the level of their education. 2.3 instruments two types of instruments—open-ended questions and video recordings—were used to collect data. first, questions based on energy-related content knowledge were used to understand the knowledge level of psts. openended content knowledge questions about energy (ckqes) were prepared by the researchers based on a turkish science education curriculum (mone, 2013). the questions' themes were determined based on the dimensions of energy literacy defined by dewaters & powers (2013). the themes were basic conceptual knowledge, energy sources, energy needs and management, social effects of energy, global energy, and energy's environmental effect. ckqes examples are presented in table 1. ckqes were reviewed by three experts, two of whom are professors of science education and a physics professor. after expert views corresponded, the final form of questions created. the final form of ckqes consists of 15 open-ended questions. to ensure study accuracy and credibility, criteria suggested by johnson (2012) were taken into consideration. therefore, the researcher took notes when necessary, describing the instruments' design process in detail and creating a table of specifications to evaluate and confirm the content validity. secondly, argumentation videos were recorded by four groups in the argumentation process. videos were recorded for four weeks, and each week had its topic. the researchers decided on energy-related argumentation topics based on the learning outcomes of a turkish science education curriculum (mone, 2013) as ckqes. for the first week, power plants were held as an argumentation topic (for 12–24 minutes in different groups). for the second week, the topic was heat insulation (13–28 minutes). solar energy was discussed in the third week (11– 20 minutes), and in the last week, the topic was recyclingenergy scenarios (14–29 minutes). 2.4 process in this research, the researchers developed a teaching module for use in their course. the researchers grounded their module based on sadler's (2011b) framework and presley et al. (2013). the module was designed based on the turkish science education curriculum (mone, 2013). the module had three parts, covering eight weeks. the first part of the module (two weeks) included activities based on table 1 ckqes examples according to learning outcomes and energy literacy dimensions ckqes examples learning outcomes energy literacy dimensions briefly describe how energy is produced in hydroelectric power plants, thermal power plants, wind power plans, geothermal power plants and nuclear power plants (q1). investigate and present how electricity is generated in power plants (lo7.6.2.4) energy sources you've learned that illegal electricity use is high in the area where your school is appointed. you want to give your students a few suggestions to share with their parents. what advice would you give them to prohibit the use of illegal electricity? offer 4 suggestions. (q2) discuss the importance of conscious and efficient use of electrical energy in terms of family and country economy (lo7.6.2.5) energy needs and management which criteria are used for the heat insulation materials? why? (q8) determines the selection criteria of heat insulation materials used in buildings (lo6.6.1.3) basic conceptual knowledge as residents of the site, you would like to have a meeting about whether to have heat insulation. but the apartment superintendent says it will be costly and nobody will take a positive decision. how do you convince the apartment superintendent in terms of the contribution of heat insulation to energy savings for the family and the national economy? (q9-10) discuss the importance of thermal insulation in buildings in terms of family and country economy and effective use of resources (lo6.6.1.2) social effects of energy journal of science learning article doi: 10.17509/jsl.v4i1.23378 84 j.sci.learn.2020.4(1).80-90 plays about ssis' nature and features. the second part (four weeks) consisted of energy-related ssi argumentation scenarios (power plants, heat insulation, solar energy, and recycling energy). in this part, psts worked in four groups. in each group, psts were assigned certain roles (e.g., scientist, industrialist, father, environmentalist, and apartment superintendent) as in the study by evagorou, guven, & mugaloglu, (2014). one of the psts in each group took notes to support the video recordings. for the last two weeks, psts prepared a lesson plan about ssis. according to eilks & ralle (2002), par can improve educational strategies and development curriculum. the implementation of the ssi-based module was carried out in a single cycle as the researchers aimed to improve the table 2 action plan action plan steps process development of teaching strategies and materials literature review (studies about teaching ssis, designing module and energy-related topics) expert opinions (expert opinion about the subject of the module and activities to be used during implementation) meeting pre-service teachers (collecting ideas of pre-service teachers about the ssis and module) describing research study case (defining and classifying the objectives in the curriculum whether they are related with ssis or not, and also defining energy-related objectives) looking through the studies related with the research problem (problem situation was defined as energy-related topics and the studies related with it were analyzed) designing module to teach ssi-based on research questions (determining the framework of the module, deciding on the topics and activities for the process and schedule for the module) testing in practices pilot study revising the module after pilot study revising data collecting tools after pilot study collecting pre-test data according to research questions eight-week-implementation process (play activities, argumentation process and designing lesson plans) collecting post-test data according to research questions evaluation analyzing data interpreting the findings reflection and revision sharing the results figure 1 action research cycle planning (development) -literature review -expert opinion -meeting pre-service teachers -determining research case and problem -designing module action (testing) -pilot study -analyzing and evaluation pilot study data -revising the module -revising data collecting tools observe -eight-week-implementation process -collecting pre/post-test data -meeting pre-service teachers evaluate/reflection -analyzing data -interpreting the findings -sharing the results journal of science learning article doi: 10.17509/jsl.v4i1.23378 85 j.sci.learn.2020.4(1).80-90 content knowledge and argumentation quality of psts. this cycle consists of four-part: planning (development), action (testing), observe and evaluation/reflection (figure 1). in broad terms, the action plan designed according to eilks & ralle’s (2002) “par model for science education” for the research process is presented in table 2. according to table 2, literature review, expert and psts opinion studies, determination of research problem and study case, design of ssi-based module were carried out in the development of teaching strategies and materials step. in the second step, testing in practices, pilot study, revision of module and data collection tools, pre/post-test collect, and eight-week implementation was done. in the last steps, evaluation, reflection, and revision analyzing data, interpreting the findings, and sharing the results were carried out. 2.5 data analysis two instruments, a rubric, and an inventory were used in the data analysis. firstly, an energy-related content knowledge rubric (eckr) developed by the researchers was used to analyze data related to content knowledge about energy. the criteria were determined according to the themes used in ckqe. the rubric was designed with a review of three measurement and evaluation experts and three science and physics experts. the reliability was determined using kappa reliability coefficients. the reliability value based on the formula defined by fleiss (1971) was calculated as 0.68, which means that “substantial”. according to landis and koch (1977), the extent of the agreement is perfect if the kappa values fall between (.81) and (1.00); substantial if they fall between (.61) and (.80); moderate if they fall between (.41) and (.60); fair if they fall between (.40) and (.21); slight if they fall between (.00) and (.20); and poor if they are less than (.00). this result indicates that the agreement value between the six raters was enough to use the analysis tool for the data related to content knowledge about energy. eckr was an analytical rubric defining four levels: “inadequate (1), developable (2), acceptable (3) and exemplary (4)” and the category names were “basic conceptual knowledge, energy sources, energy needs and management, social effects of energy, a global energy, and the environmental effect of energy.” data were analyzed by two researchers based on eckr. the inter-rater reliability was determined using kappa reliability coefficients. the reliability value based on the formula defined by cohen (1960) was calculated as 0.69, which means that “substantial”. this result indicates that the agreement value between the two raters was enough to categorize the findings. secondly, to define the argumentation quality, the argumentation quality video analysis inventory (aqvai) was used. this inventory was based on tap and designed by erduran, simon, & osborne, (2004). in this study, aqvai was preferred because it is both comprehensive and appropriate for the subject of argumentation. aqvai defines the levels of arguments regarding elements like claims, data, reasoning, and rebuttal (see levels in table 3). level 1 defines the argument comprising the claim or claim with counterclaims; level 2 defines arguments including the claim with data, warrant, and backing; level 3 describes claims with data, warrant, backing, weak rebuttal, and counterclaims; level 4 defines arguments with a strong rebuttal, and level 5 is defined by more than one rebuttal or arguments with all elements of argumentation. the argumentation videos of each group were analyzed by two researchers, according to aqvai. the inter-rater reliability was determined using kappa reliability coefficients. the reliability value based on the formula defined by cohen (1960) was calculated as 0.71, which means that “substantial”. this result indicates that the analyses of both researchers have corresponded; the findings were obtained. 3. result and discussion the results below comprise psts’ knowledge level about energy-related content knowledge and their argumentation quality. 3.1 psts’ knowledge about energy-related content the knowledge level of psts was determined by using the same questions before and after the argumentation process, established by six criteria: basic conceptual knowledge (bck), energy sources (es), energy needs and table 3 argumentation quality video analysis inventory (adapted from erduran et al., 2004) aql criteria level 1 claim or claim with counterclaim (not include data, warrant, backing and rebuttal) level 2 claim with data, warrant or backing (not include rebuttal) level 3 claim or counterclaims with data, warrant, backing and weak rebuttal level 4 strong rebuttal level 5 more than one rebuttal and all elements of argumentation table 4 psts’ scores according to eckr criteria test scores (�̅�) basic conceptual knowledge pre-test 2.07 post-test 3.10 energy sources pre-test 1.64 post-test 2.89 energy needs and management pre-test 2.46 post-test 3.26 social effects of energy pre-test 1.93 post-test 3.00 global energy pre-test 1.57 post-test 2.63 environmental effect of energy pre-test 1.71 post-test 3.21 journal of science learning article doi: 10.17509/jsl.v4i1.23378 86 j.sci.learn.2020.4(1).80-90 management (enm), social effects of energy (see), global energy (ge), and environmental effect of energy (eee). the pre-test score of psts on bck was 2.07 (level 2), and the post-test score was 3.10 (level 3); for es, the pretest score was 1.64 (level 1), and the post-test score was 2.89 (level 3); the pre-test score for the criterion enm was 2.46 and the post-test score was 3.26; for see, the pre-test score was 1.93 (level 2), and the post-test score was 3.00 (level 3); the pre-test score of the criterion ge was 1.57 (level 1) and the post-test score was 2.63 (level 3); and the pre-test score of the last criterion, eee, was 1.71 (level 1) and the post-test score was 3.21 (level 3). these findings are shown in table 4. according to the findings, the module improved psts’ energy-related content knowledge, as it stepped up each criterion by a minimum of one level. 3.2 psts’ argumentation quality the findings of the argumentation quality of psts according to aqvai over four weeks are shown in figure 2. according to figure 2, it can be seen that arguments of psts in the first week (power plant-related argumentation) were at level 1 and 2; in the second week (heat insulationrelated argumentation), all levels of arguments were voiced, but levels 1 and 2 were more intensely voiced; in the third week (solar energy-related argumentation), arguments from levels 1 to 3 were expressed and levels 1 and 2 were dramatically decreased compared to the first two weeks; and in the fourth week (recycling energy-related argumentation), when all levels of arguments were expressed, the amount of level 3 and level 4 arguments increased. moreover, although the number of arguments decreased weekly, the argumentation quality increased. for table 5 psts’ argumentation samples (heat insulation-related argumentation samples) argumentation samples criteria level heat insulation should not be done. because the materials used there are damaged after some time and they become garbage. furthermore they become carcinogen (pst3) claim with warrant level 2 we have no data effect of these materials on human health and i didn’t see anyone with cancer due to heat insulation (pst1) counterclaim to claim level 1 as heat insulation is newly applied in our country, we may not have encountered such effects yet (pst3) claim with backing level 2 heat insulation is not new in our country. applied since 2001. heat insulation also prevents mould growth. growth moulds can affect airways in uninsulated houses. in addition to, 5% heat loss through the window, 6% from ground, 17% thorough doors and 23% through roof in uninsulated houses. this leads to loss of energy and economy (pst4) claim with data, warrant, backing and counterclaim to claim level 3 ok. materials used in heat insulation will quickly flash and flash-over in a fire (pst2) counterclaim to claim level 1 what you say is banned in 2007. since the materials used for heat insulation are covered with stone does not damage in case of fire. lastly, use of fossil fuels decreases by 50% with heat insulations (pst4) rebuttal level 4 figure 2 argumentation quality of psts according to aqvai 31 29 12 11 22 27 21 19 3 4 5 1 1 0 5 10 15 20 25 30 35 40 week i week ii week iii week iv a r g u m e n ts ( f) weeks level i level ii level iii level iv level v journal of science learning article doi: 10.17509/jsl.v4i1.23378 87 j.sci.learn.2020.4(1).80-90 detailed argumentation, samples of psts, according to aqvai, are shown in table 5. at the end of the ssi-based module implementation, we can say that the module included ssi-based activities that improved psts’ energy-related content knowledge. based on their studies, çetin (2014) inferred that participating in the argumentation process positively affects the content knowledge level of psts. other studies have reached the same inference (venville & dawson, 2010; zohar & nemet, 2002). according to zohar & nemet (2002), teaching argumentation skills improves argument skills in the context of ssi and enables participants to understand science. von aufschnaiter, erduran, osborne, & simon (2008) indicated the reason for this situation as that participants spend more time on the topic during argumentation practice. in this study’s teaching module, four weeks were spent with argumentation practice. after two weeks of play activities about ssis, the argumentation process was held, aiming to prepare psts for topics. after the argumentation process, psts formed their argumentation lesson plan over two weeks. thus, overall the argumentation process was intensely experienced by psts, which can be one reason for their improved content knowledge level, as seen in other studies. for example, jimenez-aleixandre & pereiro-munoz (2002) indicated that participating in the argumentation process advances knowledge about the topic practiced during argumentation implementation. additionally, psts might state more accurate knowledge during the argumentation process because they are far less uneasy about failing to use an argument's components. for example, at the beginning of the implementation process, psts were hesitant while expressing their ideas, and they used sentences like “what i am saying now is a claim or data component for my argument”. within weeks, psts were confident in expressing their ideas because they learned the components of an argument. this situation may cause an improvement in their content knowledge level. additionally, jho, yoon, & kim (2014) indicated that ssi-based teaching practices step up content knowledge of psts, as ssi-based education requires understanding and reflecting knowledge. in our study, the questions used to understand psts’ content knowledge were versatile, comprising skills such as creating projects, designing concept maps, reading graphs, and decision-making, which triggers the use of diverse skills. therefore, the improvement in psts’ content knowledge level in our study should not be seen as a one-dimensional improvement but an improvement that includes different skills. another remarkable finding based on eckr is that the module mostly affected the environmental effects of psts’ content knowledge's energy dimension. according to sandell, öhman, & östman (2003), when individuals participate in an ssi-based decision-making process, they have the chance to evaluate the information that has effects both on their individual and their social lives, which represents excellent progress in terms of education for sustainable development. moreover, simonneaux & simonneaux (2012) defined sustainable development issues as ssis. ratcliff & grace (2003) defined environmental topics as ssis. argumentation topics held in the module (power plants, solar energy, and the relation between recycling and energy) were positively related to environmental issues. during the weeks, these topics were debated, some of the psts played environmentalists' role, and they got involved in the decision-making process to refute counterarguments. this result can be the reason for the improvement in environmental issues-related energy topics. jegstad & sinnes (2015) concluded that dealing with green chemistry topics, which are chemistry topics based on environmental issues, may increase pupils’ understanding of the scientific process. similarly, sustainable development-related topics in our module may bring about an improvement in energy-related content knowledge. our study's argumentation quality dimension shows that the arguments with elements like data, warrant, and rebuttal increased in number when the number of arguments expressed by psts decreased. thus, it can be inferred that the module enhanced the argumentation quality of psts. the inventory used in this study has two methodological approaches while the arguments are evaluated, which are the number of arguments and rebuttal of the counterargument based on tap (erduran, simon, & osborne, 2004). it can be said that argumentation quality can be defined by the number of arguments and a rebuttal and the nature of the rebuttal. in other words, the amount of argumentation does not guarantee its quality; rebuttal is a crucial criterion to evaluate the argumentation quality. therefore, our study's decreased number of arguments should not be evaluated as a degradation in argumentation quality. according to jimenez-aleixandre & pereiro-munoz (2002), the argumentation process enhances knowledge about argumentation practice. for the same reason, psts might use a reduced number of claims and many other argumentation elements. in our study, the more they learned about argumentation practice, the less they tended to produce level 1 and level 2 claims. although the number of claims was reduced, the last weeks' claims were more substantial than in previous weeks. moreover, warrant, backing, and rebuttal elements of argumentation were more evident, defined as an advancement of argumentation. an improvement in content knowledge can also cause this situation. according to sampson & clark (2011), content knowledge is an essential factor in argumentation quality. thus, the improvement in psts’ journal of science learning article doi: 10.17509/jsl.v4i1.23378 88 j.sci.learn.2020.4(1).80-90 content knowledge within weeks could also have affected their argumentation quality. during the module design, the researchers recognized five learning outcomes about ssis or argumentation topics that could be used as ssis in the turkish science education curriculum (mone, 2013), even though there are many learning outcomes about energy topics, which are generally about energy usage. besides, ssis are only mentioned in the goals of the renewed curriculum (2017). in the 21st century, energy is one of the factors that define countries’ development level, and it is an essential part of their budgets, which is significant for citizens. besides, citizenship consciousness is emphasized in science literacy (sakschewski, eggert, schneider, & bögeholz, 2014). in this situation, energy-related ssis can be the intersection topic of both citizenship consciousness and science literacy. thus, it can be essential to rank this topic higher in the curriculum. when this is done, individuals will be supported in terms of decision-making, evaluating issues in the media or daily life, speaking knowledgeably about world affairs, being open to new ideas, listening to different views, and showing respect for different ideas. therefore, ssis should be covered in the curriculum. this module, whose aim was to test energy-related ssis, can be used as a guide for future ssi modules. also, the similarities and differences can be presented among ssi-based modules. 4. conclusion it can be an inference from this study that ssi-based teaching modules can use practical tools to improve energy content knowledge and enhance the argumentation quality of psts’. besides, it is essential to bear in mind that this study was designed as an action research study. action research studies are used to fill the gap between the practical world and the research literature and help teachers create effective learning environments (johnson, 2012). action research studies do not require experimental and control groups, dependent and independent variables, or hypotheses. the action study aims to take a snapshot of the situation to understand it (johnson, 2012). in this study, the researcher was a participant-observer, and the subject was created by purposive sampling. thus, the results are limited concerning psts. however, the study could be adapted for teachers and students, and other experimental studies could be implemented to understand the module's impact better. in our study, we aimed to improve energy content knowledge and the quality of argumentation of psts. since we achieved our aim in the first cycle, we completed the study in one cycle. action research could complete two or more cycles in action research. this result can be a limitation of this study that we will explore further. it can be suggested that participants' practices could include in the “reflection and revision” step of the action plan and the practices could compare with different cycles of action research. since action research provides one-to-one solutions to problems encountered in schools, we need to pay attention to promoting ssis in participants’ classrooms. the other limitation is argumentation contents discussed in four weeks are different from each other, although all are energy-related ssis. this situation may have affected the results of psts’ content knowledge and argumentation qualities. thus, other studies could be practiced on focusing on one issue. references andrews, r., costello, p., & clarke, s. (1993). improving the quality of argument: final report. hull. uk: university of hull. bartosch, i. (2018). learning about energy. a real-life approach challenging the present culture of science & engineering. visions for sustainability. 9, 27-40. doi: 10.13135/2384-8677/2771 bencze, j.l. (2000). procedural apprenticeship in school science: constructivist enabling of connoisseurship. science education, 84(6), 727-739. doi: 10.1002/1098-237x(200011)84:6<727::aidsce3>3.0.co;2-o borgerding, l.a., & dagistan, m. (2018) pre-service science teachers’ concerns and approaches for teaching socio-scientific and controversial issues. journal of science teacher education, 29(4), 283306, doi: 10.1080/1046560x.2018.1440860 cohen, j. (1960). a coefficient of agreement for nominal scales. educational and psychological measurement. 20(1), 37-46. council of ministers of education, canada (cmec). (1997). common framework of science learning outcomes k to 12: pan-canadian protocol for collaboration on school curriculum for use by curriculum developers. toronto, canada. creswell, j.w. (2009). research design: qualitative, quantitative, and mixed methods approaches. los angeles: sage. çetin, p.s. (2014). explicit argumentation instruction to facilitate conceptual understanding and argumentation skills. research in science & technological education. 32(1), 1-20. doi: 10.1080/02635143.2013.850071 dawson, v. (2001). addressing controversial issues in secondary school science. australian science teachers journal. 47(4), 38-44. dawson, v., & carson, k. (2017). using climate change scenarios to improve grade 10 students’ argumentataion skills. in o. finlayson, e. mcloughlin, s. erduran, & p. childs (eds). electronic proceedings of the esera 2017 conference. research, practice and collaboration in science education, part 8 (co-ed. j.a. nielsen & m. lindahl), dublin, ireland: dublin city university. isbn 9781-873769-84-3 dewaters, j., & powers, s. (2013). establishing measurement criteria for an energy literacy questionnaire. the journal of environmental education. 44(1), 38-55. doi: 10.1080/00958964.2012.711378 duschl, r.a., schweingruber, h.a., & shouse, a.w. (2007). taking science to school: learning and teaching science in grades k-8. washington dc: national academy press. eilks, i. (2018). action research in science education: a twenty years personal perspective. action research and innovation in science education. 1(1), 3-14. doi: 10.12973/arise/98909 eilks, i., & ralle, b. (2002). participatory action research in chemical education. in b. ralle & i. eilks (eds.), research in chemical rducation-what does this mean? (pp. 87-98). aachen: shaker. eisenkraft, a., nordine, j., chen, r.f., fortus, d., krajcik, j., neumann, k. (2014). why focus on energy instruction? in r.f. chen, a. eisenkraft, d. fortus, j. krajcik, k. neumann, j. nordine & a. scheff (eds). teaching and learning of energy in k-12 education (pp. 1-14). switzerland: springer. ekborg, m., ottander, c., silfver, e., & simon, s. (2013). teachers’ experience of working with socio-scientific issues: a large scale and in depth study. research in science education. 43, 599-617. doi: 10.1007/s11165-011-9279-5 elliot, j. (1991). action research for educational change. buchingam: open university press. journal of science learning article doi: 10.17509/jsl.v4i1.23378 89 j.sci.learn.2020.4(1).80-90 erduran, s., simon, s., & osborne j. (2004). tapping into argumentation: developments in the application of toulmin’s argument pattern for studying science discourse. science education. 88(6), 915-933. doi: 10.1002/sce.20012 evagorou, m. (2011). discussing a socio-scientific issue in a primary school classroom: the case of using a technology supported environment in formal and nonformal settings. in t. d. sadler (ed.). socio-scientific issues in the classroom: teaching, learning and research (pp. 133-159). new york: springer. evagorou, m., guven, d., & mugaloglu, e. (2014). preparing elementary and secondary pre-service teachers for everyday science. science education international. 25(1), 68-78. fleiss, j.l. (1971). measuring nominal scale agreement among many raters. psychological bulletin. 76(5), 378-382. friedrichsen, p., sadler, t., graham, k., & brown, p. (2016). design of a socio-scientific issue curriculum unit: antibiotic resistance, natural selection and modelling. international journal of design for learning. 7(1), 1-18. doi: 10.14434/ijdl.v7i1.19325 gresch, h., hasselhorn, m., & bögeholz, s. (2013). training in decisionmaking strategies: an approach to enhance students’ competence to deal with socio-scientific issues. international journal of science education. 35(15), 2587-2607. doi: 10.1080/09500693.2011.617789 hanley, p., ratcliffe, m., & osborne, j. (2007). teachers’ experiences of teaching ‘ideas-about-science’ and socio-scientific issues. paper presented at the 7th conference of the european science education research association. malmö, sweden. jegstad, k.m., & sinnes, a.t. (2015). chemistry teaching for the future: a model for secondary chemistry education for sustainable development. international journal of science education, 37(4), 655-683. doi: 10.1080/09500693.2014.1003988. jho, h., yoon, h.g., & kim, m. (2014). the relationship of science knowledge, attitude and decision aking on socio-scientific issues: the case study of students’ debates on a nuclear power plant in korea. science and education, 23, 1131-1151. jimenez-aleixandre, m.p., & pereiro-munoz, c. (2002). knowledge producers or knowledge consumers? argumentation and decision making about environmental management. international journal of science education, 24(11), 1171-1190. johnson, a.p. (2012). a short guide to action research. (4th ed.). boston ma: allyn and bacon. keefer, m.w. (2003). moral reasoning and case-based approaches to ethical instruction in science. in d. l. zeidler (ed.), the role of moral reasoning on socio-scientific issues and discourse in science education (pp. 241259). dordrecht: springer. klosterman, m.l., & sadler, t.d. (2010). multi-level assessment of scientific content knowledge gains associated with socio-scientific issues based instruction. international journal of science education. 32, 1017-1043. klosterman, m. l., sadler, t. d., & brown. j. (2012). science teachers’ use of mass media to address socio-scientific and sustainability issues. research in science education, 42(1), 51-74. doi: 10.1007/s11165-011-9256-z landis, j.r., & koch, g. (1977). the measurement of observer agreement for categorical data. biometrics, 33, 159-174. lederman, g., antink, a., & bartos, s. (2014). nature of science, scientific inquiry, and socio-scientific issues arising from genetics: a pathway to developing a scientifically literate citizenry. science and education. 23, 285-302. doi: 10.1007/s11191-012-9503-3 lee, r.p. (2015). analysis of psychological determinants and factors influencing energy evaluations and preferences: implications for managing the human dimension of energy system (doctoral dissertation). tu bergakademie freiberg, germany. magnusson, s., krajcik, j., & borko, h. (1999). nature, sources, and development of pedagogical content knowledge for science teaching. in j. gess-newsome & n. g. lederman (eds.), examining pedagogical content knowledge: the construct and its implications for science education (pp. 95-132). boston: kluwer. mcintyre, d. (2005). bridging the gap between research and practice. cambridge journal of education. 35, 357-382. mctaggart, r. (1997). reading the collection. in r. mctaggart (ed.), participatory action research (pp. 1-12). albany, ny: suny press. ministry of national education of turkey (mone) (2013). i̇lköğretim kurumları fen bilimleri dersi öğretim programı [elementary schools science education curriculum for grades 3, 4, 5, 6, 7, and 8]. ankara: milli eğitim bakanlığı talim ve terbiye kurulu başkanlığı. ministry of national education of turkey (mone) (2017). fen bilimleri dersi öğretim programı (i̇lkokul ve ortaokul 3, 4, 5, 6, 7 ve 8. sınıflar) [science education curriculum for grades 3, 4, 5, 6, 7, and 8]. ankara: milli eğitim bakanlığı talim ve terbiye kurulu başkanlığı. nielsen, j.a. (2012). science in discussions: an analysis of the use of science content in socio-scientific discussions. science education, 96(3), 428-456. osborne, j., erduran, s., & simon, s. (2004). enhancing the quality of argumentation in school science. journal of research in science teaching. 41(10), 994-1020. doi: 10.1002/tea.20035 presley, m.l., sickel, a.j., muslu, n., merle-johnson, d., witzig, s.b., izci, k. et. al. (2013). a framework for socio-scientific issues based education. science educator, 22, 26-32. pitiporntapin, s., yutakom, n., & sadler, t.d. (2016). thai pre-service science teachers’ struggles in using socio-scientific issues (ssis) during practicum. asia-pacific forum on science learning and teaching, 17(2), 1-20. ratcliffe, m., & grace, m. (2003). science education for citizenship: teaching socio-scientific issues. philadelphia: open university press. sadler, t.d. (2009). situated learning in science educaton: socioscientfc issues as contexts for practce. studies in science educaton. 45(1), 1-42. doi: 10.1080/03057260802681839 sadler, t.d. (2011a). situating socio-scientific issues in classrooms as a means of achieving goals of science education. in t. d. sadler (ed.). socio-scientific issues in the classroom: teaching, learning and research (pp. 1-10). new york: springer. sadler, t.d. (2011b). socio-scientific issues-based education: what we know about science education in the context of ssi. in t. d. sadler (ed.). socio-scientific issues in the classroom: teaching, learning and research (pp. 355-369). new york: springer. sadler, t.d., friedrichsen, p., graham, k., foulk, j., tang, n., & menon, d. (2015). the derivation of an instructional model and design processes for socio-scientific issues-based teaching. paper presented at the annual meeting of the national association for research in science teaching, chicago il. sadler, t.d., & zeidler d.l. (2005). patterns of informal reasoning in the context of socio-scientific decision making. journal of research in science teaching, 42(1), 112-138. doi: 10.1002/tea.20042 sakschewski, m., eggert, s., schneider, s., & bögeholz, s. (2014). students’ socio-scientific reasoning and decision-making on energy-related issues-development of a measurement instrument. international journal of science education. 36(14), 2291-2313. doi: 10.1080/09500693.2014.920550 sandell, k., öhman, j., & östman, l. (2003). education for sustainable development-nature, school and democracy. malmö: studentlitteratur. sampson, v., & clark, d. (2011). a comparison of the collaborative scientific argumentation practices of two high and two low performing groups. research in science education, 41, 63-97. doi: 10.1007/s11165-009-9146-9 simon, s., & amos, r. (2011). decision-making and use of evidence in a socio-scientific problem on air quality. in t. d. sadler (ed.). socioscientific issues in the classroom: teaching, learning and research (pp. 167192). new york: springer. simonneaux, j., & simonneaux, l. (2012). educational configurations for teaching environmental socioscientific issues within the perspective of sustainability. research in science education, 42(1), 7594. doi:10.1007/s11165-011-9257-y journal of science learning article doi: 10.17509/jsl.v4i1.23378 90 j.sci.learn.2020.4(1).80-90 topçu, m.s. (2008). pre-service science teachers’ informal reasoning regarding socio-scientific issues and the factors influencing their informal reasoning (doctoral dissertation). middle east technical university, ankara. topçu, m.s. (2015). sosyobilimsel konular ve öğretimi [socioscientific issues and their instruction]. ankara: pegem akademi. topçu, m.s., muğaloğlu, e.z., & güven, d. (2014). fen eğitiminde sosyobilimsel konular: türkiye örneği [socioscientific issues in science education: the case of turkey]. educational sciences: theory and practice, 14(6), 2327-2348. venville, g.j., & dawson, v.m. (2010). the impact of a classroom intervention on grade 10 students’ argumentation skills, informal reasoning, and conceptual understanding of science. journal of research in science teaching. 47(8), 952-977. doi: 10.1002/tea.20358 von aufschnaiter, c., erduran, s., osborne, j., & simon, s. (2008). arguing to learn and learning to argue: case studies of how students’ argumentation relates to their scientific knowledge. journal of research in science teaching. 45(1), 101-131. doi: 10.1002/tea.20213 windschitl, m., thompson, j., braaten, m., & stroupe, d. (2012). proposing a core set of instructional practices and tools for teachers of science. science education, 96, 878-903. doi: 10.1002/sce.21027 western australia curriculum council (wacc). (1998). the curriculum framework for kindergarten to year 12 education in western australia. perth, western australia. zeidler, d.l., sadler, t.d, simmons, m., & howes, e. (2005). beyond sts: a research-based framework for socio-scientific issues education. science education, 89(3), 357-377. doi: 10.1002/sce.20048 zohar, a., & nemet, f. (2002). fostering students’ knowledge and argumentation skills through dilemmas in human genetics. journal of research in science teaching, 39, 35-62. a © 2021 indonesian society for science educator 257 j.sci.learn.2021.4(3).257-266 received: 24 august 2020 revised: 8 march 2021 published: 5 july 2021 promoting student's problem-solving skills through stem project-based learning in earth layer and disasters topic firdha sarah kartini1*, ari widodo2, nanang winarno1, lia astuti3 1department of science education, universitas pendidikan indonesia, bandung, indonesia 2department of biology education, universitas pendidikan indonesia, bandung, indonesia 3department of computer science and information engineering, national quemoy university, kinmen, taiwan *corresponding author: firdhasarahk@gmail.com abstract located in the pacific ring of fire, indonesia is endangered by natural disasters. through the changing of learning activity in a proper way by the application of stem project-based learning, future generations are expected to develop the knowledge and thinking skill to solve the problem. therefore, this study aimed to investigate implementing stem project-based learning on student's problem-solving skills. this research used a pre-experiment method with a one-group pretest-posttest design and essay questions as data collection tools. in this research, 30 7th-grade students at one school in bandung, indonesia, were chosen as the participants. the data analysis showed significant improvement between student's problem-solving skills pretest and post-test (n-gain=0.73). in addition, idea-finding is the most significant improvement during the fact-finding on the lowest. based on the result, stem project-based learning is recommended to improve students' problem-solving skills since they can use their integrated knowledge to solve a real-world problem. keywords stem project-based learning, problem-solving skills, earth layer and disaster 1. introduction indonesia has to struggle with the risk of volcanic eruption, earthquake, flooding, and tsunami since indonesia is situated on the pacific ring of fire within the australian plate, eurasian plate, pacific plate region, and the philippine plate that makes the nation becoming the world's most seismically active region. in indonesia, over 677 earthquakes have occurred in the past 365 days. based on earthquake track (2020), the 1,628 accidents, 60 people died, 545,000 were displaced, more than 9,000 were destroyed, and hundreds of public buildings have been destroyed in the last six months (bnpb, 2020). this phenomenon shows that problem-solving in disasters is still low with a large number of victims repeatedly. through education, disaster risk reduction by developing student attitudes, skills, and knowledge can be gained. in facing unexpected challenges, we must prepare the future generation to solve real-world problems such as natural disasters by studying the phenomenon, using their expertise, and adjusting under conditions to the latest modifications. however, that will never happen because we have never improved how the learning practice is carried out. in 2019, indonesia had a pisa ranking of 396 points in education, compared to an average of 489 in oecd countries (oecd, 2019). it indicates the low quality of indonesian education. one of the reasons is because the learning system only focuses on remembering. students now have excellent memorization, but they would not be able to adapt what they have learned to real-world problems (arisanti, sopandi, & widodo, 2016). the low quality of indonesia education also affects the percentage of students' problem-solving skills in central java. the percentage was 52.93% based on the test and indicated that students' problem-solving skill is categorized low (jua, sarwanto, & sukarmin, 2018). problem-solving is defined as a cognitive-affectivebehavior method involving finding the right ways to cope with daily life problems, and it leads people to solutions for the problems (maydeu-ollivares & d'zurilla, 1997). problem-solving is a crucial skill to affect any move in life, from essential to complex (deniz, 2004). use the problem solving skill test (psat) to assess the pupils' problemmailto:firdhasarahk@gmail.com journal of science learning article doi: 10.17509/jsl.v4i3.27555 258 j.sci.learn.2021.4(3).257-266 solving abilities and the abilities of science-processing. chang and weng (2002) designed and constructed the psat according to the imaginative problem solving (cps) model, emphasizing convergent (or critical) and divergent (or innovative) thinking skills of students contained by the following four-stage problem-solving processes; (1) fact-finding stage: students can find the number and variety of facts, (2) problem-finding stage: the students can recognize problems and degree of clarity problems, (3) idea-finding stage: students can put forward with number and variation of ideas and degree of creativity of the ideas; (4) solution-finding stage: the students can propose rationality, accuracy, and the possibility of solutions. indonesia curriculum emphasizes interdisciplinary dimensions that promote the learning process using logical, conceptual, and systemic approaches. triggered by documented low-performance mathematics and science rates, a new curriculum strategy is being created and named stem education. the stem curriculum strives to teach independent and specific skills of science, engineering, and mathematics. another method to address the essential for stem education with practical action is introducing stem project-based training, which relates to the learning paradigm constructed on the stem education approach and combined with the project-oriented set of course templates (tseng, chang, lou, & chen, 2013). stem project-based learning allows students to cultivate idea understanding and critical thinking skills in keeping with the theory of lifelong learning that corresponds to the four foundations of universal education; namely learning to know, learning to do, learning to live together, and learning to be (arisanti et al., 2016). stem project-based learning is divided into five stages, which can be adopted for the teacher based on lou, chou, shih, and chung (2017) described as the preparation stage, implementation stage, presentation stage, assessment, and correction stage. stem projectbased learning also combines science curriculum and engineering design concepts; this combination of design principles develops applicability in the real world. many studies have been conducted as concerns implementing stem project-based learning to see its effectiveness, including investigated problem solving skills such as hanif, wijaya, and winarno (2019), also research on improving student's creativity by using stem project-based learning. apriyani, ramalis, and suwarna (2019) have also researched analyzing the student's problem-solving skills using stem-based learning on direct current electricity. the research on the development and validation of science, technology, engineering, and mathematics (stem) is based on instructional material already carried by gustiani, widodo, and suwarma (2017). the study using stem projectbased learning on student concept mastery and creativity skills was also have done by arisanti et al. (2016). nonetheless, there have not been several studies on stem project-based learning research found to investigate student's problem-solving skills. besides, student's problem-solving skills in disasters, especially earthquakes, are still lacking. due to the demand for improving students' problem-solving skills, the researcher decided to conduct the research titled "promoting student's problem-solving skills through stem projectbased learning in earth layer and disasters." 2. method 2.1 research design this study's research method is the pre-experimental method, which uses one research group without the classroom's control. this research method is appropriate for this research, which investigates the stem projectbased learning as the independent variable for students' problem-solving skills as the dependent variable in learning earth layers and disaster topics. one group pretest posttest is the research design used, which observed and measured the impact of the treatment not only in the last part but also before the stem project-based learning treatment is applied. therefore, the research design to compare the pretest result to know their prior knowledge and after the concept gave the treatment of the final post-test from one group is possible. the design is as shown in table 1. 2.2 participant the characteristic of the selected schools is that have implemented the 2013 national curriculum in the learning process. the location where this research was carried out is in bandung city, indonesia. all 7th-grade students were the population of this study, and 30 students from school x are used as the sample. convenience sampling is the sampling method used in this research. the selected participants are chosen because they are available to be studied. there are 13 male and 17 female students with an age average of 12-13 years old. the distribution of participants shown in table 2. 2.3 research instrument in this research, the students' problem-solving skills were measured based on criteria on the problem solving table 1 experiment design o x o pretest treatment post-test table 2 participants distribution gender experiment class number of students percentage male 13 43.3% female 17 56.7% total 30 100% journal of science learning article doi: 10.17509/jsl.v4i3.27555 259 j.sci.learn.2021.4(3).257-266 ability test (psat) by chang and weng (2002). the psat was developed and built the creative problem solving (cps) model (osborn, 1963) that underlines the convergent (or critical) and divergent (or creative) thinking skills of students in the following four-stage problem-solving processes: fact-finding, problem-finding, idea-finding, and solution-finding (treffinger, isaksen, & stead-dorval, 2005). all questions are adapted from the earthquake problem titled 'bmkg: jepang sudah wajibkan bangunan anti gempa sejak 1990' in cnn indonesia (2019). the objective is then arranged and prepared before being validated, judged by the experts, and tested to students who have learned the earth layer and disaster topic before. in this situation, they are the grade 9 students from 3 classes. the validation of the test consists of a validity and reliability test using anates v4. consisting with 5 open-ended essay questions, the items are reliable with the reliability value (α) = 0.78 and the validity value (r) = 0.63. after being judged by experts, problem solving skills objective test is deemed suitable and therefore does not change. students' problem-solving skills pretest and post-test were then given a score to see the difference in essay questions as quantitative data. 2.4 data analysis each correct answer for a question is worth four marks in the objective test. however, an incorrect answer will be 0 marks. as for problem solving skills, that has four components to be measured. every component has different total marks. the most suitable answer is worth four marks, while a not related answer will obtain 0 marks. to further investigate how stem project-based learning has a significant outcome on student's problemsolving skills, a statistical approach has been used to measure the data derived from the pretest and post-test of students. the summary of the analysis results is presented in table 3. the statistical analyses used in this analysis used the program spss version 25.0. the normality test results are shown in table 4, on the value of the problemsolving skills test. because of the sig value. (p) is more significant than 0.05; a significance value gained is 0.345 for the pretest data and 0.114 for the post-test data. it can be assumed that the data are normally distributed. from the homogeneity test results shown in table 5, the stem literacy pretest and post-test data have a significance value more significant than the α = 0.05, which is 0.043, so that the data has a homogeneous variance. therefore, a null hypothesis (h0) is that there is no difference in mean scores between pretest and posttest. meanwhile, an alternative hypothesis (ha) is a difference in mean scores between pretest and post-test. according to the paired samples test output shown in table 6, the significance value (2-tailed) is 0.000, and it is less than 0.05, as explained beforehand. thus, the null table 3 summary of student's problem solving skills objective test component problem solving skills pretest post-test participant 30 30 �̅� 69 92 sd 8.36 7.19 highest score 87 97 lowest score 57 67 normality test (shapiro-wilk) significance 0.345 0.114 interpretation normal normal homogeneity test (levene's test) significance 0.043 interpretation homogenous paired t-test significance (2-tailed) 0.00 interpretation significant normalized gain test n-gain score 0.73 interpretation high table 4 results of normality test on students' problem-solving skills treatment kolmogorov-smirnova shapiro-wilk statistic df sig. statistic df sig. pre-test 0.211 30 0.116 0.930 30 0.345 post-test 0.184 30 0.200 0.895 30 0.114 table 5 homogeneity test of students' problem-solving skills levene statistics df1 df2 sig 0.085 1 50 0.043 journal of science learning article doi: 10.17509/jsl.v4i3.27555 260 j.sci.learn.2021.4(3).257-266 hypothesis (h0) is rejected, and the alternative hypothesis (ha) is accepted. the interpretation is that there is a difference between the mean scores of the pretest and post-test. after that, the n-gain score test is performed by calculating the scores pre-and post-test. n-gain score gained is 0.73, which in the range of 0.7 ≤ n-gain scores ≤ 1, and the category is a significant improvement based on the hake (1998) criteria shown in table 7. 2.5 research procedure there are three stages in research procedures. the first stage is the preparation stage, which includes problem identification and educational bodies' problem in the teaching and learning process. it also decides the investigated same independent and dependent variables to be in the research. next, the researcher needs to reflect on some components that will improve the research's strength, such as curriculum, learning instruments, and teaching schedules. after that, the researcher arranges research instruments and learning tools that will be used. finally, the experts judge those instruments and do the trial test of an objective test instrument to discover the instrument's quality. the next stage is the implementation stage by conducting the stem project-based learning in three meetings, involving four stages, which are the preparation stage, implementation stage, presentation stage, and evaluation stage. the learning activity was carried online due to the covid-19 pandemic, which allowed us to conduct the learning activity as usual. the application use is zoom meeting and google classroom. the teacher and student will also use the whatsapp application as the communication tool. pretest starts the first meeting activity to identify preliminary student skills. on the central part, the steps of stem project-based learning are conducted. they are started by the preparation stage, where the student has introduced the concept material and problem-related to the scientific phenomenon around them to investigate. in the second meeting, there is the implementation stage where students need to generate the solution by integrating their knowledge in science, technology, engineering, and mathematics. finally, they have to make the project design and made the prototype on their home. the presentation stage was also conducted in this second meeting. the selected student need to present their idea while the other gives feedback. the last meeting is used to conduct the evaluation stage. the student and teacher give a score, evaluate the weakness, and give some recommendations for a better future project. the posttest is given at the end of the learning activity to measure the result of the implementation of stem project-based learning, and the data gained is analyzed. 3. results and discussion according to whitten and graesser (2003), the central aspect that affects problem-solving skills is promoting students' capability to explain problems and analyze solutions. the learning activity using stem project-based learning in this research through four stages is the preparation stage, implementation stage, presentation stage, and evaluation stage. this stage requires students to seek and find out the general information and the data related to the earthquake. then, the teacher provides several questions to increase students' ability to think critically. in this stage, the problem-solving skills process emphasizes fact-finding and problem finding. this activity lets the student identify problem-related to scientific phenomena and the impact on the environment. the next stage of stem project-based learning is the implementation stage. students have to use their knowledge in science, technology, engineering, and mathematics to generate a solution to the problem. while in this stage, idea-finding and solution finding of the problem-solving process in a student is developed. finally, the student is trained to make the prototype solving problems, which is an earthquake-resistant building. in process making, students will face several problems and use their scientific knowledge to solve them. from four aspects shown in table 3, almost all aspects are categorized as a significant improvement in hake's rule based on their normalized gain result. this result represents in general, student's problem-solving is improved, and stem project-based learning is effective to enhance problem-solving skills. it is shown that the idea-finding aspect has the highest normalized gain in 0.94, represent the improvement of student generating ideas to solve the problem is the most influence on the student problem-solving process. this result is because stem project-based learning trained students to solve daily life problems through doing the project. in comparison, the fact-finding aspect showed the lowest improvement than others. the normalized gain is 0.62, although categorized as a medium improvement based on hake's rule, this aspect has the smallest result. however, table 6 result of paired t-test sample treatment paired differences 95% confidence interval of the difference t df sig paired pretest-posttest -19.247 -12.533 29 0.000 table 7 interpretation of indicator of problem-solving skills pretest post-test n-gain interpretation 69 92 0.73 high journal of science learning article doi: 10.17509/jsl.v4i3.27555 261 j.sci.learn.2021.4(3).257-266 it can be said that the student problem-solving process has the positive enhancement that they can make a connection on cognitive knowledge on stem projectbased learning activity (bhakti et al., 2020). all the problem-solving aspects will be described in further detail. 3.1 fact-finding according to chang and weng (2002), the fact-finding aspect refers to the problem-solving process requiring students to determine the information available and relevant to the topic. students more emphasized investigating the fact that exist in daily life as much as they can using observation and investigation. students also need to understand the consequences to the environment. the result is measured by the open-ended essay question in a case study on indonesia's earthquake and compared to other countries. students are asked to explain their perfection due to the problem offered. the result is shown in figure 1. there is an improvement in students' fact-finding aspect after implementing stem project-based learning, according to figure 1. student results are classified as high in the pretest, that the mean score is 80.4, then it increases by 13.3 points to the mean score of 93.3 in the post-test. based on table 1, the fact-finding aspect has the lowest improvement. nonetheless, this aspect has an n-gain score of 0.63, categorized as a medium improvement based on hake's rule. the study carried out by nuraziza and suwarna (2018) revealed that the percentage of the average score obtained by student's fact-finding is 44.44%, and it is included in the sufficient category. the students' science problemsolving abilities for the project-based learning model group (mpbp) were 26.67% highly qualified, 23.33% sufficiently qualified, and 50% less qualified. the scores of students' science problem-solving abilities for the conventional learning model group were 10% sufficiently qualified, 40% low-qualified, and 50% very poorly qualified. the average achievement of each problemsolving component in the project-based learning model (mpbp) was categorized sufficiently, while the conventional model learning is in the poor category. the stem based learning activity was applied in one junior high school in bandung. after the implementation, the students can find new facts related to the waste in their environment. throughout the fact-finding aspect, students are inquired to examine their understanding of the question and identify or mention all the components of information they may think about (treffinger et al., 2005). in this research, students have faced real-life study, and students can mention the fact and the information they get in the case discussed, as seen in figure 2. students can already identify the number and variety of facts to the earthquake in yogyakarta and suruga. they can use their prior knowledge to mention that the earthquake victim in yogyakarta is higher than in suruga, even though it has the same earthquake strength in skala richter. it is showed from student answers on the post-test in figure 2. their ability to analyze the scientific phenomena includes five ws and h (who, why, what, when, where, and how). they also need to make an effort to find out what happens behind any problem, as a parallel with the statement that students make a pattern on their efforts in addressing any problem (heffernan & teufel, 2018). however, student development in this aspect is helped in stem project-based learning on the preparation stage. students are introduced to earthquakes and other phenomena related to the earth layer and disaster deeply in this stage. many information relevant to the earthquake is provided through several engaging learning media such as video, animation, and powerpoint. however, it is impossible to identify it directly because the learning activity is running online. the information gained on the preparation stage will be used on the next stem project-based learning stage to generate their creative thinking skill to propose the solution. this evidence is in line with cropley (2006) that states fact-finding is the root of creative ideas. in project making, the fact-finding ability is also trained on the first stage of the earthquake-resistant building-making process. students are introduced to one figure 1 students' fact-finding result 80.4 93.3 60 70 80 90 100 pretest posttest m e a n s c o re test figure 2 student's answer in post-test of fact-finding question journal of science learning article doi: 10.17509/jsl.v4i3.27555 262 j.sci.learn.2021.4(3).257-266 video about the application of earthquake-resistant buildings in real life. they are forced to find out the various information about their project before making the prototype by themselves. after the teacher showed the video, students are asked some questions about the earthquake-resistant building directly? how is it works? what is needed to build it? where is it used? and other related questions. in the learning meeting, some students can answer it. while the other needs a little time to think to answer it. this result can be concluded that stem project-based learning can facilitate student fact-finding ability to improve because it is not only used for the learning topic but also in the project making process, so this results in the post-test increase. 3.2 problem-finding the student problem finding focused on recognizing the number of problems and the degree of clarity of describing problems (chang & weng, 2002). the teacher promotes the chance of students understanding, investigating, and discovering answers to their problems. students must be able to find the central problem through the text or photo presented by the teacher. furthermore, the targeted source can be on the internet, reviews, books, and tests can be used to improve comprehension. problem finding value projected idea finding quality, however for another task, the quality of the problems establish expected both idea finding excellence and novelty. nonetheless, several researchers asked participants to develop some potential issues (arreola & reiter-palmon, 2016). there is an improvement in students' problem-finding aspect after implementing stem project-based learning, according to figure 3. student result classified low since it is under the minimum criteria score in the pretest, which is the mean score is 59.3, then it increases to the mean score of 92.5 in the post-test. the fact-finding aspect improves in the medium category with an n-gain score of 0.81, categorized as a significant improvement based on hake's rule. according to the student, the problem raised is close to their daily life. then the student has a clear idea about the problem and its impact. another study has been carried out on undergraduate students participating in the information technology program registered in the academic year 2016 for the strategic data management exam, carried by netwong (2018). the relation of the learning achievement in problem-solving of the experiment between pretest and post-test was higher than the pretest at the point of significance of 0.05 problem-solving skills after study. stem education can develop the problem-solving skills of the students. the learners broke the community into project-dependent problems in the first phase. the implementation of the project was to establish novel approaches by using information technologies across organizational challenges. the student shared his or her views when carrying out the study. it can promote a learning culture by practice, work, project, and work more as a team. the high score of the problem-finding aspect indicates students' ability to find the study case's main problem in the learning activity. if students already possess these skills, they can automatically formulate problems resulting from errors in disaster response behavior. to evaluate student problem-finding, they are given the news of the real cause of indonesia's earthquake. then, they are required to analyze and mention the problem that may happen because of the earthquake. according to figure 4, students can mention their analysis in three aspects of life in detail. first, students can be highlighted by a reformulation of problem statements that contain only the main problem and finally found the cause. thus, following schlegel et al. (2019) statement, stem learning increases efficacy in identifying potential causes of problems. this result can be the cause of stem project-based learning to help students identify and analyze earthquake cases deeply. an alternative to the old-style curriculum, where learning is usually highly segregated by subject field, is the interdisciplinary stem project-based learning. in stem project-based learning, learning is more intuitive and appropriate for developing a deep understanding of the definition. therefore, in real life, the principles individually taught can then be remembered as valid figure 3 students' problem-finding result 59.2 92.5 50 60 70 80 90 100 pretest posttest m e a n s c o re test figure 4 student's answer in post-test problem-finding question journal of science learning article doi: 10.17509/jsl.v4i3.27555 263 j.sci.learn.2021.4(3).257-266 experiences. this evidence can motivate students to study more (guthrie, schafer, von secker, & alban, 2000). the student has a high motivation to solve the problem related to the earthquake because the phenomenon is familiar with their daily life. some of them already experience it and become the victims of this disaster. this finding is parallel with tseng et al. (2013) revealed that stem project-based learning could develop students' learning motivation, create meaningful learning, help students solve daily life problems, and support future careers. the stem project-based learning trained students to develop their ability in problem-finding on the implementation stage. this stage is where the student can build their project by themselves. there will be many problems to make this project do as it is dreamed of, the most substantial building during an earthquake. they are faced with two big problems: making the building comfortable to live in, the beautiful shape, and the material needed to make it. from several problems in this stage, they concluded that the main problem is choosing the suitable material for their project. this result shows that students already have a great problem-finding ability to use their knowledge to identify and find the central problem by integrating their knowledge in science, technology, engineering, and mathematics. lou et al. (2014) stated that pbl-stem would enable students to systematically integrate the facets of engineering and technology and gain in-depth knowledge of science and mathematics to improve students' expertise and experience to apply their knowledge to everyday life. 3.3 idea-finding the amount and diversity of ideas planned by the student using the student's creative ideas refer to the ideafinding ability (chang & weng, 2002). this stage calls for idea-producing and idea-development to pursue proposals. the development of ideas involves dreaming up preliminary proposals as soon as possible. according to the pretest and the post-test results in figure 5, idea-finding improvement shows a big difference with 29 points. in the pretest, the mean score is 69.2 and shows that the idea-finding of students is low. after the application of stem project-based learning, the post-test highly increase due to the result shows 98.3. the n-gain score of 0.94 also shows the highest students' improvement in the problem solving skills aspect based on 1 4.3 and categorized as a significant improvement in hake's rule. another study carried by yuliati, riantoni, and mufti (2019) in tulungagung, east java, found that after the performance of stem project-based learning, the experiment class has a higher n-gain score and categorized moderate improvement since it is 0.43 bigger than the control class, which only 0.19 and categorized low. students in the stem project-based learning class developed a science product to resolve a question indicated in the worksheet for the students. once a group displayed their concept in front of the community, it only allowed other groups to provide suggestions, opinions, or critique. only instructions to elaborate on the result of the stated problem are included in the worksheet. the different treatments in the two classrooms triggered different behaviors culminating in the significant enhancement in both courses' problem-solving abilities. the high score on the post-test shows that students already possess these skills, then students can find ideas that might be used to solve problems that arise from a lack of readiness to face earthquakes. the idea revealed is produced because of the deep understanding of the topic's principle and student creativity. a positive relationship between problem finding and idea finding outcomes of undergraduates was initiated by arreola and reiter-palmon (2016), but this correlation was different across diverse problem tasks. the previous result on the problem finding gives the good representative, which influences the idea-finding aspect to be greater than others. in figure 6, students can propose some possible ideas for the earthquake problem in detail. these proposed ideas are from their point of view and from adult and government views in facing these problems. this evidence can be concluded that stem project-based learning can improve student idea-finding ability. figure 5 students' idea-finding result 69.2 98.3 60 70 80 90 100 pretest posttest m e a n s c o re test a figure 6 student's answer in idea-finding question journal of science learning article doi: 10.17509/jsl.v4i3.27555 264 j.sci.learn.2021.4(3).257-266 in this research, the stem project based-learning trained students to develop their creativity through the implementation stage to use their knowledge in science, technology, engineering, and mathematics to solve the problem. in addition, it will enable students to have more responsibility for their learning. this result is in line with capraro and jones' (2013) statement that for building a deep understanding of the concept, stem project-based learning can also meet the diverse learning needs by enabling learners with responsibility towards their learning. students also focused on increasing their idea-finding related to the earthquake-resistant building along the project-making process in stem project-based learning. this stage involves students designing and assemble the prototype by a hands-on activity. the earthquake-resistant building should have a strong foundation and the lightest material on its wall. regarding this problem, students already propose some possible ideas to be used in their future prototype. all these ideas are directly presented in a learning activity in a zoom meeting, on the presentation stage of stem project-based learning. students can describe their idea clearly. student ideas on earthquakeresistant building prototypes can be seen in figure 7. this student used straw because it is the lightest material, and shear walls using a yellow straw are used on this design to reinforcing a building. the student answer is relevant to bigrentz (2020), which states that to resist collapse, buildings require rearranging the forces that travel through them during a seismic event used, sheer walls, cross braces, diaphragms, and moment-resisting edges are central to strengthening a building. 3.4 solution-finding students are asked to consider their innovative ideas and select the most creative ones during the solutionfinding process. some studies explore the identification of new ideas in the past few years (benedek et al., 2016). this aspect refers to evaluation and adoption. approval calls for determining and implementing the final solution. grohman, wodniecka, and kłusak (2006) led an exploratory analysis of how the self-rating of primary school students' creativity correlated with official creativity scores on a visual, verbal, and science mission. outcomes revealed students could distinguish their success through realms of innovation and across levels of efficiency. in addition to that, their scores were favorably correlated with the professional imagination scores. under this grohman, wodniecka, and kłusak's (2006) analysis, the proposals' creativity was reviewed. however, the innovation elements can be isolated to examine why some innovation aspects are valued rather than others. students are then able to apply the approach they have discovered by selecting the most innovative ideas. there is a significant improvement in students' solution-finding aspect after implementing stem project-based learning, according to figure 8. the normalized gain is 0.85 and is categorized as a significant improvement based on hake's rule. student result classified medium since it is slightly above the minimum criteria score in the pretest, which is the mean score is 70.8. then it increases to the mean score of 95.8 in the post-test that shows the improvement of the student's solution-finding ability. to evaluate student solution-finding, they are given the essay question that forces them to analyze the most suitable solution. it can be seen in figure 9, and students are already able to propose a complex solution from a different perspective. to produce solutions, this method figure 7 student earthquake resistant building prototype figure 8 students' solution-finding result 70.8 95.8 60 70 80 90 100 pretest posttest m e a n s c o re test figure 9 student's answer in solution-finding question journal of science learning article doi: 10.17509/jsl.v4i3.27555 265 j.sci.learn.2021.4(3).257-266 involves student imagination or divergent thinking abilities, and students can assess the evaluative abilities of divergent thinking tasks. that evaluative capacity is correlated with divergent thinking ability. using stem project-based learning, students are introduced to the presentation and evaluation stage. the student can also present their design and the final project to all the students and teachers. students need to mention their project strengths and weaknesses and receive advice from other students and teachers to make their project results better in the future. this learning activity developed student solution-finding. therefore, when they face another problem, they are already familiar with the evaluation and developing solution. earthquake resistant building is planned to resist the largest earthquake of a distinct possibility that is expected to happen at their location. this suggests that the loss of life can be minimized by preventing unusual earthquake buildings' collapse (bigrentz, 2020). their project needs to answer this challenge. however, along with the projectmaking process, the students also faced many problems choosing the suitable material, the right length, choosing the right glue, and another problem. because this project is an individual task, they have to solve it by themselves. if students already possess these skills, they can propose the best solutions to solve the problems. students will sense the utility as students consider the relationship between their expertise and the actual problem. this also improves their (cognitive) and (psychomotor) abilities. this finding is in line with the statement by henderson (2005) that the stem pbl explains the topic better conceptually. the method of product design enables students through stem to position and express their understanding of the concepts. therefore students with good understanding and skill have a better chance of solving the difficulties they face. 4. conclusion the implementation of stem project-based learning can develop students' problem solving skills. the n-gain result suggests a substantial improvement between the pretest and the post-test treatment can be verified. the idea-finding aspect has the most significant improvement, and the lowest improvement is in the factfinding aspect, although categorized as a medium improvement. this suggests that the problem-solving abilities of students are greatly enhanced by implementing stem project-based learning. stem project-based learning trained students to define the problem and analyze the solution using their knowledge in science, technology, engineering, and mathematics. according to these research results, some recommendations can be used as potential guides for other research. this is important to allow the student to adjust to the new treatment and for the instructor to fully understand the need of the student and what kind of instructions need to be provided to the students to apply their best stem project-based learning treatment, thus achieve better data and results. acknowledgment the authors acknowledge smp alfa centauri for permission to obtain the research data. references apriyani, r., ramalis, t. r., & suwarma, i. r. (2019). analyzing student's problem solving abilities of direct current electricity in stem-based learning. journal of science learning, 2(3), 85–91. https://doi.org/10.17509/jsl.v2i3.17559 arisanti, w. o. l., sopandi, w., & widodo, a. (2016). analisis penguasaan konsep dan keterampilan berpikir kreatif siswa sd melalui project based learning. eduhumaniora|jurnal pendidikan dasar kampus cibiru, 8(1), 82-95. arreola, n. j., & reiter-palmon, r. (2016). the effect of problem construction creativity on solution creativity across multiple everyday problems. psychology of aesthetics, creativity, and the arts, 10(3), 287–295. https://doi.org/10.1037/a0040389 bnpb. (2020). data informasi bencana indonesia (dibi). retrieved from https://bnpb.cloud/dibi/ benedek, m., nordtvedt, n., jauk, e., koschmieder, c., pretsch, j., krammer, g., & neubauer, a. c. (2016). assessment of creativity evaluation skills: a psychometric investigation in prospective teachers. thinking skills and creativity, 21, 75–84. https://doi.org/10.1016/j.tsc.2016.05.007 bhakti, y. b., astuti, i. a. d., okyranida, i. y., asih, d. a. s., marhento, g., leonard, l., & yusro, a. c. (2020). integrated stem project based learning implementation to improve student science process skills. journal of physics: conference series, 1464(1). https://doi.org/10.1088/1742-6596/1464/1/012016 bigrentz. (2020). how earthquake-proof buildings are designed. retrieved from https://www.bigrentz.com/blog/earthquake-proofbuildings capraro, m. m., & jones, m. (2013). interdisciplinary stem projectbased learning. stem project-based learning an integrated science, technology, engineering, and mathematics (stem) approach, 51–58. https://doi.org/10.1007/978-94-6209-143-6_6 chang, c. y., & weng, y. h. (2002). an exploratory study on students' problem-solving ability in earth science. international journal of science education, 24(5), 441–451. https://doi.org/10.1080/09500690110066502 cnn indonesia. (2019). bmkg: jepang sudah wajibkan bangunan anti gempa sejak 1990. retrieved from https://www.cnnindonesia.com/teknologi/20191003112012199-436323/bmkg-jepang-sudah-wajibkan-bangunan-anti-gempasejak-1990 cropley, a. (2006). in praise of convergent thinking. creativity research journal, 18(3), 391-404. deniz, m. (2004). investigation of the relation between decision making self-esteem, decision making style and problem solving skills of university students. eurasian journal of educational research (ejer), (15). earthquake track. (2020). recent earthquakes near indonesia. retrieved from https://earthquaketrack.com/p/indonesia/recent gustiani, i., widodo, a., & suwarma, i. r. (2017). development and validation of science, technology, engineering and mathematics (stem) based instructional material. in aip conference proceedings (vol. 1848, no. 1, p. 060001). aip publishing llc.. https://doi.org/10.1063/1.4983969 guthrie, j. t., schafer, w. d., von secker, c., & alban, t. (2000). contributions of instructional practices to reading achievement in https://doi.org/10.17509/jsl.v2i3.17559 https://doi.org/10.1037/a0040389 https://bnpb.cloud/dibi/ https://doi.org/10.1016/j.tsc.2016.05.007 https://doi.org/10.1088/1742-6596/1464/1/012016 https://www.bigrentz.com/blog/earthquake-proof-buildings https://www.bigrentz.com/blog/earthquake-proof-buildings https://doi.org/10.1007/978-94-6209-143-6_6 https://doi.org/10.1080/09500690110066502 https://www.cnnindonesia.com/teknologi/20191003112012-199-436323/bmkg-jepang-sudah-wajibkan-bangunan-anti-gempa-sejak-1990 https://www.cnnindonesia.com/teknologi/20191003112012-199-436323/bmkg-jepang-sudah-wajibkan-bangunan-anti-gempa-sejak-1990 https://www.cnnindonesia.com/teknologi/20191003112012-199-436323/bmkg-jepang-sudah-wajibkan-bangunan-anti-gempa-sejak-1990 https://earthquaketrack.com/p/indonesia/recent https://doi.org/10.1063/1.4983969 journal of science learning article doi: 10.17509/jsl.v4i3.27555 266 j.sci.learn.2021.4(3).257-266 a statewide improvement program. journal of educational research, 93(4), 211–225. https://doi.org/10.1080/00220670009598710 grohman, m., wodniecka, z., & kłusak, m. (2006). divergent thinking and evaluation skills: do they always go together?. journal of creative behavior, 40(2), 125–145. hake, r. r. (1998). interactive-engagement versus traditional methods: a six-thousand-student survey of mechanics test data for introductory physics courses. american journal of physics, 66(1), 6474. https://doi.org/10.1119/1.18809 hanif, s., wijaya, a. f. c., & winarno, n. (2019). enhancing students' creativity through stem project-based learning. journal of science learning, 2(2), 50. https://doi.org/10.17509/jsl.v2i2.13271 henderson, c. (2005). strategies for the development of student problem solving skills in the high school physics classroom. am. j. physics, 59(10), 891-897. heffernan, k., & teufel, s. (2018). identifying problems and solutions in scientific text. scientometrics, 116(2), 1367–1382. https://doi.org/10.1007/s11192-018-2718-6 jua, s. k. , sarwanto, & sukarmin. (2018). the profile of students' problem-solving skill in physics across interest program in the secondary school. in journal of physics: conference series (vol. 1022, no. 1, p. 012027). iop publishing. lou, s. j., chou, y. c., shih, r. c., & chung, c. c. (2017). a study of creativity in cac 2 steamship-derived stem project-based learning. eurasia journal of mathematics, science and technology education, 13(6), 2387–2404. https://doi.org/10.12973/eurasia.2017.01231a maydeu-olivares, a., & d'zurilla, t. j. (1997). the factor structure of the problem solving inventory. european journal of psychological assessment, 13(3), 206–215. https://doi.org/10.1027/10155759.13.3.206 nuraziza, r., & suwarma, i. r. (2018). menggali keterampilan creative problem solving yang dimiliki siswa smp melalui pembelajaran ipa berbasis stem [exploring the creative problem solving skills of junior high school students through stem-based science learning]. wapfi (wahana pendidikan fisika), 3(1), 55. https://doi.org/10.17509/wapfi.v3i1.10941 netwong, t. (2018). development of problem solving skills by integration learning following stem education for higher education. international journal of information and education technology, 8(9), 639–643. https://doi.org/10.18178/ijiet.2018.8.9.1114 oecd. (2019). pisa 2018 results: combined executive summaries, volume i, ii & iii. retrieved from https://www.oecd.org/pisa/combined_executive_summaries_p isa_2018.pdf osborn, a. f. (1963). applied imagination; principles and procedures of creative problem-solving. scribner. schlegel, r. j., chu, s. l., chen, k., deuermeyer, e., christy, a. g., & quek, f. (2019). making in the classroom: longitudinal evidence of increases in self-efficacy and stem possible selves over time. computers & education, 142, 103637. https://doi.org/10.1016/j.compedu.2019.103637 tseng, k. h., chang, c. c., lou, s. j., & chen, w. p. (2013). attitudes towards science, technology, engineering and mathematics (stem) in a project-based learning (pjbl) environment. international journal of technology and design education, 23(1), 87-102 https://doi.org/10.1007/s10798-011-9160-x treffinger, d. j., isaksen, s. g., & stead-dorval, k. b. (2005). creative problem solving: an introduction. prufrock press inc. whitten, s., & graesser, a. c. (2003). comprehension of text in problem solving. in j. e. davidson & r. j. sternberg (eds.), the psychology of problem solving (pp. 207–229). new york, ny: cambridge university press. yuliati, l., riantoni, c., & mufti, n. (2018). problem solving skills on direct current electricity through inquiry-based learning with phet simulations. international journal of instruction, 11(4), 123–138. https://doi.org/10.12973/iji.2018.1149a https://doi.org/10.1080/00220670009598710 https://doi.org/10.1119/1.18809 https://doi.org/10.17509/jsl.v2i2.13271 https://doi.org/10.1007/s11192-018-2718-6 https://doi.org/10.12973/eurasia.2017.01231a https://doi.org/10.1027/1015-5759.13.3.206 https://doi.org/10.1027/1015-5759.13.3.206 https://doi.org/10.17509/wapfi.v3i1.10941 https://doi.org/10.18178/ijiet.2018.8.9.1114 https://www.oecd.org/pisa/combined_executive_summaries_pisa_2018.pdf https://www.oecd.org/pisa/combined_executive_summaries_pisa_2018.pdf https://doi.org/10.1016/j.compedu.2019.103637 https://doi.org/10.1007/s10798-011-9160-x https://doi.org/10.12973/iji.2018.1149a a © 2020 indonesian society for science educator 123 j.sci.learn.2021.4(2).123-133 received: 22 maret 2020 revised: 10 desember 2020 published: 27 january 2021 nanotechnology attitude scale development study for pre-service science fields teachers tuba şenel zor1*, adnan kan2 1science education department, gazi university, turkey 2guidance and psychological counseling department, gazi university, turkey *corresponding author tubasenell@gmail.com abstract this study aims to develop a measurement tool to measure pre-service science fields teachers’ (pssft’) attitudes towards nanotechnology. for this purpose, a five-point likert-type scale consisting of 55 items was applied to 373 pssft who enrolled in science fields (science, chemistry, biology, physics) at the grades 1st, 2nd, 3rd, and 4th at a public university in turkey. the exploratory factor analysis (efa) determined that the scale had a 3-factor structure consisting of 24 items and the factors explained 55.854% of the attitude variable's total variance. verification of the model was tested by applying confirmatory factor analysis (cfa). the cfa data were obtained from 770 pssft enrolled in science fields at the grades 1st, 2nd, 3rd, and 4th at two different public universities in turkey. the results obtained from cfa were in agreement with the model obtained by efa. cronbach's alpha (cr-α) reliability of the scale was calculated to be 0.926. findings from the validity and reliability analyses show that the scale is a valid and reliable measurement tool that can measure pssft’ attitudes towards nanotechnology. keywords attitude, nanotechnology, pre-service science fields teachers, scale development 1. introduction attitude research has been one of the main topics of social psychology for many years (kağıtçıbaşı & cemalcılar, 2014; oppenheim, 2001). the reason for this great interest shown in the attitude studies is that the attitudes of the individuals affect both individual's social perceptions and behaviors (kağıtçıbaşı & cemalcılar, 2014). the accurately measuring of a person’s attitudes towards any subject or object firstly depends on correctly identifying this characteristic (tezbaşaran, 2008). it has been described by many researchers in the literature (allport, 1935; droba, 1933; i̇nceoğlu, 2010; tezbaşaran; 2008). according to thurstone (1931), attitude is the degree of emotions towards an object or person. for an individual, anything such as a house, neighbor, loved or disliked people, friends, and profession can be a psychological object. therefore, the individual may have certain attitudes towards them (kağıtçıbaşı & cemalcılar, 2014). attitudes are composed of three components: cognitive, affective, and behavioral (i̇nceoğlu, 2010). kağıtçıbaşı and cemalcılar (2014) described the cognitive component as the knowledge and thoughts that the individual has towards the attitude object, the affective component as the emotions of the individual towards the attitude object, and the behavioral component as the behavior tendency of the individual triggered by the attitude object. among these components constituting the attitude, it is generally assumed that an organization brings internal consistency. according to this, what the individual knows about the subject (the cognitive component) determines what kind of emotion they will approach it with (affective component) and what kind of an attitude they will put against it (the behavioral part) (i̇nceoğlu, 2010). attitudes have an essential place in many of our social life, such as education, business, politics, entertainment, fashion, marriage, and communication. that is the main reason for social psychologists’ efforts to measure attitudes (oppenheim, 2001). because measuring the attitudes and the level of attitudes individuals have towards the object or situation is a desirable situation in many areas (erkuş, 2003). various methods have been developed to measure attitudes that play such an important role in social interactions (kuppuswamy, 1965). however, they are not directly observable because most of the individual's attitudes are usually dormant and are expressed in the form mailto:tubasenell@gmail.com journal of science learning article doi: 10.17509/jsl.v4i2.23753 124 j.sci.learn.2021.4(2).123-133 of speech or behavior only when the relevant object is perceived. attitudes can be measured indirectly by making inferences from these observable behaviors of the individual. this measurement process requires an appropriate scale (erkuş, 2003; kuppuswamy, 1965). 1.1 nanotechnology and attitudes towards nanotechnology nanotechnology is a new, perhaps somewhat unclear, and controversial field attracting the attention of many nations and the community of scientists interested in science communication and society’s perceptions of science (stephens, 2005). it is also a rapidly developing interdisciplinary field to be a major influence on the lives of future generations (nerlich, clarke, & ulph, 2007). researches on nanotechnology show that there can be revolutionary developments in materials and manufacturing, electronics, medicine, health services, energy, biotechnology, information technologies, and national security. for this reason, nanotechnology is widely referred to as “the industrial revolution of the future” (bhushan, 2010; çıracı, 2006). at this point, society’s views will significantly impact research and development activities (nerlich, clarke, & ulph, 2007). in other words, the future position of nanotechnology may shape by society’s attitude towards nanotechnology (acceptance of nanotechnology, resistance to nanotechnology, or rejection of nanotechnology) (roco & bainbridge, 2001). therefore, it is considered that the measurement of attitudes towards nanotechnology is of great importance to fully utilize the potential of nanotechnology. however, the studies on nanotechnology attitudes' determination are relatively limited numbers when considered the relevant literature (bainbridge, 2002; lee, scheufele, & lewenstein, 2005; nerlich, clarke, & ulph, 2007). the studies on nanotechnology attitudes were carried out with participants of different ages and education levels using various data collection tools and methods. for example, bainbridge (2002) collected data from 3909 participants via an internet questionnaire. another research was conducted by lee, scheufele, & lewenstein (2005) in the us, and the data collected via a telephone survey. liang et al. (2015) compared public attitudes towards nanotechnology in the united states and singapore. the data collected through an online survey for the us and computer-assisted telephone interviewing software for singapore. the superior results of these studies showed that (i) the participants had high levels of enthusiasm for the potential benefits of nanotechnology and little concern about possible dangers (bainbridge, 2002); (ii) the participants’ awareness of nanotechnology generally was low, and the media was one of the strongest predictors of attitudes towards nanotechnology (lee, scheufele, & lewenstein, 2005); (iii) singaporeans tend to be more knowledgeable about and familiar with nanotechnology than the us public, and they also have more positive attitudes towards nanotechnology (liang et al., 2015). 1.2 significance a deeper understanding of society's attitude towards nanotechnology will undoubtedly prove beneficial in furthering nanotechnology's responsible development worldwide (zhang, wang, & lin, 2015). at this point, teachers and pre-service teachers educating the youth of society who will become the responsible decision-makers and leaders of the next generation can have a crucial role. teachers' knowledge, opinions, and beliefs directly influence their classroom practice (brickhouse, 1990; pajares, 1992). when nanotechnology's interdisciplinary nature (tessman, 2009) considers, it is concluded that determining especially science fields teachers and preservice teachers' attitudes towards nanotechnology is essential. therefore, it is necessary to determine the attitudes towards nanotechnology by using appropriate tools and providing the support needed to develop these attitudes following the results obtained. when the studies investigating the attitudes towards nanotechnology are examined, it is seen that the use of measurement tools that are specifically developed to measure the attitudes towards nanotechnology and which provide quantitative data are relatively limited. in line with this situation, the scarcity of the likert-type measurement tools designed to measure nanotechnology attitudes is also noteworthy (kurnaz & bayraktar, 2012; seçken, 2009; lan, 2012). thus, it can be said that a variety of measurement tools are needed. by looking at this deficiency, the present study aims to develop a scale that can be used to measure the attitudes of pssft towards nanotechnology. 2. method this study consisted of two main parts. in the first part of the study, the development of the nanotechnology attitude scale was focused. in this process, validity (efa for construct validity) and reliability analyzes explained in the following sections were carried out. in the second part, whether the scale developed in the first part gives the same structure on a different group with similar characteristics was examined (cfa for construct validity). 2.1. study group the first part of this study was conducted with 373 pssft enrolled in science (n=274), chemistry (n=46), biology (n=35), and physics (n=18) education departments at the grades 1st, 2nd, 3rd, and 4th at a public university in turkey’s central anatolia region. the second part of the study was conducted with 770 pssft enrolled in science (n=509), chemistry (n=101), biology (n=90), and physics (n=70) education departments at the grades 1st, 2nd, 3rd, and 4th, at two different public universities in turkey’s central anatolia region. journal of science learning article doi: 10.17509/jsl.v4i2.23753 125 j.sci.learn.2021.4(2).123-133 the following criteria were taken into consideration in the selection of this study group: ensuring heterogeneity to obtain variance: these universities have student diversity from different parts of turkey and the world. target group: the scale to be developed in this study is for pssft, and the selected universities have adequate pssft providing a sample size explained at the end of this section. nanotechnology experience: target groups have different experiences in nanotechnology. in general, 1st and 2nd grade pssft did not have any nanotechnology experience, whereas 3rd and 4th grade pssft encountered nanotechnology in different courses' content. in the literature about scale development, various researchers have reported different opinions about the sample size. for example, tinsley & tinsley (1987) suggested sample size should be 5-10 respondents per item up to about 300 subjects. according to comrey and lee (1992), the sample size might be evaluated as 50-very poor; 100-poor; 200-fair; 300-good; 500-very good; and 1000 or more excellent. in light of these previous studies, it can be concluded that the present study has an adequate sample size with 373 respondents for efa and 770 respondents for cfa. 2.2. development of the scale in the development of the scale, the scale development phases expressed by crocker & algina (2006) were taken as the basis. firstly, the literature about the structure, indicators, measurement of the attitude, and characteristics of attitude scales was reviewed. later, attitude scales developed by other researchers on similar and different topics were examined. considering these scales and the structure of the attitude, 69 items were listed to measure the attitudes towards nanotechnology. attention has been paid because these points represent the attitude's components, and the positive and negative expressions are at similar ratios (see table 1). after completing the item writing process, 14 items were removed from the scale and the expression of 5 items was rearranged in line with the field experts' opinions examining the items. as a result, a 55-item trial form was created. while setting the order of the items within the scale, attention has been paid to randomly distribute the items belonging to different components and reporting positive and negative attitudes. five response categories (strongly agree=5, strongly disagree=1) were formed to determine the agreement levels of the pssft on the items in the scale. by adding the instruction that includes the purpose of the scale and the application information, the trial scale form has been made ready. 2.3. collection of data stage first, the purpose of the study was explained to the pssft, and they were informed that participation in the study was based on volunteerism. the data collection process lasted for two weeks, during which the pssft used about 10-12 minutes to reply to the trial scale form. 2.4. analysis of data the data obtained by applying the trial scale to the pssft were analyzed by statistical package for the social sciences (spss) version 21 and linear structural relation statistics package program (lisrel) version 8.71. the spss is a computer program that can be used to calculate many of the descriptive (e.g., standard deviations, z scores, and correlations) and inferential (e.g., independent and repeated measures, t-tests, analysis of variance) statistics (fraenkel & wallen, 2009). lisrel is a computer program developed by jöreskog and sörbom (1978). it allows for various analyses such as path analysis, efa and cfa, cross-lagged panel analysis, and markov modeling (tinsley & tinsley, 1987). the values reverse items included in the scale were corrected by re-grading after the data was transferred to the software in data analysis. various analyzes were made on the obtained data to establish reliability and validity. these analyzes can be summarized as follows. efa and cfa to provide evidence of the scale’s validity: the factor analysis is often used to develop and validate psychometric instruments or testing theories about tools (tinsley & tinsley, 1987). to understand the factor analysis, it may help explain the concept of "factor" first. a factor is a linear combination or cluster of related observed variables representing a specific underlying dimension of a construct correlated with one another but mostly independent of other subsets of variables (pett, lackey, & sullivan, 2003; tabachnick & fidell, 2013). factor analysis attempts to achieve parsimony by explaining the maximum amount of shared variance in a correlation matrix using the smallest number of exploratory constructs known as a factor (field, 2013). there are two basic types of factor analysis: efa and cfa. efa aims to determine the factor structure or model for a set of variables (stevens, 2009). this analysis is implemented when the researcher does not know how many factors must clarify the interrelationships among a set of characteristics, indicators, or items (pedhazur & schmelkin, 1991; pett, lackey, & sullivan, 2003; tabachnick & fidell, 2013). in contrast, cfa is used to confirm a particular pattern of relationships predicted based on theory or previous analytic results (devellis, table 1 distribution of the items according to components of attitude items components total cognitive affective behavioral positive 13 12 17 42 negative 10 10 7 27 total 23 22 24 69 journal of science learning article doi: 10.17509/jsl.v4i2.23753 126 j.sci.learn.2021.4(2).123-133 2003). it is preferred when the researcher "knows" how many factors exist and whether they should be correlated. the researcher also generally forces items to load only on a specific factor and wishes to "confirm" a hypothesized factor structure with data (stevens, 2009). while making efa, generally more traditional statistical computer packages programs like spss, statistical analysis software, and bio-medical data package are usually used (pett, lackey, & sullivan., 2003). in this study, the efa was carried out through the spss program to determine the scale's factor structure and provide evidence for construct validity. in this process, rotated principal component analysis (pca) was used as an analysis method. the varimax (orthogonal) rotation was selected as the rotation method and was not limited to factor number. pca tries to explain the maximum amount of total variance (not just common variance) in a correlation matrix by transforming the original variables into linear components (field, 2013). rotation is ordinarily used after extraction to maximize high correlations between factors and variables and minimize low ones. in other words, rotation aims to obtain more specific factor loads (fl). varimax is a variancemaximizing procedure. the varimax rotation goal is to maximize factor loadings' variance by making high loadings higher and low ones lower for each factor (tabachnick & fidell, 2013). cfa can be achieved through various statistical programs that require the use of structural equation models (pedhazur & schmelkin, 1991). lisrel is the most commonly used program for this purpose (jöreskog & sörbom, 1993; tabachnick & fidell, 2013). in line with relevant literature, lisrel was used for cfa in this study. the kaiser-meyer olkin (kmo) coefficient and the barlett sphericity test to determine the suitability of the data obtained from the scale for pca: kmo coefficient (kaiser, 1970) is a statistical method which is used to determine whether the data and sample size are appropriate and adequate for the selected analysis. the kmo statistic varies between 0 and 1, and as the kmo coefficient approaches 1, it means that the data is suitable for the analysis, and 1 is the perfect fit (field, 2013). the acceptable kmo coefficient is expected to be greater than .5 (kaiser, 1974). in the parametric method, the measured property should have a normal distribution in the universe. the barlett sphericity test is a statistical technique used to check whether the data comes from a multivariate normal distribution. chi-square test statistic is obtained end of this test. if the chi-square test statistics has a significant value, it indicates that the data comes from the multivariate normal distribution (field, 2013; raykov & marcoulides, 2008). item-test correlations (itcs) were examined to provide evidence of item validity. the cr-α coefficient was calculated to provide evidence of reliability. there are various methods used to calculate the reliability coefficient. these methods are grouped under two headings; (i) the methods based on single test administration and (ii) the methods based on two test administrations, depending on the situation where the data to be used in estimating the reliability coefficient is obtained. the cr-α coefficient is figure 1 scree-plot graphic of the scale journal of science learning article doi: 10.17509/jsl.v4i2.23753 127 j.sci.learn.2021.4(2).123-133 one of the methods based on a single application. it can be used to estimate the reliability of the dichotomously scored items or items with a wide range of scoring weights. it is a measure of item scores' consistency with total test scores (crocker & algina, 2006). 3. result and discussion 3.1. results about the validity of the scale to collect information about the construct validity of the scale, rotated pca was used. the suitability of data for pca was examined with the kmo coefficient and barlett sphericity test. at the end of the examination, the kmo coefficient was calculated as 0.946, and the chi-square test statistic obtained with the barlett sphericity test was found significant (x2=4330.579, df=276, p<0.05). these values were accepted to indicate that the scale's data provided the prerequisites for factor analysis (kaiser, 1974), and efa was conducted to reveal the scale's factor structure. as a result of the efa performed on 55 items constituting the scale, it was determined that the items were collected in 8 factors having an eigenvalue greater than 1. these eight factors explained 58.99% of the variance of the table 2 the results of validity and reliability analysis of attitude scale towards nanotechnology items fl / itc / cr-αwir  f1 f2 f3 p o si ti v e c o m p o n e n t i54 if i have an opportunity, i organize nanotechnology activities. .803/.713/.884 i53 i would like to create a website/blog on nanotechnology. .799/.745/.882 i40 i research about nanotechnology. .763/.670/.888 i55 if i have an opportunity, i provide that nanotechnology is given as a course. .719/.608/.893 i49 i follow publications related to nanotechnology. .689/.606/.893 i29 i want to make a career in the nanotechnology field. .671/.697/.886 i16 i want to prepare a nanotechnology curriculum. .655/.662/.888 i34 i like talking about nanotechnology. .624/.657/.889 i48 i feel comfortable in nanotechnology themed training or activities. .600/.637/.890 b e n e fi t c o m p o n e n t i19 nanotechnology affects economic activities positively. .752/.647/.880 i13 nanotechnology provides to obtain more efficient products. .707/.625/.882 i30 i believe that nanotechnology will make our life easier. .688/.687/.877 i25 i find nanotechnology researches useful. .668/.687/.877 i27 i believe that nanotechnology researches are necessary. .640/.679/.877 i7 nanotechnology contributes to social development. .636/.650/.880 i1 nanotechnology increases the quality of life. .633/.596/.884 i22 nanotechnology is a revolutionary development. .628/.706/.875 i45 nanotechnology helps us to understand the natural world. .568/.577/.886 n e g at iv e c o m p o n e n t i26 i move away from the environment where nanotechnological applications are talked .762/.602/.777 i20 i change the channel when i meet news or advertisement related to nanotechnology on television. .707/.639/.769 i12 i get bored when i hear the news, advertising, etc. about nanotechnology. .680/.541/.790 i28 i get uncomfortable with nanotechnology research. .630/.699/.758 i23 i do not explain my opinions in conversations or discussions about nanotechnology. .624/.530/.793 i52 nanotechnology is not worth learning. .602/.468/.812 eigenvalue 5.161 4.668 3.576 ratio of variance explanation (%) 21.503 19.451 14.900 55.854 cr-α .899 .892 .813 .926 *cr-α when the item is removed journal of science learning article doi: 10.17509/jsl.v4i2.23753 128 j.sci.learn.2021.4(2).123-133 attitude variable. after initial efa, 29 items extract from the scale one by one, and efa has performed again after each removal. while the items were extracted, three criteria were taken into consideration. the first one is that the item does not load to any factor. as a rule of thumb, it is recommended to interpret the items with factor loadings of 0.30 (raykov & marcoulides, 2008) or 0.40 and above (field, 2013; stevens, 2009). for this reason, items that have factor loadings less than 0.40 were excluded from the scale because they did not provide this criterion. the second one is that the item is loaded to more than one factor. for example, if an item is loaded to two factors and the difference between the loads of these factors is less than 0.20 (or 0.15) (dilorio, 2005), this item is considered as cross-loading and removed from the scale. lastly, the third one, two items collected under one factor, is also removed from the scale because a factor must contain at least three items (comrey & lee, 1992). in this case, the remaining 24 items were found to have a 3-factor structure with an eigenvalue greater than 1 (kaiser, 1960). an eigenvalue represents the amount of information captured by a factor (devellis, 2003). the first factor's eigenvalue is 5.161, which accounts for 21.503% of the attitude variable's total variance. when we examine the items in this factor consisting of 9 items whose fl ranging from 0.803 to 0.600, it was found that the items generally contain positive behaviors and emotional tendencies towards nanotechnology, hence the factor called “positive component.” the second factor has an eigenvalue of 4.668, accounting for 19.45% of the attitude variable's total variance. when the items in this factor, consisting of 9 items whose fl are ranging from 0.752 to 0.568, were examined, it was found that the items generally contained cognitive tendencies for the benefits of nanotechnology in daily life. hence, the factor is called the “benefit component.” finally, the third factor's eigenvalue is 3.576, which accounts for 14.90% of the attitude variable's total variance. when the items in this factor consisting of 6 items whose fl ranged from 0.762 to 0.602 were examined, it was found that these items generally had negative behavioral, emotional, and thought tendencies towards nanotechnology. the factor was named as a “negative component” for this reason. these three factors together account for 55.854% of the attitude variable’s total variance. these findings can be used as proof that the developed scale provides construct validity at a satisfactory level and that it has a three-factor structure (see table 2). figure 1 shows the scree plot graph as another widely used method for determining the scale's factor number (cattell, 1966). the screen test is also based on eigenvalues. still, it uses their relative values rather than absolute values as a criterion, and then suddenly drop in eigenvalue magnitude can be used for determining the “right” number of factors (cattell, 1966). figure 1 shows three factors whose eigenvalues are more than 1 (kaiser, 1960). the eigenvalues obtained after the varimax rotation techniques are presented in table 2. finally, the parallel analysis, which uses eigenvalue to determine the number of factors, was performed. in this method, many datasets are simulated, each containing the same number of variables and respondents. the difference is that each dataset is made up of completely random data. the eigenvalues are computed for each dataset, and the mean is computed across all of the datasets (johnson & morgan, 2016). the eigenvalues of the factors obtained from the actual data set are compared with the eigenvalues of the factors obtained due to the parallel analysis. as a result, the eigenvalues of the factors derived from the actual data set are expected to be higher (devellis, 2003). the results based on 1,000 parallel analyses conducted with o’connor spss syntax (o'connor, 2000) were represented in table 3. table 3 shows that the first three factors have eigenvalues higher than the mean eigenvalues from the randomly generated datasets. for this reason, the three factors should be evaluated according to parallel analysis. when the harmony between the results obtained from the three methods used to determine the number of factors is considered, it can be concluded that the scale has a threefactor structure. after analyzing the validity of the structure, itcs were calculated concerning the item validity of the scale. result calculations made, itcs were found to have values ranging between; 0.745 and 0.606 for the items in the first factor, 0.706 and 0.577 for the items in the second factor, and 0.699 and 0.468 for the items the third factor (see table 2). these values indicate that there is a positive and high level of correlation between item-test scores (cohen, 1992). these findings can be used as proof that the item validity is sustained and that the items are measuring the same structure. table 3 results obtained from the parallel analysis factor number eigenvalues obtained from real data mean eigenvalues from 1,000 datasets of random numbers 1 9.178 1.567259 2 2.826 1.467013 3 1.401 1.398923 4 .869 1.340365 table 4 the correlation between the scores of the factors constituting the scale and the scores obtained from the whole scale factors factor 1 factor 2 factor 3 whole scale factor 1 1.00 .556** .414** .853** factor 2 1.00 .635** .866** factor 3 1.00 .756** journal of science learning article doi: 10.17509/jsl.v4i2.23753 129 j.sci.learn.2021.4(2).123-133 the correlation between the scores of the factors constituting the scale and the scores obtained from the whole scale indicates the presence of a high level and meaningful relationship (0.414 ≤ r ≤ 0.866, p < 0.01) in the positive direction (cohen, 1992). this finding can be regarded as a demonstration that the three factors establishing the scale are components of the attitude towards nanotechnology. the detailed data on fl of the items and itcs are presented in table 2. the correlation coefficients between the scores of the factors constituting the scale and the scores obtained from the whole scale are presented in table 4. verification of the model for the 3-component structure obtained from the efa was tested by applying cfa through the lisrel software. for the fit indices, the values of χ2/df (chi-square/degree of freedom), root mean square error of approximation, normed fit index, comparative fit index, adjusted goodness-of-fit index, goodness-of-fit index, and standardized root mean square residual were also examined. the goodness-of-fit indices obtained without performing any modification on the model as a result of cfa performed on the structure consisting of three factors, and 24 items are as follows: [χ2=983.46, df=249, χ2/df=3.95 (p=0.000), root mean square error of approximation=0.062, normed fit index=0.96, comparative fit index=0.97, adjusted goodness-of-fit index=0.88, goodness-of-fit index=0.90, standardized root mean square residual=0.053]. when the model’s goodness-of-fit indices were examined, it can be said that the ratio of chi-square value to the degree of freedom (χ2/df=3.95) was acceptable (kline, 2005), root mean square error of approximation had a good fitness (brown, 2006; jöreskog & sörbom, 1993), comparative fit index and normed fit index values had excellent fitness (sümer, 2000; thompson, 2004), standardized root mean square residual value had an good fitness (brown, 2006; byrne, 1994), the goodness-offit index value has an excellent agreement, and adjusted goodness-of-fit index values were at acceptable levels (jöreskog & sörbom, 1993). in conclusion, these findings can be interpreted that efa's model agrees with the structure obtained by cfa. the model obtained as a result of cfa was given in figure 2 with standardized solution values. finally, the numbers of the items in the final form of the scale and the mean, standard deviation, skewness, and kurtosis values are presented in table 5. 3.2. results about the reliability of the scale the cr-α reliability coefficient was calculated to provide evidence of the reliability of the scale. in this process, all of the factors establishing the scale and the whole scale are table 5 descriptive statistics of the items in the final form of the scale item no �̅� s skewness kurtosis in trial form in final form statistic std. error statistic std. error i1 i1 4.321 .6737 -.705 .126 .598 .252 i7 i2 4.116 .7932 -1.022 .126 1.831 .252 i12 i3 3.759 .9044 -.864 .126 .757 .252 i13 i4 4.046 .7336 -.524 .126 .451 .252 i16 i5 3.142 1.0389 -.085 .126 -.444 .252 i19 i6 3.814 .8285 -.291 .126 -.166 .252 i20 i7 3.987 .8934 -.860 .126 .717 .252 i22 i8 3.908 .8964 -.651 .126 .307 .252 i23 i9 3.517 .8961 -.351 .126 .082 .252 i25 i10 4.005 .7441 -.953 .126 2.260 .252 i26 i11 4.035 .8242 -1.052 .126 1.633 .252 i27 i12 4.105 .7912 -.976 .126 1.553 .252 i28 i13 4.172 .8153 -1.182 .126 1.918 .252 i29 i14 3.103 1.0877 -.069 .126 -.289 .252 i30 i15 4.040 .8302 -.785 .126 .733 .252 i34 i16 3.472 .9768 -.434 .126 -.108 .252 i40 i17 3.035 .9899 -.071 .126 -.618 .252 i45 i18 3.694 .8536 -.435 .126 .265 .252 i48 i19 3.485 .8717 -.481 .126 .434 .252 i49 i20 3.054 1.0303 -.036 .126 -.590 .252 i52 i21 4.005 1.0347 -1.008 .126 .352 .252 i53 i22 2.919 1.0074 .180 .126 -.374 .252 i54 i23 3.075 1.0211 -.030 .126 -.501 .252 i55 i24 3.395 1.0586 -.447 .126 -.281 .252 journal of science learning article doi: 10.17509/jsl.v4i2.23753 130 j.sci.learn.2021.4(2).123-133 handled separately. cr-α reliability of the first factor is calculated as 0.899, the second factor as 0.892, the third factor as 0.813, and the whole scale as 0.926. these findings can be used as evidence that the scale has a satisfactory level of reliability (bland & altman, 1997). the details of the data obtained from the reliability analysis are presented in table 2. 3.3 discussion nieswandt (2005) stated that an overview of attitude definitions in related literature shows that attitude is not a uni-directional term but a multi-directional construct consisting of affective, cognitive, and behavioral components. therefore, it is a selection of researchers to define their understanding of attitude depending on the figure 2 the model obtained as a result of confirmatory factor analysis journal of science learning article doi: 10.17509/jsl.v4i2.23753 131 j.sci.learn.2021.4(2).123-133 research objectives. in this study, based on the multidirectional structure given in the literature, the items were written for all cognitive, affective, and behavioral components in the item writing process. according to the findings obtained from the analysis, the developed scale also included items for three dimensions. it is considered that these findings support the view of the multi-directional structure. it was also found that these components' items were placed in different proportions in the factors forming the scale. the items related to cognitive and behavioral components were relatively higher. according to erkuş (2003), the number and structure of attitude components may vary depending on the statistics used in structure validation, the selection of indicators used to measure the attitude structure's components, and the nature of the attitude object used. within this study's scope, a new scale that aims to measure pre-service science teachers' attitudes towards nanotechnology has been developed. participants' attitudes towards nanotechnology at various ages and levels of education were measured using various scales that have been developed. for example, kurnaz and bayraktar (2012) have developed a scale that can determine high school students' attitudes towards nanotechnology subjects. the developed scale has a 2-factor structure consisting of 19 items. these factors, which account for 73.3% of the attitude variable's total variance, were named by researchers as “valuation to nanotechnology” and “nanotechnology awareness.” the cr-α reliability coefficient of the scale was 0.88. moreover, in the study conducted by seçken (2009), a measurement tool was developed to determine pre-service chemistry teachers' attitudes towards nanotechnology. the developed scale has a 4-factor structure consisting of 16 items. these factors, which account for 59.89% of the attitude variable’s total variance, were named by the researcher as “labor,” “anxiety,” “life,” and “education.” the cr-α reliability coefficient of the scale was 0.86. furthermore, lan (2012) has developed a scale that can measure k-12 teachers’ attitudes towards nanotechnology. the developed scale has a 3-factor structure consisting of 23 items. these factors, which account for 64.11% of the attitude variable’s total variance, were named by the researcher as “the importance of nanotechnology,” “affective tendencies in science teaching,” and “behavioral tendencies towards teaching nanotechnology.” the cr-α reliability coefficient of the scale was 0.94. although the items in the factors of the scale developed in the present study are similar to the items in the related studies' sub-dimensions, these scales differ in terms of the obtained factor structures and naming of the factors. it is seen that the structure of attitude scales for nanotechnology differs considerably from each other, and a complete consensus among researchers cannot be obtained from this point of view. that may be due to different perspectives that researchers have adopted in the scale development process. besides, due to the limited number of likert-type scale development studies that can be used to determine attitudes towards nanotechnology, a clear and standard structure may not yet be achieved. the current study's focus group is pssft, which is a different aspect than the studies in the literature. it is thought that the study can contribute to the related literature in this regard. 4. conclusion in this study, a scale was developed that can be used to measure the attitudes of pssft towards nanotechnology. it has been determined that the developed scale has a 3factor structure consisting of 24 items. the researchers named these factors as “positive component,” “benefit component,” and “negative component.” the factors' eigenvalues are determined as 5.161, 4.668, and 3.576, respectively, and they account for 21.503%, 19.451%, and 14.90% of the attitude variable’s total variance, respectively. also, three factors account for 55.854% of the attitude variable’s total variance. as a result of the reliability analysis conducted to establish evidence of the scale's reliability, the cr-α reliability of the scale was found to be 0.926; and cr-α reliability for each factor found as 0.899, 0.892, and 0.813, respectively. the lowest score that can be taken from the scale is 24, and the highest score is 120. it can be said that the attitude towards nanotechnology increases as the score obtained from the scale increases. this study shows that the psychometric properties of the nanotechnology attitude scale have a valid and reliable structure to measure the attitudes of pssft towards nanotechnology. however, this study has several limitations. first, the nanotechnology attitude scale findings presented in this study are limited to the data obtained from the study group selected from turkey’s central anatolia region. in further studies, the researchers using different groups with similar characteristics can contribute to generalizing the results of this study and the validity and reliability of the scale. second, the pssft is identified as the target group in developing the scale in this study. if the scale is intended to use for pre-service teachers or teachers in different areas and different countries or cultures, validity and reliability analyses should be performed with the data that is going to be obtained from these groups. as another important point, to better understand the structure of the attitude towards nanotechnology, researchers can develop an attitude scale for participants of different ages and education levels or apply the scales available in the literature to contribute to these scales' validity and reliability. journal of science learning article doi: 10.17509/jsl.v4i2.23753 132 j.sci.learn.2021.4(2).123-133 notes a part of this study was presented at the iii. ines international education and social science congress in alanya, 28 april-1 may, 2018. abbreviations efa, exploratory factor analysis; cfa, confirmatory factor analysis; kmo, the kaiser-meyer olkin; spss, statistical package for the social sciences; lisrel, linear structural relation statistics package program; fl, factor loads; itcs, item-test correlations; cr-αwir, cr-α when the item is removed; pssft, pre-service science fields teachers; pca, principal component analysis. references allport, g. w. (1935). attitudes. in c. murchison. (ed.), handbook of social psychology (pp. 798-844), worcester, ma: clark university press. bainbridge, w. s. (2002). public attitudes toward nanotechnology. journal of nanoparticle research, 4, 561–570. doi: 10.1023/a:1022805516652 bhushan, b. (2010). introduction to nanotechnology. in b. bhushan. (ed.), springer handbook of nanotechnology (pp. 1-19). springer handbooks. springer, berlin, heidelberg. bland, j. m., & altman, d. g. (1997). statistics notes: cronbach's alpha. bmj, 314, 572. doi:10.1136/bmj.314.7080.572 brickhouse, n. w. (1990). teacher beliefs about the nature of science and their relationship to classroom practices. journal of teacher education, 41, 53-62. brown, t. a. (2006). confirmatory factor analysis for applied research. new york: guilford press. byrne, b. m. (1994). structural equation modeling with eqs and eqs/windows: basic concepts, applications, and programming. thousand oaks, ca: sage publications. cattell, r. b. (1966). the scree test for the number of factors. multivariate behavioral research, 1(2), 245–276. çıracı, s. (2006). ulusal nanoteknoloji araştırma merkezi (unam): nanobilim ve nanoteknolojide türkiye’nin bir mükemmeliyet merkezi. bilim ve teknik: yeni ufuklara, 2-4. http://www.bilimteknik.tubitak.gov.tr/system/files/nanoteknoloj i_t.pdf cohen, j. (1992). a power primer. psychological bulletin, 112(1), 155–159. doi:10.1037/0033-2909.112.1.155 comrey, a. l., & lee, h. b. (1992). a first course in factor analysis (2nd ed.). new jersey: lawrence erlbaum associates. crocker, l., & algina, j. (1986). introduction to classical and modern test theory. holt, rinehart and winston, 6277 sea harbor drive, orlando, fl 32887. devellis, r. f. (2016). scale development: theory and applications (vol. 26). sage publications. dilorio, c. (2005). measurement in health behaviour: methods for research and education. san francisco: jossey-bass. droba, d. d. (1933). the nature of attitude. journal of social psychology, 4(4), 444–463. doi: 10.1080/00224545.1933.9919338 erkus, a. (2003). psikometri uzerine yazilar. turk psikologlar dernegi yayinlari, 24, 34-72. field, a. (2013). discovering statistics using ibm spss statistics (4th ed.). london: sage publications. fraenkel, j. r., & wallen, n. e. (2009). how to design and evaluate research in education (7th edition). new york: mcgraw-hill. i̇nceoğlu, m. (2004). tutum algı iletişim (attitude, perception, communication). ankara: elips kitap kesit tanıtım. johnson, r. l., & morgan, g. b. (2016). survey scales: a guide to development, analysis, and reporting. new york: the guilford press. jöreskog, k. g., & sörbom, d. (1993). lisrel 8: structural equation modeling with the simplis command language. chicago: ssi scientific software international inc. jöreskog. k. g., & sörbom. d. (1978). analysis of linear structural relationships by the method of maximum likelihood. chicago: national educational resources. kağitçibaşi, ç., & cemalcilar, z. (2014). dünden bugüne insan ve insanlar: sosyal psikolojiye giriş. baskı-cilt: sistem matbaacılık yılanlı ayazma yolu, (8), 146-147. kaiser, h. f. (1960). the application of electronic computers to factor analysis. educational and psychological measurement, 20, 141-151. doi:10.1177/001316446002000116 kaiser, h. f. (1974). an index of factorial simplicity. psychometrika, 39, 31-36. doi:10.1007/bf02291575 kaiser, h. f. (1970). a second-generation little jiffy. psychometrika, 35, 401–415. kline, r. b. (2005). principles and practice of structural equation modeling (2nd ed.). new york: guilford press. kuppuswamy, b. (1965). an introduction to social psychology. bombay, india: asia publishing house. kurnaz,, m. a., & bayraktar, g. (2012). nanoteknoloji tutum ölçeği: geliştirilmesi, geçerliliği ve güvenilirliği. bayburt üniversitesi eğitim fakültesi dergisi, 7(1), 41–53. http://dergipark.gov.tr/befdergi/issue/23151/247273 lan, y-l. (2012). development of an attitude scale to assess k-12 teachers' attitudes toward nanotechnology. international journal of science education, 34(8), 1189-1210, doi:10.1080/09500693.2011.651657 lee, c., scheufele, d. a., & lewenstein, b. v. (2005). public attitudes toward emerging technologies. science communication, 27(2), 240– 267, doi:10.1177/1075547005281474 liang, x., ho, s. s., brossard, d., xenos, m. a., scheufele, d. a., anderson, a. a., hao, x., & he, x. (2015). value predispositions as perceptual filters: comparing of public attitudes toward nanotechnology in the united states and singapore. public understanding of science, 24(5) 582–600, doi.org/10.1177/0963662513510858 nerlich, b., clarke, d. d., & ulph, f. (2007). risks and benefits of nanotechnology: how young adults perceive possible advances in nanomedicine compared with conventional treatments. health, risk & society, 9(2), 159-171, doi:10.1080/13698570701306856 nieswandt m. (2005). attitudes toward science: a review of the field. in: cobern w.w. et al. (eds.) beyond cartesian dualism: encountering affect in the teaching and learning of science (pp.41-52). science & technology education library, vol 29. springer, dordrecht. o'connor, b. p. (2000). spss and sas programs for determining the number of components using parallel analysis and velicer's map test. behavior research methods, instrumentation, and computers, 32, 396402. doi.org/10.3758/bf03200807 oppenheim, a. n. (2001). questionnaire design, interviewing and attitude measurement. london: continuum. pajares, m. f. (1992). teacher’ beliefs and educational research: cleaning up a messy construct. review of educational research. 62, 307-332. https://doi.org/10.3102/00346543062003307 pedhazur, e. j., & schmelkin, l. p. (2013). measurement, design, and analysis: an integrated approach. psychology press. pett, m. a., lackey, n. r., & sullivan, j. j. (2003). making sense of factor analysis: the use of factor analysis for instrument development in health care research. ca: sage. raykov, t., & marcoulides, g. a. (2008). an introduction to applied multivariate analysis. new york: routledge-taylor & francis. roco, m. c., & bainbridge, w. s. (2001). societal implications of nanoscience and nanotechnology. nset workshop report, arlington, virginia. seçken, n. (2009). attitudes scale towards “nano technology” for chemistry teachers. journal of baltic science education, 8(3), 157-171. http://oaji.net/articles/2014/987-1404740194.pdf https://doi.org/10.1023/a:1022805516652 https://doi.org/10.1023/a:1022805516652 https://doi.org/10.1136/bmj.314.7080.572 http://www.bilimteknik.tubitak.gov.tr/system/files/nanoteknoloji_t.pdf http://www.bilimteknik.tubitak.gov.tr/system/files/nanoteknoloji_t.pdf http://psycnet.apa.org/doi/10.1037/0033-2909.112.1.155 http://doi.org/10.1080/00224545.1933.9919338 https://doi.org/10.1177%2f001316446002000116 https://doi.org/10.1007/bf02291575 http://dergipark.gov.tr/befdergi/issue/23151/247273 https://doi.org/10.1080/09500693.2011.651657 http://doi.org/10.1177/1075547005281474 https://doi.org/10.1177%2f0963662513510858 https://doi.org/10.1080/13698570701306856 https://doi.org/10.3102%2f00346543062003307 http://oaji.net/articles/2014/987-1404740194.pdf journal of science learning article doi: 10.17509/jsl.v4i2.23753 133 j.sci.learn.2021.4(2).123-133 stephens, l. f. (2005). news narratives about nano s & t in major u.s. and non-u.s. newspapers. science communication, 27(2), 175-199. doi:10.1177/1075547005281520 stevens, j. p. (2009). applied multivariate statistics for the social sciences (5th ed.). taylor & francis group. sümer, n. (2000). yapisal eşitlik modelleri: temel kavramlar ve örnek uygulamalar. türk psikoloji yazilari. 3(6), 49-74. tabachnick, b., & fidell, l. (2013). using multivariate statistics, (6th ed). saddle river, nj: pearson education. tessman, j. m. (2009). students' conceptions of nanoscience phemonena: the beginning of a nanoscience concept inventory [doctoral dissertation], purdue university, west lafayette, indiana. tezbaşaran, a. (2008). likert tipi ölçek hazırlama kılavuzu (3. sürüm ekitap). erişim tarihi, 15. ankara: türk psikologlar derneği publications. thompson, b. (2004). exploratory and confirmatory factor analysis: understanding concepts and applications. washington, dc: american psychological association, doi:10.1037/10694-000 thurstone, l. l. (1931). the measurement of social attitudes. the journal of abnormal and social psychology, 26(3), 249-269, doi: 10.1037/h0070363 tinsley, h. e. a., & tinsley, d. j. (1987). uses of factor analysis in counseling psychology research. journal of counselling psychology, 34(4), 414-424. zhang, j., wang, g., & lin, d. (2015). high support for nanotechnology in china: a case study in dalian. science and public policy, 43(1), 115– 127, doi: 10.1093/scipol/scv020 https://doi.org/10.1177%2f1075547005281520 http://psycnet.apa.org/doi/10.1037/10694-000 http://psycnet.apa.org/doi/10.1037/h0070363 http://psycnet.apa.org/doi/10.1037/h0070363 http://doi.org/https:/doi.org/10.1093/scipol/scv020 microsoft word 2020 4 (2) kalemkus.docx a © 2021 indonesian society for science educator 113 j.sci.learn.2021.4(2).113-122 received: 28 august 2020 revised: 27 november 2020 published: 27 january 2021 comparative effects of argumentation and laboratory experiments on metacognition, attitudes, and science process skills of primary school children jale kalemkuş1*, şule bayraktar2, sabahattin çiftçi3 1science education, kafkas university, kars, turkey 2science education, giresun university, giresun, turkey 3department of basic education, necmettin erbakan university, konya, turkey *corresponding author: jale.kalemkus@yahoo.com abstract the purpose of this study is to investigate and compare the effects of laboratory experiments and argumentation-based science teaching on science process skills, metacognitive awareness levels, and attitudes towards the science of 4th-grade elementary school students. in this study conducted according to the quantitative research method, a pre-test and post-test quasi-experimental design was used with two experimental groups and a control group. students from three classes of an elementary school participated in the study (n = 98). "science process skills test," "what do i really think about science scale," and "metacognitive awareness scale" were employed to collect data for the research. the study results showed that the science process skills of the 4th-grade students improved significantly in both experimental groups, which were taught by employing experiments and argumentation. students' metacognitive awareness levels and attitudes towards science developed in all three groups. however, it was observed that the development was higher in the groups in which science teaching based on experiments and argumentation-based science teaching was performed, compared to the control group. keywords argumentation, experiments, metacognition, science process skills, science teaching 1. introduction since the development of information technology facilitated access to information, today's schools aim to help students acquire higher-order skills rather than directly giving them the information. the 21st century requires individuals to have digital-age literacy, critical thinking, creative thinking, effective communication, and high efficiency (ncrel, 2003). therefore, today's schools' primary goal is to equip students with these skills by making necessary arrangements in educational environments by utilizing teaching approaches considered useful in acquiring these skills. one of these approaches is argumentation-based teaching. the importance of experiments is well known, and they are preferred by teachers in almost all levels of education in science education. however, although it has been shown to develop students' higher-order cognitive skills, teachers are reluctant to practice argumentation (jimenez-aleixandre, rodriguez, & duschl, 2000; newton, driver, & osborne, 1999). furthermore, argumentation is hardly utilized in primary grades. a review of research on argumentation shows that although a significant amount of research has been done at the middle and high school level, the number of studies conducted with primary school children is limited (bağ & çalık, 2017). more studies on the applicability of argumentation-based teaching in primary grades are needed. furthermore, comparing the effectiveness of experiments and argumentation on particular cognitive and affective variables is valuable since the questions of journal of science learning article doi: 10.17509/jsl.v4i2.27825 114 j.sci.learn.2021.4(2).113-122 "which is more effective?" or "could one of them be an alternative to the other?" will be answered. the purpose of this research is to investigate and compare the effects of laboratory experiments and argumentation-based science teaching on science process skills, metacognitive awareness level, and attitudes toward science of 4th-grade students. 1.1. theoretical framework the argumentation process involves making claims, using data to support the claims, warranting the claims with scientific evidence, and further backing the warrants (simon, erduran, & osborne, 2006). accordingly, an argument is developed when a claim is made, providing evidence that supports this claim. (simon et al., 2006; van eemeren & grootendorst, 2004). according to toulmin (2003), an argument is an assertion and its accompanying justification. in a simple argument, there are three elements: data, claim and warrant. toulmin also provides a more complex argument structure consisting of data, claim, backing, warrant, rebuttal, and qualifier, as seen in figure 1. among these elements, data refer to evidence or specific information used to support the claim. a claim is an assertion that the individual would like to prove. warrants explain the relationship of data with the claim. the backing is the generally widely accepted basic assumptions that justify strengthening the warrant. qualifiers represent the special cases where the claims are correct and the limits of accuracy of the claim. on the other hand, rebuttals represent special cases where the claim is not valid (driver, newton, & osborne, 2000; lazarou, 2010; roberts & gott, 2010; simon et al., 2006; von aufschnaiter, erduran, osborne, & simon, 2008). an example covering all toulmin's argument model elements is given in figure 2 (kalemkus, bayraktar, & çiftçi, 2019). as an element of scientific language, argumentation is essential both in the creation and transmission of scientific knowledge (jimenez-alexiandre et al., 2000). considering that argumentation is one of the dimensions of acculturation in scientific discourse, it should be encouraged in science education (jimenez-aleixandre & erduran, 2007). by developing arguments and evaluating others' arguments, students can build an understanding of how scientific knowledge is created (driver et al., 2000). figure 1 toulmin's argument model (toulmin, 2003: 97) figure 2 an example covering all the elements of toulmin's argument model journal of science learning article doi: 10.17509/jsl.v4i2.27825 115 j.sci.learn.2021.4(2).113-122 inclusion of argumentation in the teaching process supports students' critical thinking skills, improves their knowledge, thoughts, and judgment, and increases their ability to use scientific language (osborne, erduran, simon, & monk, 2001). according to driver et al. (2000), argumentation in science classes can help students understand the social dimension of science, improve their understanding of science's epistemology, and develop conceptual understanding and research skills. students find a chance to compare their ideas and others' on a particular topic during the argumentation process. in a condition that their ideas contradict others, they evaluate the others' claims based on the supporting evidence. suppose the supporting data, warrants, and backings are sufficient. in that case, students realize that their previous conceptions are not acceptable and reorganize their conceptual framework to accommodate the new knowledge, indicating that conceptual change occurs. the fact that science generally includes abstract concepts, using teaching methods based on active learning seems to be mandatory. laboratory experiments are essential in science education since they engage students with hands-on learning experiences (özdener, 2005). experiments could be defined as actions carried out in an appropriate environment with the necessary tools and equipment to demonstrate a known fact or test a hypothesis. in a learning environment based on laboratory experiments, students will have the opportunity of establishing a cause-effect relationship for the phenomena they encounter, identifying the problem and thinking about both causes and solutions of this problem, designing experiments to implement the solutions they thought of, and performing these experiments to draw conclusions based on the data they collected. using experiments, students would have a chance to know the world of scientists. students, who observe their teacher or peers while constructing and testing a hypothesis, might develop a tendency to construct their hypotheses and test them. while testing the hypothesis, students think about variables that might affect the result, collect data, and reach the result by evaluating the data obtained. in this way, the students acquire higher-level cognitive skills by establishing a cause-effect relationship. if the students' hypothesis is not valid, they enter a different thinking process and feel a need to search for different solutions for the problem or reconsider their variables. when students need to construct a different hypothesis about the problem, they refer to social communication with peers and reach different solutions, thoughts, or ideas by interacting with them. students' constructing and testing hypotheses enable them to discover the way the knowledge is constructed. in this process, where the student is active, learning by doing is supported. the students' psychomotor skills are also supported by the experiments carried out as the students experience the process similar to that of scientists, their interest, curiosity, and motivation towards science increase. experiments carried out both in the classroom and in the laboratory environment lead students to gain the ability to explain the events they encounter in daily life using scientific language. also, through the experiences, the knowledge acquired by students becomes more concrete. furthermore, students could relate the knowledge with their existing concepts, and meaningful learning takes place. science process skills have been defined as a set of practical skills for many scientific disciplines and reflect the skills scientists use in scientific research (anagün & yaşar, 2009; padilla, 1990). science process skills are grouped under two categories: necessary process skills and integrated process skills. basic process skills are fundamental for scientific inquiry. integrated skills, on the other hand, are grounded on the necessary skills and more complex (padilla, 1990). necessary process skills are recording data, classifying, measuring, communicating, observing, using space-time relationship between number and space, estimating (predicting), and drawing conclusions. integrated process skills are: hypothesizing, determining/controlling variables, interpreting data, designing/conducting experiments, modeling, and defining operationally. many reports on learning science emphasized that students should not only gain conceptual and quantitative information but also need to develop science process skills such as hypothesizing, designing experiments and making conclusions based on data and observations, and working with other people as a team to solve complex and open-ended problems (etkina, karelina, & ruibal-villasenor, 2008; yang & heh, 2007). metacognition is defined by flavell (1976) as the information that a person has about their cognitive processes and products of these processes. for example; if i realize that i have more difficulty in learning a than b; if i think i should check again before i conclude that c is correct; if i think i should carefully examine each one before deciding which one is the best for a multi-choice job, this is a metacognitive process (flavell, 1976). flavell (1979) believes that cognitive monitoring interventions occur through interactions between four concepts: metacognitive experience, goals/tasks, metacognitive knowledge, and strategies/actions. an individual acquires metacognitive knowledge as a cognitive creature related to humans and their various cognitive tasks, goals, actions, and experiences. a child who is aware that, unlike his friend, he is better in mathematics than essay writing can be given as an example of this knowledge. metacognitive experiences are cognitive or affective experiences that accompany and relate to any cognitive intervention (flavell, 1979). it is an example of a metacognitive experience to think of the journal of science learning article doi: 10.17509/jsl.v4i2.27825 116 j.sci.learn.2021.4(2).113-122 possibility of failure in some attempts to take place or to think that the previous attempt was performed very well (flavell, 1979). aims/tasks represent the goal of cognitive intervention. actions/strategies, on the other hand, actions/strategies explain the cognition or behaviors used to achieve them (flavell, 1979). an individual's positive or negative emotional tendency towards objects, people, places, events, and ideas expresses the concept of attitude (i̇pek & bayraktar, 2004). having a positive attitude towards any lesson increases the student's motivation to learn that lesson. thus, academic success can be expected to increase. research by downing and filer (1999) and zeidan and jayosi (2015) reveal a significant positive relationship between attitudes and science process skills towards science. 1.2. literature review a study by asterhan and schwarz (2007) examining the effect of argumentation on understanding concepts related to evolution determined that students participating in the argumentation process acquired more knowledge and retained the knowledge compared to the control group students. it was determined that the control group participants either lost their gains or failed to improve their conceptual understanding. von aufschnaiter et al. (2008) examined the argumentation processes and cognitive development of secondary school students in science and sociology courses. the research results determined that the students used their previous experiences and knowledge when they participated in the argumentation. besides, students reinforced their existing knowledge and could elaborate on science concepts with such activities. kıngır (2011) investigated the effects of argumentation-based science learning approach on students' understanding of the concepts of chemistry and their academic achievement. the study results showed that this approach is more effective in understanding concepts related to chemical change and mixtures than traditional instruction. türkoguz and cin (2014) concluded that argumentation-based teaching utilizing concept cartoons was more effective in increasing students' science process skills than traditional instruction. similar results were found by gültepe and kılıç (2015) for chemistry class. aslan (2016) showed that laboratory applications based on argumentation improved students 'science process skills, found to be more effective, specifically in the students who have a low level of science process skills. furthermore, the attitudes of the students in both groups towards the laboratory course increased positively. karakuş and yalçın (2016), in their meta-analysis study, concluded that argumentation-based science teaching has a positive and very large scale in terms of both academic achievement and science process skills. aydın and kaptan (2014) determined that the students' metacognition and logical thinking skills in the group where the lessons were based on argumentation were positively affected. the inquiry laboratory's effect on the development of metacognitive skills among chemistry students was examined by kipnis and hofstein (2008). during the research activities, it was observed that the students used their metacognitive abilities at various stages of the research process. based on these results, the researchers stated that inquiry laboratory applications could provide students with metacognitive skills opportunities. durmuş and bayraktar (2010) found that both the experimental and conceptual change texts were more effective in overcoming the misconceptions than traditional instruction. kanlı and yağbasan (2008) investigated the deductive laboratory approach's effectiveness and the 7e model-based laboratory approach in developing science process skills. results revealed that the science process skills of the students in both groups improved. aydoğdu, buldur, and kartal (2013) examined the effect of open-ended and closedended experiments on acquiring science process skills. it was determined that there was a significant difference in the students' primary and integrated science process skills in both groups. the researchers stated that it could be said that open-ended science experiments based on scenarios are more effective in terms of developing primary and integrated science process skills than laboratory activities performed using closed-ended experiments. in the study conducted by celep and bacanak (2013), it was carried out to get teachers' opinions about scientific process skills and acquire these skills. as a result of the interviews with the teachers; it was determined that they believe that the laboratory method, inventive path, 5e model, and guess-observe-explain teaching methods are useful in gaining scientific process skills. it was also determined that teachers believed that question-answer technique, brainstorming, discussion, six-hat technique, demonstration experiment, open-ended experiment, deductive experiment, inductive experiment, project, and case study techniques were influential developing scientific process skills. in the study conducted by freedman (2001), laboratory programs to increase scientific knowledge achievement and develop an attitude towards science were examined. as a result of the research, it was determined that the students who regularly conduct laboratory activities have higher scientific knowledge achievement levels than the students without laboratory experience. however, it was determined that there was no significant difference in attitude towards science between the two groups. in the study conducted by çetin and şahin-taşkın (2015), the effects of verbal feedback effectively given by journal of science learning article doi: 10.17509/jsl.v4i2.27825 117 j.sci.learn.2021.4(2).113-122 the teacher in the learning-teaching process on the metacognitive awareness, academic achievement, and attitudes of the primary school students were examined. at the end of the study, it was determined that the useful feedback of the teacher in the learning-teaching process significantly affected the students' academic achievement, attitudes towards the lesson, and metacognitive awareness. this research examines the effects of science teaching with experiments and argumentation-based science teaching on the scientific process skills, metacognitive awareness levels, and attitudes toward science in 4th-grade primary school students. for this purpose, answers to the following questions were sought: 1. what is the effect of science teaching with experiments on primary school 4th-grade students' scientific process skills, metacognitive awareness, and attitudes towards science? 2. what is the effect of argumentation-based science teaching on primary school 4th-grade students' scientific process skills, metacognitive awareness, and attitudes towards science? 3. what is the effect of science teaching conducted with the current curriculum on primary school 4th-grade students' scientific process skills, metacognitive awareness, and attitudes towards science? 4. do the effects of science teaching with experiments, argumentation-based science teaching, and science teaching carried out according to the current curriculum on the scientific process skills, metacognitive awareness, and attitudes towards the science of primary school 4thgrade students differ? 2. method 2.1. research design this study adopted a pre-test and post-test quasiexperimental design with two experimental groups and one control group. to separately evaluate the effects of argumentation-based science teaching and experimental science teaching on variables (metacognitive awareness, scientific process skills, and attitude towards science) and to make a comparison between argumentation-based teaching and experimental teaching, research was conducted in two different experimental groups. the first experimental group (e1) was taught by utilizing experiments, the second experimental group (e2) was taught by utilizing argumentation activities, and the control group was taught by following the current curriculum. 2.2. research group the research was carried out with 98 elementary school students studying 4th grade of a public school in kars, turkey. the first experimental group (e1) consisted of 29 students (15 girls, 14 boys), the second experimental group (e2) consisted of 34 students (17 girls, 17 boys), and the control group consisted of 35 students (12 girls, 23 boys). 2.3. data collection instrument "what i really think of science" scale: "what i really think of science" scale consisting of 21 items, developed by pell and jarvis (2001) and adapted into turkish by kırıkkaya (2011), was used to determine students' attitudes towards science. cronbach alpha coefficient was calculated for this study as 0.794. metacognitive awareness inventory for children: metacognitive awareness inventory for children (maic), which was developed by sperling, howard, miller, and murphy (2002) to measure the metacognitive skills of 3rd–5th-grade students, and adapted into turkish by karakelle and saraç (2007), was used to determine the metacognitive awareness levels of primary school 4thgrade students. in this study, the cronbach alpha coefficient was calculated as 0.89. science process skills test: to determine the science process skills of 4th-grade students and to evaluate the developments in these skills, multiple-choice test items for measuring classification, measurement, observation, collecting data, space/time relation, prediction, determining variables, interpreting data, drawing a conclusion, hypothesis, modeling, and designing table 1 pre-test scores on the data collection tools by participant groups (anova) the data collection tools groups descriptive statistical anova n x ss f p "what i really think of science" scale experiment 1 29 40.45 6.015 0.149 0.862 experiment 2 34 39.68 7.619 control group 35 39.63 5.976 metacognitive awareness inventory for children experiment 1 29 22.97 6.62 0.521 0.596 experiment 2 34 24.38 5.609 control group 35 23.37 5.047 science process skills test experiment 1 29 16.14 5.579 0.462 0.632 experiment 2 34 15.03 5.26 control group 35 16.17 5.737 *significant at the level of p<.05 journal of science learning article doi: 10.17509/jsl.v4i2.27825 118 j.sci.learn.2021.4(2).113-122 experiment skills were developed by the researchers, based on the review of the relevant literature. there are 27 items in this multiple-choice test. each item has four different options and one correct answer. two faculty members working in science education, three science and technology teachers, and two classroom teachers were consulted for expert review of the test. after receiving expert opinions and making revisions in the test items in line with these opinions, the test was applied to 254 primary school 4th-grade students. at the end of the application, scoring was made by giving 1 point to each correct answer given by the students to the test items. each wrong answer was given to the students' test items, and the items left blank or marked with more than one option were scored by giving 0 points. after scoring for each item, item difficulty indexes and item discrimination indices of the test items were calculated. the test items' discrimination indexes ranged between 0.34 and 0.77, and item difficulty indexes varied between 0.28 and 0.85. the kr-20 reliability coefficient of the test was calculated as 0.82. 2.4. data collection process the research was carried out during the instruction period of the "let's know the matter" unit of the science class for 4th-grade students and was completed in thirteen weeks, including pre-test and post-test studies. in the first experimental group (e1), closed-ended and open-ended experiments were carried out with group and demonstration experiments. the experiments were planned together with the first experimental group teacher, and the necessary tools and equipment were provided in advance (see appendix a for a sample). in the second experimental group (e2), various argumentation activities such as competing theories with concept cartoons, expressions tables, and experiment reports were used for the instruction (see appendix b for a sample). no intervention was made to the lessons in the control group, but it was carried out by following the current curriculum with approaches other than experiments and argumentation methods. 2.5. data analysis one way analysis of variance (anova) was used to test whether there are significant differences among e1, e2, and control group students, concerning metacognitive awareness scale, science process skills test, and "what i really think of science" scale for both pre-test and post-test. 3. result the anova test results to determine whether statistically significant differences existed among the groups regarding the scores on the data collection instruments before the intervention are shown in table 1. as seen in table 1, the anova test results showed no statistically significant difference among the three different groups regarding attitudes, metacognitive awareness, and science process skills. the post-test mean scores for the students in the e1, e2, and control group on the "what i really think of science" scale, metacognitive awareness inventory for children, and science process skills test post-test are shown in table 2. anova test results revealed that there were statistically significant differences among the groups for all three tests. scheffe test was used to determine which groups caused the difference. according to the results of the scheffe test at table 3 for "what i really think of science" scale, "metacognitive awareness inventory for children inventory", and "science process skills test" scores of the participant groups showed that the significant differences were between the students in both experimental groups and the control group (p<.05). there was not a significant difference between the scores of the two experimental groups. 4. discussion this research was conducted to investigate the effects of laboratory experiments and argumentation-based science teaching on 4th-grade students' science process skills, meta-cognitive awareness levels, and attitudes table 2 post-test scores on the data collection tools by participant groups (anova) the data collection tools group descriptive statistical anova n x ss f p "what i really think of science" scale experiment 1 29 57.14 4.462 39.158 0.000 experiment 2 34 54.59 5.411 control 35 43.91 8.455 metacognitive awareness inventory for children experiment 1 29 33.62 1.45 24.05 0.000 experiment 2 34 32.94 2.37 control 35 27.57 5.95 science process skills test experiment 1 29 24.48 2.68 19.972 0.000 experiment 2 34 23.09 4.18 control 35 17.69 6.06 *significant at the level of p<.05 journal of science learning article doi: 10.17509/jsl.v4i2.27825 119 j.sci.learn.2021.4(2).113-122 towards science. results of the study showed that both experiments and argumentation activities were more effective than traditional instruction. the studies conducted on the experimental method support the results obtained in this study. in the study conducted by kanlı and yağbasan (2008), the 7e model-based laboratory approach's effectiveness and the deductive laboratory approach in developing scientific process skills were examined. at the end of the study, it was determined that both the 7e model-centered laboratory approach and the deductive laboratory approach improved the students' scientific process skills. in the study conducted by bilen and aydoğdu (2012), the activities prepared based on the strategy of "predict-observation-explain" in the general biology laboratory were examined by comparing it with the validation laboratory approach to the development of scientific process skills. for the research, while the proposed laboratory approach was carried out with the experimental group, the validation laboratory approach was applied to the control group. at the end of the research, it was observed that there was an improvement in the scientific process skills of both the experimental group students and the control group students. it was also determined that this improvement was more in the experimental group. in environments where laboratory experiments occur, students can experience science skills such as hypothesizing, designing an experiment to test their hypothesis, determining and controlling the experiment variables, conducting the experiment, observing, classifying, and recording data. when these opportunities are provided, it will be possible to expect for the students to develop their science process skills over time. when the literature is examined, it is seen that there is a study that examines the effect of the experimental method on metacognition. this study was conducted by ulu and bayram (2014). this study investigated whether the laboratory applications in the science and technology course with activities based on the science writing tool cause a difference in terms of metacognitive knowledge and skills. at the end of the research, there was a significant difference in favor of the experimental group in terms of explanatory knowledge, methodological knowledge, conditional knowledge, planning, and cognitive strategy dimensions of the students' metacognitive knowledge and skills. still, there was no difference between the experimental and control groups in terms of self-control, self-assessment, and selfmonitoring. students may not feel a need to overthink their self-learning when they are passive receivers of information. however, they can follow their self-learning processes when they are given opportunities to be more active. therefore, the development of metacognitive awareness is expectable in an environment that includes experimental activities. previous studies determined a positive relationship between attitude and science process skills. one of these studies was done by downing and filer (1999). in the study, the relationship between teacher candidates' scientific process skills and their attitudes towards science was examined. at the end of the study, it was determined that there is a significant positive relationship between primary school teacher candidates' science process skills table 3 scheffe test the data collection tools group (i) group (j) variationin average (i-j) standard error p "what i really think of science" scale experiment 1 experiment 2 2.550 1.631 0.299 control 13.224* 1.620 0.000 experiment 2 experiment 1 -2.550 1.631 0.299 control 10.674* 1.554 0.000 control experiment 1 -13.224* 1.620 0.000 experiment 2 -10.674* 1.554 0.000 metacognitive awareness inventory for children experiment 1 experiment 2 0.680 0.987 0.790 control 6.049* 0.981 0.000 experiment 2 experiment 1 -0.680 0.987 0.790 control 5.370* 0.941 0.000 control experiment 1 -6.049* 0.981 0.000 experiment 2 -5.370* 0.941 0.000 scientific process skills test experiment 1 experiment 2 1.395 1.168 0.493 control 6.797* 1.160 0.000 experiment 2 experiment 1 -1.395 1.168 0.493 control 5.403* 1.112 0.000 control experiment 1 -6.797* 1.160 0.000 experiment 2 -5.403* 1.112 0.000 *significant at the level of p<.05 journal of science learning article doi: 10.17509/jsl.v4i2.27825 120 j.sci.learn.2021.4(2).113-122 and their attitudes towards science. another research was conducted by zeidan and jayosi (2015). in the study, the relationship between students' attitudes towards science and their scientific process skills was examined. at the end of the study, it was determined that there is a significant positive relationship between scientific process skills and attitude towards science. based on this finding, it is possible to link the development of attitudes towards students' science to develop science process skills. additionally, experiments in teaching environments lead students to a position where they could create knowledge by being physically and mentally active rather than passive receivers of information. when students become active in teaching environments, they feel the lesson is fun and develops positive attitudes. results of this study revealed that argumentationbased science teaching also has a positive effect on science process skills. previous research results support this result. in the study conducted by çınar and bayraktar (2013), the effect of argumentation-based science teaching on students' scientific process skills was examined. it was determined that the students' scientific process skills in the group in which the argumentation-based science teaching was carried out were statistically significantly higher than the students in the group where the current teaching was applied. in the study conducted by türkoguz and cin (2014), the effect of argumentation based on concept cartoons activities on students' scientific process skills was examined. at the end of the study, it was determined that the students' scientific process skills in the experimental group in which argumentation-based teaching based on concept cartoons was carried out developed more than the control group students. in the study conducted by gültepe and kılıç (2015), the effect of scientific argumentation on students' scientific process skills in chemistry teaching was examined. as a result of the research, it was determined that both the traditional teaching approach and the argumentation-based teaching approach improved students 'scientific process skills. simultaneously, it was determined that argumentationbased teaching was more effective than the traditional teaching approach in developing students' scientific process skills. in the study conducted by aslan (2016), the effect of argumentation-based laboratory applications on scientific process skills and attitude towards laboratory course was examined. at the end of the study, it was determined that argumentation-based laboratory applications improve students' scientific process skills and more effective in developing students' skills with lower scientific process skill levels. it was determined that the students' attitudes in both groups towards the laboratory course increased positively with the application. through argumentation, students are involved in the process of making claims. to support a claim, they need to access some data, make observations, design and apply experiments, record the data, and interpret it. besides, students have to determine the variables that will affect their claims to determine their claims' limitations and refutations. students feel the need to use these skills for their claims and evaluate others' claims. through argumentation activities, students use science process skills in experimental teaching, which will improve these skills over time. in the study conducted by aydın and kaptan (2014), argumentation on the metacognition and logical thinking skills in teacher candidates' education was examined. at the end of the research, it was determined that the students' metacognitive and logical thinking skills in the group where the lessons were conducted based on argumentation were positively affected. in the study conducted by erenler (2017), the effect of argumentbased inquiry research applications on pre-service teachers' metacognitive awareness was examined. at the end of the study, it was determined that the argumentbased inquiry research method applications were statistically significant in all sub-dimensions of metacognitive awareness. in the study conducted by ulu (2019), the argumentation-based science learning approach on students' metacognitive knowledge and skills was investigated. laboratory activities were carried out in the control group based on the traditional approach, in the experiment-1 group based on the open inquiry-based argumentation-based science learning approach, in the experiment-2 group, based on the guided inquiry-based argumentation-based science learning approach. at the end of the study, two results were obtained. the first of these is that argumentation-based science learning-based laboratory applications are more successful than traditional-based laboratory applications in increasing students' metacognitive knowledge and skills. the other result is that the guided inquiry-based argumentationbased science learning laboratory applications in increasing students' metacognitive knowledge and skills are more successful than the open inquiry-based argumentation-based science learning laboratory applications. argumentation practices might lead students to evaluate their self-learning. it is possible for the student to question his or her learning, claiming "i learned" or i did not learn" and evaluate the argument in the light of the supporting evidence similar to the questioning process of evaluating whether a claim is true or false in the context of the subjects they learn. individuals' ideas or evaluations of their learning are related to meta-cognition. therefore, it is possible to conclude that argumentation activities will increase students' metacognitive awareness levels. it was determined that argumentation positively affected students' attitudes, and this result was supported by previous research. in the study conducted by yalçın çelik and kılıç (2014), the effect of argumentation-based journal of science learning article doi: 10.17509/jsl.v4i2.27825 121 j.sci.learn.2021.4(2).113-122 instruction on students' conceptual understanding, attitude towards chemistry, and argumentation tendencies was examined. at the end of the study, it was determined that the students studying with argumentation had significantly higher levels of conceptual understanding, attitude towards chemistry, and argumentation tendency than the students studying with the traditional teaching method. through argumentation activities, students enter making and strengthening their claims and assessing the contradictory claims. a learning environment in which the student is active is expected to positively affect the lesson's attitude towards the lesson. therefore, argumentation activities will also have a positive effect on attitudes. control group students' scores on "what i really think of science scale" and "meta-cognitive awareness inventory for children" and "science process skills test" were also increased during this study; however, this increase was low in science process skills. this result suggests that the current curriculum also positively affects metacognitive awareness levels and attitudes towards science. the low increase in science process skills might result from their not engaged with experiments during the study period. 5. conclusion this study showed that laboratory experiments and argumentation were useful in developing metacognitive awareness, science process skills, and attitudes. considering the positive effects of argumentation on students, teachers might be encouraged to teach science classes based on argumentation when the experiments cannot be used. in other words, argumentation-based teaching can be preferred as an alternative to the experimental method. for teachers to utilize argumentation-based teaching, they should be familiar with argumentation-based teaching activities and then be able to create their argumentation activities. teachers also need to gain awareness regarding the effects of metacognition on the learning process. this way, they can motivate their students to think about their self-learning, leading to improved learning outcomes. teachers' awareness of the positive effects of experiments and argumentation should be developed considering the results of this study and the previous studies on the subject. laboratory conditions in schools should be improved to comply with today's technological developments, and the use of these laboratories should be encouraged. especially in elementary school science lessons, students should be actively involved in the thinking process by including experimental practices and simple equipment types. in this process, students should be given time for making observations, thinking about a situation, offering solutions, evaluating the offered solutions, and, if needed, developing or changing these solutions, and they should be motivated for this. acknowledgment this study is a part of the first author's doctoral thesis. references anagün, ş. s., & yaşar, ş. (2009). i̇lköğretim beşinci sınıf fen ve teknoloji dersinde bilimsel süreç becerilerinin geliştirilmesi [developing scientific process skills at science and technology course in fifth grade students]. i̇lköğretim online, 8(3), 843-865. aslan, s. (2016). argümantasyona dayalı laboratuar uygulamaları: bilimsel süreç becerilerine ve laboratuar dersine yönelik tutuma etkisi [argumentation based activities in the laboratory: effects on science process skills and attitudes towards laboratory course]. hacettepe üniversitesi eğitim fakültesi dergisi, 31(4), 762-777. asterhan, c. s., & schwarz, b. b. (2007). the effects of monological and dialogical argumentation on concept learning in evolutionary theory. journal of educational psychology, 99(3), 626. aydın, ö., & kaptan, f. (2014). fen ve teknoloji öğretmen adaylarının eğitiminde argümantasyonun biliş üstü ve mantıksal düşünme becerilerine etkisi ve argümantasyona ilişkin görüşler [effect of argumentation on metacognition and logical thinking abilities in science and technology teacher education and opinions about argumentation]. eğitim bilimleri araştırma dergisi, 4(2), 163188. aydoğdu, b., buldur, s., & kartal, s. (2013). the effect of open-ended science experiments based on scenarios on the science process skills of the pre-service teachers. procediasocial and behavioral sciences, 93, 1162-1168. bağ, h., & çalık, m.(2017). a thematic review of argumentation studies at the k-8 level. education and science, 42(190), 281-303. bilen, k., & aydoğdu, m. (2012). tahmin etgözleaçıkla (tga) stratejisine dayalı laboratuar uygulamalarının öğrencilerin bilimsel süreç becerileri ve bilimin doğası hakkındaki düşünceleri üzerine etkisi [a study on the efficiency of predict-observe-explain strategy on general biology laboratory applications]. gaziantep üniversitesi sosyal bilimler dergisi, 11(1), 49-69. kırıkkaya, e. b. (2011). grade 4 to 8 primary school students' attitudes towards science: science enthusiasm. educational research and reviews, 6(4), 374-382. celep, a., & bacanak, a. (2013). yüksek lisans yapan öğretmenlerin bilimsel süreç becerileri ve kazandırılması hakkındaki görüşleri [perceptions of teachers who are attending on their master’s degree regarding the science process skills and their attainment]. türk fen eğitimi dergisi, 10(1), 56-78. çetin, m. f., & şahin-taşkın, ç. (2015). sözlü dönütün ilkokul öğrencilerinin akademik başarı, derse yönelik tutum ve üstbilişsel farkındalığına etkisi [the effect of verbal feedback on primary school students' academic achievement, attitude towards the lesson and metacognitive awareness]. sakarya üniversitesi eğitim fakültesi degisi, 29, 39-67. çınar, d., & bayraktar, ş. (2013). argümantasyon temelli fen öğretiminin 5. sınıf öğrencilerinin öğrenme ürünlerine etkisi [the effect of argumentation based science instruction on 5th grade students' learning outcomes] (doctoral dissertation, necmettin erbakan university, turkey). retrieved from https://acikerisim.konya.edu.tr/xmlui/handle/20.500.12452/782 downing, j. e., & filer, j. d. (1999). science process skills and attitudes of pre-service elementary teachers. journal of elementary science education, 11(2), 57-64. driver, r., newton, p., & osborne, j. (2000). establishing the norms of scientific argumentation in classrooms. science education, 84(3), 287-312. durmuş, j., & bayraktar, ş. (2010). effects of conceptual change texts and laboratory experiments on fourth grade students’ journal of science learning article doi: 10.17509/jsl.v4i2.27825 122 j.sci.learn.2021.4(2).113-122 understanding of matter and change concepts. journal of science education and technology, 19(5), 498-504. erenler, s. (2017). argüman temelli sorgulayıcı araştırma uygulamalarının fen bilgisi öğretmen adaylarının üstbilişsel farkındalık düzeyine ve yazma becerilerine olan etkilerinin incelenmesi [the effect of argument driven inquiry (adi) method on pre-service science teachers' metacognitive awareness and scientific writing skills] (master’s thesis, abant i̇zzet baysal university, turkey). etkina, e., karelina, a., & ruibal-villasenor, m. (2008). how long does it take? a study of student acquisition of scientific abilities. physical review special topics-physics education research, 4(2), 1-15. flavell, j. h. (1976). metacognitive aspects of problem solving. in l. resnick (eds.), the nature of intelligence (pp. 231-236). hillsdale, nj: erlbaum. flavell, j. h. (1979). metacognition and cognitive monitoring: a new area of cognitive-developmental inquiry. american psychologist, 34(10), 906-911. freedman, m. p. (2001). the influence of laboratory instruction on science achievement and attitude toward science among ninth-grade students across gender differences. retrieved from eric database (ed454070). gültepe, n., & kilic, z. (2015). effect of scientific argumentation on the development of scientific process skills in the context of teaching chemistry. international journal of environmental & science education, 10(1), 111-132. i̇pek., c., & bayraktar, ş. (2004). aday öğretmenlerin fen bilimleri ve sosyal bilimlere bakışları [prospective teachers' views on science and social sciences]. yüzüncü yıl üniversitesi eğitim fakültesi dergisi, 1(1). jimenez-aleixandre, m. p., rodriguez, a. b., & duschl, r. a. (2000). “doing the lesson” or “doing science”: argument in high school genetics. science education, 84(6), 757-792. jimenez-aleixandre, m. p., & erduran, s. (2007). argumentation in science education: an overview. in s. erduran, & m. p. jimenezaleixandre (eds.), argumentation in science education: perspectives from classroom-based research (pp. 3-27). dordrecht, the netherlands: springer. kalemkus, j., bayraktar, ş., & çiftçi, s. (2019). eğitimde sosyal, zihinsel ve sözlü-yazili bir aktivite: argümantasyon [a social, mental and verbal-written activity in education: argumentation]. electronic turkish studies, 14(4). kanlı, u., & yağbasan, r. (2008). 7e modeli merkezli laboratuvar yaklaşımının öğrencilerin bilimsel süreç becerilerini geliştirmedeki yeterliliği [the efficacy of the 7e learning cycle model based on laboratory approach on development of students’ science process skills]. gazi üniversitesi, gazi eğitim fakültesi dergisi, 28(1), 91-125. karakelle, s., & saraç, s. (2007). çocuklar için üst bilişsel farkındalık ölçeği (übfö-ç) a ve b formları: geçerlik ve güvenirlik çalışması[validity and factor structure of turkish versions of the metacognitive awareness inventory for children (jr. mai) a and b forms]. türk psikoloji yazıları, 10(20), 87-103. karakuş, m., & yalçın, o. (2016). fen eğitiminde argümantasyon temelli öğrenmenin akademik başarıya ve bilimsel süreç becerilerine etkisi: bir meta-analiz çalışması [the effect of the argumentation-based learning in science education to the academic achievement and scientific process skills: a metaanalysis study]. anadolu üniversitesi sosyal bilimler dergisi, 16(2), 120. kıngır, s. (2011). using the science writing heuristic approach to promote students understanding in chemical change and mixtures (doctoral dissertation, middle east technical university, turkey). retrieved from https://open.metu.edu.tr/handle/11511/21489 kipnis, m., & hofstein, a. (2008). the inquiry laboratory as a source for the development of metacognitive skills. international journal of science and mathematics education, 6, 601627. lazarou, d. (2010). learning to tap: an effort to scaffold students’ argumentation in science. in g. çakmakcı, & m. f. taşar (eds.), contemporary science education research: scientific literacy and social aspects of science (pp. 43-50). ankara: pegem akademi. ncrel (2003). engauge 21st century skills for 21st century learners. retrieved from http://www.cwasd.k12.wi.us/highschl/newsfile1062_1.pdf newton, p., driver, r., & osborne, j. (1999). the place of argumentation in the pedagogy of school science. international journal of science education, 21(5), 553-576. osborne, j., erduran, s., simon, s., & monk, m. (2001). enhancing the quality of argument in school science. school science review, 82(301), 63-70. özdener, n. (2005). deneysel öğretim yöntemlerinde benzetişim (simulation) kullanımı [use of simulations in experimental teaching methods]. the turkish online journal of educational technology (tojet), 4(4), 93-98. padilla, m. j. (1990). the science process skills. research matters…to the science teacher. retrieved from eric database (ed266961). pell, t., & jarvis, t. (2001). developing attitude to science scales for use with children of ages from five to eleven years. international journal of science education, 23(8), 847-862. roberts, r., & gott, r. (2010). a framework for practical work, argumentation and scientific literacy. in g. çakmakcı, & m. f. taşar (eds.), contemporary science education research: scientific literacy and social aspects of science (pp. 99-105). ankara: pegem akademi. simon, s., erduran, s., & osborne, j. (2006). learning to teach argumentation: research and development in the science classroom. international journal of science education, 28(2-3), 235-260. sperling, r. a., howard, b. c., miller, l. a., & murphy, c. (2002). measures of children's knowledge and regulation of cognition. contemporary educational psychology, 27(1), 51-79. toulmin, s. (2003). the uses of argument (3rd ed.). new york: cambridge university press. türkoguz, s., & cin, m. (2014). effects of argumentation-based concept cartoon activities on students' scientific process skills. mersin university journal of the faculty of education, 10(2), 142-156. ulu, c., & bayram, h. (2014). araştırma sorgulamaya dayalı bilim yazma aracı kullanımının üstbilişsel bilgi ve becerilere etkisi [effects of implementing inquiry based approach known as science writing heuristic on metacognitive awareness and skills]. turkish international journal of special education and guidance & counseling, 3(1), 68-80. ulu, c. (2019). argümantasyon tabanlı bilim öğrenme yaklaşımına dayalı laboratuvar uygulamalarının üstbilişsel bilgi ve becerilere etkisi [the effects of argumentation-based science learning approach based laboratory applications on metacognitive knowledge and skills]. uluslararası bilim ve eğitim dergisi, 2(1), 1123. van eemeren, f. h., & grootendorst, r. (2004). a systematic theory of argumentation. new york: cambridge university press. von aufschnaiter, c., erduran, s., osborne, j., & simon, s. (2008). arguing to learn and learning to argue: case studies of how students’ argumentation relates to their scientific knowledge. journal of research in science teaching, 45(1), 101-131. yalçın çelik, a., & kılıç, z. (2014). the impact of argumentation on high school chemistry students’ conceptual understanding, attitude towards chemistry and argumentativeness. eurasian journal of physics and chemistry education, 6(1), 58-75. yang, k. y., & heh, j. s. (2007). the impact of internet virtual physics laboratory instruction on the achievement in physics, science process skills and computer attitudes of 10th-grade students. journal of science education and technology, 16(4), 451-461. zeidan, a. h., & jayosi, m. r. (2015). science process skills and attitudes toward science among palestinian secondary school students. world journal of education, 5(1), 1324. a © 2020 indonesian society for science educator 174 j.sci.learn.2020.3(3).174-184 received: 03 march 2020 revised: 14 july 2020 published: 31 july 2020 discovery learning with the solar system scope application to enhance learning in middle school students atika zahara1*, selly feranie2, nanang winarno1, nurhadi siswantoro3 1international program on science education (ipse) study program, faculty of mathematics and science education, universitas pendidikan indonesia, bandung, indonesia 2department of physics education, faculty of mathematics and science education, universitas pendidikan indonesia, bandung, indonesia 3department of material science and engineering, faculty of engineering, eskişehir technical university, eskişehir , turkey *corresponding author zaharaatika@gmail.com abstract in education, “industrial revolution 4.0” refers to utilizing technology to present students with complex situations that will develop their critical thinking and problem-solving skills. the use of technology in the science curriculum should be designed according to established learning models. this study aimed to investigate the effect of implementing a discovery learning model, supported by the solar system scope computer application, on the ability of students to master essential concepts. this research used a one-group pre-test-post-test design. participants were 31 7th-grade students at one school in bandung, indonesia. the results showed a medium enhancement of concept mastery in students from pre-test to post-test (n-gain=0.48). no gender difference in outcomes after the implementation of discovery learning with the solar system scope application was found. based on these results, discovery learning supported by the solar system scope application could be an alternative teaching approach to enhance students' skills in mastering concepts. keywords discovery learning, solar system scope application, students’ concept mastery, solar system 1. introduction curriculum 2013 is one of indonesia’s education systems, which is chosen as an education system for the time being. based on kementerian pendidikan dan kebudayaan (2014) curriculum 2013 is one of the highly effective teaching models because this curriculum can accommodate and approach students’ emotional, physical, and academic. besides, they also stated that in addition to delivering instructional materials, teachers also need to be able to develop the value and skills of students. it is not easy to achieve the purpose of curriculum 2013, especially learning in science. based on the comprehensive survey study by oecd (organization for economic co-operation and development) using pisa (program international for student assessment) test as the international study of the achievements of reading and math. the data shows indonesia is ranked 62nd with a score of 403 from 70 countries that participated in the pisa test in 2015 while neighboring country singapore is in the rank 1 with a score of 556. according to the international survey of trends in international mathematics and science study (timss) in 2015 shows that the average rating of indonesian students’ science achievement is below the international average score. the data of timss indicates that indonesia is ranked 45th out of 48 countries participating in this survey. the global score is 600, while indonesia’s average rating is 397 (martaida, bukit, & ginting, 2017). from the statement, it means that learning science still becomes one of the hardest subjects in indonesia. the statement also supported by several problems related to the implementation of science learning in indonesia. some of these studies state that there are problems related to students’ conceptual understanding (sani, rochintaniawati, & winarno, 2019; vania, setiawan, & wijaya, 2018; furqani, feranie, & winarno, 2018). before the treatment done, the researcher conducts an interview and observation with some students in the mailto:zaharaatika@gmail.com journal of science learning article doi: 10.17509/jsl.v3i3.23503 175 j.sci.learn.2020.3(2).174-184 school. the interview was conducted by asking ten students some questions regarding how they learn science in their school. then the researcher observes teachinglearning activity in the class. they were stated that physics is the most difficult subject in science. the statement also supported by ornek, robinson, & haugan (2007), who say the most challenging topic in science is physics since the students have to comprehend the science content with different representations. different representations, in this case, means concepts and laws in physics. science and technology teaching curriculum should accordingly be developed with the implementation of learning methods because teaching students with the notion of discovering, critical thinking, questioning, and problem-solving skills is one of the main principles of science and technology teaching (balim, 2009). there are a lot of learning models that can be used by the teacher in the learning activity, especially in learning science. but one of these models that developed based on constructivists is discovery learning. discovery learning is one of the learning models because it has stages of learning, such as stimulation, problem statement, data collection, data processing, verification, and generalization. the basis of science teaching is understanding that natural phenomena and the nature of science require inquiring and discovering; it is consists of experiments and inquiring natural phenomena by discovery learning (bruner, 1961). several studies about discovery learning were done by researchers who used it on their teachinglearning process. balim (2009) stated using the discovery learning is considered to increase students’ success and inquiry learning skills more than the traditional teaching methods. mukherjee (2015) also found that during the exercise using discovery learning, students found themselves engaged in a relatively challenging cognitive activity; it is useful in getting students interested and curious. in learning science, especially solar system topics, students will get instanced content that has to be understood. the hardest part of learning the solar system is not every single object or phenomenon in the solar system could be observed directly, such as the character and movement of each planet and the process of the lunar and solar eclipse (furness, winn, & yu, 1997). the teaching-learning process more efficient and accurate, it is better to apply technology such as simulation for a solar system. virtualization on science has a way to observe natural phenomena, which, perhaps because of their location, duration, and size, are impossible to direct observe (furness, winn, & yu, 1997). talking about cognitive activity and skills, pillow (2008), has examined the gender differences between students on their academic performance showed that their cognitive and non-cognitive effect is one of the most important and influential characteristics in academic performance in individual context characteristics. the topic of gender has become the talk of the forum today. other research was conduct by goni (2015) the result shows that there were no significant differences exist between gender in students’ academic performance. it is quite interesting to observe whether any differences between male and female students according to their cognitive skills in learning science since there has been no study about it previously. solar system scope application is one of the planetarium software to see the virtualization of the solar system to help the students identify or observe the solar system. this application was first presented in 2010, supported by excellent scope team, adrian bayer, who came up with the idea, marian bayer, as the first one to make real programming for solar, etc. solar system scope application has many features for students as users to learning solar systems in order to make them learn better. some features are night sky features that allow students to see the sky by using handphone, near stars, and messier objects feature is to explore any object besides planet and stars in the solar system, and planet explore feature in the application to learn about characteristics of each component in the solar system. by using virtual planetarium environments, students will get the experience of a phenomenon or a place that seems so complicated and impossible to see directly in real life (dede, 2000). one as exciting examples of virtual learning environments, planetarium software, offers many possibilities for learning the solar system in a new way and beneficial ways, as the software provides students the structure, details, and complexity of the solar system through 3d visualization (gilbert, 2008; mikropoulos & natsis, 2011). de jong and van joolingen (1998) also stated that learning with simulations based on discovery learning can take place in education and instruction as a new line of learning environments, based on technology, in which more emphasis is given to the learner’s initiative. previously, several studies about discovery learning were done by the researchers. balim (2009) research the effects of discovery learning on students’ success and inquiry learning skills. mukherjee (2015) investigates the effective use of discovery learning but focusing on improving understanding of factors that affect quality. mostafae (2015) also research discovery learning towards learners’ speaking ability. another research by saab, van joolingen, and van hout‐wolters (2009) investigates the relation of learners’ motivation with the process of collaborative scientific discovery learning. in 1984, andrews researching discovery and expository learning compared: their effects on independent and dependent students. singer and pease (2013) also study about discovery learning by comparing discovery learning and guided instructional strategies on motor skill learning, retention, and transfer. de jong and van joolingen (1998) journal of science learning article doi: 10.17509/jsl.v3i3.23503 176 j.sci.learn.2020.3(2).174-184 study about discovery learning with computer simulations but focusing on conceptual domains. and similar research conducted by dalgarno, kennedy, and bennett (2014) on research discovery learning using computer-based simulations to investigate students’ exploration. there are several studies about discovery learning supported by technology, which is a computer-based simulation. however, the differences between this research with those previous research are implementing discovery learning as a learning model using the solar system scope application as technical support. knowing about the demands of science education that the learners should increase their concept mastery, this study aims to determine the effect of implementing discovery learning supported by solar system scope application to enhance students’ concept by analyzing students’ gender differences. solar system scope application can be the technical support in discovery learning for solar system topics. because discovery learning can be used as a learning model in learning the solar system and solar system scope application is the facilitation to help students to observe the solar system. it will be a way to solve the problem in learning the solar system since students should identify the object of the solar system and analyze the phenomenon in the solar system. by implementing discovery learning that supported by the solar system scope application, hopefully, teachers can deliver the material of the solar system in another way with an easily understandable, more creative, and interesting way other than a traditional teaching-learning activity. through the implementation of the discovery learning model with a supporting app, which is solar system scope application in the classroom, students will understand this concept easily and give students a new experience in learning solar systems too. also, this research can be used as a reference for further research. other researchers can investigate the implementation of discovery learning that supported by another planetarium software or implementing other learning models since there is still a lot of learning model and planetarium software that can be used, or researchers can investigate the other students’ skills in the teaching-learning process. this study aimed to investigate the effect of implementing discovery learning supported by the solar system scope application on students’ concept mastery. 2. method 2.1 method design the method used was experimental research. according to fraenkel, wallen, and hyun (2011), a poor experimental design involving one group that is pretested, exposed to a treatment, then post-tested. that is related to the purpose of this study, which is to investigate the effect of the implementation of discovery learning supported by solar system scope application towards students' concept mastery in learning solar system. the design used in this research is one-group pretest-post-test design (fraenkel, wallen, & hyun, 2011). one-group pre-test-post-test design measuring a single group not only after being exposed to the treatment of some sort but also before the procedure. treatment in this research is learning solar system using discovery learning supported by solar system scope application. solar system scope application is one of the planetarium software to see the virtualization of the solar system to help the students identify or observe the solar system. this application was first presented in 2010, supported by excellent scope team, adrian bayer, who came up with the idea, marian bayer, as the first one to make real programming for solar, etc. table 1 shows the research design used in this research. 2.2 population and sample the location of this research is at one of secondary school in bandung. the curriculum that is used in this school is curriculum 2013, which is fitted with the implementation of discovery learning. this curriculum requires students to not only understand the concept but also be active and contribute to the learning activity. it can be done through some steps in discovery learning implemented in this research. the population in this research is all the students in one of the classes in 7th grade who have not studied the solar system topic before. the sampling technique used in this research was convenience sampling. according to fraenkel, wallen, & hyun (2011), convenience sampling is a group of individuals who are available for study. the researcher selects participants because they are available to be studied. there are 31 students in one class at this school with 13 male students and 18 female students with an age average of 12-13 years old. the students as the sample to be studied because they are available, and the table 1 one-group pre-test-post-test design o x o pre-test treatment post-test table 2 data of the sample population sample percentage (%) total (%) 7th grade male 13 41.93% 100% female 18 58.06% table 3 test item of students’ concept mastery no indicator test item 1 remembering 1, 2, 6, 7, 8 2 understanding 4, 5, 9, 10, 11, 13, 14, 17 3 applying 3, 12,16 4 analyzing 15, 18, 19, 20 journal of science learning article doi: 10.17509/jsl.v3i3.23503 177 j.sci.learn.2020.3(2).174-184 researcher has the permission of the principal. table 2 shows the recapitulation of the sample. 2.3 research instrument in this research to measure students’ concept mastery in learning the solar system, researcher use objective test, which consists of 20 multiple choices. the questions cover all the content and concept mastery that is used in this research, so five questions for each cognitive domain. twenty multiple choices distributed before and after conducting learning activity. pre-test distributed to the students before they learn about the solar system. solar system topic and post-test distributed after learning activity measure the enhancement in students’ concept mastery. there are c1 (remembering), c2 (understanding), c3 (applying), and c4 (analyzing) as a cognitive domain that used in this research. test items of students’ concept mastery can be seen in table 3. the data were analyzed in terms of its discriminating power, difficulty level, and validity by using anates. the recapitulation of the test item is tabulated in table 4. the total number of multiple-choice questions is 26 questions. the questions were distributed to be tested to 32 students of 8th grade in junior high school who already learned about solar system topics. those questions researchers use based on some books, national exam exercises, and some items are from the research that previously validated. after we analyze using anates, regarding the result of reliability and validity (appendix), there are five questions rejected, and 20 questions are used with two questions are need some revision before used as an objective test in this research. 2.4 data analysis analysis of the objective test to measure students’ concept mastery was done by microsoft excel calculation to determine the score of pre-test and post-test. the process of data calculation is explained as follows: scoring of test item pre-test and post-test test items were calculated to know the result of each student. the question of each test item pre-test and post-test is 20 multiple-choice questions. calculate the gain and normalized gain the normalized gain, introduced by hake 1998, has become the standard measure for reporting scores on research-based concept inventories. after scoring the test item, the data was processed to know the gain score and normalized gain score. hake defined the average normalized gain as: < 𝑔 >= %𝑆𝑓 − %𝑆𝑖 100 − %𝑆𝑖 description: <g>= normalized gain sf= post-test score si= pre-test score (hake, 1998) normality test a normality test is a test about the normal distribution of data. this test is the most widely performed test for parametric statistical analysis. the data that is normally distributed is a requirement for parametric tests. the data analyzed using spss 24.00 and was distributed normally. homogeneity the homogeneity test aims to determine whether the measured score variance in both samples has the same variation or not. populations with equal variation are called populations with homogeneous variance, whereas populations of unequal variance are called populations with heterogeneous variance. independent t-test an independent sample t-test was used to test the significance of the average difference between the two groups. this test is used to test the effect of the independent variable on the dependent variable. the significance value is 0.05 and determines the hypothesis. 2.5 research procedure there are three stages of this research procedure: preparation, implementation, and completion stages. preparation stage as the first stage, including literature review, conducted to analyze the information about curriculum 2013, discovery learning, solar system scope application, students’ concept mastery, and solar system topic. after analyzing the researcher arrange instruments and observation sheet instructional tools that will be used; lesson plans and worksheets that arranged to support the implementation of this research. then, experts will conduct the judgment of instrumentation. to do the trail test of an objective test instrument that will be conducted to identify the quality of the instruments. the result of the trial test was revised based on judgment results and test item analysis. after the preparation stage, the implementation stage is done by conducting a pre-test to identify students’ preliminary skills first. then come to the main part, which is treatment. in this part, students are learning using discovery learning supported by the solar system scope application that conducted since the first meeting until the last meeting of learning the solar system. in the end, a post-test will be conducted to analyze students’ understanding. the data that the researcher got will be obtained then calculated; this is the first step to do in the completion stage. then, the result of data calculation will be analyzed, journal of science learning article doi: 10.17509/jsl.v3i3.23503 178 j.sci.learn.2020.3(2).174-184 and discussion will be done to elaborate on the result of the analysis—the researcher concluding the result of this study based on the analyzed data. 3. result and discussion 3.1 the effect of discovery learning supported by solar system scope application on students’ concept mastery the purpose of this research is to investigate the effect of discovery learning supported by solar system scope application on students’ concept mastery. previous studies reported that students had difficulties in learning physics (afriani, agustin, & eliyawati, 2019; winarno, rusdiana, riandi, susilowati, & afifah, 2020). in this study, the students’ concept mastery of solar system is measured by using 20 multiple choice questions, which represent variance cognitive level domain. the cognitive level based on a revision of that tested in this research is c1 (remembering), c2 (understanding), c3 (applying), and c4 (analyzing). there are three topics of the solar system that involves in this research. those are the characteristics of solar systems component, rotation, and revolution of the earth, and solar and lunar eclipse. the result of the pre-test and post-test score, which is used to be analyzed, is the score of one tier test. the data were analyzed using spss for windows version 24.0 program to know whether the data is normally distributed or not. after that, the data is analyzed using a parametric test or non-parametric test based on the result of the normality test. the statistic test was done to know the difference concept mastery between pre-test and posttest. the result of the normality test of the pre-test and post-test is shown in table 4. from the result in the kolmogorov-smirnov test, the significance value (α) of the pre-test is 0.103, and the significance value of the post-test is 0.110. the kolmogorov-smirnov test is the test of the difference between data that is tested and the standard normally distributed data. the level of significance use on the test is 0.05. the result of the significant value of the data tested shows > 0.05, which means that the data is normally distributed. since the data is normally distributed, then the test continued with a homogeneity test. the detail of the data result of the homogeneity test can be seen in table 5. based on the results of the levene statistic test, the significance value (α) is 0.981. if the significant value compared with the level of significance use on the test, which is 0.05, resulting in 0.981 > 0.05, then h0 is accepted. it means that the data value is homogeneous. the sample of the research comes from one class, and the data that is compared is pre-test and post-test in the class. the paired samples t-test is used because it compares the means for two related units on a continuous outcome that is normally distributed. the result of the paired sample t-test is shown in table 6. the statistical test is used to test two hypotheses statements below: h0: there is no difference in students’ scores on pretest and post-test. h1: there is a difference in students’ scores on pretest and post-test. the level of significant value used in the test is 0.05. if the result of the analysis shows the significant value, which is more than 0.05, it means that h0 is accepted, while when it is less than 0.05, h1 is accepted. the result of the test shows that the significant value is 0.000 or less than 0.05, which means that there is a difference in students’ scores on pre-test and post-test. the data gained is also analyzed using a normalized gain test based on the rule of hake (1998). the average ngain score of students’ conceptual mastery in pre-test and table 4 results of normality test on students’ concept mastery kolmogorov-smirnova shapiro-wilk statistic df sig. statistic df sig. pre-test 0.144 31 0.103 0.961 31 0.318 post-test 0.143 31 0.110 0.945 31 0.115 table 5 homogeneity test of variance on students’ concept mastery levene statistic df1 df2 sig .002 1 60 .981 table 6 result of paired sample t-test paired differences t df sig. mean std. deviation std. error mean 95% confidence interval of the difference lower upper pair 1 pre-test post-test -24.35 8.73 1.56 -27.55 -21.15 -15.53 30 .000 table 7 students’ score on pre-test and post-test average gain n-gain category pre-test 49.19 24.36 0.48 medium posttest 73.55 journal of science learning article doi: 10.17509/jsl.v3i3.23503 179 j.sci.learn.2020.3(2).174-184 post-test showed in table 7. based on the data analysis in table 7, it can be found that the average pre-test score was 49.19, while the average post-test score was 73.55. there is an enhancement of students’ concept mastery in learning solar system concepts since the average score on the post-test is higher than the pre-test. from the graph, it is seen that there is an enhancement in students’ concept mastery from pre-test to post-test. the initial average score is 49.19, from the average ideal score 100. the students’ pre-test scores show that the students’ prior knowledge about the solar system is still lack after the implementation of discovery learning supported by solar system scope application. there is an enhancement in the average score, which is 73.55 from the average ideal score 100. the score also has gain normalization with the value of n-gain 0.48 between the average of pre-test and posttest scores. according to hake’s rules of normalization, the value of n-gain 0.48 means that it categorized as medium. students’ concept mastery enhancement in learning solar system proved by the result of n-gain score. students could enhance their understanding (concept mastery) is because, in this research, discovery learning is supported by solar system scope application as the technology called planetarium software to help students learning the solar system better. the technical term is similar to the research done by de jong and van joolingen (1998). they used computer simulation as the technology; the result of their study is discovery learning with computer simulation leads to knowledge that is more intuitive and deeply rooted in a learner’s knowledge base. dalgarno, kennedy, and bennett (2014), in their research, also stated using discovery learning with computer simulation as an active exploration process is more effective than passive observation. it could happen because students are more active by discovering the information by themselves based on discovery learning and solar system scope application as the technology support called planetarium software to help students observe and analyze solar system. prima, putri, and sudargo (2017) also stated that the implementation of the stellarium virtual observatory could improve students’ understanding of learning solar systems. in this research, discovery learning has a role as a learning model to guide students learning the solar system. discovery learning in this research refers to guided discovery through six steps (syntax) during a learning activity. the syntax of discovery learning could be the main reason students can improve their understanding of learning the solar system. according to rivers and vockell (1987), discovery learning involves planning (designing an experiment), executing (carrying out the experiment and collecting data), and evaluating (analyzing the data and developing a hypothesis). this research used syntax from joyce, weil, & showers (1992). six learning steps are used: stimulation, problem statement, data collection, data processing, verification, and generalization. stimulation is the first step during a learning activity; in this step, students are stimulated by some questions or problems. the teacher has a role in stimulating them with the question based on some phenomena in daily life that related to the solar system. students will feel interested and curious to find the answer, and it will motivate the students to start to learn. making a hypothesis is also essential for students. hence, they know what topic or concept that they want to learn, or students have some ideas so they can prove their ideas by observing, and they can develop the hypothesis. for example, is in the learning characteristics of solar system components, the first step is a simulation. in this step first, students are asked about what they already know about the solar system. after that, students are given a short story about the process that happens in the solar system. this step will stimulate them to give more attention and curios to the topic. the second step is the problem statement. students are asked to identify the questions, problems, or making a hypothesis. in this research, questions are stated in the worksheet, and students have to complete it. while the hypothesis is based on the phenomenon related to the solar system, the phenomenon in daily life could be the effect of rotation and revolution of the earth, so students will be more understanding of the problem or question because the phenomenon happens every day. in learning characteristics of solar system components, students were given the simulation about the solar system by using solar system scope. after that student was given the worksheet that contains some questions to answer (table of the component of the solar system and they should fill it), in this step, students need to understand first what they have to learn and find the information using the application. figure 1 solar system scope application features journal of science learning article doi: 10.17509/jsl.v3i3.23503 180 j.sci.learn.2020.3(2).174-184 data collection is the third step. this step asked students to answer the question or problem. students were given a chance to gain some relevant information by reading and observing. in this step, the role of the solar system scope application as planetarium software is essential. students freely explore the application to obtain information and data. when students were seeking information by themselves, it could help them to understand better rather than just memorizing the concept or topic. the data collection step could help students to solve their problems. for characteristics of solar system components topic, students were asked to observe and analyze the component in the solar system as much as they want using the solar system scope application. not only to answer the questions in the worksheet but also to seek new information as much as they can. in case they have something to see and observe besides the questions in the worksheet. they want to find the data, for example, the other planets beside eight main planets or the phenomena of a falling star and constellation shape based on their astrological sign. so learning activity not only focuses on the worksheet, but they can observe and analyze anything they want to see but related to the topic. data processing is the step where students process all the information by reading and observing in the previous step, or if necessary for some questions, students have to calculated first then interpreted it. this step will teach students to learn the information that they found by themselves. after getting the data, students need to think and analyze fist before concluding. it makes students learning the concept twice by collecting the data and process it. in the characteristic of the solar system components topic, students were discussing, write, and answer the questions in worksheets with their group. they explain this step is based on the data they get in the data collection step. they can share their idea with other members before concluding and share it with the other students. the next step is verification, based on the result of the previous step, which is processing the data, the questions, problems, or hypothesis that has been formulated earlier, is checked. in this research, students need to make their predictions about the phenomenon in the solar systems, such as the planet movement, it could be earth’s rotation and revolution or by analyzing the process of the solar and lunar eclipse. the verification step is essential to make the student concluding what they have already learned by themselves. it will improve understanding— the steps in line with the next step, which is a generalization. for example, in learning about characteristics of solar system components, students were present about the result of their data, because each group writes the different data. the first group is writing about the eighth planets, the second group writes the planet, and their satellite and the other groups are writing the other object beside planets. so they can share and tell their idea of the component in a solar system based on what they get. after collecting the data, processing the data, and verification to check the hypothesis or the problem that we’re in the first step, then it comes to the concluding. each group is asked to go in front of the class and share what information, conclusion, and concept. they have already obtained it. since the term of discovery learning in this research is guided discovery learning, the teacher could correct if there is a wrong concept that students get. in the last step, teachers and students concluded about what they have learned about components of the solar system. since in this research, the term discovery learning is guided discovery so the teacher can make some corrections for student’s answers. by sharing and concluding the topic together, it prepares students are more understanding so that learning activity will be more fun. those six steps could improve students’ understanding and learn better since the n-gain score in learning solar system after using discovery learning supported by solar system scope application is improved. this statement is supported by some studies before. the steps that are used for learning activity based on discovery learning is generally can make students explore acquire a deeper understanding of the world because rather than just memorizing the concept, they discover it by themselves figure 2 planet explore features in solar system scope application figure 3 earth in planet explore features journal of science learning article doi: 10.17509/jsl.v3i3.23503 181 j.sci.learn.2020.3(2).174-184 (westwood, 2008). based on großmann and wilde (2019), in their research about discovery learning, they found that students who worked with guided discovery had higher conceptual and procedural knowledge than the other who is not. balim (2009), in his research, also stated that using the discovery learning method. which is one of the various teaching methods in which the students are active and are guided by the teacher, is considered to increase students’ success and inquiry learning skills more than the traditional teaching methods. another research also could support that discovery learning could enhance the understanding of students. mostafaee (2015) stated that discovery learning has a significant effect on students as learners. it is because students found themselves engaged in a relatively challenging cognitive activity, and learning using a discovery model could be quite useful in getting students interested and curious (mukherjee, 2015). another statement also stated by singer and pease (2013), in their research, learners that were administered discovery instructional strategies in the initial learning situation were superior to the guided learners in retention performance and in the early stages of learning a second related task. it could happen because, during learning activity, using the discovery model. students are asked to solve the problems, questions, or making hypotheses, and learners who often formulated theories had better learning results while working with the discovery learning environment saab, van joolingen, & van hout-wolters (2006). they using discovery learning, the performance of a dyad existing of a high, and a lowly motivated learner can be influenced positively by the highly motivated peer saab, van joolingen, & van hout‐wolters (2009). it means learning based on the discovery model could make the students affect the other students to understand about the topic. the other research could support this statement, andrews (1984) states there is effectiveness using guided learning approach for not only independent students, but it still applies to the dependent individuals. the other reason that could enhance students’ concept mastery in this research is using planetarium software as technology support through discovery learning. solar system scope application is an interactive way for students to observe and analyze solar systems. through syntax in discovery learning, students can seek information, concept, and explore the solar system by themselves using the solar system scope application. solar system scope application has many features for students as users to learning the solar system to make them learn better. some features in the solar system scope application used by students in this research can be seen in figure 1. figure 1 shows features that are used by students, which are the solar system to explore every single object and movement in the solar system. the features night sky is to see the sky by using a handphone, so students only place their phone directly to the sky, and they will see what objects in the sky at that time. planet explores students to explore each planet in the solar system, the characteristics until their movement. besides the planet explore, there is star explore which provides students to see the stars in the solar system in detail. the other features are near stars, and messier objects are to examine any object besides the planet and stars in the solar system, it will improve their knowledge in learning the solar system. one topic that can be applied using the solar system scope application is the characteristics of solar system components. students can use the planet to explore features in the app to learn about the characteristics of solar system components subtopic. some students still did table 8 results of normality test on students’ gender differences kolmogorov-smirnova shapiro-wilk statistic df sig. statistic df sig. male female 0.171 0.192 13 18 0.200 0.078 0.932 0.934 13 18 0.359 0.226 table 9 homogeneity test of variance on students’ gender differences levene statistic df1 df2 sig 4.080 1 29 0.053 table 11 students’ pre-test and post-test scores on gender average n-gain category male pre-test 50.38 0.48 medium posttest 73.85 female pre-test 48.33 0.48 medium posttest 73.33 table 10 result of paired sample t-test paired differences t df sig. mean std. deviation std. error mean 95% confidence interval of the difference lower upper pair 1 male & female -0.020 0.259 0.072 -0.177 -0.137 -0.278 12 0.786 journal of science learning article doi: 10.17509/jsl.v3i3.23503 182 j.sci.learn.2020.3(2).174-184 not know the arrangement of the solar system and the name of each planet. through this application, students freely explore what they want to know. figure 2 and figure 3 show the feature of the planet explore in solar system scope application. there are eight planets that students can explore, such as mercury, venus, earth, mars, jupiter, saturn, uranus, neptune. figure 3 shows the explore features of each planet. they can observe the earth in the planet system, gain information about earth genuinely, and see the structure of the earth in detail. not only planets that can be explored by students, sun as the star is also can be explored in detail. it helps the students learn more comfortable and better since this application comes with great virtualization and animation through some features. using the solar system scope application as planetarium software to help students learning the solar system through discovery learning could improve their understanding since the n-gain score of students’ concept mastery is developed after the implementation. this statement is supported by the result of some studies before, by using the planetarium software, students are better equipped to learn fundamental aspects of the universe (persson & eriksson, 2016). yu, sahami, sahami, and sessions (2015) stated that using visualization can accurately represent the correct scale, orientation, and position of solar system objects. the other researcher yu, sahami, denn, sahami, and sessions (2016) agree with that, the state after using planetarium software, the students show more significant learning gains when comparing to another group with no visualization, even when the instructors, lecture content, and visuals are constant. by using virtual planetarium environments, students will get the experience of a phenomenon or a place that seems so complicated and impossible to observe directly in real life (dede, 2000). 3.2 the effect of discovery learning supported by solar system scope application on students’ gender differences to analyze deeply the effect of discovery learning supported by solar system scope application in learning the solar system. an analysis was done to see the differences between male and female students. then a statistic test was done for pre-test and post-test. the result of the normality test of the pre-test and post-test is shown in table 8. from the result of the kolmogorovsmirnov test, the result of the significance value (α) of male students is 0.200. meanwhile, the significance value of female students is 0.078. the kolmogorov-smirnov test is the test of the difference between data that is tested and the standard normally distributed data. the level of significance use on the test is 0.05. the result of the significant value of the data tested shows (α) > 0.05, which means that the data is normally distributed. since the data is normally distributed, then the test continued with a homogeneity test. the detail of the data result of the homogeneity test can be seen in table 9. based on the results of the levene statistic test, the significance value (α) is 0.053. if the significant value compared with the level of significance use on the test, which is 0.05, resulting in 0.053 > 0.05, then h0 is accepted. it means that the data value is homogeneous. thus, the data were analyzed using the paired samples t-test to compares the means for two related units on a continuous outcome that is normally distributed. the result of the paired sample t-test is shown in table 10. the statistical test is used to test two hypotheses statements below: h0: there is no difference in students’ pre-test and post-test scores on gender. h1: there is a difference in students’ pre-test and post-test scores on gender. the level of significant value used in the test is 0.05. if the result of the test shows the significant value, which is more than 0.05, it means that h0 is accepted, while when it is less than 0.05, h1 is accepted. the result of the test shows that the significant value is 0.786 or more than 0.05, which means that there is no difference in students’ gender after the implementation of discovery learning supported by the solar system scope application. the result of the acceptance of h0 is also supported by using n-gain. the data gained analyzed using a normalized gain test based on the rule of hake (1998). the average n-gain score of students’ conceptual mastery in pre-test and post-test showed in table 11. based on the data analysis in table 11, it can be found that the average pre-test score is 50.38, and the post-test score is 73.85 for male students. meanwhile, the average pre-test score is 48.33, and the post-test score is 73.33 for females students. there is an enhancement of students’ concept mastery in learning solar system concepts for both male and female students since the average score on the posttest is higher than the pre-test. comparison of the average of pre-test and post-test scores for both male and female students, the score also has gain normalization with the value of n-gain 0.48. according to hake’s rules of normalization, the value of n-gain for male students is 0.48 and categorized as medium. it is the same as the n-gain score for female students, which is 0.48 and also classified as medium. the result of the n-gain score for both male and female students shows that there are no differences between male and females students in learning solar systems after the implementation of discovery learning supported by the solar system scope application. gender issue has become the talk of today's forum. the result of this research is similar with other study conducted by baharudin and luster (1998) they stated that the gap between the average scale scores of males and females in mathematics journal of science learning article doi: 10.17509/jsl.v3i3.23503 183 j.sci.learn.2020.3(2).174-184 achievement was quite small and has fluctuated only slightly over the past ten years. goni (2015) also stated that there was no significant differences exist between gender in students’ academic performance. those statements also supported by the result of research conducted by güzel (2010), who says that there was no significant difference between male and female students of their success in a physics lecture. in other words, those statements mean there was no significant difference by gender in learning. during learning activity, students are divided into some groups, and the member of the groups is consists of both male and female students. this could be a reason for the result that stated that there are no differences in students’ gender differences in learning solar systems based on the data of n-gain. using discovery learning could improve both male and female students in learning the solar system. großmann and wilde (2019), in the research, stated that the students who worked with guided discovery had higher conceptual and procedural knowledge than the students who are not. saab, van joolingen, & van hout-wolters (2006) also said that with collaborative discovery learning, the performance of a dyad existing of a high and a lowly motivated learner could be influenced positively by the highly motivated peer. since in this research, both male and female students are work in the same group with the same treatment, they could have a positive impact on another student (different gender) such as the motivation so the other students can be influenced positively in the learning activity. the groups were made randomly by shuffle students' names to make fair for all groups. 4. conclusion based on the discussions, results, and analysis of previous chapters, the researcher summed up several findings. implementation of discovery learning supported by the solar system scope application could enhance students’ concept mastery. it can be proved by the acceptance of h1, which means that there is a significant effect of learning solar system using discovery learning towards students’ concept mastery. the improvement of students’ concept mastery, also supported by the results of n-gain, is 0.48, which means that it is categorized as medium. this research also concludes that there is no difference in students’ gender after the implementation of discovery learning supported by the solar system scope application. from the analysis result of the research, there are some suggestions. first, the instruments for implementing the discovery learning should be prepared and made well. it includes worksheets during a learning activity. the application as the technical support also should be available in bahasa, since not all of the students. the recommendation for further research is discovery learning supported by solar system scope application that can be implemented to investigate other variables such as critical thinking, creative thinking, or others. acknowledgment the author would like to express a great appreciation to several parties involved in this research—particularly grateful for the assistance given by the school in which the research has been conducted. several parties also include the teacher who has helped the author to conduct the implementation of this treatment. this paper may not have been accomplished without the help of those several parties. references afriani, t., agustin, r. r., & eliyawati, e. (2019). the effect of guided inquiry laboratory activity with video embedded on students’ understanding and motivation in learning light and optics. journal of science learning, 2(4), 79-84. andrews, j. d. w. (1984). discovery and expository learning compared: their effects on independent and dependent students. the journal of educational research, 78(2), 80–89. balım, a. g. (2009). the effects of discovery learning on students’ success and inquiry learning skills. eurasian journal of educational research (ejer), (35), 1-20. baharudin, r., & luster, t. (1998). factors related to the quality of the home environment and children’s achievement. journal of family issues, 19(4), 375-403. bruner, j. s. (1961). the act of discovery. harvard educational review. dalgarno, b., kennedy, g., & bennett, s. (2014). the impact of students’ exploration strategies on discovery learning using computer-based simulations. educational media international, 51(4), 310–329. dede, c. (2000). emerging influences of information technology on school curriculum. journal of curriculum studies, 32(2), 281-303. de jong, t., & van joolingen, w. r. (1998). scientific discovery learning with computer simulations of conceptual domains. review of educational research, 68(2), 179-201. fraenkel, j. r., wallen, n. e., & hyun, h. h. (2011). validity and reliability, how to design and evaluate research in science education. mc graw–hill companies. furness, t. a., winn, w., & yu, r. (1997). global change, vr and learning, a report for the nsf of workshops, the impact of three dimensional immersive ve on modern pedagogy. nsf. furqani, d., feranie, s., & winarno, n. (2018). the effect of predict-observe-explain (poe) strategy on students’ conceptual mastery and critical thinking in learning vibration and wave. journal of science learning, 2(1), 1-8. gilbert, j. k. (2008). visualization: an emergent field of practice and inquiry in science education. in visualization: theory and practice in science education (pp. 3-24). springer. goni, u. (2015). gender difference in students’ academic performance in colleges of education in borno state, nigeria: implications for counselling. journal of education and practice, 6(32), 107-114. großmann, n., & wilde, m. (2019). experimentation in biology lessons: guided discovery through incremental scaffolds. international journal of science education, 41(6), 759-781. güzel, h. (2010). profiles of university students based on multiple intelligence theory and its effect on their success in physics lecture. world applied sciences journal, 10(6), 665-674. hake, r. r. (1998). interactive-engagement versus traditional methods: a six-thousand-student survey of mechanics tests data for journal of science learning article doi: 10.17509/jsl.v3i3.23503 184 j.sci.learn.2020.3(2).174-184 introductory physics courses. american journal of physics, 66(1), 6474. joyce, b., weil, m., & showers, b. (1992). models of teaching (4th ed.). ally and bacon. mikropoulos, t. a., & natsis, a. (2011). educational virtual environments: a ten-year review of empirical research (1999– 2009). computers & education, 56(3), 769-780. mostafaee, l. (2015). the impact of form focused discovery approach on efl learners’ speaking ability. modern journal of language teaching methods, 5(1), 10-19. mukherjee, a. (2015). effective use of discovery learning to improve understanding of factors that affect quality. journal of education for business, 90(8), 413–419. ornek, f., robinson, w. r., & haugan, m. r. (2007). what makes physics difficult?. science education international, 18(3), 165-172. persson, j. r., & eriksson, u. (2016). planetarium software in the classroom. physics education, 51(2), 025004. pillow, b. (2008). a comparison of academic performance in alevel economics between two years. international review of economics education, 2(1), 8-24. rivers, r. h., & vockell, e. (1987). computer simulations to stimulate scientific problem-solving. journal of research in science teaching, 24(5), 403-415. prima, e. c., putri, c. l., & sudargo, f. (2017). applying pre and post role-plays supported by stellarium virtual observatory to improve students’ understanding on learning solar system. journal of science learning, 1(1), 1-7. saab, n., van joolingen, w. r., & van hout-wolters, b. h. a. m. (2006). supporting communication in a collaborative discovery learning environment: the effect of instruction. instructional science, 35(1), 73–98. saab, n., van joolingen, w. r., & van hout‐wolters, b. h. a. m. (2009). the relation of learners’ motivation with the process of collaborative scientific discovery learning. educational studies, 35(2), 205–222. sani, a., rochintaniawati, d., & winarno, n. (2019). using brainbased learning to promote students’ concept mastery in learning electric circuit. journal of science learning, 2(2), 42-49. singer, r. n., & pease, d. (1976). a comparison of discovery learning and guided instructional strategies on motor skill learning, retention, and transfer. research quarterly. american alliance for health, physical education and recreation, 47(4), 788-796. vania, p. f., setiawan, w., & wijaya, a. f. c. (2018). edmodo as webbased learning to improve student’s cognitive and motivation in learning thermal physics. journal of science learning, 1(3), 110115. winarno, n., rusdiana, d., susilowati, e., & afifah, r. m. a. (2020). implementation of integrated science curriculum: a critical review of the literature. journal for the education of gifted young scientists, 8(2), 795-817. westwood, p. s. (2008). what teachers need to know about teaching methods. aust council for ed research. yu, k. c., sahami, k., sahami, v., & sessions, l. c. (2015). using a digital planetarium for teaching seasons to undergraduates. journal of astronomy & earth sciences education, 2(1), 33-50. yu, k. c., sahami, k., denn, g., sahami, v., & sessions, l. c. (2016). immersive planetarium visualizations for teaching solar system moon concepts to undergraduates. journal of astronomy & earth sciences education, 3(2), 93-110. a © 2021 indonesian society for science educator 101 j.sci.learn.2021.4(2).101-112 received: 11 october 2020 revised: 3 december 2020 published: 4 january 2021 investigation of middle school students’ attitudes towards science, technology, engineering and mathematics (stem) education and determination of the predictors cennet göloğlu demir1*, nagihan tanik önal2, nezih önal2 1bandirma onyedi eylul university, bandirma, turkey. 2nigde omer halisdemir university, nigde, turkey *corresponding author nagihanta@gmail.com abstract the purpose of the current study is to investigate middle school students’ attitudes towards science, technology, engineering, and mathematics (stem) education and to determine the predictors of these attitudes. the study was designed according to the relational survey model, one of the quantitative research designs. the sample of the study is comprised of 408 middle school 6th, 7th, and 8th-grade students. the data of the current study were collected using a stem-oriented attitude scale. an independent samples t-test, one-way variance analysis, and stepwise multiple regression analysis were used to analyze the collected data. as a result of these analyses, it was determined that the students’ attitudes towards stem vary significantly depending on the students’ gender, grade level, participation in in-school and out-of-school social activities, science and mathematics achievement. the most effective three predictors of stem were found to be science achievement, being a 6th grader, and being female. the state of being female was found to be negatively correlated with the prediction of the attitudes towards stem. as a conclusion of the study, suggestions were made to eliminate gender-based differences in stem attitudes, increase stem activities in upper grades, and for career planning. keywords stem, attitude, middle school student, predictors 1. introduction training individuals having 21st-century skills can be seen as a prerequisite for struggling with today's competitive conditions, producing technology, and ensuring economic growth and well-being. in particular, to strengthen education for innovation, more people should be directed to the fields of science, technology, engineering, and mathematics (oecd, 2018). implementation of an interdisciplinary approach such as stem education has been on the agenda of many countries such as the united kingdom, australia, ireland, norway, germany, sweden, denmark, particularly in the past decade (bybee, 2010a; ejiwale, 2012; gonzalez & kuenzi, 2012). stem education can be defined as the integrated teaching of the four disciplines by establishing connections between science, technology, mathematics, and engineering (sanders, 2009; smith & karr-kidwell, 2000). stem education makes it possible for individuals to reach the information they need, to gain skills that can meet their daily needs with the information they have reached, and to be trained in a multidimensional manner in different fields (yıldırım, 2016). in other words, it is believed that wellqualified and well-equipped individuals will be trained with stem education (turner, 2013). it is also thought that stem education will enable countries to develop technologically and economically (bybee, 2010b; bybee, 2013; fan & ritz, 2014; west, 2012). seen from this perspective, the importance of stem education is highly apparent. when the national literature is examined in turkey, it is seen that a great emphasis has been put on the importance and necessity of stem education (akgündüz et al., 2015; çorlu, 2014). while çorlu (2014) emphasized that quality stem manpower is needed to increase turkey's innovation capacity, akgündüz et al. (2015) stated that turkey has to train an innovative generation with 21st-century skills and feels interested in stem fields. the stem education report of the ministry of nastional education (2016) mailto:nagihanta@gmail.com journal of science learning article doi: 10.17509/jsl.v4i2.28859 102 j.sci.learn.2021.4(2).101-112 strongly emphasized that stem education is very important for turkey and should be put into effect without delay. this report also emphasized that students should be made interested in stem fields and directed to professions related to these fields. a study conducted in korea stated that stem studies had increased students' interest in stem fields over time, but although students' interest has increased, they do not prefer to choose stem-related professions (jon & chung, 2013). this indicates that students' interest in stem fields does not directly cause them to prefer a career in stem fields and that other factors such as attitude, ability, personal characteristics, and employment opportunities may also have a mediating effect. attitudes towards stem education can also be seen as one of these factors. attitude towards stem education is defined as the behaviors individuals are expected to display in stem fields (yıldırım, 2016). as can be understood from its definition, the attitude towards stem is reflected in the behaviors. on the other hand, students' attitudes towards science and mathematics are considered among the most critical factors affecting their success in these fields (abell & lederman, 2007; oecd, 2007; oral & mcgivney, 2011). with the determination of the attitude towards stem education, especially at young ages, it is possible to shape the education to be given to students accordingly and to direct students to stem-centred professions. it is thought to be essential to investigate students' attitudes towards stem education and the variables that affect these attitudes in this context. there are some studies conducted to determine attitudes towards stem in the related literature (aydın, saka, & guzey 2017; gülhan & şahin, 2016; gündüz & tarhan, 2017; mahoney, 2009; sivrikaya, 2019; unfried, faber, stanhope, & wiebe, 2015). in the current study, investigation of middle school students’ attitudes towards stem education concerning the variables of students’ grade level, gender, participation in in-school and out-of-school social activities and math and science achievement and determination of the predictors of these attitudes are believed to make contributions to the relevant literature. working with a younger age group like middle school students in the current study has a special value since this period represents the first years when individuals' career awareness begins to form. the literature generally reported that as the grade level increases, students' attitudes towards stem become more negative (karakaya & avgın, 2016; unfried, faber, & wiebe, 2014). this indicates that something should be done for students to choose stem professions in their career plans. it is known that there are gender differences in attitudes towards stem and career choices in stem fields. in a report prepared by unesco (2017), it was stated that the rate of women in stem-related fields in higher education is 35%. the reason for gender differences in stem fields is explained as the social, cultural, and gender norms that shape the identities, beliefs, behaviors, and choices of women and men. as in other countries that strive to achieve gender equality in stem fields, some projects are made in turkey. for example, the “honey bees becoming engineers” project is carried out in cooperation with ford otosan, one of the important automotive industry institutions, the ministry of national education and flying broom women's communication and research association. within the scope of the project, city visits called hive days were made, and in 22 provinces, the engineering profession was introduced to a total of 4535 students, 2668 of whom were women (url-1). as a result, gender should be investigated to develop a positive attitude towards stem and direct them to make a career in these fields. it is aimed at students who are expected to acquire 21stcentury skills to produce solutions to daily life problems with stem education. in this connection, students' participation in social activities inside and outside the school on the attitude towards stem education is curiosity. finally, investigation of the effect of the achievement in math and science courses on the attitudes towards stem education based on the relationship between the attitudes towards mathematics and science courses and the academic achievement in these courses (oral & mcgivney, 2011) will provide essential data for mathematics and science curriculums. in this connection, the current study investigates middle school students’ attitudes towards stem education and determines these attitudes' predictors. to this end, answers to the following sub-problems were sought: 1) do middle school students’ attitudes towards stem vary significantly depending on; gender, grade level, participation in out-ofschool social activities, participation in in-school social activities, math achievement, science achievement?. 2) what is the extent to which gender, grade level, participation in out-of-school and in-school social activities, and math and science achievement together predict middle school students’ attitudes towards stem? 2. method 2.1 research model the current study aiming to investigate middle school students’ attitudes towards stem and to determine the predictors of these attitudes employed the relational survey model. relational survey studies are conducted to determine the relationship between two or more variables without any treatment to the participants without taking the variables under control (fraenkel & wallen, 2009). through relational survey studies, it is possible to determine the differences between groups or the predictor relationships between the variables. journal of science learning article doi: 10.17509/jsl.v4i2.28859 103 j.sci.learn.2021.4(2).101-112 2.2 population and sample the current study population consists of 2542 6th, 7th, and 8th-grade students attending 9 state middle schools in a city located in turkey's western black sea region. according to cohen, manion, and morrison (2018), while determining the sample size, the confidence level should be taken as 95%, and the confidence interval should be taken as 5%; thus, the current study's sample size was determined to be 333. in selecting the participants, the convenience sampling method, one of the non-probability sampling methods, was used. in the convenience sampling method, the researcher starts constructing the sample with the most easily reachable respondents (cohen, manion & morrison, 2018). as a result, 553 middle school students volunteering to participate in the current study constitute the study sample. of the 553 students, 145 were discarded from the study as they left many items not responded, and as a result of the outlier and missing data analyses. as a result, the analyses were conducted on the data collected from 408 students. the variables included in the current study are gender, grade level, participation in in-school and out-of-school social activities, math achievement, and science achievement. information about the demographic features of the students participating in the current study is given in table 1. the distribution of the participating students across genders and grade levels seems to be balanced. of the participating students, 47.3% stated that they regularly participate in out-of-school social activities while 52.7% stated that they do not, and while 38.2% of them stated that they participate in in-school social activities, 61.8% stated that they do not. when the students’ math and science achievement is examined, it is seen that more than half of them are highly successful in science. in the classification of the students’ achievement, the five-point classification system used in turkey [poor (1), okay (2), medium (3), good (4), very good (5)] was taken as reference, and thus a three-point categorization was constructed (poor, okay=low, medium=medium, good, very good=high). 2.3 data collection tool the current study data were collected by using the scale of attitudes towards stem developed by the friday innovation in education institute (2012) and adapted to turkish by özcan & koca (2018). the scale is a five-point likert scale consisted of 37 items and 4 factors. the adaptation study of the scale into turkish was conducted by özcan & koca (2018) on 1323 middle school 6th, 7th, and 8th-grade students attending middle schools in three cities located in different turkey regions. in the study, the cronbach’s alpha coefficient was calculated to be .91 for the whole scale, .86 for the math factor, .87 for the science factor, .86 for the engineering and technology factor, and .88 for the 21st-century skills factor. in this way, the validity and reliability of the scale were established. the lowest score to be taken from the scale is 37, while the highest score is 185. cronbach’s alpha (α) reliability coefficient was calculated for the whole scale and its sub-factors in the current study. for the whole scale, the cronbach’s alpha (α) coefficient was found to be .91, while it was found to be .89 for the math factor, .87 for the science factor, .83 for the engineering and technology factor, and .86 for the 21stcentury skills factor. thus, as the cronbach’s alpha internal consistency coefficients were found to be in the range of .80≤ α <1.00 for the whole scale and it is all subdimensions, it can be argued that the scale is highly reliable in this sample (pallant, 2016). 2.4 data analysis before analyzing the data collected from the scale of attitudes towards stem, the normality assumption was tested to understand whether the variables are distributed normally. one criterion to satisfy the normality assumption is that the skewness and kurtosis coefficients should be between -1 and +1 (morgan, leech, gloeckner & barrett, 2004). as a result of the calculations made in this context, table 1 some demographic features of the participants variables groups frequency % gender male 206 50.5 female 202 49.5 grade level 6th grade 173 42.4 7th grade 107 26.2 8th grade 128 31.4 participation in out-of-school social activities yes 193 47.3 no 215 52.7 participation in in-school social activities yes 156 38.2 no 252 61.8 math achievement low 60 14.7 medium 180 44.1 high 168 41.2 science achievement low 40 9.8 medium 132 32.4 high 236 57.8 journal of science learning article doi: 10.17509/jsl.v4i2.28859 104 j.sci.learn.2021.4(2).101-112 mean scores and standard deviations obtained for the whole scale and its sub-dimensions are given in table 2. the skewness and kurtosis values for the whole scale and its sub-dimensions are between -1 and +1. thus, it can be said that the data distributed normally. the lowest mean score was obtained from the scale's mathematics subdimensions from among the sub-dimensions of the scale on (x̄mat.= 28.26), while the highest mean score was obtained from the 21st-century skills sub-dimension (x̄21stskills.= 28.26). the mean score obtained from the whole scale was found to be x̄stem=135.95. independent samples t-test was used to determine whether the mean scores are taken from the whole scale and the sub-dimensions vary significantly depending on gender, participation in out-of-school social activities, and participation in in-school activities. one-way variance analysis (anova) was used to determine whether the mean scores vary significantly depending on grade level and math and science achievement. the analyses were interpreted by taking the significance level as .05 and based on percentages, frequencies, means, and standard deviations. cohen’s d statistics calculated to determine the extent to which the significant difference is affected by the mean difference. when the eta squared effect size was found to be .01 then it was considered to be a small effect size; when it was found to be .06, then it was considered to be a medium effect size, and when it was found to be .1, then it was considered to be a large effect size (cohen, 1988). for the predictor analysis, the stepwise multiple regression analysis was used. while the study's categorical variables were included in the regression analysis by being coded as “dummy variables,” the continuous variables were included in the analysis with their original values. information about the dummy coding of all the variables included in the analysis is given in table 3. in the multiple regression analysis, the assumptions of a normal distribution, linearity, constant variance, absence of autocorrelation, and absence of multicollinearity between independent variables were tested (kalaycı, 2009). the relationships between standardized estimated values and non-standardized error values were examined with graphs (figure 1 and figure 2) for the normality and linearity assumptions. according to figure 1, it can be argued that the histogram established for the standardized predicted values and normal distribution curves shows a distribution converging to the normal, and according to figure 2, it can be argued that there is a linear and positive correlation between the variables. when the indicators of multicollinearity between the predictor variables were examined, tolerance values were found to be ranging from 0.337 to 1.00, and variance inflation factor (vif) values were found to be varying between 1.00 and 2.969, and the highest condition index (ci) value was found to be 8.47. according to pallant (2016), to be able to talk about multicollinearity problems in analysis, the vif value must be smaller than 10, and the tolerance value must be bigger than 0.10. thus, it can be figure 1 linearity distribution of stem figure 2 linearity distribution of stem table 2 mean scores and standard deviations for the whole scale and its sub-dimensions factors n min. max x̄ ss skewness kurtosis factor 1: mathematics 408 8.00 40.0 28.26 8.435 -.327 -.932 factor 2: science 408 11.00 45.00 32.38 7.987 -.121 -.809 factor 3: engineering and technology 408 11.00 45.00 32.22 7.753 -.330 -.538 factor 4: 21st century skills 408 15.00 55.00 43.08 8.128 -.667 .414 stem 408 71.00 185.000 135.95 23.398 -.059 .665 journal of science learning article doi: 10.17509/jsl.v4i2.28859 105 j.sci.learn.2021.4(2).101-112 said that there is no multicollinearity problem. the durbin watson value used to test autocorrelation means that the correlation between error terms is desired to be between 1.5 and 2.5 (kalaycı, 2009). the durbin watson value was found to be 1.78, which shows that there is no autocorrelation. the standardized residual value was found to be between -2.815 and 2.144. tabachnick & fidell (2013) stated that this value must be between +3.3 and 3.3. ‘mahalanobis distance values were found to be ranging from 2.963 and 13.261. this value is lower than 13.82, determined for the minimum independent variable number 13.82 (pallant, 2016). for cook’s distance, the max value is 0.022. this value is lower than 1, showing that the data is suitable for regression (tabachnick & fidell, 2013). 3. result and discussion 3.1 investigation of the attitudes towards stem education concerning gender independent samples t-test was used to investigate whether the scale and sub-dimensions' attitude scores vary significantly depending on gender. the results of this analysis are presented in table 4. only the mean attitude scores taken from the subdimension of engineering and technology were found to be varying significantly depending on gender in favor of the male participants (t=6,687; p<.05). the effect size of the sign was found to be high (eta square=.104). similarly, the mean attitude scores taken from the whole scale were found to be varying significantly depending on gender in favor of the male participants (t=3.816; p<.05). here, the effect size of the sign was found to be low (eta square=.035). 3.2 investigation of the attitudes towards stem education concerning grade level arithmetic means and standard deviations of the attitude scores taken from the scale of attitudes towards stem education were calculated to grade level. these results are presented in table 5. the mean scores decrease with increasing grade level. one-way variance analysis (anova) was conducted to determine whether mean attitude scores vary significantly depending on grade level, and the results of this analysis are presented in table 6. the students’ attitudes vary significantly depending on grade level in all sub-dimensions. then, to determine the source of the difference found with anova, complementary post-hoc analysis techniques were employed. table 3 coding of dummy variables categorical variables level dummy variable coding categories kept outside gender 1. male 2. female gender female:1 male:0 male grade level 1. 6th grade 2. 7th grade 3. 8th grade grade6 grade7 grade6:1 grade7:0 grade6:0 grade7:1 8th grade participation in out-ofschool social activities 1. yes 2. no ouf-of-schoolyes yes:1 no:0 no participation in in-school social activities 1. yes 2. no in-schoolyes yes:1 no:0 no math achievement 1. low 2. medium 3. high matmedium mathigh medium:1 high:0 medium:0 high:1 low science achievement 1. low 2. medium 3. high sciencemedium sciencehigh medium:1 high:0 medium:0 high:1 low table 4 results of t-test conducted to determine whether attitudes vary significantly depending on gender gender n x̄ ss sd t p eta square factor 1: mathematics male 206 29.03 8.907 406 1.866 .063 - female 202 27.48 7.870 factor 2: science male 206 32.98 8.207 406 1.512 .131 - female 202 31.78 7.731 factor 3: engineering and technology male 206 34.69 7.037 406 6.867 .000* .104 female 202 29.70 7.651 factor 4: 21st-century skills male 206 43.55 8.252 406 1.187 .236 - female 202 42.59 7.990 stem male 206 140.25 24.358 406 3.816 .000* .035 female 202 131.55 21.567 journal of science learning article doi: 10.17509/jsl.v4i2.28859 106 j.sci.learn.2021.4(2).101-112 after anova, to decide which post-hoc multicomparison technique to use, first levene’s test was used to test the hypothesis of the homogeneity of the variances. after it was determined that the variances are homogenous, scheffe's multi-comparison technique was used. according to the test results, significant differences were found between the 6th graders and 8th graders in the subdimensions of mathematics, science, engineering and technology, and 21st-century skills and the whole scale in favor of the 6th graders (p<.05). while in the subdimension of mathematics, the effect size was large (eta square = .135), it was found to be medium in the whole scale (eta square = .079) and was found to be small in the other sub-dimensions. 3.3 investigation of the attitudes towards stem education concerning participation in out-of-school social activities t-test was run to determine whether the students’ stem attitude scores vary significantly depending on participation in out-of-school activities. the results of this analysis are presented in table 7. the mean attitude scores taken from all the sub-dimensions except for the mathematics sub-dimension and the mean attitude scores taken from the whole scale were found to be varying significantly depending on participation in out-of-school social activities in favor of those participating in these activities (p<.05) and the effect sizes of these differences are small. 3.4 investigation of the attitudes towards stem education concerning participation in in-school social activities t-test was run to determine whether the students’ stem attitude scores vary significantly depending on participation in in-school activities. the results of this analysis are presented in table 8. the mean attitude scores taken from the sub-dimension of 21st-century skills and the whole scale vary significantly depending on participation in in-school social activities in favor of those participating in these activities (p<.05) and the effect sizes of these differences are small. 3.5 investigation of the attitudes towards stem education concerning achievement arithmetic means and standard deviations of the attitude scores taken from the scale of attitudes towards stem education were calculated in relation to achievement. these results are presented in table 9. the mean scores vary significantly in the whole scale and the sub-dimensions of the scale. one-way variance analysis (anova) was conducted to determine whether there is a significant difference between the mean scores. the results of this analysis are presented in table 10. the mean attitude scores taken from the sub-dimensions of the attitude scale except for the engineering and technology sub-dimension vary significantly depending on mathematics achievement. to determine the source of the significant differences, table 5 frequencies, mean scores and standard deviations to grade level grade level n mathematics science eng.&tec. 21st cent. skills stem x̄ ss x̄ ss x̄ ss x̄ ss x̄ ss 6th grade 173 31.56 7.04 33.04 8.32 33.65 7.30 44.71 7.82 142.97 21.94 7th grade 107 27.65 8.17 32.38 7.71 31.67 8.00 42.78 8.29 134.50 23.46 8th grade 128 24.32 8.63 31.50 7.72 30.74 7.86 41.10 7.99 127.66 22.48 table 6 results of the variance analysis (anova) conducted to determine whether attitudes vary significantly depending on grade level sum of squares sd mean square f p a significant difference (tukey) eta square m a th between-groups 3910.72 2 1955.36 31.61 .000* 6>7, 6>8 .135 within-groups 25048.69 405 61.85 7>8 total 28959.41 407 s c ie n c e between-groups 174.58 2 87.29 1.37 .255 - within-groups 25790.01 405 63.68 total 25964.59 407 e n g .& t e c . between-groups 666.91 2 333.46 5.68 .004* 6>8 .027 within-groups 23795.23 405 58.75 total 24462.15 407 2 1s t c e n . s k il ls between-groups 973.787 2 486.894 7.610 .001* 6>8 .036 within-groups 25910.857 405 63.977 total 26884.645 407 s t e m between-groups 17542.656 2 8771.328 17.305 .000* 6>7, 6>8 .079 within-groups 205286.158 405 506.879 total 222828.814 407 journal of science learning article doi: 10.17509/jsl.v4i2.28859 107 j.sci.learn.2021.4(2).101-112 complementary post-hoc analysis techniques were employed. after anova, to decide which post-hoc multicomparison technique to use, first levene’s test was used to test the hypothesis of the homogeneity of the variances. the scheffe multi-comparison technique was used for the sub-dimensions of science and 21st-century skills. moreover, the variances are homogenous. the dunnet-c multi-comparison technique was used for the subdimension of mathematics. as a result, the whole scale where the variances are not homogenous. the obtained findings showed that in all the sub-dimensions and the whole scale, the students with high and medium achievement have significantly higher attitude scores than those with low achievement (p<.05). only in the subdimension of mathematics and on the whol were scale were significant dbetween the students having high achievement and medium achievement (p<.05). when the effect sizes were examined, effect sizes in the sub-dimension of mathematics (eta square = .22) and the whole scale (eta square = .10) were found to be medium, while in all the other sub-dimensions, the effect sizes were found to below. one-way variance analysis (anova) was run to determine whether the participants’ stem attitude scores vary significantly depending on science achievement, and the results of this analysis are presented in table 11. the students’ attitudes towards stem vary significantly depending on science achievement in all the subdimensions except for the sub-dimension of engineering and technology. to determine the source of the significant table 7 results of the t-test conducted to determine whether attitude scores vary significantly depending on participation in outof-school activities participation n x̄ ss sd t p eta square factor 1: mathematics yes 193 28.49 8.40 406 .516 .606 - no 215 28.06 8.48 factor 2: science yes 193 33.28 7.49 406 2.154 .032* .011 no 215 31.58 8.35 factor 3: engineering and technology yes 193 33.24 7.23 406 2.542 .011* .016 no 215 31.30 8.10 factor 4: 21st century skills yes 193 44.22 7.72 406 2.709 .007* .018 no 215 42.05 8.37 stem yes 193 139.23 22.418 406 2.709 .007* .018 no 215 132.99 23.914 table 8 results of the t-test conducted to determine whether attitude scores vary significantly depending on participation in inschool activities participation n x̄ ss sd t p eta square factor 1: mathematics yes 156 29.27 7.93 406 1.899 .058 - no 252 27.64 8.69 factor 2: science yes 156 32.41 7.91 406 .051 .960 - no 252 32.37 8.05 factor 3: engineering and technology yes 156 33.11 7.44 406 1.826 .069 …. no 252 31.67 7.91 factor 4: 21st century skills yes 156 44.12 7.96 406 2.040 .042* .01 no 252 42.43 8.18 stem yes 156 138.90 22.70 406 2.016 .044* .01 no 252 134.12 23.68 table 9 frequencies, mean scores and standard deviations to mathematics and science achievement academic achievement n mathematics science eng.&tec. 21st cent. skills stem x̄ ss x̄ ss x̄ ss x ̄ ss x̄ ss math. low 60 19.35 6.21 29.02 6.54 31.07 6.20 39.75 7.75 119.18 15.61 medium 180 28.24 8.19 32.16 7.86 32.50 7.86 42.93 7.61 135.83 22.37 high 168 31.48 7.00 33.83 8.24 32.33 8.13 44.42 8.48 142.06 23.95 science low 40 22.30 7.96 28.18 6.02 31.35 6.91 40.00 8.55 121.83 18.34 medium 132 26.07 8.63 29.92 7.21 31.80 7.17 41.64 7.59 129.43 20.99 high 236 30.50 7.57 34.47 8.04 32.60 8.20 44.40 8.10 141.98 23.50 journal of science learning article doi: 10.17509/jsl.v4i2.28859 108 j.sci.learn.2021.4(2).101-112 differences found in anova, complementary post-hoc tests were initiated. after anova, to decide which post-hoc multicomparison technique to use, first levene’s test was used to test the hypothesis of the homogeneity of the variances. scheffe multi-comparison technique was used for the subdimensions of mathematics and 21st-century skills where the variances are homogenous, and the dunnet-c multicomparison technique was used for the sub-dimension of science and the whole scale where the variances are not homogenous. the obtained findings showed that in all the sub-dimensions and the whole scale, the students with high achievement have significantly higher attitude scores than those who have a medium and low achievement (p<.05). only in the sub-dimension of mathematics, a significant difference was found between the students having medium achievement and low achievement (p<.05). in all the subdimensions, effect sizes were found to be small. 3.6 investigation of the predictors of the attitudes towards stem education the data obtained regarding the variables investigated as the factors affecting the attitudes towards stem education (gender, grade level, participation in out-ofschool and in-school social activities, math achievement, and science achievement) were subjected to multiple regression analysis by using the stepwise model, and the results are presented in table 12. as can be seen in table 12, the stepwise regression analysis did not include the variables of participation in out-of-school and in-school social activities and math achievement as they did not predict significantly and thus, only the variables of gender, grade level, and science achievement was included in the stepwise regression analysis. as shown in table 12, when the predictor variables were included in the regression analysis stepwise, five models were formed. in the first model, while the high science achievement of the student is a predictor variable, in the second model, the 6th-grade variable is added, in the third model female, in the fourth model 7th grade and the fifth model medium level for science achievement are added, and these predictor variables were found to be explaining 21% (r²= .211) of the variance in the predicted variable. the model constructed to explain the attitude scores taken from stem is significant at the level α=.05 (f=21.470, p<.05). according to the standardized regression coefficient in the fifth model, which explains the variance in the predicted variable to the greatest extent, from among the predictor variables, science achievement is the variable making the greatest contribution to the stem attitude scores (β=.45), followed by the 6th-grade level variable (β=.30), the female variable (β=-.19), the medium level science achievement variable (β=.17) and the 7thgrade level variable (β=.13). the regression equation concerning the prediction of the stem attitude according table 10 results of the one-way variance analysis (anova) conducted to determine whether the attitude scores vary significantly depending on mathematics achievement sum of squares sd mean square f p a significant difference (tukey) eta square m a th . between-groups 6501.12 2 3250.57 58.62 .000 high>medium high>low medium>low 0.22 within-groups 22458.28 405 55.45 total 28959.41 407 s c ie n c e between-groups 1039.28 2 519.64 8.44 .000 high>low medium>low 0.04 within-groups 24925.30 405 61.54 total 25964.58 407 e n g .& t e c . between-groups 96.080 2 48.04 .80 .451 within-groups 24366.06 405 60.16 total 2446.14 407 2 1s t c e n . s k il ls between-groups 972.33 2 486.17 7.60 .001 high>low medium>low 0.04 within-groups 25912.30 405 63.98 total 26884.64 407 s t e m between-groups 23140.76 2 11570.38 2.47 .000 high>medium high>low medium>low 0.10 within-groups 199688.05 405 493.06 total 222828.81 407 journal of science learning article doi: 10.17509/jsl.v4i2.28859 109 j.sci.learn.2021.4(2).101-112 to the fifth model of the stepwise multiple regression analysis is as follows: stem attitude= 117.38+21.41sciencehigh+14.07grade6+8.92female+6.92gra de7+8.70sciencemedium it is known that there are gender differences in the choice of profession related to stem fields and, therefore, in the education given in stem fields at universities. in the current study, the attitudes towards stem education in general and the sub-dimension of engineering and technology were found to be varying significantly depending on gender in favor of the male students. this shows that this difference in individuals' attitudes towards stems starts to emerge in middle school years. parallel to these findings, mahoney (2009) and unfried, faber, stanhope, & wiebe (2015) found that females have weaker attitudes towards the sub-dimension of engineering and technology in stem. on the other hand, there is a limited amount of research showing that stem attitudes do not vary significantly depending on gender (aydın, saka, & guzey 2017; sivrikaya, 2019). the fact that attitudes towards stem start to vary significantly in favor of males in middle school years shows that works should be carried out for improving students’ attitudes towards stem at early ages. in this connection, in their study, gündüz & tarhan (2017) found that 4th-grade students believe that women should have a job, yet they think that some professions are not for women. this provides clues showing that the main reason for gender-based differences in attitudes towards stem is the gender perception of society. in the current study, the middle school students’ attitudes towards stem education were varying significantly depending on grade level in the subdimensions of mathematics, engineering and technology, 21st-century skills, and the whole scale in favor of the 6th graders. in the studies comparing stem attitudes of students from different grades, it has been concluded that the attitude scores of the lower grades are higher than those of the upper grades (aydın, saka & guzey, 2017; lamb, akmal, & petrie, 2015; mahoney, 2009; unfried, faber, stanhope, & wiebe, 2015). one reason for this difference in turkey can be the centralized higher school entrance exam taken by the 8th graders. in this connection, çetin & ünsal (2018) stated that due to the centralized exams, teachers do not implement all the dimensions of the curriculum, they determine their objectives and contents based on these exams, and they tend to deliver their classes through lecturing and to give tests to students. as a result, students can be distanced from stem activities. another issue is that in the centralized exams, students' anxiety to perform poorly in the science and math sections of these exams may cause them to feel alienated from these courses. it is necessary to work on the centralized exams used for measurement and evaluation purposes to reduce their negative effects on stem attitudes. 21st-century students are expected to be versatile and have some skills such as accessing scientific knowledge, producing a scientific product, and thinking analytically (mone, 2018). the development of these skills is not possible only through academic development. in this table 11 results of the one-way variance analysis (anova) conducted to determine whether the attitude scores vary significantly depending on science achievement sum of squares sd mean square f p a significant difference (tukey) eta square m a th . between-groups 3243.63 2.00 1621.81 25.54 .00 high>medium high>low medium>low 0.11 within-groups 25715.78 405.00 63.50 total 28959.41 407.00 s c ie n c e between-groups 2538.72 2.00 1269.36 21.95 .00 high>medium high>low 0.10 within-groups 23425.86 405.00 57.84 total 25964.59 407.00 e n g . & t e c . between-groups 87.61 2.00 43.80 .73 .48 within-groups 24374.54 405.00 60.18 total 24462.15 407.00 2 1s t c e n . s k il ls between-groups 1067.34 2.00 533.67 8.37 .00 high>medium high>low 0.04 within-groups 25817.30 405.00 63.75 total 26884.64 407.00 s t e m between-groups 22178.72 2.00 11089.36 22.38 .00 high>medium high>low 0.10 within-groups 200650.09 405.00 495.43 total 222828.81 407.00 journal of science learning article doi: 10.17509/jsl.v4i2.28859 110 j.sci.learn.2021.4(2).101-112 regard, the findings of the current study show that the attitude scores of the students participating in out-ofschool and in-school social activities are higher in the subdimension of 21st-century skills and the whole scale. similarly, yoder (2019) stated that co-curricular activities positively affected students' 21st-century skills development. also, dabney et al. (2012) also found a statistically significant relationship between students' participation in school clubs, which can be seen within the scope of social activities and their career interests in stem professions. participation in out-of-school and in-school social activities was found to affect the attitudes towards stem significantly. thus, students should be encouraged to participate in out-of-school and in-school social activities. educators should provide such opportunities for students and should prepare the required settings. in the current study, attitudes of the students who have higher math and science achievement towards stem were found to be more positive. parallel to this, dabney et al. (2012) stated that the students whose science and math grades are higher are more interested in pursuing careers in professions related to science, technology, engineering, and mathematics. in the prediction of stem attitudes, not mathematics, but science achievement was found to be effective. the science courses being more effective in predicting stem attitudes can be because stem activities are done by establishing more connections with science classes' content and activities. in the longitudinal study conducted by ing (2014), it was determined that mathematics and science achievement from 7th grade to 12th grade are related to stem career attainment. another important finding of the current study is that 21% of the participating students’ stem attitude scores are predicted by the variables of science achievement, being a 6th grader, being female, and having medium science achievement. the three most influential factors in predicting stem attitude scores were science achievement, being a 6th grader, and being female. these three factors explain 19% of the variance in the students’ attitudes towards stem. here, being female is in a negative correlation with the attitudes towards stem. 4. conclusion the resulting study showed that the attitudes towards stem education were found to be varying significantly only in the sub-dimension of engineering and technology depending on gender in favor of the male students. significant differences were found in stem education attitudes in favor of the student’s sixth-graders with participating students in in-school and out-of-school social activities. the students who have higher math and science achievement, attitudes towards stem education were found to be more positive. the three predictors making the most significant contribution to the students’ stem table 12 results of the stepwise multiple regression analysis concerning the variables predicting the attitudes towards stem modelpredictor variables b std. error beta t sig partial (r) part (r) r r2 f p m o d e l 1 constant 127.66 1.70 74.98 .00 .303 .092 40.921 .000 science high 14.32 2.24 .30 6.40 .00 .30 .30 m o d e l 2 constant 122.74 1.87 65.63 .00 .394 .155 37.177 .000 science high 14.09 2.16 .30 6.52 .00 .31 .30 grade6 11.92 2.16 .25 5.52 .00 .26 .25 m o d e l 3 constant 127.04 2.11 60.34 .00 .436 .190 31.532 .000 science high 14.64 2.12 .31 6.89 .00 .32 .31 grade6 11.24 2.12 .24 5.29 .00 .25 .24 female -8.74 2.10 -.19 -4.15 .00 -.20 -.19 m o d e l 4 constant 123.93 2.48 49.95 .00 .448 .201 25.272 .000 science high 14.69 2.11 .31 6.95 .00 .33 .31 grade6 14.19 2.46 .30 5.76 .00 .28 .26 female -8.46 2.10 -.18 -4.04 .00 -.20 -.18 grade7 6.44 2.76 .12 2.33 .02 .12 .10 m o d e l 5 constant 117.38 3.78 31.02 .00 .459 .211 21.470 .000 science high 21.41 3.62 .45 5.92 .00 .28 .26 grade6 14.07 2.45 .30 5.74 .00 .28 .25 female -8.92 2.09 -.19 -4.26 .00 -.21 -.19 grade7 6.92 2.75 .13 2.51 .01 .12 .11 science medium 8.702 3.814 .17 2.282 .023 .113 .101 journal of science learning article doi: 10.17509/jsl.v4i2.28859 111 j.sci.learn.2021.4(2).101-112 attitude scores are descending order: high science achievement, being a 6th grader and being female. however, being female is in a negative correlation with the attitudes towards stem education. in this context, education policies should be developed by considering women's sociological factors to turn to stem. the reasons for upper-grade students’ negative attitudes towards stem should be investigated, and taken the necessary measures in this direction. the positive effects of these activities on stem can be utilized by increasing student participation in in-school and out-of-school social activities. in light of the results, while constructing the stem career plans of students, the students' science achievement, which was found to be a strong predictor of the attitudes towards stem, can be taken into consideration by teachers. references abell, s. k., & lederman, n. g. (2007). handbook of research on science education. mahwah, nj: lawrence erlbaum associates. akgündüz, d., aydeniz, m., çakmakçı, g., çavaş, b., çorlu, m. s., öner, t. & özdemir, s. (2015). stem eğitimi türkiye raporu. akgündüz, d. ve ertepınar, h. (ed). i̇stanbul: scala basım. aydın, g., saka, m., & guzey, s. (2017). science, technology, engineering, mathematic (stem) attitude levels in grades 4th – 8 th. mersin university journal of the faculty of education, 13(2), 787-802. doi: http://dx.doi.org/10.17860/mersinefd.290319. bybee, r. w. (2010a). advancing stem education: a 2020 vision. technology and engineering teacher, 70, 30-35. bybee, r.w. (2010b). what is stem education?. science, 329,966. doi. 10.1126/science.1194998. bybee, r. w. (2013). the case for stem education: challenges and opportunities. arlington, va: national science teachers. cohen, j. (1988). statistical power analysis for the behavioral sciences. hilsdale. nj: lawrence earlbaum associates. cohen, l., manion, l., & morrison k. (2018). research methods in education. (eighth edition). london:routledge çetin, a., & ünsal, s. (2019). social, psychological effects of central examinations on teachers and their reflections on teachers' curriculum implementations. hacettepe university journal of education, 34(2), 304-323. doi: 10.16986/huje.2018040672 çorlu, m. s. (2014). fetemm eğitimi makale çağrı mektubu [call for stem education research in the turkish context]. turkish journal of education, 3(1), 4-10. dabney, k. p., tai, r. h., almarode, j. t., miller-friedmann, j. l., sonnert, g., sadler, p. m., & hazari, z. (2012). out-of-school time science activities and their association with career interest in stem. international journal of science education, part b: communication and public engagement, 2(1), 63-79. ejiwale, j. a. (2012). facilitating teaching and learning across stem fields. journal of stem education: innovations and research, 13(3), 87. fan, s-c., & ritz, j. (2014). international views on stem education. http://www.iteea.org/conference/patt/patt28/fan%20ritz .pdf sayfasından erişilmiştir. fraenkel, j.r & wallen, n.e (2009). how to design and evaluate research in education (7th ed). new york. mcgraw-hill. friday institute for educational innovation (2012). middle and high school stem-student survey. raleigh, nc: author. gonzalez, h. b., & kuenzi, j. j. (2012). science, technology, engineering, and mathematics (stem) education: a primer. congressional research service, library of congress. gülhan, f., & şahin, f. (2016). the effects of science-technologyengineering-math (stem) integration on 5th grade students’ perceptions and attitudes towards these areas. journal of human sciences, 13(1), 602-620. gündüz, h. ç., & tarhan, s. (2017). the ımpact of gender on primary school students’ attitudes towards choosing a profession. elementary education online, 16(3), ss1287-1300 doi: 10.17051/ilkonline.2017.330258 ing, m. (2014). gender differences in the influence of early perceived parental support on student mathematics and science achievement and stem career attainment. international journal of science and mathematics education, (2014)12,1221-1239. jon, j. e., & chung, h. i. (2013). consultant report securing australia’s future stem: country comparisons-stem report republic of korea. retrieved jul 02, 2020 from https://acola.org/wpcontent/uploads/2018/12/consultant-report-korea.pdf kalaycı, ş. (2009). spss applied multivariate statistical techniques [spss uygulamalı çok değişkenli istatistik teknikleri]. ankara: asil yayıncılık. karakaya, f., & avgın, s. s. (2016). effect of demographic features to middle school students’ attitude towards fetemm (stem). journal of human sciences, 13(3), 4188-4198. lamb, r., akmal, t., & petrie, k. (2015). development of a cognition‐ priming model describing learning in a stem classroom. journal of research in science teaching, 52(3), 410-437. doi: 10.1002/tea.21200 mahoney, m. p. (2009). student attitude toward stem: development of an instrument for high school stem-based programs. unpublished phd thesis, the ohio state university. ministry of national education (mone, 2016) stem education report directorate general for innovation and educational technologies. [stem eğitimi raporu. yenilik ve eğitim teknolojileri genel müdürlüğü]. retrieved from http://yegitek.meb.gov.tr/stem_egitimi_raporu.pdf ministry of national education (mone, 2018) science curriculum, (primary and secondary school 3, 4, 5, 6, 7 and 8. grades).[ fen bilimleri dersi öğretim programı, (i̇lkokul ve ortaokul 3, 4, 5, 6, 7 ve 8. sınıflar]. retrieved from http://mufredat.meb.gov.tr/programdetay.aspx?pid=325 morgan, g. a., leech, n. l., gloeckner, g. w., & barrett, k. c. (2004). spss for introductory statistics: use and interpretation. psychology press. oecd (2007). pisa 2006: science competencies for tomorrow’s world: volume 1: analysis. oecd publishing: paris. http://dx.doi. org/10.1787/9789264040014-en. oecd (2018). strengthening education for innovation. retrieved jul 02, 2020 from https://www.oecd.org/media/oecdorg/satellitesites/stie outlook/files/policyprofile/sti%20outlook%2012_%20pp%20 hr_education.pdf. oral, i. & mcgivney, e. (2011). the determinants of student performance and achievement in math and science fields in turkey [türkiye’de matematik ve fen bilimleri alanlarında öğrenci performansı ve başarının belirleyiciler]i. i̇stanbul: eğitim reformu girişimi. özcan, h. & koca, e. (2018). turkish adaptation of the attitude towards stem scale: a validity and reliability study. h. u. journal of education ,34(2),ss:384-401 doi: 10.16986/huje.2018045061. özkurt sivrikaya, s. (2019). research of high school students’ attitudes of stem. international journal of society researches 11(18), 914-934. doi: 10.26466/opus.547459 pallant, j. (2016). spss survival manual: a step by step guide to data analysis using ibm spss (6th edition). sydney: allen & unwin sanders, m. (2009). stem, stem education, stemmania. the technology teacher, 68(4), 20-26. smith, j. & karr-kidwell, p. (2000). the interdisciplinary curriculum: a literary review and a manual for administrators and teachers. http://files.eric.ed.gov/fulltext/ed443172.pdf sayfasından erişilmiştir. tabachnick, b.g., & fidell, l. s (2013). using multivariate statistics (6th edition).boston: pearson education turner, k. (2013). northeast tennessee educators’ perception of stem education implementation. doctoral dissertation. https://www.dc.etsu.edu/etd/ sayfasından erişilmiştir. unesco (2017). cracking the code: girls’ and women’s education in science, technology, engineering and mathematics (stem). retrieved jul 02, 2020 from https://unesdoc.unesco.org/ark:/48223/pf0000253479. https://acola.org/wp-content/uploads/2018/12/consultant-report-korea.pdf https://acola.org/wp-content/uploads/2018/12/consultant-report-korea.pdf http://yegitek.meb.gov.tr/stem_egitimi_raporu.pdf https://www.oecd.org/media/oecdorg/satellitesites/stie https://unesdoc.unesco.org/ark:/48223/pf0000253479 journal of science learning article doi: 10.17509/jsl.v4i2.28859 112 j.sci.learn.2021.4(2).101-112 unfried, a., faber, m., & wiebe, e. (2014). gender and student attitudes toward science, technology, engineering, and mathematics. the friday institute for educational innovation at north carolina state university retrieved jul 02, 2020 from https://miso.fi.ncsu.edu/wpcontent/uploads/2014/08/aera-2014-paper-student-attitudestoward-stem.pdf unfried, a., faber, m., stanhope, d. s., & wiebe, e. (2015). the development and validation of a measure of student attitudes toward science, technology, engineering, and math (sstem). journal of psychoeducational assessment, 33(7), 622639. doi: 0734282915571160. url-1: https://www.rmkmarine.com.tr/bal-arilari-muhendis-oluyorprojesi.html west, m. (2012). stem education and the workplace, occasional paper series, (4),4. yıldırım, b. (2016). an examination of the effects of science, technology, engineering, mathematics (stem) applications and mastery learning integrated into the 7th grade science course. dissertation, instıtue of educational sciences, gazi university ankara. https://tez.yok.gov.tr/ulusaltezmerkezi/tezsorgusonucyeni.jsp yoder, b. (2019). instructor perceptions of extracurricular and co-curricular activities on developing 21st century soft skills in students. doctoral dissertation, southwest baptist university. https://www.rmkmarine.com.tr/bal-arilari-muhendis-oluyor-projesi.html https://www.rmkmarine.com.tr/bal-arilari-muhendis-oluyor-projesi.html a © 2020 indonesian society for science educator 50 j.sci.learn.2020.4(1).50-60 received: 21 march 2020 revised: 30 september 2020 published: 28 november 2020 analysis of pre-service science teachers’ heuristic reasoning processes about hydrogen bonding gülen önal karakoyun1, erol asiltürk2* 1program of chemistry technology, muradiye vocational high school, vanyüzüncü yıl university, muradiye, van, turkey 2department of science education, faculty of education, firat university, elaziğ, turkey *corresponding author: ecil@firat.edu.tr abstract the purpose of this research was to determine the heuristics used by pre-service science teachers in understanding the details of hydrogen bonding. the reasoning processes demonstrated were evaluated based on ten heuristic models suggested by talanquer (2014). phenomenographic assessment of the 30 participants indicated that all ten heuristics were utilized to make interpretations about hydrogen bonding. it was found that most students used short-cut strategies rather than efficient analytical reasoning processes. a total of 12 answer patterns were determined based on the answers of the participants. the percentage of students who gave the correct answer was low. the frequency sequencing of participants' heuristics demonstrated in this study was fluency, associative activation, recognition, one-reason decision making, attribute substitution, overconfidence, surface similarity, generalization, rigidity, and affect. keywords science education, hydrogen bonding, educational psychology, heuristics, chemistry education 1. introduction one of the aims of science educators is to raise individuals who can use information effectively and logically (vekli, 2020). in order to understand how individuals use information, reasoning processes should be examined in detail. many researchers have revealed that students have many mistakes and biases in reasoning processes related to science and chemistry topics (ugras, 2018). one of these science/chemistry subjects that are incompletely or incorrectly interpreted by students is the subject of hydrogen bonding. hydrogen bonding is a type of interaction between particles, and it has an essential place in chemistry, biochemistry, biology, and molecular biology. the effects of this bonding type are evident in many biotic phenomena. for instance, two dna strains are connected through hydrogen bonding. antibody and antigen bonding are also based on hydrogen bonding. hydrogen bonding is efficient in the duplication process called transcription in protein synthesis. water, which has a vital role in organisms, and many sugar molecules have hydrogen bonding. students need to have correct and robust insight about hydrogen bonding to grasp and interpret different types of biological phenomena which are of vital importance to organisms; they need to know the characteristics of hydrogen bonding to analyze different chemical phenomena such as boiling and dissolution as hydrogen bonding plays a significant role in these processes. researches on students’ perceptions about hydrogen bonding indicate that they generally don’t understand the process accurately and have a lot of inadequacies about the subject (barker & millar, 2000; cooper, williams & underwood, 2015). it is important and crucial to make indepth analyses about the reasoning processes of students in science subjects; it will thus be possible to create and improve strategies that remove their misconceptions not only about the issue of hydrogen bonding but also other significant subjects in science. science educators who want to research and understand the nature of mistakes and biases in student reasoning processes on science subjects have recently started to show interest in various disciplines. there are many kinds of studies in various disciplines such as developmental psychology and cognitive science, which analyze individuals’ judgment and decision-making processes in daily and social life from different perspectives (connor & mailto:ecil@firat.edu.tr journal of science learning article doi: 10.17509/jsl.v4i1.23737 51 j.sci.learn.2020.4(1).50-60 becker, 2003; ferrari & dovidio, 2000; hastie, 2001; schwarz, 2004). based on these multi-dimensional research results, different theories are established to explain individuals’ judgment and decision-making processes (gilovich, griffin & kahneman, 2002; todd & gigerenzer, 2000; ugras, 2018). one of these theories is the “dual process” theory, which involves essential information about individuals’ intuitive judgment and decision-making (evans, 2008; evans & stanovich, 2013; gigerenzer & goldstein, 1996; ugras, 2018). this model's essential purpose is to explain individuals’ reasoning about daily life, and it has recently become a frequently-used method in explaining students' reasoning about scientific issues. according to the dual-process theory, humans have two distinct types of reasoning processes: type 1 and type 2. type 1 processes are independent of cognitive ability and involve reasoning forms that move very fast without giving importance to working memory (ugras, 2018). as there is no need for a specific effort to trigger type 1 processes, they automatically occur (stanovich & west, 2000; ugras, 2018). type 1 processes are autonomous, involving learned strategies, and naturally formed reasoning processes (stanovich & west, 2000). type 1 processes are about intuitive judgments. on the other hand, working memory is employed in type 2 processes, which work slowly and successively (talanquer, 2014). practicing type 2 processes requires extraordinary effort and deliberate intervention; these processes are about individuals’ hypothetical, analytical, or reflective thinking ways (maeyer, 2013; ugras, 2018). studies reveal that when an individual faces a new problem or situation, type 1 processes are instantly triggered (evans, 2008; graulich, 2014; mcclary & talanquer, 2011). type 1 processes are responsible for various biases formed during reasoning processes (ugras, 2018). type 1 processes are considered short-cut reasoning strategies and are called “heuristics” (graulich, 2014; ugras, 2018). as heuristics shorten the information processing path, they ensure individuals to make decisions sooner than expected; they set some implicit rules about reasoning processes intended for solving a problem, and thus ease individuals’ reasoning (todd & gigerenzer, 2000). since heuristics are unique, fast, and frugal mental structures, individuals can make rational decisions when used heuristics consciously to solve problems they encounter in science topics. thus, examining how students use heuristics in science subjects and researching methods that will enable them to use heuristics consciously will be beneficial in increasing the quality of science education. the studies on the use of heuristics in the subjects of chemistry are relatively recent. students’ use of heuristic on the subjects of addition reactions, elimination reactions, chemical problem solving, acidity strength of molecules, chemical reactivity, chemical bonding theories, classification of chemical substances, structure-property relationships of molecules, and interpretation of ir and hnmr spectra have been investigated thoroughly until today (cooper, corley & underwood, 2013; connor, finkenstaedt-quinn & shultz, 2019; graulich, hopf & schreiner, 2011a; graulich, hopf & schreiner, 2011b; maeyer, 2013; maeyer & talanquer, 2013; mcclary & talanquer, 2011; ugras, 2018). these studies reveal that students correctly and incorrectly answered questions without using the basic and important chemical knowledge because of the effects of heuristics. any kind of model (such as the ten heuristics model suggested by talanquer) or standard was not ever used in the researches mentioned above about chemistry subjects. researchers independently determined and named heuristics in each of those studies. in addition to these critical studies, talanquer has explained the frequently used heuristics in chemistry according to the employed cognitive processes and made an essential contribution to the literature by grouping these different heuristics under ten categories (talanquer, 2014). the model of talanquer is highly significant, and it provides a reference point for future studies in the field of chemistry. for instance, in miller and kim’s study, on the use of heuristics by students in hydrogen bonding, the model proposed by talanquer was used as the basis (miller & kim, 2017). this research by miller and kim is the first and the only study that explains the use of heuristics in chemistry subjects according to the model proposed by talanquer. as they did not use the interview technique in their research, they could examine only six of the ten heuristics. however, in the current study designed meticulously, interviews were conducted with students to make detailed analyses about the effects of 10 different heuristics in students’ reasoning processes about hydrogen bonding. according to talanquer’s model, ten heuristics can be efficient in students’ reasoning processes in chemistry subjects. these heuristics are associative activation, fluency, attribute substitution, one-reason decision making, surface similarity, recognition, generalization, rigidity, overconfidence, and affect (talanquer, 2014). in his theoretical study, talanquer explained each of these ten heuristics and their specific examples of the chemistry field (talanquer, 2014). the purpose of this study is to determine and explain the heuristics used by pre-service science teachers in the subject of hydrogen bonding; the study is based on the model suggested by talanquer (2014). journal of science learning article doi: 10.17509/jsl.v4i1.23737 52 j.sci.learn.2020.4(1).50-60 2. method 2.1. sample this study was conducted during the spring semester in the 2018-2019 academic year in turkey's public university. a total of 30 teacher candidates at 2nd, 3rd and 4th grades in the faculty of education science teaching program have voluntarily participated in the research. sixteen of the participants were male, while 14 were female teacher candidates. success rates of students in general chemistry i and general chemistry ii were considered while determining the ones to participate in the study. one-third of the participants were unsuccessful; one-third were midlevel successful, while one-third were highly successful in these specific classes. the participants' actual names were not used in the study; they were encoded as s1, s2, s3, and s4… 2.2. research methods, instruments and procedure of interview protocol development in this study, phenomenographic research methodology, one of the qualitative research methods, was preferred to investigate students' heuristic use in the subject of hydrogen bonding. phenomenography, used in educational researches for a long time, is defined as: “an empirical study whose aim is to discover the quantitatively different ways in which people experience, realize and understand various aspects of a phenomenon” (akerlind, 2005; didis, ozcan & abak, 2008). phenomenography is a widely accepted methodology in education research. it is used for representing the differences in the understandings of individuals and how they perceive a single concept in different ways (didis, ozcan & abak, 2008; wihlborg, 2004). generally, interviews are done for gathering detailed information in phenomenographic research studies; this is why interviews are held with participants to accurately determine their reasoning about hydrogen bonding and identify the heuristics employed in this process. students were required to answer a question developed by miller and kim (2017) containing the lewis demonstration of acetic acid. the structural demonstration of acetic acid is presented in figure 1. each atom in the molecule is numbered explicitly in the question involving the demonstration of acetic acid’s structure. students were asked to answer the question of “with which of the numbered molecules can the water molecule form the hydrogen bonding?” miller and kim define this question type as a multiple-select format assessment item that requires the ‘selection of all valid choices’ (miller & kim, 2017). during the interviews of this research, students were given 2 minutes to answer the question. in the relevant literature, it is stated that the impacts of intuitive judgment and decision-making are higher in the states where the time is limited (mcclary & talanquer, 2011); for this reason, the time given for this step during the interviews was limited. then, the participants were required to explain the reasons for their answers in detail. time-limitation was not conducted at this stage of the process. some additional questions were asked to the participants before and after they answer the question; the purpose of these questions was to determine if all of the heuristics suggested by talanquer, especially ‘rigidity, overconfidence and affect’ heuristics, took part in the answering process. interviews were recorded both audio-visually, and then these records were transcribed verbatim; interview transcripts for each student were prepared. heuristic reasoning has been identified and encoded based on the analyses of interview transcripts. previous research studies that contain interview procedures were also considered during the encoding process of the current study (miller & kim, 2017; ugras, 2018). to ensure the inter-rater reliability, 8 randomly chosen interview transcripts (approximately 25%) were evaluated separately by the researcher and the consultant, and encodings were completed. results obtained from the two raters were compared, and encodings were revised to ensure 90% coherence. after catching the coherence level, all of the remaining interview transcripts were evaluated and encoded by herself. ten heuristics suggested by talanquer (2014) were used to create an encoding scheme about heuristics. the heuristics encodings, except rigidity, overconfidence, and affect, were carried out by associating students' specific statements about the solution of the question with heuristics. specific student statements that are the basis of encodings are presented in the results section. the procedure, explained below in detail, was followed to identify and encode the heuristics about rigidity, overconfidence, and affect heuristics. rigidity: the steps followed for researching rigidity heuristic in this study started with asking the participants a few questions before directing the question about acetic acid. the students were required to answer the questions: “do you have an absolute judgment/bias about the subject of hydrogen bonding?” “for instance, do you have any approaches such as: in the subject of hydrogen bonding, i c o o h ch h h 1 2 3 4 5 6 7 8 figure 1 structural demonstration of acetic acid. each atom in the structure is specifically numbered journal of science learning article doi: 10.17509/jsl.v4i1.23737 53 j.sci.learn.2020.4(1).50-60 have some specific judgments/reasoning that i will never change whatever the question is. i always make evaluations and solve problems by using these judgments/reasoning processes.” answers of participants to this question were carefully analyzed. on the other hand, it was carefully analyzed during interviews if participants actually used their previous strategies and solved the problem accordingly, and if they were flexible while answering questions. three specific statements were determined and encoded as the indicators of rigidity heuristics; if participants solved the problem based on the encoded statements they mentioned and had no flexibility, they would fit into this category. statements encoded as rigidity heuristics were: “no matter what the question about hydrogen bonding is, i will think that f, o, n and hydrogen atoms form hydrogen bond; i will make an evaluation based on this consideration”, “no matter what the question about hydrogen bonding is, i will think that all hydrogen atoms form hydrogen bonds; i will make evaluations based on this consideration” and “no matter what the question about hydrogen bonding is, i will think that only oxygen atoms form hydrogen bonds; i will make evaluations based on this consideration”. overconfidence: the way followed for researching the effects of overconfidence in participants started with directing a question to the participants before asking/showing them the question about acetic acid: “do you believe that you can give the correct answer to a question about the subject of hydrogen bonding? what is your self-confidence level; how would you rate your selfconfidence on a scale from 1 to 10 (1 is the lowest, 10 is the highest rate)?” right after asking/showing the question about acetic acid, before students started solving the problem, they were directed another question: “do you believe that you can give the correct answer to this question. what is your self-confidence level; how would you rate your self-confidence on a scale from 1 to 10 (1 is the lowest, 10 is the highest rate)?” finally, a question was directed to participants after they solved the problem: “how much do you confide in your answer? how would you rate your confidence in your answer on a scale from 1 to 10 (1 is the lowest, 10 is the highest rate)?” overconfidence heuristics is encoded when the students scored all three questions as 8, 9, or 10. students with such answers generally had statements such as: “i trust in myself, i gave/will give the correct answer”. table 1 answer patterns obtained from the question cluster and answer pattern visual representation of answer pattern students n % a (5,7,8) c c o o h h h h 5 7 8 h 2 o s3,s6,s11,s25,s29 5 16.66 b (1,2,4,5,7,8) h 2 o c c o o h h h h 1 2 4 5 7 8 s2,s15,s17,s30 4 13.13 c (1,2,4,8) h 2 o c c o o h h h h 1 2 4 8 s5,s20,s24,s27 4 13.13 d (5,7) h 2 o c c o o h h h h 5 7 s16,s21,s28 3 10.00 e (1,2,3,4,7,8) h 2 o c c o o h h h h 1 2 3 4 7 8 s9,s14 2 6.66 f (5,6,7,8) h 2 o c c o o h h h h 5 6 7 8 s4,s13 2 6.66 journal of science learning article doi: 10.17509/jsl.v4i1.23737 54 j.sci.learn.2020.4(1).50-60 affect: in order to research the effect of affect heuristic on the subject of hydrogen bonding, a question was directed to participants before asking/showing them the question: “how would you feel when you think about the subject of hydrogen bonding; throughout your education, did you have any experience about the subject that may positively or negatively affect you? if so, does it still have any effect on you?” on the other hand, after asking/showing the question, the participants were required to answer the question: “how do you feel now once you have seen the question?” affect heuristic is encoded according to participants' answers in the cases when it was determined that they have positive or negative perceptions because of their past experiences. 3. result and discussion in this research, interviews were held with participants to evaluate how they perceive hydrogen bonding in the context of a structural demonstration. during the interview processes, participants were required to answer the question involving lewis's structural demonstration of acetic acid: “which of the atoms in acetic acid can form a hydrogen bond with water molecule?” after the carefully followed answer-analysis process, a total of 12 different answer patterns were obtained. these 12 different answer patterns and the code names of students who gave these answers were presented in table 1. the number of students in each pattern and the percentage and the visual representation of each answer pattern was also presented in table 1. numbers under the title of the cluster and answer pattern in table 1 stand for the atoms numbered on acetic acid's structural demonstration (figure 1). as shown in table 1, only 16.66% of students gave the correct answer (cluster a). according to the literature results, the number of students who comprehended the subject of hydrogen bonding correctly is relatively few; this result is similar to the current study's findings (taagepera et al., 2002; villafane et al., 2011). for example, according to the study by villafane et al., the number of students who understood the subject was few, and students had some misconceptions about hydrogen bonding (villafane et al., 2011). on the other hand, in their study, miller and kim obtained a similar result and determined that the number of students who understood the subject correctly was few (miller & kim, 2017). table 1 answer patterns obtained from the question (continued) cluster and answer pattern visual representation of answer pattern students n % g (1,2,4) h 2 o c c o o h h h h 1 2 4 s19,s26 2 6.66 h (5,8) h 2 o c c o o h h h h 5 8 s1,s12 2 6.66 i (7,8) h 2 o c c o o h h h h 7 8 s7,s18 2 6.66 j (8) h 2 o c c o o h h h h 8 s8,s10 2 6.66 k (7) h 2 o c c o o h h h h 7 s23 1 3.33 l (none) h 2 o c c o o h h h h none s22 1 3.33 journal of science learning article doi: 10.17509/jsl.v4i1.23737 55 j.sci.learn.2020.4(1).50-60 according to the researches in the literature about hydrogen bonding, the most frequent misconceptions of students are: “all hydrogen can form hydrogen bonds”, “hydrogen bond is a covalent bond”, “hydrogen bonds occur inside the molecules rather than occurring between them”, “covalent bonds are broken during phasetransition”, “hydrogen bond is any bond that involves hydrogen”, “all polar molecules can form hydrogen bonds” and “all compounds containing nitrogen, oxygen, and fluorine together with hydrogen form hydrogen bonds, regardless of whether the hydrogen is bound to these atoms (duis, 2011; henderleiter et al., 2001; tan & treagust, 1999; villafane et al., 2011; pérez et al., 2017). in this study, whose basic purpose is to analyze the use of heuristics by students in the subject of hydrogen bonding, a multipleselect format assessment item is used for analyzing the possible interactions of a molecule at the level of atoms; this selection of assessment also enabled the researcher make direct observations about some misconceptions of students. for example, the misconception that “all hydrogen can form hydrogen bonds” was determined in previous research. it is also reflected in this study in cluster c (atoms numbers 1, 2, 4, and 8). this pattern is formed based on participants' answers, who have chosen the entire hydrogen atoms rather than choosing the other atoms in the molecule. as shown in table 1, the percentage of students in this answer pattern is 13.13%. inclusion of answer patterns based on students who have chosen the hydrogen atoms exclusively, but not each hydrogen (cluster g and j), would increase this ratio to 26.66 %. the structural/visual presentations of the answer patterns in each cluster are given in table 1. cluster b table 2 heuristic codes and summaries of student statements heuristic code summary of student statements associative activation use of the existent mental constructions to fill in the blanks all hydrogen in the molecule form hydrogen bonds (hydrogen bond equals to all hydrogen) full octet should be formed water should form be formed fluency use of easily accessible cues i have searched for a dual bond atom in the molecule i have searched for a part similar to water in the molecule i have searched for hydrogen attached to fluorine, oxygen or nitrogen attribute substitution substitution of the original question with a simpler one substitution of the original question with these questions: which atoms can form water? which atoms can undergo condensation reaction? which bonds undergo addition reactions? which part of the molecule is similar to water? between which atoms is there a dual bond? one-reason decision making simplification of reasoning with the use of a single cue or factor carbon cannot form a hydrogen bond oxygen cannot form dual bond atoms with dual bond cannot form hydrogen bond surface similarity simplification of reasoning by assuming that similar objects can be chemically comparable this aspect of acetic acid is similar to water. there is hydrogen bond in water; then, atoms that form the parts of acetic acid that are similar to water can form hydrogen bond. water molecule doesn’t involve dual bond; this is why water molecule can form hydrogen bond with the atoms in acetic acid that are similar (atoms only attached with only single bond) recognition use of a specific aspect as the decision cue h,f,o,n rule generalization generalization of learned model or rules octet rule is prevalent while hydrogen bond is formed hydrogen bonding is a chemical reaction (condensation) the bond orders are taken into consideration while hydrogen bond is formed rigidity reasoning that is not flexible and creative i think that only f, o, n and hydrogen atoms can form hydrogen bond; i will make evaluations on this basis i think that all hydrogen atoms in a molecule can form hydrogen bond; i will make evaluations on this basis i think that only oxygen atoms can form hydrogen bond; i will make evaluations on this basis overconfidence exceeding the actual accuracy because of the self-confidence in decision-making processes i absolutely solved/will solve the question correctly my self-confidence level is between 8 and 10 affect positive or negative senses i like/don’t like the subject of hydrogen bonding; positive/negative sense journal of science learning article doi: 10.17509/jsl.v4i1.23737 56 j.sci.learn.2020.4(1).50-60 (13.33%) and cluster c (13.33%) are the most preferred selections after cluster a (16.66%), which is the correct answer. in cluster b, all of the atoms except for the two carbons in the structure are involved in the calculation. in cluster c, only all of the hydrogen in the structure is involved in the calculation. in cluster d (10.00%), only oxygen atoms are selected. in cluster e (6.66%), all of the atoms except for the ones with double bonds (carbon and oxygen) are selected. in cluster g (6.66%), all hydrogen except for the one attached to oxygen is involved in the calculation. these answers are related to using strategies based on simple rules such as “all atoms except for carbon” and “only hydrogen atoms”. the use of strategies based on a specific rule is an indicator of heuristics' role in answering processes (miller & kim, 2017). for this reason, student answers have been analyzed in terms of heuristics used. codes were made by associating the specific statements in the participants' responses with ten heuristics. summaries of student statements that provide a basis to encodings about these ten heuristics are presented in table 2. hydrogen bonding is generally defined as the attractive forces between hydrogen attached to an electronegative atom and another electronegative atom. to decide which atoms in a compound can form a hydrogen bond with water molecule, it is necessary to consider the factors of electronegativity and polarity. in the question, students were required to evaluate the relationship of electronegativity between two atoms. they were also required to accurately state which atoms in a compound (acetic acid) can form a hydrogen bond with water according to their assessment of the electronegativity relationship. this is the implied target attribute. however, in this study, it is determined that participants could not accurately evaluate the target attribute or evaluate some other attributes instead of the target one. to express the results visually, heuristic usage percentages are given in figure 2 as a graphical representation. participant names (pseudonyms of the participants) that have used the relevant heuristics are as follows; associative activation: s1, s4, s5, s8, s10, s12, s13, s19, s20, s22, s23, s24, s26, s27; fluency: s1, s2, s3, s4, s6, s7, s8, s9, s10, s11, s12, s13, s14, s15, s17, s18, s23, s25, s29, s30.; attribute substitution: s1, s4, s7, s8, s9, s10, s12, s13, s14, s18, s23.; one reason decision making: s2, s7, s9, s14, s15, s16, s17, s18, s21, s28, s30.; surface similarity: s1, s4, s7, s8, s10, s12, s13, s18, s23.; recognition: s2, s3, s6, s11, s15, s16, s17, s21, s25, s28, s29, s30.; generalization: s4, s5, s13, s19, s20, s21, s22, s26.; rigidity: s2, s20, s21, s25, s27, s28, s30.; overconfidence: s2, s6, s11, s15, s17, s20, s21, s24, s29, s30.; affect: s1, s8, s23, s29. it is crucial to determine the points specifically required in a question accurately. this crucial step may sometimes be perceived as a straightforward process. however, in this research, according to the reasoning of participants employed while solving the problem, heuristics revealed that heuristics affected their interpretations, and thus there are differences in participant statements about the target attribute implied in the question. the reason for these differences is the impact of the attribute substitution heuristic; through this heuristic, complications are decreased by unconsciously ignoring information that is, in fact, crucial for solving problems (ugras, 2018). this study determined that heuristics such as fluency, generalization, and surface similarity trigger the heuristic of attribute substitution; this heuristic caused 11 of the participants (36.66%) to substitute the original problem with another one. for example, the statement of “hydrogen bonding is a reaction of condensation” results from the generalization heuristic’s effect. due to the effect of this reasoning, figure 2 graphical presentation of heuristic usage percentages 46.66 66.66 36.66 36.66 30 40 26.66 23.33 33.33 13.33 0 20 40 60 80 p e rc e n ta g e ( % ) heuristics journal of science learning article doi: 10.17509/jsl.v4i1.23737 57 j.sci.learn.2020.4(1).50-60 attribute substitution heuristic is triggered, and students, thus, simplified the original question and changed it with a much simpler form: “which bonds or atoms in acetic acid can undergo condensation reaction?” these students have also focused on this simple question instead of the target attribute in the following processes. in this study, it is determined that with the effect of attribute substitution, students focused on answering these questions instead of the original one: “which atoms can form water?”, “which atoms can undergo condensation reaction?”, “which bonds undergo condensation reaction?”, “which parts of the molecule are similar to water?” and “which atoms have a dual bond?” the heuristic of attribute substitution, which caused students to focus on the questions mentioned above instead of the target attribute, is triggered and supported by other heuristics such as fluency and surface similarity. heuristics usually trigger one another; thus, more than one heuristic is efficient in the decision-making processes (ugras, 2018). this study is in line with the literature statements; participants in this study used multiple heuristics in answering processes. the problem-solving method of the participant s7 is an example of the use of multiple heuristics; s7: atoms that can form a hydrogen bond with water are the ones number 7 and 8. interviewer: will you please explain the strategy you used for answering the question? s7: i was a little surprised when i first saw the question as the question is interesting and unusual for me; this is why i had no idea about how to solve the problem at the beginning. i started to carefully analyze the structural demonstration of acetic acid on the chance of finding a solution. i thought that there is a cue in this demonstration that i can use as a starting point, and it can help me solve the problem. on the one hand, i was thinking about these; on the other, i analyzed the shape of the molecule to find a cue. then – oh group in the molecule attracted my attention. i started to think if i can use the –oh group as a cue. this part of the molecule, namely –oh group, was similar to water. this similarity made me think about the structure of the water molecule. i was sure that water has hydrogen bonding. since water can form a hydrogen bond, then molecules that structurally involve groups similar to water can form hydrogen bonds through these parts. based on this reasoning, i reanalyzed the acetic acid molecule to see if any other parts in it are similar to water. the only part similar to water was –oh group formed by the atoms number 7 and 8. as i thought that hydrogen (atom number 7) and oxygen (atom number 8) atoms in this part could form a hydrogen bond, the answer is the atoms number 7 and 8 according to me. the use of –oh group on the acetic acid molecule as an easily accessible cue indicates the efficiency of fluency heuristic in the reasoning process of participant s7. the heuristic of surface similarity also played an essential role in this process. the assumption that chemical compounds that are similar in terms of the structural demonstration are the members of the same category and believe that these types of compounds have similar features and behaviors result from the impact of surface similarity heuristic (talanquer, 2014). in this context, it can be said that the reasoning of participant s7 (since water can form a hydrogen bond, then molecules that structurally involve groups that are similar to water can form hydrogen bond through these parts”) is a result of this heuristic’s impact. participant s7 looked for a part on the acetic acid molecule that is similar to water. s/he thus changed the target attribute and focused on finding an answer to another question (which part of the molecule is similar to water?). this type of behavior is a result of the impact of attribute substitution. the same student trusted in a single reason (only water-like parts of the molecule form hydrogen bond) in the process of answering the question and s/he solved the problem based on this criterion; so, it can be said that one-reason decision-making heuristic was also efficient in the student’s reasoning process. based on these reasons, as they were efficient and dominant in the reasoning process of participant s7, fluency, surface similarity, attribute substitution, and onereason decision-making heuristics are encoded as the ones used by the student. participants frequently used surface attributes unconsciously, similar to the example mentioned above. surface attributes are easily accessible cues, and their unconscious use is also related to the heuristic of fluency. students' tendency towards easily accessible information is considered a result of the fluency effect (talanquer, 2014). fluency effect manipulates strategies or cues used for a duty; thus, it is the most common heuristic that plays a role in decision-making processes. besides, it enforces the other heuristics (ugras, 2018). in this research, it is determined that the fluency effect played a significant role in the decision-making processes of 20 participants while solving the problem (66.66%, the most commonly used heuristic). some participants used double bonds or –oh groups in the acetic acid molecule as an easily accessible cue; this process results from the fluency heuristic. along with the fluency heuristic, some other heuristics triggered and enforced one another in these processes. the problemsolving method of participant s4 is an example that shows the occurrence of many other heuristics because of fluency. s4: atoms that can form a hydrogen bond with water are numbers 5, 6, 7, and 8. interviewer: will you please explain the strategy you used for answering the question? s4: …….. at that point, the double bond on the molecule attracted my attention. i suddenly remembered the other reaction when i was thinking about the double bond. i do not know if it is correct or reasonable, but i think that the hydrogen bond formation is an addition reaction; this is why i started thinking about addition reactions. as much as i remember, compounds with double bonds could undergo an addition reaction. on the other hand, water was formed as a by-product of addition reactions. the question was about journal of science learning article doi: 10.17509/jsl.v4i1.23737 58 j.sci.learn.2020.4(1).50-60 the atoms that could form a hydrogen bond with water. water bonds to atoms that form the dual bond on acetic acid (atoms number 5 and 6) to form hydrogen bond. as a result of this addition reaction, namely the formation of the hydrogen bond, some atoms should be separated from the structure of acetic acid and form water. –oh group connected to carbon atom number 6 is similar to water. almost certainly, the oh group causes water formation by separating from the molecule during the addition reaction. another water molecule could have formed a hydrogen bond with the oxygen and hydrogen atoms (atoms number 7 and 8) in –oh group to separate this –oh group. i believe that atoms in –oh group can form hydrogen bonds because this group is water-like. i thought that if water can form a hydrogen bond, then molecules that involve water-like groups in their structure can form hydrogen bond through these parts. i am not sure if my explanations are correct, but i decided based on these thoughts i mentioned above. i checked to see if there is any other dual bond in the molecule before making my final decision. there was no other dual bond. as a result, in my opinion, the answer is atoms numbers 5, 6, 7, and 8. as shown above, participant s4 used the double bond and –oh group as a cue in the answering process; this use is an example of the result of fluency heuristic. the student's approach is built based on the perception that “hydrogen bonding is an addition reaction,” which indicates that the generalization heuristic was also efficient in the process. the reasoning of “water is formed in the process of hydrogen bonding” implies the heuristic of associative activation. instead of the target attribute, the student-focused on finding answers to some other questions such as “between which atoms is there dual bond” and “which part of the molecule is similar to water?” focusing on some other and more straightforward questions result from the impact of the attribute substitution heuristic. the student thought that –oh group could also form hydrogen bonds similar to water; this has resulted from the surface similarity heuristic impact. as a result, the heuristic of fluency, which was efficient in the first approach of participant s4, triggered the heuristics of generalization, associative activation, attribute substitution, and surface similarity. some of the students stated that they specifically focused on h, f, o, and n atoms in problem-solving processes; when they were asked about the reason behind this, they said that h, f, o, and n atoms would appear in their mind as soon as they heard about hydrogen bonding and they somehow related these atoms with this type of bonding. the researcher asked these students about the relationship between h, f, o, and n atoms and hydrogen bonding. they stated that when the topic of hydrogen bonding was mentioned in chemistry classes, these atoms would commonly be mentioned; they were thus familiar with them, and they had the perception that only h, f, o, and n atoms could form a hydrogen bond. the participants said that this perception is called “h, f, o, n rule”. individuals’ tendency to trust in the information that is easier to recall from the memory is considered as the effect of recognition heuristics (talanquer, 2014); thus, the students’ trust in h, f, o, and n atoms in problem-solving is evaluated as a result of this heuristic. these students trusted in h, f, o, and n atoms as they have already known them from earlier classes, and it was easy to bring them from memory. after analyzing the obtained data, it was determined that most of the students who employed this rule for solving the problem did this unconsciously and thus gave the wrong answer. besides, a few of the students who used the rule for solving the problem gave a correct answer. some students unconsciously over-generalized some patterns and rules that they have learned before solving the problem. this situation is a result of the generalization. the student statements indicating the over-generalization of patterns of rules are as such: “octet rule applies in the formation of hydrogen bond”, “hydrogen bonding is a chemical reaction (condensation)” and “the number of bonds is taken into consideration during the formation of hydrogen bond”. the heuristic of generalization, which becomes evident in such reasoning, played an essential role in triggering other heuristics. the reasoning process of participant s4 presented above is an example of how generalization heuristic triggers other heuristics. one of the participants stated that (s29) s/he made a project work about hydrogen bonding when s/he was in high school, he liked doing the project, completed it successfully, got positive feedback from his/her teacher, and got a high score at the end of the project. s/he mentioned that s/he liked the subject, s/he has always been interested in it, and s/he has more than positive feelings about hydrogen bonding. affect heuristic is encoded based on these statements of s29. three of the students (s1, s8, s23) mentioned that they generally hated non-numerical chemistry classes, they felt close to the mentality of numerical classes, subjects in chemistry classes mostly require chemical and mathematical processes, they do not like abstract topics and relationships between these topics and as a result of this, they do not like the topic of hydrogen bonding. the students specifically stated that they are not interested in hydrogen bonding as it is a non-numerical chemistry field. these statements of s1, s8, and s23 are taken into consideration, and the affect heuristic is encoded based on them. the ratio of students who gave correct answers to the question about hydrogen bonding is 16.66%, which is a considerably low value. it was also determined that the correct answer ratio is similar to research in the literature about why reasoning processes in chemistry issues were similarly low. for example, according to research about “chemical bonding theories and molecular structures”, the total correctness ratio of participator answers was 36% (ugras, 2018). moreover, there is little research in the literature about chemistry topics with a high total correctness ratio of participator answers. for example, this journal of science learning article doi: 10.17509/jsl.v4i1.23737 59 j.sci.learn.2020.4(1).50-60 ratio is 77% in the research analyzing acid strengths and student reasoning about the topic. besides, although the correctness ratio was high in the research, it was determined that only less than 8% of students chose the correct answer based on scientific reasoning (mcclary & talanquer, 2011). except for miller and kim's research, none of the research about students’ intuitive judgment in chemistry classes employed a model (such as the model of 10 heuristics in chemistry suggested by talanquer) to classify heuristics. in their research, miller and kim used a form involving a structural demonstration of acetic acid to analyze the use of heuristic by students in hydrogen bonding. miller and kim handed the form to the participants and required them to write down their answers and explain why they preferred the answer. the researchers did not conduct interviews in their research; they solely analyzed the written answers of students. because of this process, they stated that their study method was proper for determining 6 of the ten heuristics suggested by talanquer; the researchers mentioned that their study was not proper for determining fluency rigidity, overconfidence, and effect (miller & kim, 2017). as the most proper method for determining the heuristic use of students is the interview technique, specifically designed interviews were held with the participants to determine all of the heuristics suggested by talanquer in this research study; this is why it was possible to determine the impacts of the entire ten heuristics suggested by talanquer on students’ reasoning processes in the subject of hydrogen bonding. 4. conclusion cognitive science theories have been used to analyze individuals’ judgments, decision-making processes, and reasoning types in these processes about daily life issues for years. besides, there has been a very recent increase in the number of studies researching the impacts of heuristics, frequently mentioned in dual-process theory, on student reasoning in chemistry/science subjects. on the other hand, there is no study in the literature about how heuristics cause biases in students’ judgment and decision making processes in chemistry/science subjects based on the interview data obtained from students in line with talanquer’s ten heuristics. in this respect, this research that evaluates and analyzes hydrogen bonding, a significant chemistry/science subject, will make a significant contribution to the literature in the context of cognitive psychology theory. students' heuristics while answering the question of “which atoms on acetic acid can form a hydrogen bond with water?” is determined for the first time in this study. students commonly used the determined heuristics while answering the question indicates that most of them employed shorter strategies, precisely surface similarity, instead of scientific/chemical reasoning. requiring a specific duty from students by presenting structural demonstration triggered the use of heuristic reasoning strategies. during the interviews, most of the students mentioned that they preferred the strategy that required the minimum amount of time. participants who answered the question through the use of strategies based on the heuristics gave incorrect answers. on the other hand, some students answered the question correctly by using heuristics. students alone are not responsible for the frequent use of heuristic strategies; throughout their educational background, they are generally taught to use short-cut problem-solving strategies, which sometimes decrease their tendency to use cognitive ways or scientific reasoning skills in solving a problem. because of this, they may have built the habit of solving problems by using short-cut strategies. intuitive judgment is one of the most common reasoning types. educators' duty is not to prevent intuitive judgment; they are to research the impacts of this judgment on the understandings and interpretations of students, carefully analyze the data obtained at the end of the research process and form domain-specific, successful reasoning and articulation methods. while teaching a subject in general chemistry classes, it might be advantageous to explain the possible incorrect reasoning encountered in using the common short-cut strategies in a specific subject. to ensure students have the habit of solving problems using chemical processes instead of short-cut strategies, which have nothing to do with scientific reasoning, it is suggested to require them to solve new and different types of chemistry questions. the question used in this research is an uncommon type for the participants. although they encountered a different question, they frequently preferred to use familiar short-cut strategies; that is why we believe that there is a need to carry out more studies to determine which types of questions support students in terms of using chemical reasoning. it is indispensable to develop question types that encourage students to use chemical reasoning in chemistry questions with surface attributes such as structural demonstration. on the other hand, to correct the biases caused by type 1 processes in different chemistry subjects, it is necessary to conduct more studies to determine how type 2 processes can become more active. there are various researches on students' understanding of intermolecular forces and their role in different chemical phenomena. all of these researches have focused on analyzing how students perceive the subject and phenomenon conceptually. however, it is also essential to determine how visually presented structural demonstrations affect students’ conceptual understandings. at the end of this research, based on structural demonstration, it is determined that students overused the heuristics in solving the problem they were asked. this significant finding of the research should be considered; the fact that structural demonstrations trigger heuristics is once again revealed. this study shows that a multiple-select format assessment item can help journal of science learning article doi: 10.17509/jsl.v4i1.23737 60 j.sci.learn.2020.4(1).50-60 understand how students evaluate hydrogen bonding. the use of open-ended interviews and a multiple-select format assessment item in this research has been highly beneficial for analyzing the relationship between structural demonstration and heuristic use. this research reveals how different heuristics are efficient in students’ reasoning processes about hydrogen bonding. this research will be beneficial for educators in determining strategies to prevent students’ incorrect reasoning approaches. this research does not aim to decrease the use of heuristics or prevent the impacts of them; we suggest that it is also necessary to research different teaching strategies designed to decrease the negative impacts of heuristics, which are efficient during intuitive judgments. notes this article has been prepared from the relevant parts of gülen önal karakoyun's doctoral dissertation. gülen önal karakoyun’s orcid number: https://orcid.org/0000-0002-7675-0006. erol asiltürk's orcid number: https://orcid.org/0000-0001-81267812. references akerlind, g. s. (2005). variation and commonality in phenomenographic research methods. higher education research & development, 24(4), 321–334. barker, v., & millar, r. (2000). students’ reasoning about basic chemical thermodynamics and chemical bonding: what changes occur during a context-based post-16 chemistry course? international journal of science education, 22(11), 1171– 1200. connor, p. e., & becker, b. w. (2003). personal value systems and decision-making styles of public managers. public personnel management, 32(1), 155-180. connor, m.c., finkenstaedt-quinn, s.a., & shultz, g.v. (2019). constraints on organic chemistry students’ reasoning during ir and 1h nmr spectral interpretation. chemistry education research and practice, 20, 522-541. cooper, m. m., corley, l. m., & underwood, s. m. (2013). an investigation of college chemistry students’ understanding of structure-property relationships. journal of research in science teaching, 50(6), 699−721. cooper, m. m., williams, l. c., & underwood, s. m. (2015). student understanding of intermolecular forces: a multimodal study. journal of chemical education, 92(8), 1288–1298. didis n., ozcan o., & abak a. (2008). quantum physics from students’ perspective: a qualitative study. hacettepe üniversitesi eğitim fakültesi dergisi, 34, 86-94. duis, j. m. (2011). organic chemistry educators’ perspectives on fundamental concepts and misconceptions: an exploratory study. journal of chemical education, 88(3), 346–350. evans, j. s. b. t. (2008). dual-processing accounts of reasoning, judgment, and social cognition. annual review of psychology, 59, 255−278. evans, j. s. b. t., & stanovich, k. e. (2013). dual-process theories of higher cognition: advancing the debate. perspectives on psychological science, 8, 223−241. ferrari, j. r., & dovidio j. f. (2000). examining behavioral processes in decision: decisional procrastination and decision-making style. journal of research in personality, 34, 127–137. gigerenzer, g., & goldstein, d. g. (1996). reasoning the fast and frugal way: models of bounded rationality. psychological review, 103, 650−669. gilovich, t., griffin, d., & kahneman, d. (2002). eds. heuristics and biases: the psychology of intuitive judgment. cambridge: cambridge university press. graulich, n. (2014). intuitive judgments govern students’ answering patterns in multiple-choice exercises in organic chemistry. journal of chemical education, 92, 205-211. graulich, n., hopf, h., & schreiner, p. r. (2011). heuristic chemistryaddition reactions. chemistry a european journal, 17, 30-40. graulich, n., hopf, h., & schreiner, p. r. (2011). heuristic chemistryelimination reactions. chemistry an asian journal, 6, 3180–3188. hastie, r. (2001). problems for judgment and decision making. annual review of psychology, 52, 653-683. henderleiter, j., smart, r., anderson, j., & elian, o. (2001). how do organic chemistry students understand and apply hydrogen bonding? journal of chemical education, 78, 1126-1130. maeyer, j., & talanquer, v. (2013) making predictions about chemical reactivity: assumptions and heuristics. j. res. sci. teach. 50, 748−767. maeyer, j.r. (2013). common-sense chemistry: the use of assumptions and heuristics in problem solving. phd thesis, department of chemistry and biochemistry, the university of arizona. mcclary, l. k., & talanquer, v. (2011). heuristic reasoning in chemistry: making decisions about acid strength. international journal of science education, 33, 1433−1454. miller, k., & kim, t. (2017). examining student heuristic usage in a hydrogen bonding assessment. biochemistry and molecular biology education, 45(5), 411-416. pérez, j. r. b., pérez, m. e. b, calatayud m. l., garcía-lopera r., montesinos, j. v. s., & gil, e. t. (2017). student’s misconceptions on chemical bonding: a comparative study between high school and first year university students. asian journal of education and elearning, 5(1), 1-15. schwarz, n. (2004). metacognitive experiences in consumer judgment and decision making. journal of consumer psychology, 14(4), 332-348. stanovich, k. e., & west, r. f. (2000). individual differences in reasoning: implications for the rationality debate. behavioral and brain sciences, 23(5), 645-726. taagepera, m., arasasingham, r., potter, f., soroudi, a., & lam, g. (2002). following the development of the bonding concept using knowledge space theory. journal of chemical education, 79, 756-762. talanquer, v. (2014). chemistry education: ten heuristics to tame. journal of chemical education, 91, 1091-1097. tan, k. c. d., & treagust, d. f. (1999). evaluating students’ understanding of chemical bonding. school science review, 81, 75– 83. todd, p. m., & gigerenzer, g. (2000). précis of simple heuristics that make us smart. behavioral brain sciences, 23(5), 727−741. ugras, m. (2018). heuristic reasoning of pre-service science teachers in chemistry topics. journal of baltic science education, 17(2), 343-356. vekli, g. s. (2020). examination of prospective science teachers’ levels of designing project directed to national support programs: a profile of turkey. journal of science learning, 3(2), 36-45. villafane, s. m., bailey, c. p., loertscher, j., minderhout, v., & lewis, j. e. (2011). development and analysis of an instrument to assess student understanding of foundational concepts before biochemistry coursework. biochemistry and molecular biology education, 39(2), 102–109. wihlborg, m. (2004). student nurses’ conceptions of internationalism in general and as an essential part of swedish nurses’ education. higher education research & development, 23(4), 433-453. https://orcid.org/0000-0002-7675-0006 https://orcid.org/0000-0001-8126-7812 https://orcid.org/0000-0001-8126-7812 a © 2020 indonesian society for science educator 20 j.sci.learn.2020.4(1).20-30 received: 10 september 2020 revised: 22 november 2020 published: 28 november 2020 investigating turkish pre-service science teachers’ moral reasoning in genetics related socioscientific issues ümran betül cebesoy1* 1department of science education, faculty of education, usak university, usak, turkey *corresponding author: ubetulcebesoy@gmail.com abstract in this study, turkish pre-service science teachers' moral reasoning patterns and the factors which influence their decisions while discussing genetics-related socio-scientific issues (ssi) were investigated. a basic qualitative approach was adopted for this purpose. seven third-grade pre-service science teachers enrolled in the study. semi-structured interviews for different genetics related ssi were conducted. the results revealed that decisions were generally based on the consequences of genetic applications (consequentialist) or based on moral principles or prescripts (principle-based). most participants used consequencebased moral reasoning in their decisions, while principle-based moral reasoning was less used. they also used emotion-based moral reasoning. their decisions were influenced by emotions, including empathy or sympathy toward the characters, or the unborn baby, in the scenarios. additional and varied factors, including legal, ethical, economic, and technological concerns, were revealed as influential. participants' decisions were also shaped by their own experiences, media resources, and faith in science. the implications for science teacher education programs are discussed. keywords decision-making, genetics, pre-service science teachers, socioscientific issues 1. introduction scientific literacy is essential for several reasons: first, it helps people deal with the issues and challenges in their daily lives. it also enhances making informed decisions when confronted with their health, lifestyles, or consumption habits by considering scientific understanding. second, it prepares individuals as knowledgeable civic decision-making participants (national academies of sciences, engineering, and medicine, 2016; organisation for economic co-operation and development [oecd], 2012). thus, science education has long desired to support scientific literacy development (bossér, lundin, lindahl, & linder, 2015). on the other hand, socioscientific issues (ssi, hereafter) are considered an indispensable part of scientific literacy (zeidler & keefer, 2003; sadler, 2004a). ssi includes a wide range of socially controversial issues linked to science (sadler & zeidler, 2005; sadler, 2004a). these issues are ill-structured and open-ended, often requiring negation by considering multiple perspectives (chang rundgren & rundgren, 2010; sadler & donnely, 2009; zeidler, sadler, applebaum, & callahan, 2009; zeidler, 2003). a wide range of social issues which are connected to science, environment, and technology are considered as ssi: cloning, gene therapy, stem cell research, genetic engineering, nuclear power plantation, and global climate change (lee, chang, choi, kim & zeidler, 2012; sadler & zeidler, 2005; 2004; sadler & donnely, 2009). while dealing with the issues mentioned above, individuals engage in the decision-making process (fowler & zeidler, 2016). while making-decisions in ssi, individuals need to consider ethical and moral dimensions in addition to the scientific content (bell & lederman, 2003; sadler, 2004a, 2004c). zeidler & keefer (2003) indicated that morality could not be omitted while discussing and negotiating ssi. one of the essential topics of ssi is genetics related issues. genetics is developing fast with current technological innovations, giving rise to various controversies (gericke & smith, 2014). thus, it is frequently used by researchers (i.e., sadler & zeidler, 2004; 2005). also, it provides context for exploring participants' use of morality while making-decisions in ssi. for instance, sadler and zeidler (2004) explored college students' decision making on genetic engineering issues. the results showed that college students made decisions by referring mailto:ubetulcebesoy@gmail.com journal of science learning article doi: 10.17509/jsl.v4i1.28155 21 j.sci.learn.2020.4(1).20-30 to consequence-based and principle-based moral reasoning patterns. the researchers also revealed that emotions and intuition were influential in their decisions. their choices were also influenced by a wide range of factors, including family bias, personal experience, religion, extra background knowledge, and pop culture. in another study, sadler (2004a) explored 30 college students' moral sensitivity enrolling in biology and psychology courses using two genetic engineering issues (cloning and human gene therapy). the results revealed that participants had a variety of moral considerations when confronted with genetics related ssi. participants were concerned about individuals' health, and well-being and the sufferings that the people would go through. besides health-related reasons, they were concerned about proposed genetic technologies were altering the natural order. another concern identified was the slippery slope arguments indicating using the genetic application in one proper context might lead to its usage in other unacceptable contexts. also, minor concerns such as doctors' responsibilities, patients' right to treatment, making decisions for someone who cannot decide on his/her own, and creating social classes among society were revealed. topçu, yılmaz-tüzün, and sadler (2011) explore 39 turkish pre-service science teachers' informal reasoning explored in different ssi, including gene therapy, human cloning, and global warming. they revealed pre-service science teachers' multidimensional informal reasoning patterns, including rationalistic, emotive, and intuitive, interwoven with each other. the results also showed that pre-service science teachers' moral reasoning was influenced by different factors, including personal experiences, social, moral, and ethical considerations. in another recent study, cebesoy (2014) explored science teachers' decision-making skills in genetics-related ssi. the researcher found out that many factors influenced science teachers' decisions on genetics-related ssi. while moral considerations were the most influential factor, other factors including economic, legal, technological, and religious concerns, were also significant. sadler and donnely (2009) explored high school students' argumentation quality in genetic engineering issues and revealed that students considered genetic engineering issues as moral problems. the abovementioned studies conducted with college students (sadler & zeidler, 2004; sadler, 2004b), preservice science teachers (topcu, yilmaz-tuzun & sadler, 2011), science teachers (cebesoy, 2014) or with students (sadler & donnelly, 2006). these studies showed the importance of including the morality dimension while dealing with ssi. besides, studies also reported that ssibased teaching has fruitful outcomes (i.e., eggert, ostermayer, hasselhorn, & bögeholz, 2013; gresch, hasselhorn, & bögeholz, 2017). for instance, studies reported that ssi-based teaching improved students' decision-making skills (eggert et al., 2013; gresch et al., 2017). however, ozden (2020) indicated that students use low-quality reasoning while discussing ssi, which shows the need to include ssi-based teaching into the science classes, which is frequently emphasized in the literature (sadler & donnelly, 2006; ozden, 2020; zohar & nemet, 2002). here, teachers' and future teachers' skills to introduce ssi-based teaching into their classes come to the fore. previous research indicated that teachers' beliefs and attitudes towards ssi influence their ssi-based education (khishfe, 2014; walker & zeidler, 2007). however, the literature reports many teachers perceived ssi-based teaching as challenging (sadler, amirshokoohi, kazempour, & allspaw, 2006; rundgren & chang rundgren, 2018). many teachers were reported to feel uncomfortable introducing ethics and values along with scientific content (sadler et al. 2006), to have lack of knowledge (aivelo & uitto, 2019), and lack of confidence for dealing with ssi (bryce & gray, 2004). making informed decisions on controversial issues does not develop naturally unless training in decision-making is provided to the students (eggert et al., 2013; gresch et al., 2017; hsu & lin, 2017). thus, the teachers and future teachers are expected to develop their students' decisionmaking skills in the controversial issues to prepare them for their future roles as active individuals of society (bingle & gaskel, 1994). at this point, science teacher education programs play a crucial role in developing science teachers' ssi-based practices (sadler et al. 2006). however, when the role of teacher education programs is considered, there seems to be one crucial unanswered question: how do future teachers decide these controversial issues? this question remains unanswered while dealing with genetics related controversies like gene therapy or other genetics applications. thus, the present study aims to explore preservice science teachers' decision-making processes in genetics related to controversial issues. specifically, two research questions guided this study: 1. what are the moral reasoning patterns that pre-service science teachers refer to while handling genetics related controversial issues? 2. what other factors (other than moral reasoning) influence pre-service science teachers' decision-making in genetics related controversial issues? 2. method this study adopted the basic qualitative generic qualitative approach (kahlke, 2014; merriam, 2009). in this approach, the researcher focuses on "(1) how people interpret their experiences, (2) how they construct their worlds, and (3) what meaning they attribute to their experiences" (merriam, 2009). as a result, the studies adopting this approach are not directed by a set of grounded assumptions in other forms of qualitative methodologies like grounded theory or ethnography (caelli, ray, & mill, 2003). this method seeks to journal of science learning article doi: 10.17509/jsl.v4i1.28155 22 j.sci.learn.2020.4(1).20-30 understand people’s interpretation of the world (kahlke, 2014). likewise, this study explored how pre-service science teachers interpret a series of genetics related to controversial issues based on their worldviews and experiences. 2.1. participants seven third-grade pre-service science teachers who enrolled at a large public university in turkey's central anatolia region voluntarily participated in the study. typical purposive sampling was used for determining the participants (maxwell, 1997; teddlie & tashakkori, 2009). in this technique, the researcher collects data from particular settings or participants to provide vital information for the study (maxwell, 1997). in this study, third-grade pre-service science teachers were chosen purposefully. they had completed a series of formal courses about biology, genetics, and biotechnology in their previous semesters and thus were considered as informed about the topics being investigated in the study. the researcher explained the study's aim in a course and invited all the third-grade pre-service science teachers (a total of 32 students at the time) for an interview in a private office. willingness to participate in the study and the grade level was the only criteria for participant selection. seven preservice teachers (all were females) agreed to join the study voluntarily. 2.2. data collection and issue selection the semior full-structured interviews are a standard data collection tool in the basic qualitative approach (percy, kostere, & kostere, 2015). semi-structured interviews conducted by the researcher collected the data in this study. the interviews focus on a series of genetics related to controversial issues, including genetics applications. the researcher mainly chose these scenarios for two reasons: first, one of the objectives of the 8th-grade turkish primary science curriculum stated that development of students' discussion of controversial issues arising from future genetics and biotechnology applications (ministry of national education [mone], 2018). as a result, the four scenarios used in the present study included a discussion of genetics applications. second, they are pedagogically appropriate and used in previous studies conducted with both science teachers (cebesoy, 2014) and pre-service science teachers (karisan & cebesoy, 2020) in turkey. the researcher conducted semi-structured interviews with participants in a private office. the interviews lasted between 37 minutes to 62 minutes and audio recorded after taking permission of the participant. each interview began with a brief description of the scenario, and the participant handled the procedures if she wanted to read it herself. then, the participants were asked to answer a series of questions in each scenario. the scenarios were briefly explained below: scenario 1: fetal tissue transplantation this scenario was developed by bell and lederman (2003). it begins with a fictitious newspaper article mentioning dr. acar's treatment (fetal tissue transplantation) of alzheimer's disease using unborn fetuses' brain cells. suzan, in her late 30s, had an unexpected pregnancy considers aborting the fetus and donating it to dr. acar's study to be used for her father's treatment. should she abort the fetus for her father's treatment? scenario 2: cystic fibrosis this scenario was developed by zohar and nemet (2002). there is a brief description of cystic fibrosis disease and the symptoms of the disease. two treatment options are only in the clinical trials phase, with no success at that moment. a couple whose brothers had cystic fibrosis disease learns that they are expecting a baby. should the couple decide to end the pregnancy? scenario 3: huntington disease this scenario was developed by zohar and nemet (2002) to show a brief description of huntington's disease and the symptoms of the disease. there is no successful treatment at that moment. a fictitious character, lale, whose father was diagnosed with huntington's disease, learns that she is pregnant. should she decide to end the pregnancy? scenario 4: gene therapy this scenario was developed by sadler and zeidler (2004). there is a brief description of gene therapy and how it could treat huntington's disease and enhance human intelligence. should gene therapy be used for these purposes? each scenario included a series of questions to get a more in-depth understanding of pre-service science teachers' perspectives. each scenario was translated and adapted into turkish. experts checked the equivalence of original and translated versions in biology education and science education. a pilot study was done with one science teacher to check the appropriateness of the given information's language and sufficiency. 2.3. data analyses the data were transcribed verbatim. thematic analysis (braun & clarke, 2013) was used to identify the data's familiar themes. we preferred to use theoretical thematic analysis among different thematic analysis types as there were some predetermined codes and themes in the previous studies. these predetermined codes and themes were derived from previous studies (bell, 1999; bell & lederman, 2003; sadler, 2004a; sadler & zeidler, 2004; topcu, 2008; zohar & nemet, 2002). then, we applied the quantizing approach (transforming qualitative data into numeric values) (miles & huberman, 1994) to determine which theme was more emergent in the data. participants’ morality-based decisions in geneticsrelated scenarios were analyzed by sadler and zeidler’s (2004) framework (see, figure 1). journal of science learning article doi: 10.17509/jsl.v4i1.28155 23 j.sci.learn.2020.4(1).20-30 while making decisions in genetics related ssi, participants might display three categories of moral choice as moral reasoning, emotive-based, and intuitive-based. according to sadler and zeidler (2004), moral reasoning can also be classified under two subheadings: (1) consequentialist moral reasoning and (2) principal based moral reasoning. while consequentialist moral reasoning was based on their utilitarian analyses of the consequences of processes in the genetics applications, a principal-based ethical decision was mainly based on the moral principles or prescripts. under some circumstances, participants' decisions might be found on neither consequences nor principles. here, their choices were influenced by emotions such as empathy or sympathy towards the scenarios' fictitious characters. lastly, in moral intuition, participants expressed their opinions but did not explain their decisions. while exploring pre-service science teachers' consequentialist and principal-based moral reasoning, some patterns were revealed based on the participants' responses to genetics related ssi during the interview. the patterns of consequentialist moral reasoning were presented in table 1, and the patterns of principle-based moral reasoning were shown in table 2, respectively. the researcher coded participants' responses to the interviewer's questions. after coding, the frequency of appearance of each pattern was calculated. to ensure the participants' anonymity, each participant was given an id number (from 1 to 7). while 'pt' refers to pre-service science teacher, pt-5 refers to the fifth pre-service science teacher in the study. different scenarios were coded as s-1 for fetal tissue transplantation, s-2 for cystic fibrosis, s-3 for huntington's disease, and s-4 for gene therapy scenarios. 2.4. trustworthiness of the study the trustworthiness of the study was ensured by using several techniques. first, investigator triangulation was used to prevent misinterpretation of the data and researcher bias (archibald, 2016; guion, 2002). for this, the researcher coded the first three pre-service science teachers' audio-transcribed data based on the framework developed by sadler and zeidler (2004). then, another researcher who has expertise in science education and socioscientific issues coded the same data. lastly, both researchers assembled to discuss the framework and built a consensus on the analysis framework. inter-coder reliability (miles & huberman, 1994) was calculated to show the compatibility and the agreement between two coders by using the formula: total number of agreements/ (total number of agreements + the total number of disagreements) this value was calculated as 86%. then, the researcher completed the rest of the data analysis. table 1 the patterns of consequentialist moral reasoning (adapted from sadler & zeidler, 2004) consequentialist moral reasoning description health improvement statements that emphasize the importance of improvement in the health of individuals slippery slope statements that indicate concerns about the allowance of the application of genetics technologies in one acceptable context would lead to the use of that technology in unacceptable contexts. societal betterment statements that imply the use of genetic technologies will improve society overall. social stratification statements showing concerns regarding that use of genetic technologies may segregate a population by creating classes of “genetic haves” and “genetic have nots” diversity statements that indicate participants’ concerns about the genetics application will reduce the diversity and will cause erosion of diversity table 2 the patterns of principle-based moral reasoning (adapted from sadler & zeidler, 2004) principle-based moral reasoning description taking human life statements showing concerns regarding the status of an embryo as a human being. therefore, sacrificing embryos violates the principle of taking human life means to an end statements showing concerns about the use of embryos as resources or tools when needed disrupting natural order statements showing concerns regarding the applications of genetics change natural process unexpectedly parental rights statements that indicate that a fetus does not have rights, it is parents’ responsibility to decide whether to abort or not abort the fetus. figure 1 the morality aspect of socioscientific issues (adapted from sadler & zeidler, 2004) journal of science learning article doi: 10.17509/jsl.v4i1.28155 24 j.sci.learn.2020.4(1).20-30 3. result and discussion the four scenarios regarding genetics related ssi confronted pre-service science teachers with various opinions during the interviews. first, pre-service science teachers' moral reasoning patterns were examined. then, what kind of considerations other than ethical considerations influenced their decisions and opinions were examined. 3.1 pre-service science teachers’ moral reasoning patterns a total of 289 statements were coded. while 228 of the comments were coded under the moral reasoning theme (78.89%), 61 were coded as influencing factors (21.19%). among 228 statements, 159 statements (69.74) were coded under moral reasoning, and 69 comments (30.26%) were coded under moral-emotion based reasoning (see table 3 for the frequency and percentages). the results revealed that the pre-service science teachers in this study showed two moral reasoning categories while making decisions about genetics related ssi: (a) moral reasoning and (b) moral-emotion based reasoning. while examining their moral reasoning patterns, two significant themes also emerged: pre-service science teachers either made decisions based on the consequences of genetics applications (consequentialist) or based on the moral principles or prescripts (principle-based). pre-service science teachers’ consequentialist moral reasoning patterns pre-service science teachers indicated the consequences of genetics applications while making decisions in genetics related ssi. they evaluated the benefits and advantages as well as disadvantages of genetics applications presented in the scenarios. thus, their reasoning was coded as consequentialist moral reasoning. moreover, pre-service science teachers' consequentialist moral reasoning patterns were also explored, and other sub-themes emerged in the data. as mentioned in the method section, the researcher delineated which sub-themes were more emergent based on the frequency analysis. table 4 shows the sub-themes emerged in the consequentialist moral reasoning patterns: pre-service science teachers demonstrated a wide range of consequentialist patterns in their decisions while dealing with ssi genetics. when table 4 was examined, it could be seen the major sub-theme that emerged in the data was health improvement. pre-service teachers favored the importance of health while making decisions. even though health improvement was vital for them to support genetics applications, some were concerned that these genetics table 3 the total number of statements with respect to each reasoning pattern frequency (n) percentage (%) subdimensions frequency (n) percentage (%) moral reasoning 228 78.89 moral reasoning consequencebased 116 50.88 principle-based 43 18.86 moral emotionbased 69 30.26 other factors 61 21.19 total 289 100 228 100 table 4 pre-service science teachers’ consequentialist moral reasoning patterns consequentialist moral reasoning frequency sample excerpts health improvement 73 ‘i think there is no problem to use fetus in this treatment [referring fetal tissue transplantation]. the priority for me is this method is used for treating the patient.’ (pt-3, s-1) ‘if they do not want the fetus, they can consider aborting the fetus as they need this treatment [referring to fetal tissue transplantation]. i think, this treatment option can be considered.’ (pt-6, s-1) social stratification 15 ‘the only thing that comes to my mind with the possible treatment options offered in the scenario is the accessibility of the treatment by all people who do need it. i mean if some people can access and others could not, then, there would be a stratification among society who can afford or access the treatment and who cannot.’ (pt-3, s-2) societal betterment 12 ‘with this method [referring to gene therapy], the next generations will be healthy which is beneficial for the society as a whole.’ (pt-5, s-4) slippery slope 12 ‘i think parents should be given to change their children’s genes in case of diseases. except for diseases, using gene therapy for beauty or intelligence is worrisome.’ (pt-7, s-4) diversity 4 ‘i believe that society needs all kinds of people except smarter people. i think we do need farmers as much as we need scientists as a society. thus, there should be some kind of diversity among levels of intelligence.’ (pt-2, s-4) journal of science learning article doi: 10.17509/jsl.v4i1.28155 25 j.sci.learn.2020.4(1).20-30 applications might create social classes (those who can access the genetics applications and those who cannot) reflected in the social stratification theme. another theme was the slippery slope, which indicated that pre-service science teachers indicated that genetic engineering allowance in one proper context (i.e., use it for diseases) might lead to unexpected consequences (i.e., use it for beauty or intelligence). lastly, some statements highlighted the importance of heterogeneity in society, coded under the diversity theme. these pre-service teachers stated that there should be diversity in society as society needs different talents and different needs. pre-service teachers’ principle-based moral reasoning patterns in addition to pre-service teachers' consequentialist moral reasoning patterns, the pre-service teacher sometimes made decisions based on some prescripts or ethical principles such as the right to live or natural order. with this respect, their decisions were considered independent of genetics applications' consequences but dependent on some moral rules. here, four sub-themes emerged reflecting the principle-based moral reasoning theme: (a) parental rights, (b) taking human life, (c) a means to an end, and (d) distrusting natural order. table 5 shows the most frequently stated sub-themes emerged in the data: according to table 5, the most emergent theme was parental rights. while some pre-service science teacher (i.e., pt-1) expressed that parents had the right to choose what is the best for their children, other pre-service teachers (i.e., pt-4 and pt-5) reflected that parents should not have the right to change their children's genes by using gene therapy to make them smarter (s-4) or to let their unborn baby (fetus) to be used in experimental research (s-1). another emergent theme was taking human life, indicating participants' concerns as embryo's status and rights. they considered the fetus a human being and thus, believed that the fetus has the right to live. some participants had concerns about using the fetus as a tool or resource for the scenario's treatment (s-1). they perceived the fetus's use in an experimental trial as unexpectable if the fetus is only perceived as a tool for the trial. lastly, some participants indicated that there should be no external intervention to the natural order, which was coded under a disturbing natural order theme. these participants believed that there is an order in the natural which needs to be undisrupted. 3.2. pre-service science teachers’ emotion-based reasoning patterns participants' decisions about genetics related ssi did not always rely on consequent reasons or principles. preservice science teachers sometimes were influenced by emotions, including empathy or sympathy toward the characters or the unborn baby in the scenarios. sixty-nine statements (30.26%) were coded as emotive based. the parents considered donating the fetus for an experimental trial for their father, who has parkinson's disease. in cystic fibrosis and huntington disease scenarios, the symptoms are severe and mostly fatal. sample excerpts are provided below: 'if lale [referring to the fictitious character] gave birth to her child when the child turned to be the 50s, his/her parents might not be with him what i mean, and they could be deceased. i think it as myself. then, you are giving birth to someone with whom you will not be by the side. that is a responsibility. i consider making an abortion with this respect.' (pt-3, s-3) ‘i think it is a bad feeling to know when a person will die. thus, it is better to abort the fetus before it is too late.’ (pt-1, s-2) table 5 pre-service science teachers’ principle-based moral reasoning patterns principle-based moral reasoning frequency sample excerpts parental rights 16 ‘i think what family goes through is more important. thus, i believe that the family has the right to choose what is best for their unborn baby.’ (pt-1, s-3) ‘if parents were allowed to change their children’s genes, their responsibility would definitely change, in fact, it would increase. thus, parents should not be given the right to use gene therapy to increase the intelligence of their children.’ (pt-4, s-4) ‘we have no rights over individuals. thus, parents should not able to let their fetus be used in this experimental research [referring to fetal tissue transplantation].’ (pt-5, s-1) taking human life 14 ‘a fetus even in its embryo stage has the right to live. it makes no sense to abort the fetus, just as it might be sick.’ (pt-2, s-2) ‘while deciding on abortion, lale [referring to the fictitious character], she would be taking the fetus’ right to live.’ (pt-4, s-3) means to an end 7 ‘suzan [referring to the fictitious character] should not be allowed to abort the fetus if she only intends to donate it to an experimental trial.’ (pt-3, s-1) ‘if the main aim is to provide tissue for the experimental trial, this does not make sense. then, she would be killing her unborn baby for her father who already lived for a certain time.’ (pt-6, s-1) disrupting natural order 6 ‘we should not interfere with nature. we are evolving more and more as time passes. as we complete our evolution, our intelligence will also develop more. i think this process should be natural and there should be no external intervention.’ (pt-1, s-4) journal of science learning article doi: 10.17509/jsl.v4i1.28155 26 j.sci.learn.2020.4(1).20-30 'if the genetic test results show that the embryo is sick, reyhan, and semih [referring to the fictitious couple] should consider abortion. i am putting myself in their shoes. i decide to abort the fetus because the pain they would go through would be terrible for them.' (pt-7, s2) when the above-presented quotations were examined, it could be seen that the participants considered the emotional situations that the families would go through in the scenarios. especially the pain and the empathy towards the fictitious characters influenced their decisions in these scenarios. 3.3. the factors influencing pre-service science teachers’ decision-making processes the second research question explored the other factors (other than moral reasoning and emotion-based reasoning) that influence pre-service teachers' decisionmaking processes in genetics-related ssi. to answer this research question, the researcher dug into the data and investigated other influential factors in participants' decisions. there were 61 statements (21.19%) coded in this part (see table 6). here, there a wide range of influencing factors was revealed from the data. the most influential factor was the legal concerns reflected by participants. they table 6 the factors influencing pre-service science teachers’ decision-making processes influencing factors description frequency sample excerpts legal concerns participants referred to the need for standards or legal regulations in the genetics applications. 14 ‘if conducting gene therapy is allowed, there needs to be some kind of regulations by government or ministry of national health.’ (pt-2, s-4) pop culture participants referred to the films, media sources, or documentaries while making decisions in the genetics related dilemmas 9 ‘i have heard many rumors from media about the medical trials of these genetic applications are conducted with the convicts in the prisons. the films i have watched also portray similar issues. thus, i am affected by these kinds of issues.’ (pt-3, s-4) technological concerns participants referred to their technology-related concerns while making decisions in the genetics related dilemmas. 8 ‘the development of technology makes both happy and concerned. i am happy for them finding cures for diseases but worried about the technology might be used in an unwanted way and might be resulted in unwanted situations.’ (pt-1s-4) development of science participants referred their reliance on science and scientific developments while making decisions in the genetics related dilemmas 8 ‘i think the couple should not abort the fetus because any new treatment might come up at any time. science is progressing rapidly. thus, i believe treatment can be developed at any time.’ (pt-5, s-2) economic concerns participants indicated their concerns about economic reasons which would end up some people would access the genetics applications in the scenarios while some people could not. 6 ‘if this method [referring to gene therapy] is developed, then, it would be expensive. using this method would most likely be related to wealth. then, only wealthy people would use it and would be smarter.’ (pt-5, s-4) family bias participants indicated that their decision may change if the situation involved themselves or their family members 5 ‘i do make decisions in these hypothetical scenarios but i do not go through these situations. my decisions can be very logical when viewed outside. however, my decisions may change when i, myself go through it.’ (pt-4, s-4) personal experience participants used their previous experiences while making decisions in genetics related dilemmas 4 ‘there is a family who has a child with severe disabilities we know closely. the child lies down all the time. the family could not provide anything needed for the child. i think families should consider what they can provide to a sick child while making an abortion decision.’ (pt-3, s-2) ethical concerns participants referred that the ethical aspect of genetics application in the scenarios 4 ‘i think if that kind of treatment was really existed [referring to fetal tissue transplantation], getting pregnant for this treatment did not seem ethical to me.’ (pt-3, s-1) need more information participants indicated they need more information in the scenarios to make decisions. 3 ‘i think lale [referring to the fictitious character] should abort the fetus as the disease is fatal. however, we do not know many factors that are important at this point: what is the economic situation of this family? or we do not know anything about what lale’s husband thinks about this abortion option. thus, we need more details to make a broader decision.’ (pt-6, s-3) journal of science learning article doi: 10.17509/jsl.v4i1.28155 27 j.sci.learn.2020.4(1).20-30 were concerned about the need for some standard and legal regulations to create some standard procedures for applying genetics technologies (n=14). they were also influenced by the films, media sources, or documentaries reflected in pop culture sub-theme (n=9). some statements were showing pre-service science teachers' concerns about technology. they were concerned about the ill-usage of technology (n=8). however, there were some statements showing participants' reliance on scientific developments. they showed faith in science and how scientific development can change the current treatment options for diseases (n=8). their decisions sometimes were influenced by the economic constraints that some people might not have access to the genetics application because of economic conditions (n=6). similarly, some participants indicated that their positions might change if they include themselves or their family members (n=5). even though it is not very frequent, in some scenarios, participants used their own experiences to interpret the scenarios to make decisions (n=4). in a few cases, pre-service science teachers emphasized the ethical issues. for instance, one participant (pt-3) did not think it was proper to get pregnant to donate fetus' tissues for the hypothetical treatment scenario (s-1). lastly, in a very few cases, pre-service teachers indicated that they needed more information (i.e., information about the family's economic situation or the husband's opinion about abortion) to make a more comprehensive decision (n=3). table 6 presents the description of, frequency of and excerpts exemplifying each influencing factor. 3.4. discussion this study explored pre-service science teachers' moral reasoning patterns and the factors that influence their genetics-related ssi decisions. the results revealed that pre-service science teachers' decisions in genetics related ssi were heavily affected by moral considerations. this result can be an expected outcome as morality is crucial while discussing and making decisions in ssi (zeidler & keefer, 2003). existing studies also confirmed the significant role of morality while making decisions regarding ssi (bell & lederman, 2003; chang rundgren & rundgren, 2010; karısan & cebesoy, 2020; sadler, 2004a; 2004c; sadler & zeidler, 2004; zeidler & keefer, 2003). for instance, exploring college professors' decision-making in different ssi, bell, and lederman (2003) found out that 85% of their responses included ethical, moral, and value considerations. when the moral reasoning patterns were explored, pre-service science teachers' decisions were mostly influenced by genetics applications' consequences. in another study with pre-service science teachers (n=47), karisan and cebesoy (2020) revealed that participants' decisions were mostly influenced by ethical and moral considerations (42%). close examination of consequentialist moral reasoning patterns revealed that participants emphasized the participant's health improvement as a significant factor influencing their decisions. health improvement of participants was also a considerable concern for both science teachers (cebesoy, 2014) and college students (sadler & zeidler, 2004; sadler, 2004a) decisions supporting gene therapy or other genetic applications the scenarios. different consequentialist moral reasoning patterns including participants' concerns were disrupting natural order, creating social classes (those accessed to the genetic applications and those who cannot access) in the society and slippery slope implications of genetic applications (allowing gene therapy in one proper context might lead its use in other unacceptable contexts) and the need of a variation among society. these concerns were also frequently reported in existing studies (cebesoy, 2014; sadler & zeidler, 2004; sadler, 2004a). even similar patterns were found between the studies mentioned earlier and this study; some differences were also seen (sadler & zeidler, 2004; sadler, 2004a). for instance, sadler and zeidler (2004) reported overpopulation reflecting participants' concerns about the effects of genetic engineering application on the human population as a consequentialist moral reasoning pattern. this theme was not found in this study. in another study, sadler (2004a) reported that college students were mostly concerned about the effects of genetic applications for others, maintaining natural order, and slippery slope concerns. revealing different consequentialist moral reasoning patterns among various studies might be related to each study's participants were different (college students majoring in another department and in-service science teachers). compared to consequentialist moral reasoning patterns, pre-service science teachers' principle-based moral reasoning patterns were less referred to in their decisions. still, pre-service teachers mostly stressed parental rights. here, we also see two different opinions (one opinion expressing it is parents' right to choose what is best for their child and the other represents parents/individuals have no right over other individuals their children-). besides parental rights, participants showed concern about the embryo's status as a human being and aborting the fetus to donate its tissues for medical reasons, and the need to maintain natural order. while the finding of using principle-moral reasoning patterns less in their decisions was a common finding in this study and other studies (cebesoy, 2014; sadler & zeidler, 2004), finding parental rights pattern more when compared to different patterns (a means to an end, making human life or disrupting natural order) was surprising. existing studies reported that participants were more concerned about the embryo's status and scarifying it for medical reasons (cebesoy, 2014; sadler & zeidler, 2004). in this study, we found vice versa. this result also might be related to participant characteristics and backgrounds. another central theme found in pre-service science teachers' reasoning patterns was emotion-based reasoning. journal of science learning article doi: 10.17509/jsl.v4i1.28155 28 j.sci.learn.2020.4(1).20-30 here, participants showed empathy (i.e., statements like 'if i were her' or 'i am putting myself in their shoes') or expressed the suffering that the families/the patients would go through. the life quality of sick people influenced their decisions. this finding was also consistent with the literature stating emotions played an important role in participants' decisions indifferent ssi, including genetic applications (cebesoy, 2014; karisan & cebesoy, 2020; sadler & zeidler, 2004, 2005; sadler, 2004a; topcu et al. 2011), nuclear power plant and generation (lee et al. 2012; jho, yoon & kim, 2014), global warming and climate change (lee et al. 2012; topcu et al. 2011), organ transplantation, recycling and use of forest areas (ozden, 2020), and stem cell research (lee et al. 2012). regardless of the ssi topic, emotions were found to influence participants' decisions. moreover, fowler, zeidler, and sadler (2009) argued that ssi's moral emotions are crucial to enhance functional scientific literacy. this study also confirmed that participants showed care feeling towards the fictitious characters in the scenarios regardless of the scenarios discussed. the studies exploring participants' reasoning patterns revealed that participants sometimes showed sudden and unsupported claims while discussing ssi, which was defined as moral intuition (cebesoy, 2014; ozden, 2020; sadler & zeidler, 2004, 2005; topcu et al. 2011). on the contrary, this study revealed no moral intuition-based reasoning pattern among participants' responses. the second research question of this study explored which factors (other than moral) influenced pre-service decisions. the findings revealed that participants' decisions were affected by a wide range of factors, including legal, ethical, technological, economic concerns, personal experiences, pop culture, and family bias. overall, the literature showed that economic factors (bell & lederman, 2003; cebesoy, 2014; chang rundgren & rundgren, 2010; karisan & cebesoy, 2020; topcu et al. 2011), personal experiences (bell & lederman, 2003; cebesoy, 2014; chang rundgren & rundgren, 2010; jho et al. 2014; sadler & zeidler, 2004; sadler, 2004a; zeidler & schafer, 1984), support of science (bell & lederman, 2003; cebesoy, 2014), pop culture and family bias (cebesoy, 2014; sadler & zeidler, 2004) were influential factors for participants' decisions in ssi showing the necessity of including multiple perspectives. indeed, existing studies also stress the importance of having multiple views of ssi-based teaching (chang rundgren & rundgren, 2010; karisan & cebesoy, 2020). there were some statements showing participants' support for and faith in scientific developments in this study. this result confirms the existing literature reporting the participants showed a degree of faith in scientific developments (bell & lederman, 2003; cebesoy, 2014). significantly few participants indicated it would be better to know more information about the scenarios to make comprehensive decisions. this finding was in line with previous studies in which participants asked for more details for making broad decisions (cebesoy, 2014; sadler & zeidler, 2004). existing literature also reported that religious considerations were a significant factor influencing participants' decisions in various ssi (bell & lederman, 2003; cebesoy, 2014; chang rundgren & rundgren, 2010; sadler & zeidler, 2004, sadler, 2004a; topcu et al., 2011). in contrast to this finding, the present study revealed no religious concerns stated by the preservice teachers. 4. conclusion the present study revealed that pre-service science teachers' decisions in genetics-related ssi are mainly influenced by consequentialist moral reasoning and emotive-based moral reasoning. they considered a wide range of consequentialism-based factors including health improvement of patients, the risk of creating subclasses in the society based on the accessibility of genetics applications, the concerns of genetics applications might be used in an unacceptable context such as beauty reasons or making people smarter, and the need of diversity in the individuals building up the society. besides, they showed empathy and sympathy toward the characters presented in the scenarios. as a second attempt to explore if other factors influenced pre-service teachers' decisions, some factors including legal, economic, ethical, and technological concerns were revealed. moreover, participants used their personal experiences to make decisions. lastly, in some scenarios, participants indicated the need for more information for making comprehensive decisions. overall, these results showed that the participants considered a wide range of factors while making genetics-related ssi decisions. all these factors during ssi-based teaching could enhance pre-service science teachers' development of decision-making skills. when the crucial role of teacher preparation for teaching ssi in their classes considered, this study has some recommendations for further research: first of all, the present study included a small number of participants to shed light on pre-service science teachers' moral reasoning and the other factors influence their decisions while dealing with genetics related ssi. the number of participants might be increased to examine if the factors (moral intuition or religious considerations) that were not revealed in this study can be revealed or not. second, as this study confirms the need to include multiple perspectives and factors during ssi-based teaching, an intervention focusing on the inclusion of multiple perspectives can be developed to develop pre-service science teachers' decision-making skills in genetics related ssi. lastly, this study only adopted genetics related to ssi topics. other ssi topics, such as climate change, organ transplantation, and nuclear power plantation, can be used to explore participants' moral reasoning patterns and crosscase investigation, revealing differences in reasoning patterns among different ssi topics. journal of science learning article doi: 10.17509/jsl.v4i1.28155 29 j.sci.learn.2020.4(1).20-30 references aivelo, t., & uitto, a. (2019). teachers' approaches to genetics teaching mirror their choice of content and avoidance of sensitive issues. [e-reader version]. retrieved from http://dx.doi.org/10.1101/350710 archibald, m. m. (2016). investigator triangulation: a collaborative strategy with potential for mixed methods research. journal of mixed methods research, 10(3), 228-250. bell, r. l. (1999). understandings of the nature of science and decision making on science and technology based issues (unpublished doctoral dissertation). oregon state university, oregon. bell, r. l., & lederman, n. g. (2003). understandings of the nature of science and decision making on science and technology based issues. science education, 87(3), 352-377. bingle, w. h., & gaskell, p. j. (1994). scientific literacy for decisionmaking and the social construction of scientific knowledge. science education, 78(2), 185-201. bossér, u., lundin, m., lindahl, m., & linder, c. (2015). challenges faced by teachers implementing socioscientific issues as core elements in their classroom practices. european journal of science and mathematics education, 3(2), 159-176. braun, v., & clarke, v. (2013). successful qualitative research: a practical guide for beginners. sage. bryce, t., & gray, d. (2004). tough acts to follow: the challenges to science teachers presented by biotechnological progress. international journal of science education, 26(6), 717-733. caelli, k., ray, l., & mill, j. (2003). ‘clear as mud’: toward greater clarity in generic qualitative research. international journal of qualitative methods, 2(2), 1-13. cebesoy, u. b. (2014). an analysis of science teachers’ genetics literacy and related decision making process (unpublished doctoral dissertation). middle east technical university, ankara, turkey. chang rundgren, s. n., & rundgren, c. j. (2010). see-sep: from a separate to a holistic view of socioscientific issues. asia-pacific forum on science learning & teaching, 11(1), 1-24. eggert, s., ostermeyer, f., hasselhorn, m., & bögeholz, s. (2013). socio-scientific decision making in the science classroom: the effect of embedded metacognitive instructions on students' learning outcomes. education research international, 1-12. fowler, s. r., & zeidler, d. l. (2016). lack of evolution acceptance inhibits students' negotiation of biology-based socioscientific issues. journal of biological education, 50(4), 407-424. fowler, s. r., zeidler, d. l., & sadler, t. d. (2009). moral sensitivity in the context of socioscientific issues in high school science students. international journal of science education, 31(2), 279-296. gericke, n., & smith, m. (2014). twenty-first-century genetics and genomics: contributions of hps-informed research and pedagogy. in m. r. matthews (ed.), international handbook of research in history, philosophy and science teaching (pp. 423–467). springer. gresch, h., hasselhorn, m., & bögeholz, s. (2017). enhancing decisionmaking in stse education by inducing reflection and self-regulated learning. research in science education, 47(1), 95-118. guion, r. m. (2002). validity and reliability. in s. g. rogelberg (ed.), handbook of research methods in industrial– organizational psychology (pp. 57–76). blackwell. hsu, y. s., & lin, s. s. (2017). prompting students to make socioscientific decisions: embedding metacognitive guidance in an e-learning environment. international journal of science education, 39(7), 964-979. jho, h., yoon, h. g., & kim, m. (2014). the relationship of science knowledge, attitude and decision making on socioscientific issues: the case study of students’ debates on a nuclear power plant in korea. science & education, 23(5), 1131-1151. kahlke, r. m. (2014). generic qualitative approaches: pitfalls and benefits of methodological mixology. international journal of qualitative methods, 13(1), 37-52. karisan, d., & cebesoy, u. b. (2020). use of the see-sep model in preservice science teacher education: the case of genetics dilemmas. in w. a. powell (ed.), socioscientific issues-based instruction for scientific literacy development (pp. 223-254). igi global. khishfe, r. (2014). explicit nature of science and argumentation instruction in the context of socioscientific issues: an effect on student learning and transfer. international journal of science education, 36(6), 974-1016. lee, h., chang, h., choi, k., kim, s. w., & zeidler, d. l. (2012). developing character and values for global citizens: analysis of pre-service science teachers' moral reasoning on socioscientific issues. international journal of science education, 34(6), 925-953. maxwell, j. a. (2012). qualitative research design: an interactive approach (vol. 41). sage publications. merriam, s. b. (2009). qualitative research: a guide to design and implementation (3rd ed.). jossey-bass. miles, m. b., & huberman, a. m. (1994). qualitative data analysis: an expanded sourcebook (2nd ed.). sage publications. ministry of national education [mone]. (2018). primary science and technology curriculum for 3th–8th grades. [e-reader version]. retrieved from http://mufredat.meb.gov.tr/programdetay.aspx?pid=325 national academies of sciences, engineering, and medicine. (2016). science literacy: concepts, contexts, and consequences. the national academies press. organisation for economic co-operation and development [oecd]. (2012). assessment and analytical framework. [e-reader version]. retrieved from https://www.oecd.org/pisa/pisa-fordevelopment/pisa-d-assessment-and-analytical-frameworkebook.pdf ozden, m. (2020). elementary school students' informal reasoning and its' quality regarding socioscientific issues. eurasian journal of educational research, 86, 61-84. percy, w. h., kostere, k., & kostere, s. (2015). generic qualitative research in psychology. the qualitative report, 20(2), 76-85. rundgren, c. j., & chang rundgren, s. n. (2018). aiming for responsible and competent citizenship through teacher professional development on teaching socioscientific inquiry-based learning (ssibl). asia-pacific forum on science learning and teaching, 19(2), article 2. sadler, t. d. (2004a). moral sensitivity and its contribution to the resolution of socioscientific issues. journal of moral education, 33(3), 339-358. sadler, t. d. (2004b). moral and ethical dimensions of socioscientific decision-making as integral components of scientific literacy. the science educator, 13, 39–48. sadler, t.d. (2004c). informal reasoning regarding socioscientific issues: a critical review of the literature. journal of research in science teaching, 4, 513–536. sadler, t. d., & donnelly, l. a. (2006). socioscientific argumentation: the effects of content knowledge and morality. international journal of science education, 28(12), 1463-1488. sadler, t. d. & zeidler, d. l. (2004). the morality of socioscientific issues: construal and resolution of genetic engineering dilemmas. science education, 88, 4–27. sadler, t. d., & zeidler, d. l. (2005). patterns of informal reasoning in the context of socioscientific decision making. journal of research in science teaching, 42(1), 112-138. sadler, t. d., amirshokoohi, a., kazempour, m., & allspaw, k. m., (2006). socioscience and ethics in science classrooms: teacher perspectives and strategies. journal of research in science teaching, 43(4), 353‐376. teddlie, c., & tashakkori, a. (2009). foundations of mixed methods research: integrating quantitative and qualitative approaches in the social and behavioral sciences. sage publications. topcu, m. s. (2008). pre-service science teachers' informal reasoning regarding socioscientific issues and the factors influencing their informal reasoning (unpublished doctoral dissertation). middle east technical university, ankara. topcu, m. s., yılmaz-tüzün, ö., & sadler, t. d. (2011). turkish preservice science teachers' informal reasoning regarding http://dx.doi.org/10.1101/350710 http://mufredat.meb.gov.tr/programdetay.aspx?pid=325 https://www.oecd.org/pisa/pisa-for-development/pisa-d-assessment-and-analytical-framework-ebook.pdf https://www.oecd.org/pisa/pisa-for-development/pisa-d-assessment-and-analytical-framework-ebook.pdf https://www.oecd.org/pisa/pisa-for-development/pisa-d-assessment-and-analytical-framework-ebook.pdf journal of science learning article doi: 10.17509/jsl.v4i1.28155 30 j.sci.learn.2020.4(1).20-30 socioscientific issues and the factors influencing their informal reasoning. journal of science teacher education, 22(4), 313-332. walker, k. a., & zeidler, d. l. (2007). promoting discourse about socioscientific issues through scaffolded inquiry. international journal of science education, 29(11), 1387-1410. zeidler, d. l., & keefer, m. (2003). the role of moral reasoning and the status of socioscientific issues in science education. in d. l. zeidler (ed.), the role of moral reasoning on socioscientific issues and discourse in science education (pp. 7-38). springer. zeidler, d. l., & schafer, l. e. (1984). identifying mediating factors of moral reasoning in science education. journal of research in science teaching, 21(1), 1-15. zeidler, d. l., sadler, t. d., applebaum, s., & callahan, b. e. (2009). advancing reflective judgment through socioscientific issues. journal of research in science teaching, 46(1), 74-101. zohar, a., & nemet, f. (2002). fostering students' knowledge and argumentation skills through dilemmas in human genetics. journal of research in science teaching, 39(1), 35-62. a © 2020 indonesian society for science educator 205 j.sci.learn.2020.3(3).205-215 received: 25 april 2020 revised: 08 july 2020 published: 31 july 2020 evaluation of the science laboratory applications course in a pre-service primary school teacher curriculum nebi altunova1, hüseyin artun2* 1department of basic education, faculty of education, van yüzüncü yıl university, turkey 2 department of basic education, faculty of education, van yüzüncü yıl university, turkey *corresponding author. nebialtunova@gmail.com abstract in this study, the science laboratory applications (sla) course given in a department of elementary school teaching in turkey was evaluated for effectiveness. a triangulation research design, with mixed methods, was employed based on research data collected via a semi-structured interview form, a science experiments evaluation rubric (seer) developed by the researchers, and the researchers’ diary notes. the study group included 66 preservice teachers in their second year of study. to select the participants, a maximum variation sampling method was used with the qualitative interviews. the seer scores were analyzed using the packaged software of spss, while interview data were evaluated using content analysis, and descriptive analysis was applied to the researcher’s diary notes. the quantitative and qualitative results obtained in the study revealed that preservice teachers achieved the course outcomes as well as the objectives of the curriculum. based on these results, several suggestions are put forward for future researchers and practitioners. keywords elementary school preservice teacher, curriculum evaluation, course of science laboratory applications 1. introduction today, developments in science regarded as the primary criterion for accepting societies to be developed. examples of these developments could be said to include those in the areas of technology, astronomy, robotics, and engineering (hodson, 1988; topsakal, 1999). in other words, it is essential to know and make use of science not only for facilitating daily life but for the development of a country as well. in this respect, it would not be wrong to say that science constitutes the basis of scientific activities that will allow the development of a country (abruscato, 1988). science can define as the effort to understand the causes of a phenomenon occurring in our natural environment, to predict the aspects that might happen in the future, and to take related precautions in advance (aydoğdu & kesercioğlu, 2005; chalmers, 2013). in turkey, science teaching starts in the preschool period. in this preschool period, simple activities used to help children develop positive attitudes towards science (çınar, 2013). in the elementary school period, science taught starting from the 3rd class grade within the scope of the course of science. turkey has frequently given to science experiments in the curriculum. the science curriculum designed to be taught in the laboratory in the majority of the courses. therefore, almost all schools have a science laboratory. however, the lack of sufficient equipment for the teachers to use the laboratory is an obstacle to the use of laboratories in science classes (mone, 2018). for high school students, science divided into three areas, such as physics, chemistry, and biology, and science taught to high school students within the scope of these three courses. however, today, students are not considered to be as successful as expected, although science-related courses intensively taught at high schools (aslan & erden, 2018). there might be various reasons for this academic failure in science. yet, related studies conducted in the field demonstrate that this failure mainly occurs due to a lack of adequate and productive use of the teaching methods and techniques applied in the education processes of high school students (karamustafaoğlu, bayar, & kaya, 2014; taşkaya & sürmeli, 2014). in the teaching/learning process of the course of science, a wide variety of methods and techniques can use as in other courses. in science teaching, besides giving theoretical information, making laboratory applications journal of science learning article doi: 10.17509/jsl.v3i3.23706 206 j.sci.learn.2020.3(3).205-215 and experiment-aided teaching could be said to be indispensable for effective education (hofstein & lunetta, 1982). in the study conducted by wardani & winarno (2017), it was stated that laboratory activities are useful in learning science. in science courses taught in the elementary school period, students face basic science concepts for the first time in their educational lives. therefore, primary school teachers and/or preservice teachers need to become qualified in the field of science. laboratory activities play an important role in the teaching of science concepts. laboratory environments, where theoretical knowledge is tested, and various experiments carried out, facilitates the reinforcement of what is learned (wakeling, green, naiker, & panther, 2017; lacey, campbell, shaw, & smith, 2020). the ability of classroom teachers to design and carry out laboratory activities depends on the education they receive during the undergraduate period. laboratory use skills developed in the undergraduate period will make it easier for classroom teachers to teach practical science in their professional life. however, science lessons to be shown in the laboratory will both attract more attention of children and enable them to carry out continual learning through experiments (ahmad, shaharim, & abdullah, 2017). during their undergraduate education, elementary school preservice teachers take several courses regarding how to teach science courses and how to select the teaching methods and techniques they will use (çepni, ayvacı, bacanak, özsevgeç, & aydın, 2007). when the undergraduate program applied in the department of elementary school teaching is examined, it is seen that one of the courses taken by elementary school preservice teachers with science teaching is the course of “science laboratory applications” (cohe, 2018). the content of the class of “science laboratory applications” should cover the purpose and importance of laboratory in science teaching, safety in the laboratory, scientific methods, scientific process skills, the way these methods and skills are acquired, laboratory experiments for elementary school students (planning and conducting the experiments and evaluating the results of these experiments), preparing experiment-related worksheets and reports, sample experiments to be performed with simple and cheap materials and group work (cohe, 2018). in this respect, the undergraduate course of “science laboratory applications” has an essential place in the acquisition of the skills in question, and elementary school teachers should develop their related qualifications in line with the objectives of the course. the goals of the program have achieved to understand, and there is a need for studies on program evaluation. therefore, assessment of the curriculum of the course of “science laboratory applications” is thought to be beneficial for the training of qualified elementary school teachers. laboratories have an essential place in science education. in this regard, it can state that the lessons that improve laboratory skills are crucial. especially the laboratory practice lessons that prospective teachers take during the undergraduate period have a significant impact on the development of these skills (shapiro et al. 2015). there are many studies on the importance of the laboratory in science teaching in the international literature. reviews stating that science teaching in the laboratory is more effective than science teaching in the classical classroom were found. hofstein and lunetta (1982), scruggs and mastropieri (1990), fraser and mcrobbie (1995), hofstein and mamlok-naaman (2007) and karacop (2017) have demonstrated the effectiveness of the laboratory on science teaching. laboratory practices in curriculum turkey in this context can be useful to set out the nature of the course. program evaluation is an essential process in education. the purpose of program evaluation should be to reveal the value of the program and to judge the related administrative practices besides determining the learning outcomes for students (ertürk, 2013; uşun, 2016). while deciding on whether to go on using a program or not, it is necessary to conduct program evaluation studies to get an idea about the effectiveness of the program and to make decisions regarding the program (demirel, 2016). erden (1998) defines program evaluation as the process of collecting data regarding the effectiveness of the program with the help of observation and various other measurement tools, interpreting the data via comparisons with the criteria set as the indicators of the effectiveness of the program, and drawing conclusions related to the effectiveness of the program. according to sanders and nafziger (2011), program evaluation is done for the following purposes: determining and improving the strong and weak aspects of the program, predicting the probable problems to be experienced within the scope of the program and dealing with them in advance, determining the educational needs and sources, determining the desired educational outcomes, collecting the information necessary for planning and decision making, and gathering the required data to decrease the educational costs. ornstein and hunkins (2016) point out that program evaluation is done to identify students’ performance levels, to determine the strong and weak aspects of the program before and after its implementation and to determine its effectiveness by comparing it with other applications. the common purpose of program evaluation done in various ways is to evaluate the program in several respects, to determine whether the program functions well, and to make the necessary eventual changes in the program. in line with this common purpose of program evaluation, the present study aimed to find answers to the following questions to reveal the effectiveness of the curriculum of the course of “science laboratory applications (sla)”: journal of science learning article doi: 10.17509/jsl.v3i3.23706 207 j.sci.learn.2020.3(3).205-215 1. what is the distribution of elementary school preservice teachers’ scores in the “science experiments evaluation rubric”? 2. what are elementary school preservice teachers’ views about the course of “science laboratory applications”? 3. what are elementary school preservice teachers’ behaviors in the process of their taking the course of “science laboratory applications” according to the researcher’s diary notes?? 2. method this study was carried out using a mixed-method research design. the methodologies and techniques applied in mixed-method research are specific to the study; therefore, this method provides the researchers with a wide variety of opportunities (plano clark & ivankova, 2018). the reasons for using this method in studies include triangulation, complementary, development, initiation, and expansion (greene, caracelli, & graham, 1989). in the present study, among these reasons, triangulation was considered to be more appropriate. data triangulation allows viewing the problem from different perspectives by combining the qualitative and quantitative data collection tools (guion, 2002). 2.1. study group the quantitative part of the study was carried out with 66 2nd grade preservice teachers attending the department of elementary school teaching in the spring term of the academic year of 2018-2019. the qualitative interviews held with six volunteering preservice teachers among the 66 participants. the preservice teachers interviewed were coded as t1, t2, t3, and so on. these six preservice teachers selected among those who got low, average, and high levels of scores from the rubric. for this purpose, the maximum triangulation sampling method was used. 2.2. data collection tools in the study, as the data collection tools, science experiments evaluation rubric (seer), a semi-structured interview form, and researcher’s diary notes were used. the quantitative aspect of the study, seer, which made up of three dimensions (things to be done before, during, and after experiments), was used as the data collection tool. the participants’ scores in seer constituted the quantitative aspect of the study. the total scores obtained via seer were evaluated in the following way: the achievement scores ranging between 90-100 referred to “very good”; achievement scores ranging between 80-89 referred to “good” achievement scores ranging between 70-79 referred to “better than average”; achievement scores ranging between 60-69 referred to “average”; and those lower than 60 referred to “unsuccessful” (altunova & artun, 2019). in the development process of seer, the procedural steps determined by andrade (1997) were followed. these steps taken in the process of developing seer were as follows: 1. step 1: a related literature review for the evaluation of science experiments. 2. step 2: the criteria, definitions, and score ranges in the rubric were determined. 3. step 3: the draft rubric prepared. 4. step 4: the draft rubric piloted. 5. step 5: feedback for the rubric was taken (from the students and the faculty member). 6. step 6: the draft rubric was revised, and the necessary changes made. 7. step 7: content and face validity studies conducted. 8. step 8: the rubric was finalized and made ready for use. as the data collection tool used for the qualitative dimension of the study, a semi-structured interview form developed by the researchers and researcher’s diary notes used. an interview is a data collection technique in which participants respond verbally to a series of questions (fraenkel, wallen, & hyun, 2015). the researcher’s diary notes were the researcher’s written statements regarding what he saw, heard, experienced, and thought during the qualitative data collection process. in this diary, the researcher takes notes regarding contextual information about the place, time, data, and focus; descriptive information about the characteristics, attitudes, and behaviors of the individuals observed; descriptive index; observer’s comments; and overall comments (uzuner, 1999). all the in-class observations were noted down in the researcher’s diary on each day when the preservice teachers taught for 12 weeks. while taking notes related to the situations in class, the preservice teachers’ behaviors examined from different perspectives. in the study, the researcher’s diary notes were coded as rd1, rd2, rd3, and so on. 2.3. data analysis the quantitative data collected via the rubric within the scope of the study were analyzed using the packaged software of spss 22. for the analysis, descriptive statistics like percentage, frequency, and mean score used to evaluate the preservice teachers’ academic achievement. the data collected via the interviews were subjected to content analysis. content analysis is defined as organizing the data reason following the concepts obtained via the conceptualization of the data and as getting the themes explaining the data accordingly. the researcher’s diary notes were analyzed using the descriptive analysis method. descriptive analysis is a data analysis technique that allows drawing the picture of a current situation or phenomena (yıldırım & şimşek, 2013). 2.4. validity and reliability to achieve validity and reliability in scientific studies, lincoln (1995) suggests using particular strategies like negative situations, obtaining sample information, longtime participation, constant observation, and participant’s approval. according to the results of the reliability analysis journal of science learning article doi: 10.17509/jsl.v3i3.23706 208 j.sci.learn.2020.3(3).205-215 conducted using the packaged software of spss 22 for the rubric used as the quantitative data collection tool in the study, the cronbach’s alpha reliability coefficient was calculated as 0.74. in the process of developing the semistructured interview form, first, the related literate was reviewed to see what kind of questions could be directed concerning science teaching. following this, three experts in the field asked for their views about the draft interview form prepared. in line with the experts’ opinions, the interview form was finalized in a way to include five questions. the validity of the interview form, a female and a male preservice teacher were interviewed within the scope of a pilot application. in order to reveal what participants understand from the questions prepared, it is essential to conduct a pilot application first, which will allow ensuring the validity of the participants’ responses to the questions (türnüklü, 2000). as a result of the pilot application, the interview form was found to be understandable and applicable as a data collection tool, and the research process started. before the analysis, the interview forms to be analyzed were read thoroughly, and the internal consistency of the views reported by the same participant was examined. it is seen that all the responses to the questions were consistent and that the participants did not make any contradictory statements. therefore, the data collected were found to be reliable. after the content analysis was conducted on the data, the codes and themes were formed in order to ensure the reliability of these codes and themes, the reliability formula suggested by miles and huberman (baltacı, 2017). the analysis of the qualitative data, and the fit value among the three different coders was calculated as 0.81. the reliability ratio found to be higher than 0.70 revealed that the analyses conducted were valid. in order to ensure the reliability of the researcher’s diary notes, the audit trail method, another data collection tool used in the study, was used. in this method, the decisions, designs, procedures, and questions used in the analysis process are written down and reflected wholly and attentively. this method involves keeping records of the operations/activities so that future researchers can make use of them. the purpose here is to show the evidence and process that help obtain detailed results (başkale, 2016). 3. result and discussion this part presents the quantitative and qualitative results in line with the sub-problems. 3.1. results related to the quantitative data the scores obtained via seer constituted the quantitative data in the study. table 1 shows the mean scores of the preservice teachers regarding the items in seer. when table 1 examined, it’s seen that the preservice teachers’ mean scores were highest for the “preexperiment” and lowest for the “post-experiment.” concerning the sub-dimension of “pre-experiment,” the preservice teachers got the highest mean score for the item of “wears clothes appropriate to the laboratory (uniforms, gloves, table 1 preservice teachers’ mean scores in seer dimensions items x̄ pre-experiment wears clothes appropriate to the laboratory (uniforms, gloves, hair restraints, shoe covers, masks, etc.) 4.89 brings all the tools to be used for the experiment 4.85 presents the report at the beginning of the lesson 4.82 writes down the report using his/her own statements 4.62 writes legibly and neatly 4.38 writes accurately considering turkish language grammar and spelling/punctuation rules 4.36 introduces the materials to the students before the experiment and explains for what purposes the materials will be used 4.35 writes a thorough report including all sections 4.08 makes comprehensible, satisfactory and full explanations of the subject 3.80 makes definitions based on students’ views instead of providing direct definitions 3.73 during the experiment behaves attentively and carefully while using the materials 4.86 conducts the experiment according to the order of the related procedures 4.53 tells the students what s/he has done during the experiment 4.17 involves the students in the experiment and makes them active in the process 4.15 directs questions to the students and informs them about the experimental process 3.77 post-experiment uses the time effectively and productively 5,00 cleans the experimental materials and stores the items appropriately 4.82 encourages students to speak and tries to learn to what extent they have understood the subject 3.91 tests whether or not the students have understood the experiment 3.52 tests whether the product obtained is appropriate to the purpose of the subject of the experiement 1,61 journal of science learning article doi: 10.17509/jsl.v3i3.23706 209 j.sci.learn.2020.3(3).205-215 hair restraints, shoe cover, mask, etc.)” (x ̄=4.89), the average mean score for the item of “the report has been written legibly and neatly” (x ̄=4.38) and the lowest mean score for the issue of “makes definitions based on students’ views instead of providing direct definitions” (x ̄=3.73). in relation to the sub-dimension of “during an experiment,” the preservice teachers got the highest mean score for the item of “behaves attentively and carefully while using the materials” (x ̄=4.86), the average mean score for the item of “tells the students what s/he has done during the experiment” (x ̄=4.17) and the lowest mean score for the item of “directs questions to the students and informs them about the experimental process” (x ̄=3.77). in relation to the sub-dimension of “post-experiment,” the preservice teachers got the highest mean score for the item of “uses the time effectively and productively” (x ̄=5.00), the average mean score for the item of “encourages students to speak and tries to learn to what extent they have understood the subject” (x ̄=3.91) and the lowest mean score for the item of “tests whether the product obtained is appropriate to the purpose of the subject” (x ̄=1.61). the preservice teachers’ total scores in seer regarded as their general achievement scores. table 2 presents the distribution of the preservice teachers’ scores at the end of the academic term. when table 2 is examined, it is seen that the preservice teachers’ total mean score in the rubric was calculated as 84.18. the preservice teachers mostly got scores in seer within the range of 80-89 (good; n=48). an equal number of preservice teachers (n=16) got scores within the fields of 90-100 (excellent) and 70-79 (better than average). also, only one preservice teacher got a score within the scope of 60-69 (average). the maximum score the preservice teachers got was 97, while the minimum score was 63. lastly, the most frequent score was 87. 3.2. results related to the qualitative data in the study, the qualitative data were collected via the interviews held with the preservice teachers and the diary notes taken by the researcher. the data collected via the observations and the interviews were used together in a blended manner. the preservice teachers were first asked for their views about the course of science laboratory applications (sla). table 3 presents the preservice teachers’ overall views about sla. according to table 3, the preservice teachers reported views about such benefits of sla as allowing students to learning via experience (n=3), increasing permanency (n=3), allowing concretizing (n=3), making lessons entertaining (n=1), increasing students’ interest with the help of experiments (n=1), developing the ability to do experiments (n=1), increasing sociality (n=1) and allowing group work (n=1). it is seen that the preservice teachers mostly emphasize such characteristics of sla as its being practice-based and its supporting learning via experience. among the participants, t2 claimed that the experiments concretized what they had learned within the scope of the science course, saying, “it concretized the abstract points for me. if we concretize these points, then our students will understand them more easily and make better use of them in their future lives.” when the diary notes taken via the in-class observations by the researcher for program evaluation were examined, it is seen that the researcher’s diary notes were consistent with the data obtained via the interview forms. the researcher’s diary notes demonstrated that the preservice teachers learned by doing and experiencing science experiments. some of the researcher’s observations in the process were as follows: “another skill that the students developed was the ability to use tools. the preservice teachers teaching today are more skilled in using tools when compared to the past when they did their first teaching (rd6)”. in the study, all the preservice teachers interviewed believed that elementary school teachers should do experiments within the scope of the course of science. table 2 seer distribution of the preservice teachers’ scores score range total score f level between 90-100 97 1 *very good 94 1 93 1 92 1 91 1 90 3 between 80-89 89 4 **good 88 3 87 9 86 5 85 2 84 3 83 5 82 6 81 8 80 4 between 70-79 79 4 ***better than average 78 3 76 1 between 60-69 63 1 ****average *achievement score between 90-100; ** achievement score 80-89; *** achievement score between 70-79; **** achievement score between 60-69 table 3 preservice teachers’ views about sla general evaluation regarding sla f codes learning by experience 3 being permanent 3 allows concretizing 3 being entertaining 1 being interesting 1 developing the ability to do experiments 1 increasing sociality 1 allowing group work 1 journal of science learning article doi: 10.17509/jsl.v3i3.23706 210 j.sci.learn.2020.3(3).205-215 table 4 presents the reasons why preservice teachers supported the need for doing experiments in science teaching. when table 4 was examined, it was seen that the preservice teachers reported views about the need for doing experiments in science teaching. all the preservice teachers interviewed believed in the necessity of doing experiments in science teaching. according to the preservice teachers, the reasons for education via experiments included the following: experiments allow concretizing in science teaching (n=2), allow making practical applications (n=2), make teaching effective (n=2), allow associating theoretical knowledge with daily life (n=1) and increase permanency. the preservice teachers interviewed agreed on the need for doing experiments within the scope of the course of science. one of the preservice teachers coded as t3 stated that doing experiments in science teaching allowed concretizing, saying, “science is generally an abstract course, and that’s why it is necessary to do experiments to concretize the lessons.” another preservice teacher coded as t5 reported that the course of science is more related to daily life than other classes are, and s/he pointed to the importance of doing experiments, saying “science is more related to daily life, and it is thus necessary to do experiments so that students can understand the lessons better.” when the researcher’s diary notes taken for program evaluation were examined, the preservice teachers thought to do experiments made teaching more effective. the preservice teachers stated that the course was suitable for making practical applications. in relation to this, some of the diary notes taken by the researcher were as follows: “in addition, the preservice teachers came to the laboratory before the lessons on the day they would teach, and they had the opportunity to do experiments in a laboratory environment before their teaching (rd7)”. the preservice teachers’ views about the way of teaching the course were gathered under two sub-themes: positive views and negative views. table 5 presents the preservice teachers’ views about teaching the course of sla. table 5 shows the preservice teachers’ views about teaching the course of sla. it was seen that the preservice teachers reported more positive aspects of teaching the course of sla than negative views. the preservice teachers had positive views about the following points related to teaching the course of sla: the number of tools used in teaching the course was sufficient (n=4); the experiments were individual (n=2); the students had the opportunity to select the experiment they would do (n=2); the lessons included practical applications (n=2); a rubric was used as a measurement and evaluation tool (n=1); the lessons were taught in a laboratory environment (n=1); the experiments were related to daily life (n=1), and the preservice teachers were relaxed while teaching (n=1). on the other hand, the preservice teachers reported negative views about those who taught the course of sla in following respects: lacking experience (n=3), lacking methodological knowledge (n=2), lacking class management skills (n=1), inefficient in using tools (n=1), lacking content knowledge (n=1) and failing to use an understandable language (n=1). some of the preservice teachers’ positive views about the flow of the lessons were as follows: “to me, the evaluation method was quite good. now, we don’t have any doubts or question marks in our minds. also, we no longer question why we got a lower mark than we had expected. i mean, we now know our mistakes. that was quite beneficial for us because we can see our weaknesses (t2)”. besides the preservice teachers who had positive views about the way the course was taught, some others were reporting negative views. the participants’ views about the negative aspects of the way the course was taught were as follows: “in fact, we do the planning, and we think a lot in advance about what to do, but while teaching, we may face difficulties because we are really novice teachers (t2)”. when the teachers’ ways of teaching were examined, it was seen that they mostly had weaknesses in terms of table 4 preservice teachers’ views about the need for doing experiments in science teaching main theme views about doing experiments f codes allows concretizing 2 allows trying practical applications 2 allows doing effective teaching 2 associating theoretical knowledge with daily life 1 being permanent 1 table 5 preservice teachers’ views about teaching sla main theme views about teaching sla f codes positive efficient tools 4 individual experiments 2 students’ selecting the experiments 2 lessons taught via applications 2 useful measurement and evaluation tools 1 lessons taught in a laboratory 1 experiments related to daily life 1 feeling relaxed 1 negative lack of experiments 3 lack of methodological knowledge 2 inefficient class management 2 lack of content knowledge 1 use of a plain language 1 journal of science learning article doi: 10.17509/jsl.v3i3.23706 211 j.sci.learn.2020.3(3).205-215 professional knowledge and skills. during the interviews, the preservice teachers pointed to their lack of professional knowledge and skills. some of the observations regarding this issue were as follows: “while teaching an experiment, the preservice teachers were quite excited. when they fail to involve the class in the lesson, students’ attention is distracted. thus, the number of students interested in the lesson is decreasing (rd1)”. the preservice teachers put forward suggestions to make the course of science laboratory applications more productive, as can be seen in table 6. according to table 6, the suggestions put forward by the preservice teachers to make the course of science laboratory applications more productive were mostly related to the physical sub-structure. it was seen that the preservice teachers’ suggestions were about increasing the numbers of microscopes (n=3), dynamometers (n=1), and visual materials (n=1). among all the preservice teachers, one thought that a u-shape seating plan could be more effective. one of the preservice teachers put forward the following suggestions to make the course of sla more productive: “also, there were not enough microscopes. say there are 35 students in one classroom, and you divide these students into four groups. therefore, almost 6 or 7 students have to use the same microscope, and to me, this decreases the effectiveness of the lesson. in addition, there are no interesting visuals on classroom walls at all. also, the desks should be organized in u-shape (t6)”. 3.3. discussion at the end of the study, it was seen that the highest mean score of the preservice teachers regarding the things to be done before the experiment belonged to the item related to laboratory tools and clothes appropriate to laboratory conditions (gloves, hair restraints, shoe cover and so on). getting a high score in this item shows that the preservice teachers paid attention to the safety rules. besides, the preservice teachers could be said to be efficient in preparing reports. studies carried out by duru, demir, önen, and benzer (2011) and by uluçınar, cansaran, and karaca (2004) demonstrate that students have difficulty in writing reports. in the present study, the fact that the students were good at writing reports could be because there were several sample reports written at the beginning of the research process and that they were already used to writing reports in other courses. the item for which the preservice teachers got the lowest mean score was related to the teaching skills. according to the results obtained in the study, the highest score of the preservice teachers regarding the subdimension of things to be done during the experiment belonged to the item related to the use of tools. kılıç, keleş, and uzun (2015) report that the ability to use laboratory materials develops depending on the frequency of using tools. in the present study, the preservice teachers could be said to become proficient in using tools since they frequently did experiments during the lessons. as mentioned in the researcher’s diary notes, the students who were initially novice at using tools became better at using the correct tools towards the end of the research process. in this respect, the preservice teachers could be said to develop themselves in time. they got an average mean score for the item related to teaching the students in class what they had done during the experiment. in a study conducted by aslan and tekin (2015), the researchers pointed out that doing experiments developed the ability to teach. in the present study, the fact that the preservice teachers developed themselves in this respect while doing the experiments might have caused them to get a good score for this item. therefore, this situation could be the reason why the preservice teachers got low scores in this item of the rubric. in the study, the highest mean score that the preservice teachers got in the dimension of the post-experiment process belonged to the item related to time management. in one study carried out by arslan and umdu-topsakal (2019), it was revealed that the preservice teachers were quite good at time management. the fact that similar findings were obtained in the present study shows that the preservice teachers were proficient in time management. in addition, for the sub-dimension of post-experiment, the preservice teachers got an average level of the mean score in the item related to giving students the right to express themselves. the lowest score that the preservice teachers got for the sub-dimension of post-experiment and the whole rubric belonged to the item related to testing whether the product obtained as a result of the experiment served the purpose of the subject or not. in one study, demir, böyük, and koç (2011) found that the preservice teachers experienced difficulties in evaluating the product they obtained at the end of the research process. in the present study, this situation might have occurred because the preservice teachers had not taken any course related to measurement and evaluation. in the present study, based on the elementary school preservice teachers’ scores in seer, on their views determined via the interviews and on the diary notes taken by the researcher in the process, it could be stated that all the preservice teachers developed their qualifications to a great extent. depending on the researcher’s diary notes, it table 6 suggestions to make the sla course more effective main theme suggestions f codes number of microscopes should be increased 3 number of dynamometers should be increased 1 number of visual materials should be increased 1 there should be a u-shape seating plan 1 journal of science learning article doi: 10.17509/jsl.v3i3.23706 212 j.sci.learn.2020.3(3).205-215 could indicate that the course of science laboratory applications developed the preservice teachers’ ability to do experiments and thus helped them. not only teach the subjects via experience but also concretize the topics for permanency. according to avery and mayer (2012), teachers’ effective science teaching depends on their field knowledge to a great extent. similarly, jacobs, martin, and otieno (2008) point out that teachers’ skills have an important place in science teaching. in one study conducted by yao and guo (2018), it was found that the effective application of a science curriculum is closely related to teachers’ qualifications. therefore, the researchers emphasize the importance of science teaching in teacher training programs. in another study, davis, janssen, and van driel (2016) reported that teachers’ knowledge and skills influenced the frequency and way of their using the materials included in the science curriculum. based on all these findings obtained in different studies, it could be stated that during their undergraduate education, developing the laboratory usage skills of elementary school teachers who will teach science is essential for effective science teaching. in the study, the preservice teachers’ views about the course of science laboratory applications mostly focused on such issues as learning via experience, permanency, concretizing, being interesting, developing the ability to do experiments, and doing group work. in one study, levinson (2018) found that learning via experience developed the ability to do experiments, which was, in turn, reflected positively onto science teaching. in another study conducted by molefe and stears (2014), the researchers concluded that the experiment-doing skills of teachers who would teach science were influential in their science teaching. in the present study, the results obtained via the rubric, interviews, and researcher’s diary notes demonstrated that the course of science laboratory applications developed the preservice teachers’ teaching skills. based on the fact that the preservice teachers got high scores in seer; that during the interviews, they reported they became more proficient in science teaching; and that positive development was observed according to the researcher’s diary, it could be stated that the goals of the course of science laboratory applications were achieved. in this respect, preservice teachers taking the course of laboratory applications could be said to become proficient in the field of science teaching. in the study, it was seen that all the preservice teachers agreed on the necessity of doing experiments in science teaching. according to the preservice teachers, the reasons for doing experiments in science teaching were as follows: allowing concretizing, facilitating association of what has been learned in class with daily life, contributing to effective science teaching, and increasing permanency. jacobs, martin, and otieno (2008), who claimed that science teaching should be supported with laboratory activities in teacher training programs, stated that science teaching without doing any experiments would always be short of something. therefore, the researchers emphasized the need for developing laboratory skills during teacher training. in one study conducted with elementary school students, eren, bayrak, and benzer (2015) reported an increase in the permanency of what the students had learned within the scope of a science course supported with experiments. in the present study, it was also found that the experiments helped the students develop positive attitudes towards the science course. in the study, it was found that the preservice teachers had positive views about the course in following respects: the number of evaluation tools was sufficient; the experiments were individual; the students had the opportunity to select the experiments themselves; there were several practical applications; a rubric was used as a measurement and evaluation tool; the course was taught in the laboratory, and the experiments were related to daily life. studies conducted by arias, bismack, davis, and palincsar (2016), wardani and winarno (2017), and kaya and böyük (2011) demonstrate that a sufficient number of tools in science laboratories is important for doing experiments. in other studies, arias, davis, marino, kademian, and palincsar (2016) and güneş, dilek, topalgermi, & nesrin (2013) point out that practical application is important in science teaching and that experiments should thus be included in science teaching. similarly, in another study carried out by kelly and erduran (2019), the researchers reported that doing experiments was necessary for science teaching. in one study, chan, rollnick, and gess-newsome (2019) found that the use of a rubric had a positive influence on the teaching skills of the participants, which was a finding supporting the results of the present study. in the present study, the evaluation of the in-class activities using seer helped the students know what they would do. the preservice teachers made great efforts to meet the criteria in seer and eventually obtained good scores. in this respect, the use of a rubric as a measurement and evaluation tool could be said to be beneficial. in the study, the preservice teachers had negative views about the way the course of science laboratory applications was taught in the following respects: lack of experience, lack of methodological knowledge, lack of an ability to use tools, and lack of content knowledge. when these negative views were taken into account, it could be stated that the preservice teachers mostly focused on teaching skills. in one study, yeigh et al. (2016) point out that professional knowledge and skills are important for the profession of teaching and that it is important to acquire these skills and knowledge during preservice training. in another study, harris and farrell (2007) found that the teachers who taught science had not been trained well during undergraduate education and that their ways of teaching were not thus sufficiently effective. in one other study, alt journal of science learning article doi: 10.17509/jsl.v3i3.23706 213 j.sci.learn.2020.3(3).205-215 (2018) reported that the science teachers lacked certain professional qualifications, and the researcher suggested that these deficiencies could be overcome with the help of such methods as distance education and in-service pieces of training. in the present study, the researcher’s diary notes, and the interviews revealed that the preservice teachers were not efficient in making practical applications in science teaching. when the items in seer were examined, it was seen that the preservice teachers got low scores, mostly in relation to professional knowledge and skills. this result could be associated with the fact that they were almost on the halfway of their whole undergraduate education. in addition, this situation might have occurred due to the fact that the preservice teachers participating in the study did not make enough practical applications. this problem could be overcome via more practical applications to be done by preservice teachers. 4. conclusion when the seer scores obtained by the preservice teachers were taken into account, it could be stated that they achieved the objectives of the course. for this reason, the overall goals of the program could be said to be achieved to a great extent. in one study conducted by luft (1999), it was found that the science lessons taught using a rubric developed the preservice teachers’ skills in doing experiments. also, the results obtained via the interviews held with the preservice teachers participating in the present study revealed that following the research process, they developed positive attitudes towards doing experiments in science teaching. moreover, the diary notes taken by the researcher during the 12-week teaching process revealed that the preservice teachers developed themselves in terms of doing experiments in science teaching. in this respect, it was seen that doing experiments in science teaching developed the preservice teachers with respect to making practical applications, associating with daily life, and teaching effectively. based on these results, the following suggestions can be made: 1. within the scope of science course, if students select the experiments themselves that they are interested in and if they do these experiments in class, then the way of teaching becomes more effective. thus, preservice teachers could be provided with the opportunity to select the experiments to be done. 2. in the study, the rubric used for the evaluation of the in-class activities was found effective in experiment evaluation. in this respect, faculty members teaching science courses could be encouraged to use a rubric in their courses. 3. at the end of the study, it was seen that supporting scientific subjects with experiments in laboratories increased permanency. therefore, laboratory applications should be used more frequently in science teaching. 4. laboratory tools should be enough in number for crowded groups. thus, increasing the number of such tools could be beneficial for teaching science more effectively. references abruscato, j. (1988). teaching children science. boston: allyn and bacon. ahmad, c. n. c., shaharim, s. a., & abdullah, m. f. n. l. (2017). teacher-student interactions, learning commitment, learning environment and their relationship with student learning comfort. journal of turkish science education, 14(1), 57-72. alt, d. (2018). science teachers' conceptions of teaching and learning, ict efficacy, ict professional development and ict practices enacted in their classrooms. teaching and teacher education, 73, 141150. altunova, n., & artun, h. (2019, april 26). rubric development study for the evaluation of experiments in science laboratory practices course [proceeding]. paper presented at the 6th international multimedia dissipative studies congress, hasan kalyoncu university, gaziantep. retrieved from: https://www.multicongress.net/ andrade, h. g. (1997). understanding rubrics. educational leadership, 54(4), 14-17. arias, a. m., bismack, a. s., davis, e. a., & palincsar, a. s. (2016). interacting with a suite of educative features: elementary science teachers' use of educative curriculum materials. journal of research in science teaching, 53(3), 422-449. arias, a. m., davis, e. a., marino, j. c., kademian, s. m., & palincsar, a. s. (2016). teachers’ use of educative curriculum materials to engage students in science practices. international journal of science education, 38(9), 1504-1526. arslan, k., & umdu-topsakal, u. u. (2019). determining time management skills levels of science teacher candidates. academic perspective procedia, 2(1), 66-75. aslan, m., & erden, r.z. (2018). evaluation of 5th grade science curriculum]. necatibey faculty of education electronic journal of science and mathematics education, 12(2), 508-537. aslan, s., & tekin, n. (2015). reporting laboratory applications in argument-based science inquiry report format effects on conceptual understanding and using modal representation. erzincan university journal of education faculty (eujef), 17(1), 73-96. avery, l. m., & meyer, d. z. (2012). teaching science as science is practiced: opportunities and limits for enhancing preservice elementary teachers' self‐efficacy for science and science teaching. school science and mathematics, 112(7), 395-409. aydoğdu, m., & kesercioğlu, t. (2005). science and technology teaching in primary education. ankara: anı publishing. baltacı, a. (2017). miles-huberman model in qualitative data analysis. ahi evran university institute of social sciences journal, 3(1), 114. başkale, h. (2016). determination of validity, reliability and sample size in qualitative studies. dokuz eylul university e-journal of nursing faculty, 9(1), 23-28. chalmers, a. f. (2013). what is this thing called science? (4th ed.). indianapolis: hackett publishing. chan, k. k. h., rollnick, m., & gess-newsome, j. (2019). a grand rubric for differentiating the quality of science teachers’ pedagogical content knowledge. in a. hume, r. cooper, & a. borowski (eds.), repositioning pck in teachers’ professional knowledge for teaching science (pp. 251–269). singapore: springer. cohe [council of higher education] (2018). primary school teacher undergraduate curriculum. https://www.yok.gov.tr/documents/kurumsal/egitim_ogretim_ dairesi/yeni-ogretmen-yetistirme-lisansprogramlari/sinif_ogretmenligi_lisans_programi09042019.pdf journal of science learning article doi: 10.17509/jsl.v3i3.23706 214 j.sci.learn.2020.3(3).205-215 çepni, s., ayvacı, h.ş., bacanak, a., özsevgeç, t., & aydın, m. (2007). science and technology laboratory practices for primary school teachers-ii. trabzon: celepler publishing. çınar, s. (2013). determining the actvities used by preschool teachers in science and nature uses. journal of research in education and teaching. 2(1), 364-371. davis, e. a., janssen, f. j., & van driel, j. h. (2016). teachers and science curriculum materials: where we are and where we need to go. studies in science education, 52(2), 127-160. demir, s., böyük, u., & koç, a. (2011). science and technology teachers' opinions about laboratory conditions and usage and tendency to follow technological innovations. mersin university journal of the faculty of education, 7(2), 66-79. demirel, ö. (2016). program development in education. ankara: pegem a publishing. duru, m. k., demir, s., önen, f., & benzer, e. (2011). the effects of inquiry-based laboratory applications to preservice science teachers’ laboratory environment perceptions, attitudes and scientific process skills. marmara university atatürk education faculty journal of educational sciences, 33(33) 25-44. erden, m. (1998). curriculum evaluation in education. ankara: anı publishing. eren, c. d., bayrak, b. k., & benzer, e. (2015). the examination of primary school students’ attitudes toward science course and experiments in terms of some variables. procedia social and behavioral sciences, 174, 1006-1014. ertürk, s. (2013). program development in education. ankara: meteksan publishing. fraenkel, j. r., wallen, n. e., & hyun, h. h. (2015). how to design and evaluate research in education. new york: mcgraw-hill education copyright. fraser, b. j., & mcrobbie, c. j. (1995). science laboratory classroom environments at schools and universities: a cross‐national study. educational research and evaluation, 1(4), 289-317. greene, j.c., caracelli, v.j., & graham, w.f. (1989). toward a conceptual framework for mixed-method evaluation designs. education evaluation policy analyze, 11, 255-274. guion, l.a. (2002). triangulation: establishing the validity of qualitative studies. florida: institute of food and agricultural services. güneş, m. h., dilek, n. ş., topal-germi, n., & nesrin, c. a. n. (2013). teacher and student assessments regarding to use of science and technology laboratory. dicle university journal of ziya gokalp education faculty, 20, 1-11. harris, k. l., & farrell, k. (2007). the science shortfall: an analysis of the shortage of suitably qualified science teachers in australian schools and the policy implications for universities. journal of higher education policy and management, 29(2), 159-171. hodson, d. (1988). experiments in science and science teaching. educational philosophy and theory, 20(2), 53-66. hofstein, a., & lunetta, v. n. (1982). the role of the laboratory in science teaching: neglected aspects of research. review of educational research, 52(2), 201-217. hofstein, a., & mamlok-naaman, r. (2007). the laboratory in science education: the state of the art. chemistry education research and practice, 8(2), 105-107. jacobs, c. l., martin, s. n., & otieno, t. c. (2008). a science lesson plan analysis instrument for formative and summative program evaluation of a teacher education program. science education, 92(6), 1096-1126. karacop, a. (2017). the effects of using jigsaw method based on cooperative learning model in the undergraduate science laboratory practices. universal journal of educational research, 5(3), 420-434. karamustafaoğlu, o., bayar, a., & kaya, m. (2014). an investigation of science teachers’ teaching methods and techniques: amasya case. journal of theoretical educational science, 7(4), 436-462. kaya, h., & böyük, u. (2011). qualifications of science lectures teachers’ towards laboratory studies. erciyes university institute of science journal of science, 27(1), 126-134. kelly, r., & erduran, s. (2019). understanding aims and values of science: developments in the junior cycle specifications on nature of science and preservice science teachers’ views in ireland. irish educational studies, 38(1), 43-70. kılıç, d., keleş, ö., & uzun, n. (2015). science teachers' self-efficacy beliefs regarding to use of laboratory: effect of laboratory applications program. erzincan university journal of education faculty (eujef), 17(1), 218-236. lacey, m. m., campbell, s. g., shaw, h., & smith, d. (2020). self‐ selecting peer groups formed within the laboratory environment have a lasting effect on individual student attainment and working practices. febs open bio, 10(2020), 1194-1209. levinson, r. (2018). realising the school science curriculum. the curriculum journal, 29(4), 522-537. lincoln, y. s. (1995). emerging criteria for quality in qualitative and interpretive research. qualitative inquiry, 1(3), 275-289. luft, j. a. (1999). rubrics: design and use in science teacher education. journal of science teacher education, 10(2), 107-121. molefe, l., & stears, m. (2014). rhetoric and reality: science teacher educators' views and practice regarding science process skills. african journal of research in mathematics, science and technology education, 18(3), 219-230. mone [ministry of national education]. (2018). science course curriculum (primary and secondary school 3, 4, 5, 6, 7 and 8th grades. ankara: meb publishing. ornstein, a. c., & hunkins, p. f. (2016). eğitim programı temeller, ilkeler ve sorunlar (çev. ed. asım arı) [curriculum foundations, principles, and issues]. konya: eğitim publishing. plano clark, v.l., & ivankova, n.v. (2018). karma yöntemler araştırması: alana yönelik bir klavuz (ö. çokluk-bökeoğlu, çev.). ankara: nobel publishing. sanders, j. r., & nafziger, d. n. (2011). a basis for determining the adequacy of evaluation designs. journal of multidisciplinary evaluation, 7(15), 44-78. shapiro, c., moberg-parker, j., toma, s., ayon, c., zimmerman, h., roth-johnson, e. a., hancock, s.p., levis-fitzgerald, m., & sanders, e. r. (2015). comparing the impact of course-based and apprentice-based research experiences in a life science laboratory curriculum. journal of microbiology & biology education, 16(2), 186. scruggs, t. e., & mastropieri, m. a. (1990). the case for mnemonic instruction: from laboratory research to classroom applications. the journal of special education, 24(1), 7-32. taşkaya, s.m., & sürmeli, h. (2014). evaluation of primary school teachers’ teaching methods used in science and technology course. gaziantep university journal of social sciences, 13(1), 169-181. topsakal, s. (1999). science teaching. (2nd edition). istanbul: alfa publishing distribution. türnüklü, a. (2000). a qualitative research technique that can be used effectively in educational science research: interview. educational administration: theory and practice, 24, 543-559. uluçınar, ş., cansaran, a., & karaca, a. (2004). the evaluation of laboratory studies in science. the journal of turkish educational sciences, 2(4), 465-475. uşun, s. (2016). curriculum evaluation in education: processes, approaches and models]. ankara: anı publishing. uzuner, y. (1999). qualitative research approach. a.a. bir (ed.). in the research methods in social sciences (p. 175-190). eskişehir: open education faculty publication no: 601. wakeling, l., green, a., naiker, m., & panther, b. c. (2017). an active learning, student-centred approach in chemistry laboratories: the laboratory as a primary learning environment. in proceedings of the australian conference on science and mathematics education (formerly uniserve science conference) (p. 134). wardani, t. b., & winarno, n. (2017). using inquiry-based laboratory activities in lights and optics topic to improve students' journal of science learning article doi: 10.17509/jsl.v3i3.23706 215 j.sci.learn.2020.3(3).205-215 understanding about nature of science (nos). journal of science learning, 1(1), 28-35. yao, j. x., & guo, y. y. (2018). core competences and scientific literacy: the recent reform of the school science curriculum in china. international journal of science education, 40(15), 1913-1933. yeigh, t., woolcott, g., donnelly, j., whannell, r., snow, m., & scott, a. (2016). emotional literacy and pedagogical confidence in preservice science and mathematics teachers. australian journal of teacher education, 41(6), 107-121. yıldırım, a., & şimşek, h. (2013). qualitative research methods in the social sciences. ankara: seçkin publishing. a © 2018 indonesian society for science educator 77 j.sci.learn.2018.1(3).77-94 received: 29 june 2018 revised: 30 july 2018 published: 31 july 2018 doi: 10.17509/jsl.v1i3.11789 learning electricity using arduino-android based game to improve stem literacy alifa irna yasin1, eka cahya prima1*, hayat sholihin2 1department of science education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia 2department of chemistry education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia *corresponding author. ekacahyaprima@upi.edu abstract the stem is an interdisciplinary approach provided learning atmosphere where students can use science, technology, engineering and math in daily life. the aim of the stem is educating the students to be stem literate. this research goal was to implement stem learning on electricity using arduino-android game based experiment to 8th-grade students. stem learning was chosen as an approach in this research by the consideration that it was developed through android game, ywrobot, and arduino uno experiment lesson plan and worksheet. the analysis of this research was focused on the effect of stem learning implementation through lesson plan and worksheet to 8th-grade students’ stem literacy on electricity topic. the method used in this research was pre-experimental with one group pretest-posttest design. the data of this research was obtained from the stem literacy objective test (pretest and posttest) based on allan zollman. then the data were analyzed based on each aspect of stem literacy such as science, technology, engineering, and mathematics. the result shows that the value of g ̅ from students’ stem literacy pretest-posttest are -0.06, -0.12, -0.06, -0.87 for science, technology, engineering and mathematics literacy respectively. the result implies that stem learning implementation was less optimal to improve science, mathematics, technology and engineering literacy. the reason was because stem learning implementation was not implement in continuously. therefore, science, technology, engineering, and technology literacy regarding electricity topic are emphasized less optimally. keywords stem learning, stem literacy, android game, arduino uno, electricity 1. introduction education in every country should follow the current development to face competition in various areas of life especially science and technology aspects. the 21st century is a century of technology, where several countries have implemented many policies which related to scientific education (lou, shih, diez, & tseng, 2011). many scholars argued that the integration of science, technology, engineering and mathematics (stem) education have the advantage to the national economy, teachers, and institutes which have been working to develop this stem education program (tseng, chang, lou, & chen, 2013). stem education provides knowledge and skills that are transferable to the future problem and can help them to approach college and their career (laboy-rush, 2011). it coherent with the goal of igcse curriculum such as help the students to understand about scientific theories and method work together, the study of practice of science is affected and limited by social, economic, technology, ethical and cultural factors, the application of science in everyday life may be both helpful and harmful to the environment and the last knowledge that science overcomes national boundaries (cambridge international examinations, 2013). by implementing stem education, the researchers believed that students can study mathematics and science and to consider their careers in engineering, science, and technology in the future (prima, oktaviani & sholihin, 2018). the stem is considered as instructional strategies, teaching method, interdisciplinary approach and part of the curriculum. bybe (2013) argued that stem is an approach functioned as a tool for facilitating students become stem literate. it can be stated that the goal of the stem is creating stem-literate students. in line with reeve (2013) that the goal of stem approach is providing learning atmosphere where the students can use science, technology, engineering, and math in daily life context that mailto:ekacahyaprima@upi.edu journal of science learning article doi: 10.17509/jsl.v1i3.11789 78 j.sci.learn.2018.1(3).77-94 can be connected it into certain occupation and surround environment until stem-literate students who can compete in the knowledge-based economy. the term stem is derived from science, technology, engineering, and mathematics. stem education integrates the four disciplines through cohesive and active teaching and learning approaches (ong, mclean, & greco, 2014). bybee (2013) stated that stem education focused on designing a solution for real-world issues and problem. bybee (2013) also added that it is an educational approach to conduct science and engineering practices combining several strategies that provide an implementation of crosscutting concepts and core disciplinary ideas in science lessons. stem is aimed to critic and support nowadays education reform. it is adapted from common international standard to prepare better teacher and enhance coordination across the whole k-12 education system. stem as an approach provides active learning activity which beneficial for both students and teacher. stem pointed on a multidisciplinary approach, including inquiry and problem-solving to prepare students in stem subject and increase the number of post-secondary graduates who are prepared for stem occupations (national research council, 2013). this stem education is argued to improve the proficiency of all students in the stem which in line with 21st-century skills such as critical thinking, problemsolving, creativity, collaboration, self-directed learning, and scientific, environmental and technological literacy. specifically, five domains have been proposed for promoting stem involvement including; interest and engagement; competence and reasoning; attitude and behavior; career knowledge and acquisition; and content knowledge (prima, oktaviani & sholihin, 2018). moreover, integrating stem subject can be engaging for students can promote problem-solving and critical thinking skills to solve a real-world problem. in recent years, the stem becomes a trending pedagogic topic in every education scoop in many countries. the interest of researchers in the transformation of game-based learning technology, especially in stem education is growing rapidly. meluso, zheng, & lester (2012) stated that science content learning and self-efficacy of the students which taught through the stem and gamebased learning significantly increased. rapini (2012) argued that the games “world of goo” will help the player to understand the topic of static equilibrium. nowadays, learning science with arduino becomes the interesting discussion. arduino or microcontroller is the device that can control how the component in the circuits are connected or disconnected independently. huang (2015) argued that providing students with exposure to arduino and open-source is able to enhance physics classroom beyond the traditional book problems and traditional demonstration labs. one of the example projects which commonly done with arduino is traffic light. arduino can be programmed by the software and control light intensity. carro, et al., (2014) added that “the color of light” project with arduino is able to generate students’ curiosity and allow the teacher to emphasize the technology of led in daily life. the subject which commonly taught by using stem approach is physics. suwarma, astuti & endah (2015) argued that “balloon powered car” as a teaching media to deliver linear motion topic could increase students’ motivations and achievement in the final examination, and gave engineering learning experience. anwari, et al., (2015) shows that stem education approach in learning magnetism, electricity and electrical energy is able to attract students’ interest in science, provide deep and meaningful learning, and improve students’ thinking and practical skills. afriana, permanasari & fitriani (2016) reported that project-based learning integrated with the stem in experimental is significantly enhanced students’ scientific literacy in learning pollution. moreover, carro, et al., (2014) suggested that the use of arduino uno as a smart device is able to generate discussion among the students and enhance curiosity of students. stem education has a close relation with stem literacy. according to bybee (2013), one of the global challenges which faced by stem education is developing stem literacy as an aim and the identification of new learning outcomes, curriculum programs, and teaching practice. currently, there is not a standard or particular agreement from any professional education organization which clarifies the definition of stem literacy. one of the researchers of stem literacy, zollman (2012) developed a deictic description of stem literacy in three strata. the first stratum is literacies of science, technology, engineering and mathematics which commonly stated as four stem strands. the four strands of stem literacy are presented in the following explanations. (a) science literacy is defined as the ability of students to use and process to understand scientific knowledge such as physics, chemistry, biology and earth and space science. (b) technology literacy is defined as the ability to operate and demonstrate technology effectively and productively to conduct research and solve the problem collaboratively. (c) engineering literacy is defined as the ability to apply scientific and mathematics principal systematically and creatively to utilize practical ends. (d) mathematics literacy is defined as the ability to identify, understand and engage in mathematics to think creatively about solving problems. one of the topics on the stem which makes confusion to the students is electricity. students considered that physics as one of the most prevailing and problematic subjects in the realm of the subject which caused their low motivation and negative attitude towards learning (guido, 2013; prima, putri, & rustaman, 2018). the problems of learning electricity have been found by the previous works journal of science learning article doi: 10.17509/jsl.v1i3.11789 79 j.sci.learn.2018.1(3).77-94 (prima, putri & sudargo, 2017). mulhall, mckittrick & gunstone (2001) stated that electricity is a problem, as it involves extremely complex and highly abstract concepts and is thus totally dependent on models/ analogies/ metaphors. cao and brizuela (2016) also added that the students are hard to explain the role of the electric field in the interplay of the different elements of a circuit. there was also an alternative conception about electric current in the electric circuit which generated by the students. according to samsudin, et al., (2016) most students think that current flow fast because of turning on or off the switch of the lamp, they didn’t even realize about drift speed of current and electric dipole. to handle these students’ problems on investigating the electrical concept and application, the novel stem learning using arduinoandroid game based experiment will be first proposed. according to the background, this research aimed to improve students’ stem literacy in electricity by applying stem learning using arduino-android game based experiment. this prior research will be conducted by designing and analyzing the lesson plan, the worksheet, and the test instrument of stem literacy on electricity implemented to 8th-grade students. 2. method experimental research is used as the method of this research. pre-experimental design is chosen as one of the experimental research methods to discover a causal relationship only by involving one group of the subject. in other words, this method is used when there is no control group or extraneous factors which can influence internal validity (fraenkel, wallen & hyun, 2011). extraneous factors are any influences in selecting participants, the producers, the statistics, or the design likely to affect the outcome and provide an alternative explanation for the result than what its expected (creswell, 2012). in this research, the pre-experimental design was used to discover the effect of stem learning approach implemented by the researcher to students’ stem literacy. therefore, the researcher will know whether any change occurred due to the implementation of stem learning approach in learning electricity. the design of this research was one-group pretestposttest design. a pretest was used to measure some attribute or characteristic of participants in an experiment before receiving treatment. while posttest was used to measure some attribute or characteristic of participants in an experiment after receiving treatment (creswell, 2012). in this research design, the ability of students’ stem literacy was measured before and after the implementation of stem learning. a diagram of this design was explained in table 1. the location of this research was one of the private junior high school, which is located in bogor, west java. because this school applied the igcse curriculum in the teaching-learning activity. the population of this research would be all 8th students. the samples in this research were one class, which consist of 16 students from the 8th grade at x private junior high school. this class consisted of 7 female and 9 male students who were 15 years old. according to the index of student’s achievement from school, this class consisted of students who categorized as medium achiever from the other two classes. in order to avoid misconception about this research, some operational definitions are explained in this research. those terminologies are explained as follow: science, technology, engineering and mathematics (stem) is an approach used to teach science and mathematics applications through engineering activities that involve the development of technology. stem approach is the independent variable in this research which implements in the class. the stem approach which implemented in learning electricity is described in the lesson plan that has been arranged. stem learning implementation in this research is investigated through an observation sheet. moreover, stem literacy is the ability to identify, apply, and integrate concept from science, technology, engineering, and mathematics to understand complex problems and to innovate to solve them. stem literacy is the dependent variable in this research which the result depends on the implementation of the stem approach in learning electricity. in this research stem literacy of the students is investigated through stem literacy instrument test. in this research, the instrument is necessary to gain the data. there are several types of research instrument used in this research. those instruments are described as follow: observation sheet was used to measure the implementation of stem learning in the class. the observation sheet contained the steps which conducted by the researcher and the students that will be assessed whether it is implemented appropriately or not. the objective test is the instrument which used to test students’ stem literacy. in this research, a stem literacy based test which covers the four aspects such as science, technology, engineering and mathematics in electricity topic used as an objective test. this test was used in both pretest and posttest. the pretest was given before conducting the treatment to know stem literacy basic of students. while posttest was given to know the improvement of students’ stem literacy. the type of question of pretest and posttest was multiple choice, which contained 27 questions. the multiple choice questions table 1 one group pretest and posttest design pre-test implementation post-test stem literacy based electricity pretest implementation of stem learning stem literacy based electricity posttest (fraenkel, wallen & hyun, 2011) journal of science learning article doi: 10.17509/jsl.v1i3.11789 80 j.sci.learn.2018.1(3).77-94 were constructed based on the four strands of stem literacy from zollman (2012). the objective test used in this research was tested before it was used as pre-test and post-test. the instrument developed started by analyzing the four stem aspects and indicators which suitable for the concept of electricity. then the researcher formulated and arranged the objective test. in the preparation stage, the objective test was validated and tested to the students who have learned about electricity, which is 10th-grade students. the analysis of objective test which used in this research consisted of the discriminating power, distractor, and level of difficulty, validity, and reliability. the content of electricity that is used in this research is limited by cambridge igsce syllabus on physics 0625 for examinations 2016 on competence 4.2 about electrical quantities and competence 4.3 about electrical circuits. moreover, arduino uno is known as a small microcontroller board with usb plug which can connect into pc, laptop or even directly to electricity sources. arduino or microcontroller as shown in figure 1 is the device that can control how the component in the circuits are connected or disconnected independently. the control command can be controlled from a computer or programmed by the computer using arduino’s software. the name of arduino is given by the original makers (monk, 2010). there's also the clone of arduino design which often used “druino” as the last name of the design such as freeduino or dfrduino. according to funduino (2016), there are different size of board that can be used with the arduino software. the official name of the board is “arduino”, this board is cheaper than others but still an equivalent arduino. officially the board of the arduino that mostly used is arduino uno, arduino mega, arduino mini, etc. the software or program that needed to control the arduino is “arduino”. this software or program available for mac, pc, and linux. it can be can freely download it from www.arduino.cc. in this research, the arduino or microcontroller is added as the component of the circuit that should be built by the student. the arduino will be programmed by the software and control the light that will turn on in the circuit. in this case, the student will be asked to make a prototype of the traffic lights that can be modified by them self. android game: mgames science as shown in figure 2 is logic games puzzle and science games. it is developed by rahul dilip ogale and udita rahul ogale and published in 4th august 2016. it is 3.2 mb game and can be download in google play store and apkpure.com. mgames is an innovative way to learn science through the science game. it will help the users to learn how to make a circuit (series and parallel), find the effective resistance for a given circuit, and calculate the current. moreover, there is introductory material about series and parallel circuits, ohm’s law, and resistance in combination circuits. therefore, the users can read it before playing the game. mgames consists of four stages and one free mode stage. in each stage, there will be 1220 levels which should be passed by the users. if the answer is wrong, the next level can’t be opened. for example, in the first stage, as shown in figure 3, by the following instruction, the game will ask to connect a wire and adding resistance in the given circuit by tapping two adjacent grid point. figure 1 aduino uno (source : store-usa.arduino.cc) figure 3 mgames science welcoming screen (sources: mgames science) figure 2 stage one (source: mgames science) journal of science learning article doi: 10.17509/jsl.v1i3.11789 81 j.sci.learn.2018.1(3).77-94 3. result and discussion 3.1 lesson plan applied to stem learning constructing a stem learning lesson plan in this research, stem learning is used as an approach to conducting the teaching-learning activity. therefore, the lesson plan is constructed as an instrument to measure the implementation of stem learning. there are the stages in constructing lesson plan, such as analyzing the curriculum, physics topics, and organizing learning activity which appropriate with stem learning. the explanation of those three stages are described as follow: curriculum analysis 2013 national curriculum and cambridge igcse curriculum are analyzed to select a curriculum that supports the implementation of stem learning. cambridge curriculum is more able to provide better scientific understanding, life-long skill using technology and applying multidiscipline subject in daily life. moreover, that objective is in line with the purpose of stem learnings which is developing students’ understanding through scientific practices in other to solve the real-world problem by crosscutting science ideas and technology. physics topics analysis after choosing cambridge igcse curriculum, the content of physics for 8th grade is determined. the researcher selects the topic of electricity and subtopics of electrical quantities and an electrical circuit. thus, it will be adequately proper with mgames science (electricity) and arduino uno which used in this research. organizing learning activity the lesson plan doesn’t not only consists of delivering the topic of electricity as stated in learning objective but also shows the approach used in the class. the approach used in this research is stem learning adopted from jolly (2014). therefore, the lesson plans are developed through the topic of electricity and adjusted with the approach of stem learning. mgames science and arduino uno experiment are chosen as a learning activity because both table 2 summary of stem learning lesson plan meeting activity science concept developed stem literacy 1. wed, 26 april 2017 stem literacy pre-test recalling the basic concept of electricity. calculating the resistance and applying ohm’s law by using mgames science introduction to ywrobot and arduino current resistance ohms’ law voltage potential difference science literacy students are able to recall how resistance and potential difference affect the current. technology literacy students are able to use mgames science application in android phone. mathematics students are able to calculate the resistance. students are able to apply ohm’s law 2. wed, 10th may 2017 recalling previous topics; electrical components. introduction to ywrobot series and parallel construction with ywrobot current and voltage measurement with multimeter combined series-parallel construction introduction to arduino uno series and parallel circuit voltage and current measurement science students are able to recall a resistor, diode, and battery as an electrical component. students are able to recall the principle of series and parallel circuit. technology students are able to identify the part of ywrobot. students are able to identify arduino uno engineering students are able to construct series and parallel circuit with ywrobot. students are able to use multimeter appropriately. 3. wed, 24th may 2017 introduction to arduino uno; blink project. constructing a traffic light circuit with arduino uno modifying the traffic light circuit. stem literacy post-test parallel circuit science students are able to recognize the scheme of parallel circuit principle. technology students are able to identify the port of arduino uno. students are to identify arduino software engineering students are able to construct a traffic light circuit by using arduino uno journal of science learning article doi: 10.17509/jsl.v1i3.11789 82 j.sci.learn.2018.1(3).77-94 experiments can be engaged with the topic of electricity and adequately adjusted with stem learning approach. after that three-stage finished, the lesson plans are constructed in three meetings to develop each of the stem literacy aspects. the full version of three lesson plans used in 8th-grade physic students is available in the attachment. while the summary of those three meetings is explained as tabulated in table 2. from table 2, it can be seen that each meeting doesn’t emphasize all stem literacy aspects. stem literacy aspects are delivered separately in three meetings. while stem learning is mostly implemented in all meetings. stem learning which developed by jolly (2014) in those three meetings could be described as follow: involve students in productive teamwork in every meeting, the students had to work collaboratively in the group to solve each problem provided in mgames science application, series-parallel construction with ywrobot and a traffic light project with arduino uno. hands-on inquiry and open-ended exploration in the first meeting, students are asked to use mgames science application. in each level of the game, the students are required to tap and type the result and directly submit the answer. the game will give the response directly, if their answer right, the game will lead the students to the next level. while if they fail, the students will not be able to continue to the next level. for the second and third meeting, students are required to construct series and parallel circuits by connecting manually jump wire or resistor and lamps into protoboard and ywrobot or arduino uno. moreover, students are asked to modify their circuit without making the lights off. having multiple right answer and reframe failure as part of learning mgames science used in the first meeting has four levels which will challenge the students. one of level in the games is “constructing resistor path” will require students to try and check their connecting path scheme in mgames science application. therefore they might have multiple right answers and experienced the failure. in the second meeting, students are asked to create a series and parallel circuit. while in third meeting students are asked to construct a traffic light circuit which more advanced than the previous experiment. a student may find an obstacle in constructing and modifying the circuit such as failure to connect jump wire and resistor into protoboard or port in arduino uno correctly which make the circuit can’t perform well. but, the students are asked to find and try an alternative solution by themselves. therefore, the students will more understand to solve the problem. apply math and science content almost all levels in mgames science application used in the first meeting have required the students to calculate resistance, voltage, and current by using ohms’ law. table 3 the result of observation sheet during stem learning implementation meeting activity observer score ( rs ms x100%) category 1 2 3 t s t s t s 1 introduction √ √ √ √ √ √ 83 less complete pre-test of stem literacy √ √ √ √ √ √ team and placement √ √ √ √ √ √ recalling previous materials: how current is affected by resistance and the potential difference √ √ √ √ √ √ applying ohm’s law by using mgames science √ √ √ √ √ √ introduction to ywrobot and arduino x x x x x x max score; 36 raw score 10 10 10 2 recalling previous materials :electrical components √ √ √ √ √ √ 100 complete introduction to ywrobot √ √ √ √ √ √ team and placement √ √ √ √ √ √ series and parallel circuit construction with ywrobot √ √ √ √ √ √ current and voltage measurement with multimeter √ √ √ √ √ √ combination circuit (series and parallel circuit with three lamps ) construction √ √ √ √ √ √ introduction to arduino √ √ √ √ √ √ max score; 42 raw score 14 14 14 3 recalling previous materials : arduino uno √ √ √ √ √ √ 100 complete team and placement √ √ √ √ √ √ introduction to blink project with arduino uno √ √ √ √ √ √ traffic light project with arduino uno √ √ √ √ √ √ modifying circuit √ √ √ √ √ √ post-test √ √ √ √ √ √ max score; 36 raw score 12 12 12 average score 94.3 journal of science learning article doi: 10.17509/jsl.v1i3.11789 83 j.sci.learn.2018.1(3).77-94 therefore this game strengthens the topic of electricity to students. in the second meeting, students are not required to calculate but to prove the principle of current and voltage in series and parallel circuit. the students need to find whether series or parallel which have the same current flow and prove whether the theory of the series-parallel circuit is coherent with experiment. the last, in the third meeting students, are required to remember the structure of the parallel circuit. learning activity is guided by engineering design process in the second meeting, the students are required to measure both current and voltage in series and parallel circuit. the students have to know how to set and operate table 4 content summary of stem learning worksheet no worksheet questions science concept developed stem literacy 1 mgames science experiment part 1 1. how is currently produced in this circuit? 2. how does ohm’s law mention about the circuit? 3. how does ohm’s law mention about the circuit? 4. what are the differences between series and parallel circuit? 5. how do you calculate the resistance of series and parallel circuit current resistance voltage ohms’ law science students are able to recall how current is affected by resistance and potential difference. technology students are able to use mgames science application in the android phone. mathematics students are able to calculate resistance. students are able to apply ohm’s law part 2 1. stage 1, connecting the circuit path. 2. stage 2, calculate the total of resistance. 3. stage 3, arranging resistors based on the total of resistance. 4. stage 4, applying ohm’s law to calculate voltage, current, and resistance. 2 ywrobot experiment 1. measuring voltage and current in the working circuit (series circuit) 2. measuring voltage and current in the working circuit(parallel circuit) 3. build your own circuit (series circuit with two lamps) 4. build your own circuit (combined series and parallel circuit) series and parallel circuit voltage and current measurem ent science students are able to recall a resistor, diode, and battery as an electrical component. students are able to recall the principle of series and parallel circuit technology students are able to identify the part of ywrobot. students are able to identify arduino uno engineering students are able to construct series and parallel circuit with ywrobot. students are able to use multimeter appropriately. 3. arduino uno experiment 1. draw the scheme of the circuit in the table. 2. modify your own project and write which part of the project that has been modified parallel circuit technology students are able to identify the port of arduino uno. students are able to identify arduino software engineering students are able to construct a traffic light circuit by using arduino uno journal of science learning article doi: 10.17509/jsl.v1i3.11789 84 j.sci.learn.2018.1(3).77-94 a multimeter to measure current and voltage. afterward, the students are challenged to construct a combined seriesparallel circuit with three lamps. if the students fail to light on three lamps, they have to check the arrangement of resistor and lamps into a negative or positive pole in a protoboard, re-arrange and reconstruct the circuit. while in the third meeting the students are challenged to develop their ideas to construct a parallel circuit with three lamps which connect to arduino uno. students have to test their construction by connecting the circuit to the laptop and if their lamps failed to glow after inputting the formulae through arduino software, they had to evaluate whether they failed to connect the wire to gnd (ground) or to the digital pit in arduino uno. after the students found their failure, they had to re-design and re-construct their circuit. focus on daily life and environmental phenomenon in the third meeting, the students are required to make a project which relates to electricity. the traffic light project was considered to relate to the topic of electricity and near to daily life phenomenon. after using ywrobot in the second meeting, in this meeting the students are required to use a more advanced electrical component such as arduino uno. stem learning lesson plan implementation the implementation of stem learning was conducted within three meetings, it included pretest and posttest. this research was conducted between aprils – may 2017 in one of the private junior high school in bogor regency with the 8th-grade students as the sample. in implementing this learning approach, the observation was conducted to check the whole learning activity whether it is consistent with the steps which written in the lesson plan or not. the template of the observation sheet for three meetings was the same, but the sequence of stem learning activity is different. it was made based on every lesson plan constructed by the author. the score of observation can be gained from the result of the observer. there were two check mark, “yes” and “no”. “yes” was written, if during observation the observer found the learning activity was done completely. while “no” was written if during observation the observer found the learning activity was done incompletely. there was column provided to write the additional description or comments related to the observation. the observation sheet was given to 8th-grade physics teachers before class activity begin. the teachers stayed in the class and observed the learning activity conducted by both teacher and students in every meeting. the full format of the observation sheet can be accessed in the attachment. the result of the observation sheet, include the sequence of stem learning activity are shown in table 3. according to the observation result from the physics teacher, the implementation of stem learning was applied almost completely. the first meeting was considered as less complete because the researcher couldn’t able to run the time management effectively. therefore the class ended without any conclusion and the step of introduction to ywrobot and arduino is missed. therefore, the first meeting obtains 83%. while the rest of the learning activity in the second and third meeting is run adequately. 3.2 worksheet applied to stem learning constructing stem learning worksheet besides developing a lesson plan, the worksheet is constructed as a compliment for delivering the procedure regarding the experiment and assessing students’ work. as well as a lesson plan, the worksheet is developed through learning objective the topic of electricity as stated in chapter ii; electricity developed stem literacy and learning activity (experiment) which adjusted with stem learning written in the lesson plan. there are three worksheets used for three meetings. as well as a lesson plan, each worksheet used belongs to one lesson plan and emphasize a certain aspect of stem literacy. the full version of three worksheets is available in the attachment. while the content summary of each worksheet and its relation with developed stem literacy are described in table 4. it can be seen from table 4 that every question in the worksheet is constructed coherently with the experiment conducted by the students. therefore the teacher can assess students’ understanding of electricity and students’ performance regarding the experiments. moreover, those three worksheet emphasizes certain stem literacy aspects in each meeting. after constructing worksheet as mentioned in the previous explanation, the three worksheets are used in three meetings. the usage of each stem learning worksheet regarding its experiment is described as follow; table 5 students' response percentage of games science worksheet (introduction) no question % correct t f total students: 16 1 how is currently produced in this circuit? 87.5 14 2 2 is there any resistor in that circuit what is the function of resistor? 75 12 4 3 how does ohm’s law mention about the circuit? 100 16 4 what are the differences between series and parallel circuit? 25 4 12 5 how do you calculate the resistance of series and parallel circuit? 100 16 average 77.5 12.5 3.6 journal of science learning article doi: 10.17509/jsl.v1i3.11789 85 j.sci.learn.2018.1(3).77-94 mgames science worksheet the use of mgames science is the main activity in the first meeting, therefore this worksheet use as a complement to engage students’ with mgames science. it can be seen from the type of questions. question 1-5 delivered as an introduction to recall students’ previous topic about basic electricity before using mgames science. if the students forget about the basic topic of electricity, students might answer those five questions by opening “learn about circuit” page in mgames science. the students’ response regarding the questions is tabulated in table 5. most of the students answer the question 1 correctly but there is a student who has a different understanding about the current flow. even in the worksheet, there is no magnet and magnetic field figure, but the student relates the question “how is currently produced in this circuit (parallel circuit)” with magnetic field concept. therefore question number 1 have answered correctly by 87.5% students. the expecting and the alternative answers of the students can be seen in figure 4. 75% of students answered the second question correctly, but there are several students who have alternative answers. those students considered led as one of the resistors to conduct the electricity. both right and alternative answers of students can be explained in figure 5. question number 3 and 5 reach 100 % correct, it is because ohm’s law and series-parallel resistance formula were learned frequently by the students in the physic class. therefore the students can answer those questions correctly. the students’ answer to both questions is shown in figure 6. question number 4 have the lowest achievement from other questions, 25 %. it happened because most of the students don’t understand clearly the main point of question number 4. but there are students who can differentiate the circuit through the voltage and the current flow. the differences between alternative and expected answers from the students are shown in figure 7. the second part of the worksheet is adopted from mgames science. it consists of four stage with five level in every stage. since students’ work on mgames science can’t figure 4 students' answer (no.1) figure 5 students' answer (no.2) figure 6 students’ answer (no. 3 and 5) figure 7 students' answers (no. 4) journal of science learning article doi: 10.17509/jsl.v1i3.11789 86 j.sci.learn.2018.1(3).77-94 be recorded, therefore adding the five level of every stage in the worksheet is aimed to assess students’ work in using mgames science easily. the students’ response percentage are tabulated in table 6. in the first stage, level 14 questions are answered correctly by 100% students. this is because the level 1-4 ask the students to draw the basic path of series and parallel which is very common for students. while for level 5, there are students who fail to connect two resistors in a different path. therefore this level is answered correctly by 87.5% of students. both alternative and expected answers from students are shown in figure 8. the second stage of mgames science is answered correctly by 100% students. all levels in the second stage ask the students to calculate the equivalent resistance in series and parallel which very common to the students. therefore this stage is completely finished by the students as shown in figure 9. table 6 students' response percentage of mgames science worksheet (part 2) no questions total students : 16 % correct stage one 1 connect a single resistance in the circuit. your circuit should have only the part 100 2 connect 4 resistance (anywhere) in a circuit. the circuit should have only one path. 100 3 connect the two given resistance in series. 100 4 draw and connect two resistors resistance in parallel. 100 5 connect the two given resistance in parallel. (each resistance should be in a different path) 87.5 average 97.5 stage two 1 each given resistance is of 10 ohms. find the equivalent resistance of the circuit. (series) 100 2 each given resistance is of 10 ohms. find the equivalent resistance of the circuit. (parallel) 100 3 the purple resistance is of 12 ohms each and the blue one is of 6 ohms. find the equivalent resistance of the circuit. 100 4 the purple resistance is of 12 ohms each and the blue one is of 6 ohms. find the equivalent resistance of the circuit 100 5 the purple resistance is of 12 ohms each and the blue one is of 6 ohms. find the equivalent resistance of the circuit 100 average 100 stage three 1 connect five resistance (10 ohms each) to get an equivalent resistance of 50 ohms. 100 2 connect two resistance of 10 ohms to get an equivalent resistance of 5 ohms. 100 3 connect three resistance (6 ohms each to get an equivalent resistance of 2 ohms. 100 4 connect 3 resistance (6 ohms each) to get an equivalent resistance of 4 ohms. 100 5 connect 5 resistance (6 ohms each) in a circuit to get an equivalent resistance of 5 ohms. 100 average 100 stage four 1 a battery of 6 volts is connected connect the given resistance in the circuit and find the current through the circuit using ohm’s law 100 2 a battery of 9 volts is connected. connect the given resistance in the circuit and find the current through the circuit using ohm’s law. 100 3 a current of two amperes is flowing through the circuit. find the voltage of the battery using ohm’s law. 100 4 connect the two resistance in series and find the current flowing through the 12-ohm resistance. 100 5 connect two resistance in series and find the current flowing through the 12-ohm circuit. 100 average 100 figure 8 students' answer (level 5) journal of science learning article doi: 10.17509/jsl.v1i3.11789 87 j.sci.learn.2018.1(3).77-94 the third and four stage are also answered correctly by 100% students. all levels in the third stage are almost the same with the second level, but the difference is the students are asked to draw circuit based on the equivalent resistance which stated in every level. while all levels in the four-stage are asked the students to apply ohms’ law. those two stages are represented in figure 10 respectively. according to the result, the first worksheet is almost 95% finished correctly by all students. since the content of the worksheet is about equivalent resistance and ohm’s law which learned frequently by the students. therefore, this worksheet support students to apply science and math content. ywrobot worksheet in the second meeting, the worksheet is used to help with the implementation of stem learning and develop science, technology, and engineering literacy. the worksheet consists of the steps as shown in figure 11 to construct series and parallel circuit using ywrobot. therefore, the worksheet guides the students’ engineering design process and open-ended exploration. there are some questions in the worksheet to assess students’ work. the percentage of students’ answer are tabulated in table 7. in the first task, students are asked to measure the voltage and current of their series circuit by using a multimeter. this question is answered 67% correctly by the students. most of the students are able to prove that the series circuit has different voltage and same current flow. while the rest of the students are not able to find completely the value of each voltage and current. both alternative and correct answer of students are shown in figure 12. 35% of the students are correct in measuring voltage and current in a parallel circuit with a multimeter. the table 7 students' answer percentage of ywrobot worksheet no questions % correct t f total students: 18 1. measuring voltage and current in the working circuit (series circuit) 67 12 6 2. measuring voltage and current in the working circuit (parallel circuit) 35 6 12 3. build your own circuit (series circuit) 100 18 4. build your own circuit (combined series and parallel circuit) 67 12 6 average 67.25 12 6 figure 9 students' answer (2nd stage) figure 10 students’ answers (3rd and 4th stage) figure 11 part of ywrobot worksheet figure 12 students‘ answer (no.1) figure 13 students’ answer (no.2) journal of science learning article doi: 10.17509/jsl.v1i3.11789 88 j.sci.learn.2018.1(3).77-94 students find the correct range value of each voltage and current flow through the circuit. moreover, students can prove that the parallel circuit has a different current flow and same voltage. the rest of the students fail to find completely the value of voltage in the parallel circuit. the students’ result as shown in figure 13 of the voltage is far from the theory and correct range value. the third and fourth questions ask the students to construct and draw schemes of the circuit. there is a series circuit with two lamps and combined series-parallel circuit with three lamps which should be constructed and drawn by the students. 100% of students answer the third question completely. this is because the figure is similar to a common series circuit which added with two lamps. the students’ scheme is presented in figure 14. the next question is answered correctly by 67% of students. the rest of the students don’t draw the correct scheme of the combined series-parallel circuit with three lamps. as shown as in figure 15 students fail to draw two resistors which connected to one lamp. according to the result, it can be stated that this worksheet is less finished correctly by students. it is because the result of the first and second questions are very influenced by the multimeter and how students operate it. it is also because human error can’t be avoided in this experiment. even the result is less complete, but this worksheet support students’ open-ended exploration, recalling science concept of electricity, and the engineering design process in constructing a series-parallel circuit with ywrobot. arduino uno worksheet the third worksheet is almost the same as the second worksheet. it is used to help the implementation of stem learning and developed engineering and technology literacy. the third worksheet consists of the steps as shown in figure 16 to lead the students in constructing a traffic light project with arduino uno. it is also added with two additional questions about traffic light circuit scheme and modifying traffic light to asses students’ work. students’ response to those two questions is tabulated in table 8. the first and second questions of the third worksheet are answered correctly by 88% students. the students can draw the symbol correctly to represent of arduino uno and traffic light circuit in parallel. the students are also able to explain their modification such as changing the resistor to the jumper wire, the color of the lamp, and inputting new “delayed time” of each lamp. while the rest of the students draw an arduino uno as square next to the symbol of the power supply, it can be table 8 students answer percentage of arduino uno worksheet no question total students: 18 % correct t f 1 draw the scheme of the circuit in the table. 88 16 2 2 modify your own project and write which part of the project that has been modified. 88 16 2 average 88 16 2 figure 14 series circuit scheme drawn by students figure 15 combined circuit scheme drawn by students figure 16 part of arduino uno worksheet figure 17 traffic light circuit scheme drawn by students figure 18 students’ answer journal of science learning article doi: 10.17509/jsl.v1i3.11789 89 j.sci.learn.2018.1(3).77-94 shown in figure 17. in fact, arduino uno is not only used to save and drive the formula but also to control the voltage and current flow to the circuit. in the modification of the circuit, the students fail to explain their modification completely. several students only write the sentence such as “the time modified to be faster” as shown in figure 18, without any additional information about the delayed time after is it modified. according to the result, this worksheet is almost finished completely by the students. moreover, this worksheet supports the students’ open-ended exploration, recalling the science concept of electricity, and the engineering design process in constructing traffic light circuit with arduino uno. table 9 stem literacy based test analysis no. stem literacy aspects learning objective developed stem literacy indicator item test 1. science literacy the ability of students to use and process to understand scientific knowledge such as physics, chemistry, biology and earth and space science. state that electric circuit transfer energy from the power source to the circuit components than to surrounding state the relation of p = iv 1 identify the type of energy transfer in electricity 3 state current at every point in a series circuit is the same, while in parallel is different identify the concept of the parallel circuit through the problem 2, 6 state qualitatively how changes in p.d or resistance affect current state the relation of resistance and current 4 relate (without calculation) the resistance of a wire to its length and to its diameter recall and use quantitatively the proportionality between resistance and length of wire 5 state that the potential difference across a circuit component identify daily life phenomena of electricity (potentially different) 8 state that the current is related to the flow of charge identify daily life phenomena of electricity (current) 9 2. technology literacy the ability of students to operate and demonstrate effectively and productively to conduct research and solve the problem collaboratively. draw and interpret circuit containing circuit component identify circuit components and its function 7, 12 use and describe the use of a voltmeter both analog and digital classify the right procedure in operating voltmeter 10 communicate the result from the voltage measurement 11 describe an experiment to determine resistance using an ammeter communicate the result from the resistance measurement 13 3. engineering literacy the ability of students to apply scientific and mathematics principal to think creatively about solving the problem action and use circuit component interpret the electrical circuit containing circuit component 14, 15, 16, 17 4. mathematics literacy the ability of students to identify, understand and engage in mathematics to think creatively about solving the problem state that the combined resistance of two resistors in parallel is less than that of either resistor by itself calculate the amount of resistance in the circuit 19, 21 recall and use r = v i calculate the amount of current flow, resistance and potential differences in circuit 20, 22, 23, 24 interpret the relation of r = v i 25 total test item 25 journal of science learning article doi: 10.17509/jsl.v1i3.11789 90 j.sci.learn.2018.1(3).77-94 3.3 stem literacy instrument test applied to stem learning constructing stem literacy based test the main objective of stem learning implementation is the improvement of students’ stem literacy. therefore, stem literacy instrument test is constructed to evaluate students’ stem literacy. stem literacy based test consists of questions that developed from learning objective of electricity topic and adjusted with four stands of stem literacy from zollman (2012). the stem literacy instrument test consists of 25 questions about the topic of electricity learned by the students. the full version of test items is attached in the attachment. thus, the relation between question content, learning objective of electricity topic and the strands of stem literacy are described in table 9. according to table 9, it can be seen that each aspect of stem literacy distributes in questions separately. it is arranged separately to assess every aspect easily. therefore, each question belongs to certain stem literacy aspects. the effect of stem learning on students’ stem literacy the effect of students’ stem literacy can be measured from the result of pretest and posttest. both pre-test and post-test covered four aspects of stem literacy such as science, technology, engineering, and mathematics. those four aspects were distributed in the questions which used as pre-test and post-test. the result of pretest and posttest in this research are processed by using ms. word excel 2007 and spss statistic version 17.0. afterward, it is analyzed through a normality test, t-test paired sample test, and normalized gain. those three analyses are described as follow: normality test normality test is used to ensure whether the data is normally distributed or not. normality test is also used to see whether the question in the test sides the higher achiever students or lower achiever students. the result of the normality test is shown in the following table 10. according to the result in table 10 both pretest and posttest from one sample kolmogorov smirnov the pvalue is 0.88 and 0.60 respectively, while the level significance (alpha) is 0.05. it can be stated that both the pretest and posttest p-value is greater than the level significance (alpha) which means the data is normally distributed. therefore, the result can be analyzed through a t-test paired sample. t-test paired sample t-test paired sample test is used to identify whether the hypothesis is rejected or accepted. the result of the t-test paired sample is shown in the following table 11. according to the result, the p-value is 0.06, while the level of significance (alpha) is 0.05. it can be stated that h0 is accepted. thus the hypothesis of h1 is rejected, which can be concluded that there is no significant main difference between students’ stem literacy after applying stem learning. it can be presumed that the data obtained did not work to prove the hypothesis. occasionally, it requires a large sample to prove the relationship between two variables. normalized gain result the normalize gain score of the students explained whether there was any improvement or not on students’ stem literacy. the analysis of students’ stem literacy is done not only in general result. the author has analyzed the four aspects of stem literacy such as science, technology, engineering, and mathematics. the aim of this analysis is to see the effect of stem learning toward students’ stem literacy (per aspects). the result of pretest and posttest of students’ and the n-gain score of stem literacy in general and each aspect is shown in table 12. based on table 12, the result of pretest is good enough but there was decreasing in the post-test score. according to interview with the physic teacher which held on tuesday, 13th of june 2017, the students could get that average score because they have already got the topic of electrical quantities and circuits in the previous meeting. therefore the students could recognize the topic well. while for the post-test, the teacher added that in the meeting when the students got post-test, they have passed the formative test in the previous week. since in the formative test they had to answer a lot of type questions table 10 test of normality (one sample kolmogorov smirnov test) details post-test pre-test n 16 16 normal parameters,b mean 50.000 54.500 std. deviation 13.54499 13.92360 most extreme differences absolute .191 .146 positive .191 .134 negative -.191 -.146 test statistic .191 .146 asymp. sig. (2-tailed) .602c,d .883 c,d table 11 result of t-test paired sample details pre-test post test mean 54.5 50 mean of differences -4.5 standard deviation of difference 8.98 standard error of difference 2.24 t count 2.002 upper confidence level 9.28 lower confidence level -0.28 significance ( 2-tailed) 0.06 journal of science learning article doi: 10.17509/jsl.v1i3.11789 91 j.sci.learn.2018.1(3).77-94 not only about electrical quantities and circuit but also magnetism and electromagnetic effect, therefore it can be stated that they might forget about the topic of electricity which has been learned in the early meetings. another factor came from the time lag of each meeting. one of the factors which influence students’ achievement is the continuation of certain approach implementation. based on the academic calendar the first meeting was conducted on week 17, the second meeting was conducted on week 19 and the last meeting was conducted on week 21. because of that time lag, the continuation of stem learning implementation run less optimally. therefore, it is not able to strengthen students stem literacy on the topic of electricity. specifically, the analysis of the effect of stem learning toward students’ stem literacy (per aspects) regarding the result of stem literacy test are described as follow. students’ science literacy analysis from the table 12, it can be seen that science literacy aspect of students has the value of -0.06 in average normalized gain. if it is compared to the result of pre-test and post-test, that value is categorized as low because there was no improvement but decreasing the score of post-test. there are several factors which caused the decreasing of students’ posttest score, especially in the science literacy aspect. from the physic teacher who observes the class during teaching-learning process that has been interviewed on 13th june 2017, the teacher assumed the content of electricity was repeated very well in the first meeting, the use of mgames science were able to strengthen students mastery. in the second meeting, the series and parallel circuit construction were delivered well, but the use of multimeter in measuring voltage and current was less emphasized to the students. for the last meeting the using of arduino uno was an attractive learning activity, but in fact, arduino uno seemed like another part of electricity. it can be very seen in the third meeting, arduino uno was good to develop electricity insight of the students but less suitable to strengthen students understanding of electricity topic. it can be concluded that stem learning implementation was conducted less optimal to improve the science literacy of students. the science literacy of students is still low due to the learning process couldn’t emphasize the concept of current, voltage and resistance well to the students. on the contrary, the learning process focused on the students on ywrobot and arduino uno experiments. in line with previous research afriana, permanasari & fitriani (2016) argued that low achievement of student’s science literacy is caused by the pjbl stem implementation which runs less optimal. this learning process makes students tend to more focus on a final project to finished at a given time. therefore science concepts are less emphasized during the lesson (afriana, permanasari & fitriani, 2016). in another research from septiani (2016) stated that the implementation of stem table 12 result of pre-test and post-test stem literacy no aspects of stem literacy question content item test score < �̅� > category pretest postt est 1. stem literacy in general 25 54.5 50 -4.5 low 2. science literacy electrical quantities : power in an electric circuit current in a parallel circuit type of energy transfer in electricity current in a series circuit the relation between the resistance of a wire to its long and diameter daily life phenomena of electricity; potential difference and current 1, 2, 3, 4, 5, 6, 8, 9 4.06 4 -0.06 low 3. technology literacy electric circuit : the use of voltmeter and ammeter circuit component and its function 7, 10, 11, 12, 13 3.06 2.93 -0.12 low 4. engineering literacy electric circuit : electrical working circuit containing circuit components. 14, 15, 16, 17 2.25 2.18 -0.06 low 5. mathematics literacy electrical quantities : resistance in parallel circuit the equation of v = i x r 18, 19, 20, 21, 22, 23, 24, 25 4.25 3.37 -0.87 low journal of science learning article doi: 10.17509/jsl.v1i3.11789 92 j.sci.learn.2018.1(3).77-94 learning approach is not quite successful to drive students in analyzing the relation between each subtopic, therefore the students have difficulties in applying science concept. septiani (2016) also added that the stem learning approach needs to implement continuously to improve science process skill and cognitive of the students. students’ technology literacy analysis technology literacy aspect of students has the value of -0.12 in average normalized gain which categorized as low. nevertheless, the implementation of stem learning in three meeting require students to use mgames science application on the android phone and electrical components in second and third meeting which purpose to engage students with the technology and its appliances. but, presumably, it can’t increase the technology literacy of students optimally due to the experiment is done only in one meeting for each experiment. therefore, several detailed explanations about the incoherent result of measurement, an obstacle in using the instrument and translating the result regarding the experiment are not explained deeply. the previous research from supahar & istiyono (2015) stated that the stem approach in physics subject is less appropriate to improve technology literacy of the students. even the students are good in planning the steps of measurement experiment, they still have difficulties to take the data through correct measurement, read the result of measurement, and analyze the result from the instrument used (supahar & istiyono, 2015). there is another factor which causes low achievement of student’s technology literacy. avsec & jamsek (2016) stated that students in grade 7-9 at slovenian secondary school perform poorly on the technology literacy test because technology education is not sufficiently included in schools curriculum. the school should provide the optional subject of technology education such as electronic or robotic to foster active learning and problem solving with the implementation of the problem and project-based learning. therefore the dimension of technology literacy such as technical knowledge, technical capacity, and critical thinking decision making can be achieved optimally by the students (avsec & jamsek, 2016). students’ engineering literacy analysis it can be seen from the table 12 that engineering literacy aspect of students has the value of -0.06 in average normalized gain, which means it is categorized as low. even the usage of ywrobot and arduino uno in second and third meetings required students to design, construct, and test and evaluate their circuits. nevertheless, the main objective of engineering literacy such as engaging the students with the engineering design process can’t be achieved greatly by the students. therefore, it cannot improve students engineering literacy optimally. in line with technology literacy, it can be presumed that the low achievement of engineering literacy is caused by the experiment is done only in one meeting for each experiment. the construction and the design of the electrical circuit containing the circuit component regarding ywrobot and arduino uno experiment can’t be evaluated deeply. frequency in studying science with stem learning approach is one of the factors which affects students engineering literacy. according to grunert and adams (2016), engineering literacy of students (non-engineering students) can be improved for at least one semester. the students need to be introduced and engaged in the engineering design process through practice with experiment, design thinking, and analysis in sufficient time. mcfarland, et al., (2017) stated that the american 8th grade students who experience engineering design process through figuring out “why something was not working in order to fix it” and “taking something apart in order to fix it or see how it works “in science task more than five times have greater scale score (one domain of engineering literacy which is design-system related activity) than the students who experience one or twice that activity. therefore, the time used to involve and engage students in the engineering design process will affect their engineering literacy. students’ mathematics literacy analysis mathematics literacy of the students has the lowest rank from other aspects of stem literacy. mathematics literacy of the students has the value of -0.87. there are a lot of decreasing from pre-test to post-test. from the discussion of the stem learning in the first, second and third meeting, it can be seen that mathematics literacy was only developed in the first meeting. in the first meeting the students required to calculate the resistance, voltage, and current in the certain circuit which provided in the mgames science. while in the next meeting, the students are concerned to develop technology and engineering literacy, and a few parts of science literacy. therefore, it can be stated that mathematics and science literacy are less emphasized in those two meetings, and it makes the score of mathematics and science literacy decrease. in line with others literacy aspects, mathematics literacy of students can’t be developed directly in only one meeting. it needs sufficient time to make students understand the structure and the symbol of formulae and its relation with a certain concept. leibowitz (2016) argued that mathematical literacy of the students can be improved in at least two months. it is included the introduction and general overview which can support students mathematics literacy. another point is a time lag which influences the continuation of stem learning implementation in the class, each aspect of stem literacy can’t be developed and emphasized optimally. according to supardi (2012), students must study certain concept related to mathematics continuously, because the previous concept will affect how students accept and understand the following concept. journal of science learning article doi: 10.17509/jsl.v1i3.11789 93 j.sci.learn.2018.1(3).77-94 therefore, the frequency-time which allows students to study will affect their literacy. 4. conclusion the finding showed that stem learning on electricity using arduino-android game based experiment has improved stem literacy by -0.06, -0.12, -0.06, -0.87 for science, technology, engineering, and mathematics literacy respectively. although the learning can adequately improve technology and engineering literacy which categorized as low and fair. according to findings, several investigations have been analyzed based on the lesson plan, worksheet, and stem literacy based test. the brief conclusion is described as follow. the lesson plan which used in three meetings was constructed to have the characteristic of stem learning. even the step of introducing ywrobot and arduino uno in the first meeting is missed, but the rest of the learning activity which started in three lesson plan were conducted adequately in the class. there are three worksheets used as a compliment in stem learning implementation. mgames science worksheet is finished correctly by 95% students. ywrobot worksheet is finished correctly by 67% students. the last arduino uno worksheet is finished correctly by 88.8% students. according to pre-test and post-test average in every sub-aspects of stem literacy, stem learning implementation was less able to improve students science, mathematics literacy, engineering, and technology literacy of the students. in addition, stem learning implementation was conducted in only three meetings and discontinuously. therefore, science, technology, engineering, and technology literacy regarding electricity topic are emphasized less optimally. acknowledgment the authors acknowledge the principal, teachers, and students of school of madania, bogor, indonesia. references afriana, j., permanasari, a., & fitriani, a. (2016). project based learning integrated to stem to enhance elementary school’s students scientific literacy. jurnal pendidikan ipa indonesia, 261-267. anwari, i., yamada, s., unno, m., saito, t., suwarma, i. r., mutakinati, l., & kumano, y. (2015). implementation of authentic learning and assessment through stem education approach to improve students’ metacognitive skills. k-12 stem education, 1(3), 123-136. avsec, s., & jamšek, j. (2018). a path model of factors affecting secondary school students’ technological literacy. international journal of technology and design education, 28(1), 145-168. bybee, r. w. (2013). the case for stem education: challenges and opportunities. arlington, virginia: nsta press. cambridge international examinations. (2013). cambridge igcse physics syllabus. united states: board examination systems (bes) pilot. cao, y., & brizuela, b. m. (2016). high school students’ representations and understandings of electric fields. physical review physics education research, 12(2), 020102. carro, g., castro, m., sancristobal, e., diaz, g., mur, f., latorre, m., . . . giller, d. (2014). the color of the light: a remote laboratory that uses a smart device that connects teachers and students. istanbul: ieee global engineering education conference (educon). creswell, j. w. (2012). educational research (fourth edition ed.). boston: pearson. creswell, j. w. (2012). educational research: planning, conducting, and evaluating quantitative and qualitative research. boston: pearson education, inc. fraenkel, j., wallen, n., & hyun, h. (2011). how to design and evaluate research in education. new york: mcgraw hill. fundoino. (2016). tutorial for arduino. fundoino service team. grunert, j., & adams, d. s. (2016). increasing engineering literacy among non-engineering students. 2016 asee annual conference & exposition (pp. 1-13). new orleans, louisiana: asee peer. huang, b. (2015). open-source hardware – microcontrollers and physics education integrating diy sensors and data. washington: asee peer. jolly, a. (2014). six characteristics of a great stem lesson. education week. laboy-rush, d. (2011). integrated stem education through project based learning. portland, united states of america: sematinticscholar.org. leibowitz, d. (2016). supporting mathematical literacy development: a case study of the syntax of introductory algebra. interdisciplinary undergraduate research journal, 7-13. lou, s.-j., shih, r. c., diez, c. r., & tseng, k. h. (2011). the impact of problem-based learning strategies. international journal technology and education, 21, 195-215. mcfarland, j., hussar, b., brey, c. d., snyder, t., wang, x., wilkinsonflicker, s., . . . hinz, s. (2017). the condition of education 2017. washington dc: u.s. department of education. meluso, a., zheng, m., a, h., & lester, j. (2011). enhancing 5th graders’ science content knowledge and self-efficacy through game-based learning. computer & education, 487-504. monk, s. (2010). 30 arduino projects for the evil genius. new york: mc graw hill. mulhall, p., mckittrick, b., & gunstone, r. (2001). a perspective on the resolution of confusions in the teaching of electricity. research in science education, 31, 575-587. national research council (2013). monitoring progress toward successful k12. a nations advancing?. washington, dc: national academies press. ong, f., mclean, j., & greco, j. (2014). innovate a blueprint for science, technology, engineering, and mathematics in california public edu. california: california department of education. prima, e. c., oktaviani, t. d., & sholihin, h. (2018). stem learning on electricity using arduino-phet based experiment to improve 8th grade students’ stem literacy. journal of physics: conference series, 1013(1), 012030. prima, e. c., putri, a. r., & rustaman, n. (2018). learning solar system using phet simulation to improve students' understanding and motivation. journal of science learning, 1(2), 60-70. prima, e. c., putri, c. l., & sudargo, f. (2017). applying pre and post role-plays supported by stellarium virtual observatory to improve students’ understanding on learning solar system. journal of science learning, 1(1), 1-7. rapini, s. (2012). beyond textbooks and lectures: digital game-based learning. mclean, virginia: texas a&m bush school of government and public service. reeve, e. m. (2013). implementing science, technology, mathematics, and engineering (stem) education in thailand. logan, utah: the institute for the promotion of teaching science and technology (ipst). samsudin, a., suhandi, a., rusdiana, d., kaniawati, i., & costu, b. (2016). investigating the effectiveness of an active learning basedinteractive conceptual instruction (albici) on electric field concept. asia-pacific forum on science learning and teaching, 17(1). septiani, a. (2016). penerapan asesmen kinerja dalam pendekatan stem (sains teknologi engineering matematika) untuk mengungkap keterampilan proses sains. seminar nasional pendidikan dan saintek 2016, 654-659. journal of science learning article doi: 10.17509/jsl.v1i3.11789 94 j.sci.learn.2018.1(3).77-94 supahar, & istiyono, e. (2015). pengembangan asesmen kinerja berbasis stem untuk meningktakan soft skill dan hard skill peserta didik pada pembelajaran fisika sma. yogyakarta: eprints universitas negeri yogyakarta. supardi. (2012). peran berpikir kreatif dalam proses pembelajaran matematika. formatif: jurnal ilmiah pendidikan mipa, 248-262. suwarma, i. r., astuti, p., & endah, e. n. (2015). balloon powered car as a stem based science teaching media. prosiding simposium nasional inovasi dan pembelajaran sains 2015, 373-376. tseng, k.-h., chang, c.-c., lou, s.-j., & chen, w.-p. (2013). attitudes towards science, technology, engineering. international journal of technology and education, 23, 87-102. zollman, a. (2012). learning for stem literacy: stem literacy for learning. school science and mathematics, 112, 12-19. microsoft word 49-52_the effect of multiple intelligence-based learning towards students’ concept mastery and interest in mat a © 2018 indonesian society for science educator 49 j.sci.learn.2018.1(2).49-52 received: 28 october 2017 revised: 23 january 2018 published: 31 march 2018 the effect of multiple intelligence-based learning towards students’ concept mastery and interest in matter wida nur w pratiwi1, diana rochintaniawati1, rika rafikah agustin1* 1international program on science education, faculty of mathematics and science education, universitas pendidikan indonesia *corresponding author. rikarafikah@upi.edu abstract this research was focused on investigating the effect of multiple intelligence -based learning as a learning approach towards students’ concept mastery and interest in learning matter. the one-group pretest-posttest design was used in this research towards a sample which was according to the suitable situation of the research sample, n = 13 students of the 7th grade in a private school in bandar seri begawan. the students’ concept mastery was measured using achievement test and given at the pretest and posttest, meanwhile the students’ interest level was measured using a likert scale for interest. based on the analysis of the data, the result shows that the normalized gain was 0.61, which was considered as a medium improvement. in other words, students’ concept mastery in matter increased after being taught using multiple intelligence-based learning. the likert scale of interest shows that most students have a high interest in learning matter after being taught by multiple intelligence-based learning. therefore, it is concluded that multiple intelligence – based learning helped in improving students’ concept mastery and gain students’ interest in learning matter. keywords concept mastery, interest, matter, multiple intelligence based learning 1. introduction every child is unique in their own way, this uniqueness depends on their intelligences which is defined as a basic aptitude for learning, or it is the ability to gain and apply knowledge or skills. gardner (1983) emphasizes that each individual has different kinds of intelligences including logical-mathematical, verbal-linguistic, musical-rythmic, bodily-kinesthetic, intrapersonal, interpersonal-social and naturalist intelligence, which individuals used in order to understand concepts, solve problems and create products (hanafin, 2014). in facing these facts, multiple intelligencebased learning plays an important role as an educational approach which considers mostly to the uniqueness in every individual as learners. in which an educator provides several learning activities based on the students’ present intelligences. according to previous studies, armstrong (1994) states that the multiple intelligence theory to be applied in classrooms was concerned with teaching to, for and through intelligences (tek and peng 2006). aikenhead (2006) stated that nowadays children’s interest in learning science is declining; therefore students become disconnected with their world and lost their interest in learning science. as students grow, their interest in science is decreased. even there are students with a strong interest in learning science, but it is found that their interest is declining because there are school subjects which are more interesting. therefore, the new challenge for science teachers is to increase students’ learning interest, because students would not be able to learn science effectively without being interested (osborne, simon and collins, 2003). interest is an essential predictor for secondary course options and therefore become an assumption for further research on science not only as a career choice but also an advice for young generations’ decision making in this technologically controlled era (maltese and tai, 2011). according to the above explanation, it is shown that students’ interest plays an important role in order for students to have an effective learning in science. the lack of interest might bring another problem such as low learning motivation and also achievement which can even cause a bigger problem in the educational world nowadays. in chemistry, matter is one of the very basic chapter and most of chemisty topics are related to matter. from the definition itself, matter is anything which occupies space journal of science learning article 50 j.sci.learn.2018.1(2).49-52 and has mass. most matter are extremely small and unseen by a human naked eye, therefore it makes chemistry as one of the difficult subject for students. chemistry curricula integrates many abstact concepts, which became the central for chemistry and other sciences (taber, 2002). meanwhile these intangible concepts are essential in order for students to understand further chemistry or other sciences concepts, students understood little about the particulate nature of matter or about chemical phenomena in their everyday lives. however, the difficulty of a topic, as perceived by students, will be a major factor in their ability and willingness to learn it (sirhan, 2007). in this study, the concepts of matter which includes states of matter, diffusion and elements, compounds and mixtures became the topic which was taught using multiple intelligencebased learning. based on the above explanation, it is understood that interest has a huge impact on students achievement in students who has different intelligences inside them. therefore, this research was aimed to investigate the effect of multiple intelligence-based learning towards students’ concept mastery and interest in matter. 2. method the research method applied in this research was the weak experimental method or usually known as preexperimental research. it is a type of research design which involves a within-group or within-individual procedure in which a single group or single individuals are studied (creswell, 2008). the research design which was used in this research was one-group pretest and posttest design where only one group of sample were treated using multiple intelligence-based learning in between the pretest and post-test. this research was conducted in a private school in bandar seri begawan, brunei darussalam which implements a combination of brunei spn 21 curriculum and cambridge international curriculum. the population of this research was 13 students of grade 7 students in the school described above. during the implementation of multiple-intelligence-based learning, students were grouped according to their strongest intelligence based on the result of a preliminary research. then students were provided with multiple-intelligence based learning activities which were selected and planned making it the most suitable with the classroom condition and sub topics. in this research, the concept of matter is limited according to brunei spn 21 curriculum which includes the following subtopics: matter around us, states of matter, diffusion and elements, compounds and mixtures. matter in this research is defined as anything which occupies space and has mass. there are three states of matter which are solid, liquid and gas, and each of them has its own characteristics and properties. matter also moves, and their movement is called as diffusion. diffusion can happen in solid, liquid and gas. finally, matter is then further classified into elements, compounds and mixtures. an element is made up of only substance which cannot be broken down into simpler substances, compound is made up of two elements which cannot be broken down or separated by physical means and mixture is also made up of two or more elements which can be separated by physical means. there were three instruments used in this research. the observation sheet is an instrument for the observers to make sure whether the activities implemented are in line as planned which consists of the planned activities with their time allocations. in this research the observation sheet was judged by experts which are our lecturers. the achievement test is the instrument which was used to measure the students’ concept mastery before and after the treatment was given, it consists of 25 multiple choice questions which were also tested for its validity, reliability and homogeneity. the third instrument is an interest likert scale adopted from glynn and koballa (2006) which consists of statements of interest and enjoying the lesson and their response towards the lesson after being taught using multiple intelligence-based learning. in the likert scale of interest, students interest are measured from 1-10 which means that the higher the number, the higher interest the student is. the score of interest is then summed up and converted into percentage. 0%-20% categorized as student has no interest or very low interest, 21% 40% is table 1 multiple intelligence-based activities no concept intelligence/ activity visual-spatial logical mathematical interpersonal verballinguistic 1. a. matter around us b. states of matter : solid, liquid and gas mind mapping learning scientific models group work listening to teacher’s instructions and explanation 2. the movement of matter particles looking at teacher’s demonstration card game simple experiments on diffusion in liquid group discussion. watching a video. 3. the classification of matter cross-word puzzle game. predicting and classifying. sharing. presentation. journal of science learning article 51 j.sci.learn.2018.1(2).49-52 categorized as low interest, 41% 60% is categorized as medium interest, 61% 80% is categorized as high interest and 81% 100% is considered as very high interest. in this study, the interest score of each student was calculated and categorized. 3. result and discussion the pretest and post-test were conducted to determine whether there is any increase or decrease of students’ concept mastery after the treatment, then the interest scale was also distributed to see whether the students’ learning interest in matter increase or decrease after the treatment. the result of the students’ achievement test and interest scale were analyzed and further explain in the following: 3.1 multiple intelligence-based learning during the implementation of this research, students were taught using multiple intelligence based activities which were considered unusual to them. the activities were planned according to the four intelligences which include logical-mathematical intelligence, verbal-linguistic intelligence, visual-spatial intelligence and interpersonal intelligence adopted from gardner’s multiple intelligence activities and has been approved by expert includes the lecturers and the science teacher. the implementation of multiple intelligence-based learning was conducted for three meetings where in each meeting students were facilitated with multiple intelligence-based activities in learning matter. in this study, the topic which was taught to the students using multiple intelligence-based learning was the concepts of matter which includes matter around us, the three states of matter, diffusion and elements, compounds and mixtures. for visual-spatial intelligence, students were provided with activities of mind mapping, demonstration, card game and crossword puzzle. for logical-mathematical intelligence, students are provided with scientific models, doing experiment on diffusion and predicting and classifying elements, compounds and mixtures. interpersonal intelligence was triggered using group work, discussion and sharing session. finally, verbal linguistic intelligence was triggered through listening, watching video and presentation. the recapitulation of the multiple intelligence-based activities which were implemented during this study is summarized in table 1. 3.2 students’ concept mastery before the questions for the achievement test was conducted, the validity and reliability of the question items have been tested. the result shows that most of the questions are valid with range very low to very high validity; meanwhile the reliability value was 0.86 which is categorized as very high reliability level. after the pretest and posttest results were analyzed, it was calculated that the average score of the pretest is 48.31 and the average score of the post-test is 80.00, therefore resulted a normalized gain value <g> is 0.61, categorized as medium. so it is said that there is a medium increase in students’ concept master after learning using multiple intelligence-based learning. based on the statistical result which will be shown in the table 2, it shows that the normality test of this data was done using the shapiro-wilk which results the sig value is more than 0.05 for both pre-test and post-test which means that the data is considered as normal. the sig value of the homogeneity test is 0.147 which means that the data is homogeneous. finally, in order to conclude whether the hypotheses of this research is rejected or accepted, a paired t-test was done which results to have the value of 0.000, therefore we can say that h0 is rejected and h1 is accepted. in other words, the hypotheses that multiple intelligence table 2 statistical data of achievement test component pretest posttest n 13 13 average score 48.31 80.00 standard deviation 14.56 9.24 highest score 76 96 lowest score 24 68 g = 31.69 <g> = 0.61 category = medium shapiro-wilk normality test signification (sig.α =0.05) 0.962 0.385 conclusion normally distribute d normally distributed homogeneity test signification (sig.α =0.05) 0.147 conclusion homogenous paired t-test signification (sig.α =0.05) (asymp sig (2-tailes) <0.05, h1 = rejected) 0.000 conclusion h0 rejected, h1 accepted, there is significant difference table 3 recapitulation of students’ interest level no code interest score (%) interest level 1 student 1 94 very high 2 student 2 80 very high 3 student 3 72 high 4 student 4 74 high 5 student 5 76 high 6 student 6 86 very high 7 student 7 82 very high 8 student 8 92 very high 9 student 9 84 very high 10 student 10 50 medium 11 student 11 46 medium 12 student 12 68 high 13 student 13 78 high journal of science learning article 52 j.sci.learn.2018.1(2).49-52 based learning has a significant effect towards students’ concept mastery and interest in learning matter. according to the above statements, it can be said that multiple intelligence – based learning has a significant effect towards students’ concept mastery in learning matter. 3.3 students’ interest level from the students’ interest likert scale as reported in table 3, it was found that 46% of students were categorized as having very high interest, 38% are having a high interest and the remaining 16% are found to be categorized as having a medium interest in learning matter after learning using multiple intelligence-based learning. in other words, multiple intelligence-based learning has brought interest to students in learning matter. 4. c onclusion since it was found that students’ concept mastery have increased after learning using multiple intelligence based learning and students’ were found to have medium to very high interest in learning matter, it can be concluded that multiple intelligence-based learning has a significant effect to both students’ concept mastery and interest in learning matter. the implementation of multiple intelligence-based learning was done systematically as planned referring to the lesson plans which have been approved by the experts. students’ concept mastery increased after being taught using multiple-intelligence based learning, referring to the result of their pretest and posttest scores in matter concepts. multiple intelligence-based learning draws students’ interest in learning matter proven by the result of the likert scale of interest which shows that most students have a high interest in learning matter after being taught using multiple intelligence-based learning. references aikenhead, g. (2006). towards decolonizing the pan-canadian science framework. canadian journal of science, mathematics and technology education, vol 6, pp 387-399. armstrong, t. (1994). multiple intelligences in the classroom. alexandria: va:ascd. cresswell, j. w. (2012). educational research: planning, conducting and evaluationg quantitative and qualitative research. boston: edward brothers. gardner, h. (1993). frames of mind: the theory of multiple iintelligences. london: fontana press. glynn, s. m., & koballa, t. j. (2006). motivation to learn college science: a handbook of college science teaching. arlington va: national science teachers press. hanafin, j. (2014). multiple intelligence theory, action, research and teacher proffesional development: the irish mi project. australian journal of teacher education, vol 39 (4), pp 126-139. maltese, a. v., & tai, r. h. (2011). pipeline persistence: examining the association of educational experiences with earned degrees in stem among us students. bloomington: willey periodicals inc. osborne, j., simon, s., & collins, s. (2003). attitudes towards science: a review of the literature and its implications. international journal od science education, vol 25, pp 1049-1079. sirhan, g. (2007). learning difficulties in chemistry: an overview. journal of turkish science education. taber, k. (2002). chemical misconceptionprevention, diagnosis and cure. london: royal scociety of chemistry. tek, o. e., & peng, y. k. (2006). the theory of multiple intelligence and its application in science classroom. seameo recsam, pp 1-6. microsoft word 44-48_the effect of brainstorming on students’ creative thinking skill in learning nutrition.docx a doi: 10.1021/xxx.xxxx.xxxxxx © 2018 indonesian society for science educator 44 j.sci.learn.2018.1(2).44-48 received: 28 october 2017 revised: 23 january 2018 published: 31 march 2018 the effect of brainstorming on students’ creative thinking skill in learning nutrition winda ismi hidayanti1,2, diana rochintaniawati1, rika rafikah agustin1* 1international program on science education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia 2pribadi bilingual boarding school, bandung, indonesia *corresponding author. rikarafikah@upi.edu abstract this research investigated the effect of brainstorming on students’ creative thinking skills in learning nutrition. the method that was used in this research is quasi experimental with pretest posttest design. the sample was taken by purposive sampling technique where one group was assigned as experimental group (n=25 students) and the other one group was assigned as control group (n=25 students). the population was 7th grade students in islamic international school in bandung. the quantitative data in this research was obtained from objective test and verbal ttct (torrance test of creative thinking), while the qualitative data was obtained from observation sheet. the findings of the study showed that there are statistical significant differences between experimental and control group in creative thinking skills test with the score of 0.000. the result is in the favor of the experimental group, indicating the effectiveness of brainstorming in developing students’ creative thinking skills in learning nutrition. keywords brainstorming, students’ creative thinking skills, nutrition. 1. introduction the world keeps on developing, in the scientific, economic and social aspects of life as well as the communication revolution results from the knowledge advancement and globalization. as a response to those developments and challenges it was crucial to prepare a generation that capable to face those challenges through changing the traditional methods of learning and teaching as well as focusing on providing students with the suitable training on different thinking styles. individuals cannot be prepared for present and future through pouring information into them through the traditional teaching methods that depend on the teacher in the first place. this must be done through guiding students towards achieving knowledge understanding in relation with everyday problems since we live in the era of openness between communities requiring us to employ information and investing it in solving problems in the environment leading to the development of the ability of thinking as well as developing innovation and creativity (al-khatib, 2012). one of such strategies is the constructivist approach which lay emphasis on the active role of the learner in constructing knowledge as well as making sense of information (owo, idode & ikwut, 2016). according to orji & ekpo (2013) constructivism is a theory that believes in human generation of knowledge and meaning from the interaction between their experience and ideas. the constructivist approach to teaching stresses on meaningful learning and knowledge building through two processes: 1) the learners’ internal (cognitive) process. in this process, new knowledge is derived from previous knowledge by the transformation, organization and reorganization of previous knowledge. (2) the interaction between both the learners’ internal and external processes. here new knowledge is constructed as a consequence of the relationship or interaction between cognitive experience or prior knowledge and the external (i.e. environmental or social) factors. the external or social factor here can be in the form of social interactions with knowledgeable adults or peers who render help or scaffolding. during scaffolding, previous knowledge is activated. in any of these processes, prior journal of science learning article 45 doi: 10.1021/xxx.xxxx.xxxxxxj.sci.learn.2018.1(2).44-48 knowledge is required for meaningful learning to take place (owo, idode & ikwut, 2016). brainstorming as one of the constructivist techniques was originally introduced by an american advertising executive, alex osborn (1953) as a technique of generating ideas from a group of people in an attempt to solve a problem. he established this strategy when he realized that the traditional modes of business meetings were unable to create new ideas. he also proposed the following four rules for effective brainstorming: (i) no criticism of ideas: during brainstorming, judgment or criticism of ideas is excluded until the end of the session. (ii) encouraging large quantities of ideas: quantity of ideas is the major goal of brainstorming. the more ideas the group or participants generate, the more the chance of having good ideas among them. (iii) building on each other’s ideas: combination and improvement of ideas are very necessary. participants should be very free to associate, build and elaborate their own ideas based on ideas from others. (iv) encourage every idea: take every idea (both silly and intelligent ones) as valid, and encourage the participants to share their ideas (owo, idode & ikwut, 2016). brainstorming can be viewed as a technique in which an individual or a group engages in critical thinking to generate wide-ranging ideas and creative solution toward solving a problem. this strategy is now widely applied in different fields of human endeavor including education (owo, idode & ikwut, 2016). brainstorming provides a free and open environment that encourages everyone to participate. quirky ideas are welcomed and built upon, and all participants are encouraged to contribute fully, helping them develop a rich array of creative solutions. when used during problem solving, brainstorming brings team members' diverse experience into play. it increases the richness of idea explored, which means that you can often find better solutions to the problems that you face. what's more, because brainstorming is fun, it helps team members bond, as they solve problems in a positive, rewarding environment. while brainstorming can be effective, it's important to approach it with an open mind and a spirit of non-judgment (al-mutairi, 2015). creative thinking skills are something that rarely considered as important in learning science. teacher usually put logic as a the most essensial point and assume that creativity is not important in learning science. creative thinking is known as a compound mental activity aiming to direct a strong desire to look for solutions or reaching original solutions that were not known before (jarwan, 2008). al-khatib (2012) defined it as the multiple thinking that includes the breaking up of old ideas, making new connections, enlarging the limits of knowledge and the onset of wonderful ideas. many researchers assume that people are creative, but their degree of creativity are different from one to another (siswono, 2010). in exercising to learn creatively, students can improve their creative thinking skills. the role of teacher is to provide the teaching and learning process that facilitates students to practice and improve their creative thinking skills. 2. method the research method that was used in this research is quasi experiments. cresswell (2012) stated that in education, many experimental situations occured in which researchers need to use intact groups. quasi-experiments include assignment, but not random assignment of participants to groups. this is because the experimenter cannot artificially create groups for the experiment (creswell, 2012). the design that was used in this research is pre-test post-test design. the location of this research was an islamic international junior high school in bandung. this school is located in urban area of bandung. this school use english as their main language and bahasa indonesia (in indonesian language, civics education, and social studies subject) and also arabic (in arabic language subject). this school use the mix of cambridge igcse and kurikulum 2013 as its curriculum. the population of this research was 7th grade students. the samples are 7th grade students from two classes, where one of them was assigned as experimental and the other as control group. the sampling technique that was used in this research is purposive sampling. purposive sampling was based on previous knowledge of a population and the specific purpose of the research, investigator use personal judgment to select a sample. researchers do not simply study whoever is available but rather use their judgment to select a sample that they believe, based on prior information, will provide the data they need. (fraenkel, wallen & hyun, 2007). in this research, the concept of nutrition is limited based on cambridge igcse curriculum. the subtopics include (1) nutrients, (2) chemical test for nutrients, (3) the use of microorganism in industry, (4) food additives. there are two instruments that were used in this research; those are verbal ttct and observation sheet. the ttct was used to measure students’ creative thinking skills that include fluency, flexibility, and originality. there are six questions that represent their creative thinking skills in asking, guessing causes, guessing consequences, product improvement, unusual uses and just suppose. the score range is from 0 to 3. the second instrument which is the observation sheet contains the stages of activities which have to be done by the researcher in implementing brainstorming which has been adapted according to activities related with nutrition. the function of the observation sheet this research was to make sure that the researcher follows the appropriate steps while implimenting the brainstorming during the research journal of science learning article 46 doi: 10.1021/xxx.xxxx.xxxxxxj.sci.learn.2018.1(2).44-48 and also acts as a guideline for the researcher. observation sheet was in the form of checklist with ‘yes’ and ‘no’ column. the observation sheet was filled by the observer who was the science teacher of the school. 3. result and discussion the results show quantitative and qualitative data. the pre-test and the post-test are conducted to determine students’ creative thinking skill before and after treatments. qualitative analysis will describe how the implementation of brainstorming in the classroom. 3.1 the implementation of brainstorming the research was done in six meeting, where the first and last meetings were used for pretest and posttest. before the first treatment, students were given the pretest in order to know their prior knowledge of nutrition as well as their creative thinking skill on the same concept. after 4 meetings of treatment, students were given the posttest to know the effect of brainstorming towards students’ creative thinking skills in learning nutrition. in the beginning of each meeting, students were presented with daily life phenomenon that was related to the topic as a stimulus for students to create ideas through brainstorming. the implementation of brainstorming was investigated by the observation sheet during the lesson. the percentage of brainstorming implementation is presented in table 1. according to table 1 about the percentage of brainstorming implementation, teacher and students implemented or done all activities that has been determined in the lesson plan. the average of implementation percentage is 100% which according to arikunto (2013) is categorized as very good. from meeting one to meeting four, students showed an improvement of confidence. this finding is in line with result from unin and bearing (2016), that brainstorming contributes to the increase in students’ motivation, confidence, and participation as reflected by the positive students’ behavior during classroom observations. this was because active participation also led to an increase in the students’ level of self-esteem or self-confidence. according to the explanation above, it can be concluded that brainstorming can improve students’ creative thinking skill. indicated by the more meeting they have, the more ideas they generated, and the more confident they were in delivering thedir ideas. this result supported the research from al-khatib (2012) that brainstorming is an effective strategy in developing creative thinking skills. this may be attributed to the nature of brainstorming strategy as a collective discussion strategy that encourage students to generate the highest number of ideas that are varied and creative in a spontaneous and free open climate that is not critical and doesn’t limit the freedom of launching ideas. this finding is consistent with the research from almutairi (2015) which showed a same result. brainstorming can enhance students’ creative thinking skill, especially in the term of generating creative ideas because it is attributed to the advantages of this strategy that are accepted among students. some of those advantages are the preparing element and making students ready to participate in the sessions as well as joy environment that provide students with free climate that doesn’t contain any critics and interference. 3.2 students’ creative thinking skills students’ creative thinking skill was measured by verbal ttct that consisted of six questions which represent their ability in asking question, guessing causes, guessing consequences, improving product, determining unusual uses and the last was ‘just suppose’. all those questions were aimed to measured fluency, flexibility and originality aspects of creative thinking skill. pretest was used to identify students’ creative thinking skill in nutrition concept before the brainstorming treatment. after students were given the treatment, students were faced with posttest to measure their improvement of creative thinking skill in learning nutrition. table 1 the implementation of brainstorming meeting topic implementation criteria 1 nutrients 100% all activities implemented 2 chemical test for nutrients 100% 3 microorganism in industry 100% 4 food additive 100% figure 1 the improvement of creative thinking skills in each aspect 0.64 0.97 0.35 0.10 0.15 0.02 0.00 0.20 0.40 0.60 0.80 1.00 1.20 f lu en cy f le xi b ili ty o ri gi n al it y f lu en cy f le xi b ili ty o ri gi n al it y experimental group control group n o rm al iz ed g ai n critical thinking skills <g> journal of science learning article 47 doi: 10.1021/xxx.xxxx.xxxxxxj.sci.learn.2018.1(2).44-48 the statistic test was done in order to identify the difference of students’ creative thinking skill between experimental and control group after the treatments. table 2 is the statistical test result of students’ creative thinking skill for both experimental and control group. from table 2, it can be concluded that there is a significant difference of students’ creative thinking skill in learning nutrition between groups that was taught using brainstorming and common discussion method, where group that was trained using brainstorming show a better performance. figure 1 also shows a better explanation regarding the improvement of each creative thinking skills (fluency, flexibility, and originality) of students. the effect of brainstorming in developing creative thinking as a whole and in its sub skills may be attributed to the advantages of this strategy that are accepted among students. some of those advantages are the preparing element and making students ready to participate in the sessions as well as joy environment that provide students with a free climate that does not contain any critics and interference. the result is in line with the research finding from taleb, hamza, & wefky (2013) that there is a statistical significant difference between the performance of the experimental group that has been taught by brainstorming strategy and the control which has been taught by traditional method even in the total score of the test or its sub skills. this may be attributed to the nature of brainstorming strategy which help in developing multiple thinking that includes the breaking up of old ideas, making new connections, enlarging the limits of knowledge and the onset of wonderful ideas. as well as, one of the important reasons for this result is the acceptance of this strategy among students and preparing element and making students ready to participate in the sessions as well as joy environment that provides students with a free climate that doesn’t contain any critics and interference. the finding was also consistent with the result from sdouh (2013), that there was statistical different of students’ creative thinking skill of those who learned by using brainstorming and those who were using computer education, where students that trained through brainstorming performed better. this may be attributed to approve the activities of creative thinking skills with the principles of brainstorming strategies based on problemsolving and decision-making and thinking about the reasons and causes to give an accurate explanation and judgment objectively away from bias and favoritism. aiamy & haghani (2012) concluded that brainstorming was more effective in improving students’ creative thinking skill compared to traditional and synectics method. they think of brainstorming as one of the best group decision making methods, that an individual in the group produces responses twice those by an individual alone. variety is a rule of brainstorming which operates the creative part of brain so that it should dominate its judgmental thinking. the latest finding from mathari (2015) also proved that brainstorming can significantly improve students’ creative thinking skill in learning science. it was because brainstorming demand students to actively getting involved in the teaching and learning process. with their active involvement, they can enriched their knowledge individually as well as in group set in solving problem that was given to them as creatively as possible. table 2 the statistic test for students’ creative thinking skills component pretest posttest control group experimental group control group experimental group n 25 25 25 25 average score 20.52 14.45 29.26 69.56 standard deviation 8.40 7.74 10,00 8.04 maximum score 40,74 38,9 48,15 85,19 minimum score 3,7 5,6 9,26 46,3 kolmogorov-smirnov normality test signification (sig.α =0.05) 0.2 0.2 0.2 0.09 conclusion normally distributed normally distributed normally distributed normally distributed levene’s homogeneity test signification (sig.α =0.05) 0.708 0.170 conclusion homogenous homogenous hypothesis test signification (sig.α =0.05) independent sample t-test sig (2-tailed) < 0.05 h0 rejected, h1 accepted 0.000 conclusion h1 accepted, there is significant difference journal of science learning article 48 doi: 10.1021/xxx.xxxx.xxxxxxj.sci.learn.2018.1(2).44-48 4. conclusion research of the comparison between pre and post brainstorming has been conducted systematically, based on the research result it is acquired some conclusions as follows. the implementation of brainstorming in learning nutrition reach the implementation percentage of 100% which means that all activity in the lesson plan are all being implemented during the learning process and considered as very good. there is significant difference between experimental and control group in the term of creative thinking skill. experimental group that was trained by brainstorming got a medium gain which means that brainstorming improve students’ creative thinking skill in medium category. the gain in fluency and originality aspects categorized as medium, means that brainstorming can improve students’ fluency and originality skill in medium level, while gain in flexibility aspect categorized as high means that brainstorming can highly improve students’ flexibility skill. references aiamy, m., & haghani, f. (2012). the effect of synectics & brainstorming on 3rd grade students’ development of creative thinking on science. procedia social and behavioral sciences, 610-613. al-khatib, b. a. (2012). the effect of using brainstorming strategy in developing creative problem solving skills among female students in princess alia university college. american international journal of contemporary research, 2. almutairi, a. m. (2015). the effect of using brainstorming strategy in developing creative problem solving skills among male students in kuwait: a field study on saud al-kharji school in kuwait city. journal of education and practice, 6. arikunto, s. (2013). dasar-dasar evaluasi pendidikan (edisi 2). jakarta: bumi aksara. creswell, j. w. (2012). educational research planning, conducting, and evaluating quatitative and qualitative research. lincoln: pearson education, inc. fraenkel, j. r., wallen, n. e., & hyun, h. h. (2007). how to design and evaluate research in education (6th ed.). new york: mcgraw-hill. hake, r. r. (1999). analyzing change/ gain score. retrieved may 21, 2017, from http://www.physics.indiana.edu. mathari, m. (2015). penerapan model pembelajaran berbasis masalah dengan metode brainstorming untuk meningkatkan kemampuan berpikir kreatif dan prestasi belajar siswa smp pada materi massa jenis. bandung: universitas pendidikan indonesia. owo, d. w., idode, v. o., & ikwut, e. f. (2016). validity of brainstorming strategy on students’ prior knowledge and academic performance in chemistry in selected secondary schools in south-south nigeria. american scientific research journal for engineering, technology, and sciences (asrjets). rizi, c. e., najafipour, m., haghani, f., & dehghan, s. (2013). the effect of brainstorming method on the academic achievement of students in grade five in tehran elementary schools. procedia social and behavioral sciences, 230-233. sdouh, w. m. (2013). the effect of using the strategies of brainstorming and computer education in academic achievement and the development of creative thinking skill of sixth grade students in jordan and their attitudes towards learning mathematics. european scientific journal. supriadi, d. (2001). kreativitas, kebudayaan dan perkembangan iptek. bandung: alfabeta. taleb, a., hamza, h., & wefky, e. (2013). the effect of using brainstorming strategy on developing creative thinking skills for sixth grade students in science teaching. econf '13 proceedings of the 2013 fourth international conference on e-learning "best practices in management, design and development of e-courses: standards of excellence and creativity". unin, n., & polin, b. (2016). brainstorming as a way to approach student-centered learning in the esl classroom. procedia social and behavioral sciences. a © 2022 indonesian society for science educator 431 j.sci.learn.2022.5(3).431-438 received: 11 february 2022 revised: 5 may 2022 published: 27 november 2022 investigating the challenges facing the teaching and learning of science and technology in selected schools in the ashanti region of ghana assem humphrey-darkeh1, kofi owusu-sekyere2*, samuel agyei mensah1 1department of science, wesley college of education, kumasi, ghana 2department of science, st. joseph college of education, bechem, ghana *corresponding author: owususekyerekofi@gmail.com abstract the heightening public apprehension about instructional deliveries and the learning of science in ghanaian schools have taken a different dimension as many shareholders keep interrogating the government's pledge to ensure equity and quality science education for primary schools in the country. the study explored and defined the standing and quality of teaching, learning, and assessment of science and technology while finding the challenges faced by science educators at the jhs level in selected schools in the ashanti region. the study randomly sampled science teachers' views and students in 27 schools from the 27 districts in the ashanti region in ghana. fourteen (14) and thirteen (13) schools were sampled from rural and urban areas. it was found that government inadequately funds most schools. as a result, few qualified science teachers can handle the integrated science topics, among many other setbacks. it was concluded that the lack of infrastructure, tlms, and other factors make the teaching and learning of science practically impossible. these challenges can be mitigated when specific, well-defined measures such as the provision of required infrastructure, teaching, learning materials (tlms), and a few more are urgently implemented. keywords science, technology, teaching, learning, challenges 1. introduction the heightening public apprehension about instructional deliveries and the learning of science in ghanaian schools have taken a different dimension as many shareholders keep interrogating government's pledge to ensure equity and quality science education for primary schools in the country (manuh, gariba, & budu, 2007). literature has shown that more significant numbers of learners appear to study little science at school, even though there is evidence that science is the spine of every developed nation (fägerlind & saha, 2016; leite, 2002; deboer, 2019). when carefully explored, one gets to appreciate that countries that have a strong base in science and technology are the ones that develop faster. examples include russia, japan, brazil, china, india, malaysia, indonesia, dubai, etc. (ramamurti, 2009). in 2001 at the 12th session of the un general assembly meeting, where the focus of discussion was on the contribution of science and technology to the development of countries in the south, leaders in the high-level committee agreed that there was an urgent need to renew and upgrade science education in schools. in that regard, they maintained that particular attention should be given to rehabilitating university laboratories in the least developed countries to inspire and facilitate learning science and technology. this proposal can equally be implemented in the second and first cycles of schools in ghana to whip up interest in the teaching, learning, and assessment of science and technology. the effective use of modern information and communications technologies (ict) among these less developed countries was also discussed. furthermore, the assembly adopted the stance of launching effective fellowship programs in universities of the south to help graduates and post-graduates to study science at that level. these interventions were essential to draw attention to the world that science is the tool for the 21st century. in addition to achieving this dream, many nations appreciate the need to include science in their educational system. science and technology's role in improving people's lives on the continent of africa cannot mailto:owususekyerekofi@gmail.com journal of science learning article doi: 10.17509/jsl.v5i3.43869 432 j.sci.learn.2022.5(3).431-438 be overemphasized since the benefits have to be reaped in supreme terms all across the globe. educationalists in science have questioned the quality of science delivery in the classrooms over the years among parents, stakeholders, and the public alike. according to adepoju and fabiyi (2007), ivowi, okebukola, oludotun, and akpan (1992), and okebukola (1997), even governments of countries over the years have lamented severally on the quality of science delivery in the classrooms. their claims have ranged from lack of teaching and learning materials to large class sizes in terms of students to teacher ratio, among others. the teaching of science and technology in ghanaian schools has been disparaged on the bases of poor performance of candidates in competitive examinations in science relative to their counterparts who had their independence in the same year. this is evident from the statistics of results published by the west africa examination council (waec) in the 2017, 2018, 2019, and 2020 academic years in both the bece and the wassce examinations (nmai, 2020). today’s science is the most potent tool at the disposal of researchers, practitioners of science, teachers, and students to reveal the many mysteries blanketing the universe in which we live (jensen, 2005). all citizens of a country must study science and technology owing to its impact on society and the socio-economic benefits we derive from it. the case of ghana requires that science and technology literacy goals be modeled to reflect the targets that ghana has set for itself within the framework of the new curriculum under the new reforms. it is worth noting that defining and applying education policy is the responsibility of every government of a country within a general framework laid down by the legislature, which, under the constitution, merely establishes 'general principles applicable to the teaching, learning and assessment system. although the teacher is one element in delivering quality education, he or she is nevertheless a crucial factor. similarly, one can also not afford to underscore the need for good infrastructure stocked with quality teaching and learning materials for use by both teachers and learners (ellis & goodyear, 2016). therefore, it goes without professing that the consequence of a new system at the implementation stage should depend, to a greater extent, on the training of qualified science and technology teachers to help curb the unqualified teachers teaching in the various schools. the players in designing a curriculum must ensure that the needed human resource designated to implement the curriculum in the classrooms are equipped with the 21st-century teaching and learning skills required in the 21stcentury classroom for the teaching, learning, and assessment of science and technology in ghana. above all, they must have a strong commitment to teaching so that they might be a source of inspiration to their students and the community at large (woods, 1993). this can be achieved through the government's commitment to providing better service conditions for qualified teachers and incentives for teachers posted to rural areas (bennell, 2004; darling-hammond, 2000). teacher education plays a crucial role in empowering a group of people to assist the more significant majority of individuals to adapt to the rapidly changing social, economic, and cultural environment to ensure the development of human capital required for the economic and social growth of societies (reinders & balcikanli, 2011; elliott, 2011; petersen & treagust, 2014; ben-peretz, 2001). it is said that "if teachers acquire the professional competence and attitudes that enable them to effectively perform their multiple tasks in the classroom, in the school and the community, they will become the single most important contributing factor in ensuring quality educational provision" (dave & rajput, 1999). a critical aspect of this professional competence is the practicum. it is the heart of teacher education and an inseparable aspect of professional training. the amount of money being sunk into the educational system, especially in training science teachers, must not be allowed to go to waste. teacher posting from colleges of education has been bedeviled with favoritism instead of being posted to where services are needed. the underlying principle of teacher education in ghana has been outlined. it includes training teachers with quality practical knowledge and skills through hands-on training for the preservice teachers and in-service teachers so that they will be capable of delivering in practical terms in the ghanaian classroom. others include giving better incentives to teachers who will apply their knowledge and skills to the advantage of learners by creating an accessible, integrated teacher education and training system that provides a structure for continuous professional development throughout their teaching careers (moe, 2017). the contextual challenges of the current system cannot be overemphasized. lack of infrastructure; spacious classrooms, science laboratories, science equipment, and apparatus cannot go without mention. there is a total disconnection between theory as taught in teacher education institutions and practice in the field; that is, the needs of the schools, including their science and technology teachers, are not matched to the curriculum of colleges of education in the country. in short, one can say that teacher education does not seem to influence the 'native theories' of preservice teachers; the native theories they enter colleges of education with remain untouched (bullough, 1997). a solid synergetic relationship among the elements, especially between the practical experiences and the theory, is required for quality science and technology teacher education in ghana. currently, all the 48 colleges of education are affiliated with all the public universities in ghana. as a result, journal of science learning article doi: 10.17509/jsl.v5i3.43869 433 j.sci.learn.2022.5(3).431-438 teachers are being trained in all major public universities, healthy competition to harness better human resources for science teaching. other problems include the posting of teachers, provision of quality science laboratories stocked with science equipment, and recruitment of qualified laboratory technicians for quality delivery of teaching and learning to the future leaders of ghana. striving to achieve "education for all" should be meticulously connected to a universal determination to advance literacy levels in science and technology. this invariably suggests a good proficiency in numeracy, a good understanding of science at the basics, and a practical approach to solving everyday problems in the environment through hands-on teaching, learning and assessment, and ict integration. the above is essential in a global village where economics, politics, ethical and social flags have become indistinguishably connected with the penalties of scientific and technological advancement. for this course to thrive, the teacher in the classroom must come on board with qualities that support the use of scientific knowledge and principles to solve human problems. according to shulman (1986), quality teachers should own the qualities in figure 1. in a related issue on the qualitative nature of teachers, gess-newsome (1999) contends that teachers perceived to be quality have content knowledge and attitudes that conform to the teaching profession's standards. he further states that quality teachers have good pedagogical knowledge and skills, information about students' learning behaviors, and sound acquaintance with curriculum issues. the two arguments, therefore, show the premium placed on quality teachers, making it imperative for every serious curriculum to invest most of its resources in obtaining quality teachers. assem (2021), however, introduced an element of ict integration into shulman’s quality teacher model (figure 2). he argues that in the wake of information, communication, and technology, teachers must not only be conversant with what they teach but also should be able to acquire adequate skills that will enable them to integrate ict in the teaching, learning, and assessment of science and technology in their classrooms. figure 1 shulman’s quality teacher model (shulman, 1986) figure 2 assem and co incorporation of ict into shulman’s quality teacher model journal of science learning article doi: 10.17509/jsl.v5i3.43869 434 j.sci.learn.2022.5(3).431-438 1.1 statement of the problem research has been conducted on science teaching, learning, and assessment in schools and many learning environments (fitzgerald & smith, 2016; hackling, goodrum, & rennie, 2001; gamoran, 2003; harlen & qualter, 2018). an inquiry into the teaching, learning, and assessment of science and technology has shown that many students find science learning problematic, difficult, boring, and energy-sapping (salau, 1995; 1996). arguably, unreasonable class sizes as in learner to teacher ratio, inadequate teaching and learning materials, including inadequate curriculum resources, non-equipped science laboratories, poor pedagogical skills and technical know-how, and inadequate support for science teachers coupled with other limiting factors impede the quality of teaching, learning and assessment in most teaching environment according to krajcik (2002), ødegaard, haug, mork, and sørvik (2014), anderman, sinatra, and gray, (2012), and okebukola (1997). teaching, learning, and assessment of science and technology are confronted with a limitless number of challenges ranging from inadequate teaching and learning materials to the nonexistent of well-equipped science laboratories. needlessly, curriculum implementation issues also take center stage in the teaching, learning, and assessment of science and technology, especially in primary schools, which should not be the case. issues of lack of textbooks and inadequate technical know-how of teachers who are supposed to be the curriculum implementers have negatively impacted the teaching, learning, and assessment of science and technology in most government schools. poor teacher confidence, inadequate pedagogical content knowledge, and lack of pedagogical skills required to deliver science and technology have been equally mentioned as challenges bedeviling the teaching, learning, and assessment of science in schools in ghana. according to the national center for education statistics in las vegas in 2007, complex issues such as the availability of appropriate textbooks and classroom resources, the preparation and training of science teachers, preservice training as well as in-service professional development, among others, have posed challenges to the teaching and learning of science in that parts of the world. it is instructive to know that in our part of the world, internet use, which otherwise would have served as a merger between the real world and the abstract world, is also bedeviled with challenges of lack of access amidst the high cost of data (madrid, 2011). this investigation was meant to explore the extent to which the enumerated problems above affect science and technology delivery in some selected schools of the ashanti region and suggest remedies necessary to curb them. 1.2 purpose of the study this study aimed to investigate and describe the status and quality of the teaching-learning of science and technology in selected junior high schools (jhs) in parts of the ashanti region and proffer suggestions to mitigate the problems. research questions the study sought to interrogate and answer the research questions below. 1. what is a perfect representation of teaching, learning, and assessment of science and technology in junior high schools (jhs) as a yardstick for comparison by various stakeholders? 2. what is the current picture of teaching, learning, and assessment of science and technology as perceived by teachers and other stakeholders in the education environment? 3. what issues make up the problems and challenges fighting quality teaching, learning, and assessment of science and technology in junior high schools? 4. are there ways by which these challenges and problems can be solved or remedied if not eliminated? significance of the study this study professes a remedy to curbing problems hindering the teaching, learning, and assessment of science and technology in the junior high schools in the ashanti region of ghana through a first-hand information approach. thus, it provides curriculum implementors in science with the needed information about the challenges of teaching, learning, and assessment of science and the needed strategy to address them. curriculum planners, stakeholders, and government will find this work worthwhile as it will form the ground rules for future development of a science-based curriculum in ghana, which will make a sound argument for soliciting support from stakeholders when the need arises. 2. method 2.1 research design the research design is a survey. data were obtained for the actual actions on the ground through responses provided by science teachers and compared with literature. data for the perfect picture of teaching, learning, and assessment of science and technology were generated through analysis of the research literature, the national science curriculum for primary schools, and a survey conducted on science teachers in some well-endowed schools in the ashanti region. 2.2 population in investigating the challenges facing the teaching, learning, and assessment of science and technology at the junior high school in the ashanti region, the study randomly sampled 100 science teachers out of a population of 107 in 27 schools made up of 14 rural schools and 13 journal of science learning article doi: 10.17509/jsl.v5i3.43869 435 j.sci.learn.2022.5(3).431-438 urban schools. ten head teachers and principals were also involved in the study. also, five stakeholders were identified as major stakeholders in the teaching, learning, and assessment of science and technology. these identifiable groups include the west african examination (waec), the ghana association of science teachers (gast), and the parent-teacher association (pta) of the selected schools. 2.3 research instrument the study made use of questionnaires to elicit data from the focus groups. meetings with groups were scheduled, and interview protocols were honored. the interview guide and the questionnaire were developed with the help of lecturers and science tutors from wesley college of education. the questionnaire was designed for the teachers, while the interview was used to obtain data from headteachers/masters and the representatives of the five stakeholders for the study. the research data collected were analyzed using both quantitative and qualitative methods. 3. result and discussion the presentation of the results follows the order in which the research questions were posed. from table 1, one hundred percent (100%) of teachers opined that government funding of the teaching, learning, and assessment of science is supposed to be continuous in the areas of building more classrooms, equipping science laboratories and libraries, maintenance of school facilities, and in procuring the necessary curriculum resources including equipment and materials such as consumables (reagents) and modern textbooks. ninety-seven percent of the respondents supported the idea of quality pedagogy curated to support a viable curriculum. at the same time, 3% did not think it was necessary to think about quality pedagogy woven around a good curriculum. ninety-seven percent (97%) of the respondents also agreed that the availability of teaching and learning materials and resources must be one of the critical pillars of acquiring knowledge in science and technology. they opined that the days when improvised materials were used in place of proper and sophisticated equipment must not be revisited if the new curriculum is to chalk any success at all. "we must spend money on providing science laboratories rich with scientific equipment and using virtual laboratories to simulate real-life applications of what we teach in theory". however, 3% of the respondents suggested that the cost of tlms was something government could not bear alone. hence, teachers must be seen as using resources in the environment to aid them in delivering their lessons. "if the government decides to use all her resources to provide for teaching and learning materials to teach science and technology, then obviously, there would be nothing left for other developmental issues, " one of the respondents commented. most of these 3% respondents prefer the government providing modern and quality textbooks for teachers and students to the cost of acquiring science tlms. they believe that teachers will have a broad knowledge of the science and technology practiced in other countries when the government provides good and quality textbooks. "teachers must be well vest in their fields of endeavour". accordingly, they indicated that small class sizes would motivate science and technology teachers to have complete control of their students hence, taking care of individual differences and learning styles through the use of various teaching styles that suit each learning style. arguably, wellventilated classrooms with good seats for the students to sit comfortably were prerequisites for effective teaching, learning, and assessment of science and technology in ghana. furthermore, the population of students offering science subjects should not outnumber the capacities of the teacher. thus, there is a need for a manageable class size with enough resources to cater to each student's needs. the learning environment should motivate the learner to learn. therefore, schools should have a conducive learning environment without distraction and noise. additional information from the interview indicates that teachers generally believe that, under idyllic circumstances, teachers should have the sound subject content knowledge and pedagogical skills to achieve the purpose and goals for which the junior high school science curriculum was borne. eighty-seven percent (87%) of the respondents believe that the curriculum is hinged around pedagogy. hence, they believe that when teachers are adequately resourced and introduced to the curriculum proper through their involvement in its drafting, sound teaching, learning, and assessment of science and technology will be achieved. thirteen percent (13%) of the respondents, however, rejected the claim saying that curriculum and pedagogy do not make any difference in what is being called ideal table 1 teachers' responses to characteristics of ideal teaching, learning, and assessment of science and technology item no. of yes percentage yes no. of no percentage no funding 100 100 0 0 curriculum & pedagogy 97 97 3 3 tlms and resources 97 97 3 3 facilities 100 100 0 0 small class size 87 87 13 13 journal of science learning article doi: 10.17509/jsl.v5i3.43869 436 j.sci.learn.2022.5(3).431-438 teaching, learning, and assessment of science and technology. "the panacea to teaching, learning, and assessment of science and technology in teaching and learning resources," they claimed. thus, an accomplished science teacher should have good professional attitudes and be continuously involved in professional development. responses from participants on practical (ideal) science teaching were organized into thematic areas. the thematic areas were similar to those for ideal science teaching, learning, and assessment. these include funding, curriculum, pedagogical skills, learning and skills expected to be obtained, available teaching and learning resources, facilities and small class size, teacher content knowledge of the subject matter, attitudes, and professional learning, including community support. from table 2, ninety percent (90%) of science teachers emphasized that schools lack proper funding for building new classrooms and science laboratories, adequate classroom maintenance, and providing resources and facilities. the respondents agreed that the government does not provide adequate funds for running schools. they further remarked that students learn under harsh conditions in small classrooms with very poor ventilation. in addition, the large class sizes have negatively impacted the teaching and learning of science and technology. on the other hand, 10% of the teachers did not agree with their colleagues. according to these respondents, the government is doing well and believes that other agencies like the church must help. ninety-five percent (95%) of the science teachers believed that science teaching, learning, and assessment must be practical, yet, theoretical aspects are emphasized in the current dispensation. thus, learners are involved in only a few practical activities. as such, it makes science teaching a teacher-centered one. however, this does not inure to the benefit of learners. respondents also echoed that the science curriculum as it stands now is overloaded with content. meanwhile, the content therein requires a carefully planned series of activities to engage learners, but the time allocation is too limited to allow for it. this has promoted abstract teaching and learning environment in the schools. according to kuiper, nieveen, and berkvens (2013), there is snowballing demand for curriculums to echo changes in society in a fast-growing world to meet evolving societal needs, including financial literacy, digital literacy, and literacy for sustainable development and computational thinking. in light of this, respondents believed that new curriculums should be developed to shift focus from examination but rather provide enough grounds for students to attain skills necessary to meet the demands of the emerging world. the respondents also questioned the academic backgrounds of science teachers. most respondents explained that most teachers do not have sound background knowledge of the science and table 2 stakeholders' responses to characteristics of today's science and technology teaching, learning, and assessment item no. of yes percentage yes no. of no percentage no funding 10 10 90 90 curriculum & pedagogy 5 5 95 95 tlms and resources 10 10 95 95 facilities 5 5 0 0 small class size 4 4 96 96 table 3 stakeholders' responses to characteristics of ideal teaching, learning, and assessment of science and technology item no. of yes percentage yes no. of no percentage no funding 1 20 4 80 curriculum & pedagogy 5 100 0 0 tlms and resources 4 80 1 20 facilities 2 40 6 80 small class size 4 80 1 20 table 4 stakeholders’ responses to characteristics of today's science and technology teaching, learning, and assessment item no. of yes percentage yes number of no percentage no funding 1 20 4 80 curriculum & pedagogy 5 100 0 0 tlms and resources 4 80 1 20 facilities 2 40 6 80 small class size 4 80 1 20 journal of science learning article doi: 10.17509/jsl.v5i3.43869 437 j.sci.learn.2022.5(3).431-438 technology they teach. square pegs have been rooted in round holes and vice-versa, courtesy of "posting-gonebad" in the ghana education service. respondents say this is detrimental to students' interest in science and technology. furthermore, not all the teachers teaching science and technology are trained to teach science. only 5% of respondents did not see that as a hindrance to the teaching and learning of science and technology in schools (table 3-4). ninety percent (90%) of the respondents believed there are inadequate teaching and learning materials for science and technology in schools, especially in rural areas. however, 10% did not agree with their colleagues. this minority group believed that teachers could improvise material to teach science and technology if they wanted, thus, attributing any diverse argument to a lack of commitment. also, 95% of respondents opined that there is too much pressure on the school facilities due to the large school population and the periods between vacation and reopening. as a result, the few available facilities have been used repeatedly and are in a state of deterioration. this indeed did not motivate them, and as far as they were concerned, their security was not guaranteed. however, 5% of the respondents believed that teaching science and technology does not necessarily require well-equipped science laboratories and big classrooms. according to the commitment to teaching, learning and assessment of science and technology depend on the teacher's interest and motivation in life. regarding class size, 96% of the respondents agree that classrooms available cannot accommodate the students for the teaching, learning, and assessing science and technology in most schools in ghana. according to respondents, the student population has increased due to the free senior high school policy. they claim that in situations where laboratories exist, many students are packed in them with little or no practical work. the remaining 4% responded that science laboratory size and classroom size did not matter in the teaching, learning, and assessment of science and technology (table 5-6). they claim committed teachers can explore the internet and the environment for other options for teaching, learning, and assessment resources. 4. conclusion the teaching, learning, and assessment of science and technology at the junior high school level have been bedeviled will numerous challenges, including large class sizes, poor funding, lack of resourced science laboratories and classrooms, curriculum implementation challenges, and teacher motivation, and security and inadequate library facilities. however, these challenges can be mitigated when the following measures are implemented—first, provision of adequate facilities such as infrastructure, which range from classrooms, and well-equipped laboratories to libraries. second, provision of teaching, learning, and assessment materials for hands-on activities. the third is a collaboration between teachers and stakeholders who see themselves as partners in development. fourth, training preservice teachers as in-service teachers aligns with the 21st-century classroom requirements. acknowledgment the researchers would like to appreciate the contributions and effort of the principals of the selected school in the ashanti region as well as all the stakeholders in education consulted during this research, including the west african examination council (waec), ghana association of science teachers (gast), parent teacher associations (pta) and prominent churches within the communities. table 5 stakeholders' responses to characteristics of ideal teaching, learning, and assessment of science and technology item no. of yes percentage yes number of no percentage no funding 1 20 4 80 curriculum & pedagogy 5 100 0 0 tlms and resources 4 80 1 20 facilities 2 40 6 80 small class size 4 80 1 20 table 6 stakeholders’ responses to characteristics of today’s science and technology teaching, learning, and assessment item no. of yes percentage yes number of no percentage no funding 1 20 4 80 curriculum & pedagogy 5 100 0 0 tlms and resources 4 80 1 20 facilities 2 40 6 80 small class size 4 80 1 20 journal of science learning article doi: 10.17509/jsl.v5i3.43869 438 j.sci.learn.2022.5(3).431-438 references adepoju, a., & fabiyi, a. (2007, december). universal basic education in nigeria: challenges and prospects. in union for african population studies fifth african population conference, arusha, tanzania (pp. 10-14). anderman, e. m., sinatra, g. m., & gray, d. l. (2012). the challenges of teaching and learning about science in the twenty-first century: exploring the abilities and constraints of adolescent learners. studies in science education, 48(1), 89-117. assem. (2021). interrogating the qualities of the 21st century teacher. research commons, wesley college, ghana. ben-peretz, m. (2001). the impossible role of teacher educators in a changing world. journal of teacher education, 52(1), 48-56. bennell, p. (2004). teacher motivation and incentives in sub-saharan africa and asia. knowledge and skills for development, brighton, 1(1), 152. bullough, r. v. (1997). practicing theory and theorizing practice. purpose, passion and pedagogy in teacher education, 13-31. darling-hammond, l. (2000). how teacher education matters. journal of teacher education, 51(3), 166-173. dave, r. h., & rajput, j. s. (1999, march). adaptation of content to address the principle of learning to live together: the challenge for teacher training. in conference report on globalization and living together: the challenges for educational content in asia. new delhi (pp. 9-17). deboer, g. (2019). a history of ideas in science education. teachers college press. elliott, j. (ed.). (2011). reconstructing teacher education (vol. 221). routledge. ellis, r. a., & goodyear, p. (2016). models of learning space: integrating research on space, place and learning in higher education. review of education, 4(2), 149-191. fägerlind, i., & saha, l. j. (2016). education and national development: a comparative perspective. elsevier. fitzgerald, a., & smith, k. (2016). science that matters: exploring science learning and teaching in primary schools. australian journal of teacher education (online), 41(4), 64-78. gamoran, a. (ed.). (2003). transforming teaching in math and science: how schools and districts can support change. teachers college press. gess-newsome, j. (1999). delivery models for elementary science instruction: a call for research. electronic journal of science education, 3(3), n3. hackling, m. w., goodrum, d., & rennie, l. j. (2001). the state of science in australian secondary schools. australian science teachers journal, 47(4). harlen, w., & qualter, a. (2018). the teaching of science in primary schools. david fulton publishers. ivowi, u. m. o., okebukola, p. a. o., oludotun, j. s. o., & akpan, b. b. (1992). raising the standard of performance in public examinations in science, technology and mathematics. stan position paper, 4. jensen, e. (2005). teaching with the brain in mind. ascd. krajcik, j. s. (2002). the value and challenges of using learning technologies to support students in learning science. kuiper, w., nieveen, n., & berkvens, j. (2013). curriculum regulation and freedom in the netherlands-a puzzling paradox. leite, l. (2002). history of science in science education: development and validation of a checklist for analysing the historical content of science textbooks. science & education, 11(4), 333-359. madrid, e. m. (2011). the latino achievement gap. multicultural education, 19(3), 7-12. manuh, t., gariba, s., & budu, j. (2007). change and transformation in ghana’s publicly funded universities. partnership for higher education in africa. oxford, uk: james currey and accra, ghana: woeli publishing services. ministry of education [moe]. (2017). national teachers’ standards for ghana: guidelines by moe under creative commons attributionsharealike 4.0 international. retrieved from www.t-tel.org/hub.html updated version, november 2017. nmai, o. s. n. (2020). a critical analysis of students' performance within the 3-year and 4-year secondary education policies in ghana. universal journal of educational research, 8(5), 1884-1898. ødegaard, m., haug, b., mork, s. m., & sørvik, g. o. (2014). challenges and support when teaching science through an integrated inquiry and literacy approach. international journal of science education, 36(18), 2997-3020. okebukola, p. (1997). old, new and current technology in education. petersen, j. e., & treagust, d. f. (2014). school and university partnerships: the role of teacher education institutions and primary schools in the development of preservice teachers' science teaching efficacy. australian journal of teacher education (online), 39(9), 153-167. ramamurti, r. (2009). 13 what have we learned about emerging— market mnes?. emerging multinationals in emerging markets, 399. reinders, h., & balcikanli, c. (2011). learning to foster autonomy: the role of teacher education materials. salau, m. o. (1995). an analysis of students' enrolment and performance in mathematics at the senior school certificate level. journal of studies in curriculum, 5, 1-8. salau, m. o. (1996). the effect of class size on the achievement of different ability groups in mathematics. journal of the science teachers association of nigeria, 31, 55-61. shulman, l. s. (1986). those who understand: knowledge growth in teaching. educational researcher, 15(2), 4-14. woods, p. (1993). critical events in education. british journal of sociology of education, 14(4), 355-371. a © 2023 indonesian society for science educator 136 j.sci.learn.2023.6(2).136-142 received: 22 november 2022 revised: 24 december 2022 published: 31 march 2023 a meta-analysis of the effectiveness of digital technology-assisted stem education feyyaz öztop independent researcher, turkey *corresponding author: feyyazoztop@gmail.com abstract the purpose of this study was to reveal the effect of digital technology-assisted stem (science, technology, engineering, and mathematics) education on academic achievement. to achieve this purpose, experimental studies, including pre-test and post-test, were examined by meta-analysis. the study's data were obtained from 34 studies that met the inclusion criteria due to the searches made in the databases of the national thesis center of the council of higher education, where turkey's postgraduate theses are located. effect sizes of 38 comparisons were calculated from these studies. the study calculated effect sizes based on cohen’s d coefficient. the effect size obtained after the analysis was found to be d=2.582. this finding shows that stem education assisted by digital technology greatly affects academic achievement. in addition, a significant difference was found between the effect sizes in the study according to the level of education and the subject discipline of academic achievement. this research is expected to contribute to educational practice and future research. keywords digital technology, stem education, academic achievement, meta-analysis 1. introduction the development of individuals also provides the development of countries. therefore, governments strive to raise qualified individuals for their products. to increase talented individuals, countries focus on primary and essential lessons in education and, at the same time, follow up-to-date educational approaches. in recent years, the stem education approach, which is based on the integration of various disciplines, has received significant attention (lo, 2021; nugroho, permanasari, firman & riandi, 2021; nugroho, permanasari & firman, 2019; zhong, liu, xia & sun, 2022). the stem education approach integrates science, technology, engineering, and mathematics. it even got its name from the initials of these fields. although its history dates back to earlier (white, 2014; widya, rifandi & rahmi, 2019), stem was first used as “smet” (science, mathematics, engineering, and technology) in the 1990s (sanders, 2009). it was first used as “stem” by biologist judith a. ramaley at the national science foundation in 2001 (kurniawan & susanti, 2021; national science foundation, 2001). in this approach, engineering and technology applications are included in the basic knowledge and skills in mathematics and science. students learn skills related to these four areas in the designed teaching process (benek & akçay, 2021). the literature emphasizes that stem education has the advantages of interdisciplinary processing of subjects, acquiring 21stcentury skills, providing mathematical modeling and computational thinking, providing a design-oriented review, and using and producing technology (akgündüz, 2019). while there is a significant increase in the research on stem education, there is also a difference in the tools used to realize this education. in some studies, it is noteworthy that digital tools are used in stem education. in stem practices, digital tools are used in material design or obtaining information about the subject. for example, in the research conducted by bircan (2019), students designed a game with the scratch program using computers in stem education activities. on the other hand, in the research conducted by çimentepe (2019), students watched videos on the interactive board within the scope of stem education activities. additionally, simulations obtained online were utilized in stem education in the research done by yılmaz (2019). journal of science learning article doi: 10.17509/jsl.v6i2.52316 137 j.sci.learn.2023.6(2).136-142 although digital tools are integrated into stem education in various ways, there are several inconsistencies regarding their effectiveness in using them in stem education. for example, digital tool-assisted stem education makes a huge difference in students' learning outcomes in some studies (e.g., aysu, 2019, i̇zgi, 2020, kağnıcı, 2019), but has a negligible effect in some studies (e.g., bahşi, 2019, bircan, 2019). therefore, there is no clear conclusion about the use of these tools. in this context, it can be said that a certain opinion can be reached on the subject by combining and analyzing the findings of the studies with methods such as meta-analysis. it is striking that there are synthesis studies examining the studies on stem education in the literature. some researchers have done a review the general trend of stem education studies (farwati et al., 2021; nuraeni, malagola, pratomo & putri, 2020; nurwahyunani, 2021; ormanci, 2020; saputri & herman, 2022; sarıca, 2020; setyaningsih, ahmad, adnan & anif, 2021; wardani & ardhyantama, 2021; zulaikha, jumadi, mardiani & lutfia, 2021) and some researchers have examined the effect of stem education on various learning outcomes by meta-analysis method (amin, ibrahim & alkusaeri, 2022; ananda & salamah, 2021; ayverdi & aydın, 2020; güder, demir & özden, 2022; izzah, asrizal & festiyed, 2021; jannah, lufri, asrizal & putra, 2022; karaşah-çakıcı, kol & yaman, 2021; kasuma, asrizal & usmeldi, 2022; kazu & kurtoglu yalcin, 2021; santosa et al., 2021; saraç, 2018; taşdemir, 2022; triani, asrizal & usmeldi, 2022; yücelyiğit & toker, 2021). in addition, it is seen that synthesis research on digital technology-assisted stem education studies is also carried out in the literature. it is seen that these studies are mainly carried out by meta-analysis method. for example, wang, chen, hwang, guan & wang (2022) examined the effect of digital game-based stem education on students' learning achievement in their research. d'angelo et al. (2014) conducted a systematic review and meta-analysis of computer simulations to support stem learning. in their study, jeong, hmelo-silver & jo, (2019) examined the effects of computer-supported collaborative learning (cscl) in stem education. belland, walker, kim & lefler (2017) examined the impact of computer-based scaffolding in stem education. it is seen that meta-analysis studies are carried out abroad. however, no study reveals the effect of digital technology-assisted stem education on students' learning outcomes in turkey. to fill the gap in the literature, the effect of digital technology-assisted stem education on students' academic achievement was examined based on the theses prepared in turkey. this study is of great importance in guiding the studies on this subject and shedding light on the studies in turkey. at the same time, it is of particular importance to address academic achievement, which is one of the ultimate goals of education. therefore, for the purpose determined in the research, answers were sought for three questions: 1) what is the overall effect of digital technology-assisted stem education on students' academic achievement? 2) does the effect of digital technology-assisted stem education on students' academic achievement differ significantly according to the education level? 3) does the effect of digital technology-assisted stem education on students' academic achievement differ significantly according to the subject discipline of academic achievement? 2. method a meta-analysis method was used in this study, which aimed to reveal the effect of digital technology-assisted stem education on academic achievement. meta-analysis studies are studies that allow reaching a general conclusion from the results of different research conducted for the same problem (büyüköztürk, kılıç-çakmak, akgün, karadeniz, & demirel, 2016). in other words, these studies' effectiveness is examined by considering the similarities or differences of the studies conducted in the meta-analysis (eser, yurtçu & aksu, 2020). this meta-analysis study tried to reach a general conclusion about the effectiveness of experimental studies based on pre-test and post-test applications. the steps taken in the research are as follows: 1) determining the subject and research questions 2) scanning the literature 3) identifying the studies that meet the inclusion criteria 4) coding 5) performing the analyses 6) examining the publication bias 7) performing the heterogeneity test 8) effect size calculations 9) reporting (dinçer, 2014; şen & yıldırım, 2020). 2.1. data collection the research data were obtained in november 2022 from the "databases of national thesis center of the council of higher education", where turkey's graduate theses are collected. to collect the data, firstly, “stem”, “education”, “learning”, “approach”, “practice”, “activity”, “achievement”, “science”, “technology”, “engineering”, “mathematics”, “effect” keywords were searched. the screening resulted in a list of 74 studies, and those that satisfied the inclusion criteria were chosen to be examined. the inclusion criteria of the study are as follows: ▪ the study examined the effect of stem education assisted by digital technology on students' academic achievement. ▪ clearly stating that digital technology (computer, interactive board, internet, smartphone, video, digital game, digital education platform, etc.) is used in its application ▪ the fact that the study is an experimental study with pre-test and post-test ▪ availability of full text ▪ include statistical information required for metaanalysis journal of science learning article doi: 10.17509/jsl.v6i2.52316 138 j.sci.learn.2023.6(2).136-142 ▪ conducting with typically developing students after the examinations, it was seen that there was 34 research that met the inclusion criteria listed above. thirtyeight effect sizes (comparison) from these identified studies were examined. 2.2. data analysis şen & yıldırım (2020) state that a data table for the studies should be created after the appropriate studies are determined and quality reviews are done. therefore, a data table was constructed in the digital environment before moving on to the data analysis. this data table includes the year, author information, study name, education level, the subject discipline of the achievement test, and statistical information about the groups. the coding process was carried out with the help of the categories in the data table. for reliability, the researcher conducted the coding process twice on different days. as a result, the coefficient of agreement between the codings was 1.0. in other words, it was observed that there was 100% agreement in the coding. after the coding process, the data were analyzed. analyzes can be made using formulas and meta-analysis software (dinçer, 2014). this study used comprehensive meta-analysis software (cma) for data analysis. the standardized mean difference was used to calculate the effect sizes. in this context, the change in academic achievement was revealed using the experimental study group's post-test and pre-test statistics. in the study, analyses were done for the overall effect size and the moderator variables. effect sizes were calculated based on cohen's d coefficient. in the interpretation of the effect sizes found, the criteria sawilowsky (2009) suggested were based on 0.01=very small, 0.2=small, 0.5=medium, 0.8=large, 1.2=very large, and 2.0 =huge. 2.3. publication bias reviews publication bias in the study was examined using the funnel plot and rosenthal safe n method. the results of the funnet plot test performed are given in figure 1. as can be seen in table 1, it is seen that the other studies, except for the few studies, are distributed close to the vertical line symmetrically. to be sure, the publication bias analysis continued with the rosenthal safe n method. therefore, the rosenthal safe n method findings are given in table 1. the safe n number indicates the number of studies that should be included to make the overall effect insignificant. as seen in table 1, the safe n number was found to be 6801 due to the rosenthal safe n method test. a high value indicates that there is no publication bias. 3. result and discussion in this section, firstly, findings related to heterogeneity analysis are given. then, results and discussions on general effect size calculations and analysis of moderator variables are included. 3.1 examining heterogeneity a heterogeneity test should be performed to calculate the overall effect because the general result can be found by determining the model in line with the findings obtained from the heterogeneity test (dinçer, 2014). therefore, the findings of the analyzes related to the heterogeneity test are given in table 2. multiple values were examined to test its heterogeneity. one of these criteria is the p-value found due to the heterogeneity test. the result of the heterogeneity test was found to be significant (p<0.05). in addition, the q value was also examined, which was found to be 560.135. it is seen that the q value found is higher than the critical value of 37 degrees of freedom (52.192) at the 95% significance level in the chi-square distribution table. on the other hand, the examined i2 value is also high. all findings show that the distribution of effect sizes is heterogeneous. since it is a heterogeneous feature, the random effects model was used to calculate the study's effect sizes. 3.2 calculations of overall effect size the general effect size findings calculated according to the random effects model are given in table 3, along with the lower and upper limits according to the standard error, 95% confidence interval. as seen in table 3, the overall effect size value was calculated as d=2.582 due to the analysis performed on the random effects model. the value found was significant, figure 1 funnel plot of studies table 1 rosenthal safe n method test findings z-value for observed studies pvalue for observed studies alpha tails z for alpha number of observed studies number of safe n 41.25856 0.000 0.05 2 1.95996 38 6801 table 2 findings of the heterogeneity test df q-value i2 p 37 560.135 93.394 0.000 journal of science learning article doi: 10.17509/jsl.v6i2.52316 139 j.sci.learn.2023.6(2).136-142 with a standard error of 0.218, a lower limit of 2.156, and an upper limit of 3.009 (p<0.05). the effect size shows that academic achievement is higher in favor of the post-test. therefore, the effect size can be classified as very high. findings showed that digital technology-assisted stem education significantly affects academic achievement. in other studies (d'angelo et al., 2014; wang, chen, hwang, guan & wang, 2022), it has been concluded that stem education using digital technology has positive effects on learning outcomes. therefore, it can be thought that the student's academic achievement may have increased due to the benefits of digital tools in the learning process in many ways. güleryüz, dilber & erdoğan (2020) emphasized that stem education includes 21st-century skills such as critical thinking, creativity, innovation, problem-solving, productivity, and responsibility besides being productionoriented. in addition, it has been pointed out by some researchers that students who are introduced to stem arduino, coding, robotics, and similar concepts may realize that their skills, such as critical thinking, reflective thinking, and algorithmic thinking, will contribute to them in all areas (akkaş baysal, ocak & ocak, 2020). 3.3 analysis of moderator variables the study determined the level of education and the subject discipline of academic achievement test as moderator variables. the analysis results to determine whether there is a significant difference between the effect sizes according to education level are given in table 4. as seen in table 4, the effect sizes were d=1.257 for primary school, d=2.856 for secondary school, d=3.049 for high school, and d=1.916 for university. the most significant effect size belongs to the high school level, and the least effect size belongs to the primary school level. the analysis showed a statistically significant difference between the effect sizes (qb =16.404, p<0.05). the results of the analysis show that digital technologyassisted stem education is more effective in academic achievement at the high school level. in addition, the primary school level is where stem education assisted by digital technology has a minor effect on academic achievement. in the study by güder, demir & özden (2022), in which the effect of stem education on the scientific process skills of students in turkey was examined, the primary school level was found to be the lowest impact. in general, it can be said that stem education with and without digital technology assisted in primary school has less efficiency in turkey. on the other hand, it can be thought that the results may be affected by the effect of the procedure. therefore, practices in each study may differ. however, the effectiveness of stem education increases in the following stages. findings for the calculation of effect sizes according to the subject discipline of academic achievement are given in table 5. as seen in table 5, the effect sizes were d=2.838 for science and d=1.255 for mathematics. as can be seen, the effect size of the science achievement test is larger than that of the mathematics achievement test. the analysis showed a statistically significant difference between the effect sizes (qb=11.776, p<0.05). the analysis shows that digital technology-assisted stem education is more effective in science achievement. similarly, other research found that digital technologyassisted stem education is less effective in the subject discipline of mathematics (wang, chen, hwang, guan & wang, 2022). öçal (2022) highlighted that the field of mathematics has strengths and weaknesses in the stem education approach. the author describes students' negative attitudes and perceptions towards mathematics as weaknesses; he listed some factors, such as the difficulties they experienced in interdisciplinary integration efforts and the habits of teachers to convey information directly. çakıroğlu & dedebaş (2019) emphasize that mathematical concepts can be used in activities where science concepts are fundamental, but they are often very limited in terms of table 3 findings on the calculation of the overall effect size effect size (d) standard error 95% confidence interval p lower limit upper limit 2.582 0.218 2.156 3.009 0.000 table 4 effect sizes by education level education level n effect size (d) 95% confidence interval qb p lower limit upper limit primary school 6 1.257 0.580 1.934 16.404 0.001 secondary school 24 2.856 2.307 3.405 high school 6 3.049 1.960 4.137 university 2 1.916 1.399 2.434 table 5 effect sizes of academic achievement by subject discipline field of academic achievement n effect size (d) 95% confidence interval qb p lower limit upper limit science 32 2.838 2.381 3.294 11.776 0.001 mathematics 6 1.255 0.475 2.035 journal of science learning article doi: 10.17509/jsl.v6i2.52316 140 j.sci.learn.2023.6(2).136-142 mathematical thinking processes. due to various limitations, it can be said that the effect of digital technology-assisted stem education on mathematics achievement is less than on science achievement. kanematsu & barry (2016) emphasize that stem education is important for everyone and vital for every country's future. in this context, it can be said that it is vital to bring this study to the literature, which will shed light on increasing the effectiveness of stem education. 4. conclusion digital tools are frequently used in stem education, as in all areas of education. and many studies have been carried out on this subject. however, studies on this subject have primarily focused on academic achievement, which is one of the ultimate goals of education. this meta-analysis study revealed the effect of integrating digital tools into stem education on students' academic achievement according to general and some moderator variables. the study concluded that stem education assisted by digital technology has a very high effect on academic achievement. in addition, it has been supposed that digital technology-assisted stem education is more effective on academic achievement at the high school level and least practical at the primary school level. in addition, it was concluded that stem education assisted by digital technology was more effective in science achievement. in the future, increasing studies on a primary school level and mathematics subject discipline for digital technology-assisted stem education is recommended. however, this research is limited to postgraduate theses in turkey. the effectiveness of stem education assisted by digital technology in other nations can be investigated in the future through the meta-analysis method. in addition, only academic achievement was examined in this study. in the end, the effect of digital technology-assisted stem education on various learning outcomes, such as students' 21st-century skills, attitudes, motivations, and problemsolving skills, can be examined by meta-analysis. references akgündüz, d. (2019). stem eğitiminin kuramsal çerçevesi ve tarihsel gelişimi [theoretical framework and historical development of stem education]. in d. akgündüz (ed.), okul öncesinden üniversiteye kuram ve uygulamada stem eğitimi (pp. 19-49). ankara: anı yayıncılık. akkaş baysal, e., ocak, g., & ocak, i̇. (2020). kodlama ve arduino eğitimleri ile ilgili lise öğrencilerinin görüşleri [opinions of high school students about coding and arduino education]. elektronik sosyal bilimler dergisi, 19(74), 777-796. https://doi.org/10.17755/esosder.625496 amin, m., ibrahim, m., & alkusaeri, a. (2022). meta analisis: keefektifan stem terhadap kemampuan berpikir kreatif siswa. journal of authentic research on mathematics education (jarme), 4(2), 248-262. https://jurnal.unsil.ac.id/index.php/jarme/article/view/4844 ananda, p. n., & salamah, u. (2021). meta analisis pengaruh integrasi pendekatan stem dalam pembelajaran ipa terhadap kemampuan berpikir kritis peserta didik. jurnal penelitian pembelajaran fisika, 7(1), 54-64. https://doi.org/10.24036/jppf.v7i1.111634 aysu, g. (2019). an examination of the effects of problem-based learning stem applications on students academic achievements and permanence of learning (master's thesis). databases of national thesis center of the council of higher education. (no. 594666) ayverdi, l., & aydın, s. o. (2020). fetemm eğitiminin akademik başarıya etkisini inceleyen çalışmaların meta-analizi [meta-analysis of studies examining the effect of stem education on academic achievement]. necatibey eğitim fakültesi elektronik fen ve matematik eğitimi dergisi, 14(2), 840-888. https://doi.org/10.17522/balikesirnef.755111 bahşi, a. (2019). the effects of stem activities on scientific process skills, scientific epistemological beliefs and science achievements of 8th grade students (master's thesis). databases of national thesis center of the council of higher education. (no. 592742) belland, b. r., walker, a. e., kim, n. j., & lefler, m. (2017). synthesizing results from empirical research on computer-based scaffolding in stem education: a meta-analysis. review of educational research, 87(2), 309-344. https://doi.org/10.3102/0034654316670999 benek, i̇., & akçay, b. (2021). stem’in doğası [the nature of stem]. in h. nuhoğlu (ed.), eğitimcinin stem öğrenme yolculuğu (pp. 1-30). ankara: pegem akademi. bircan, m. a. (2019). the effect of stem education activities on attitudes towards stem, 21st century skills and mathematics achievements of fourth grade students (doctoral dissertation). databases of national thesis center of the council of higher education. (no. 600567) büyüköztürk, ş., kılıç-çakmak, e., akgün, ö. e., karadeniz, ş., & demirel, f. (2016). bilimsel araştırma yöntemleri [scientific research methods] (20th ed.). ankara: pegem akademi. çakıroğlu, e., & dedebaş, e. (2019). matematiksel bakış açısıyla stem eğitimi uygulamaları [stem education applications from a mathematical perspective]. in d. akgündüz (ed.), okul öncesinden üniversiteye kuram ve uygulamada stem eğitimi (pp. 201-219). ankara: anı yayıncılık. çimentepe, e. (2019). the effect of stem activities on academic achievement, scientific process skills and computational thinking skills (master's thesis). databases of national thesis center of the council of higher education. (no. 552833) d’angelo, c., rutstein, d., harris, c., bernard, r., borokhovski, e., & haertel, g. (2014). simulations for stem learning: systematic review and meta-analysis. menlo park: sri international. https://www.sri.com/wpcontent/uploads/2021/12/simulations-for-stem-learningbrief.pdf dinçer, s. (2014). eğitim bilimlerinde uygulamalı meta-analiz [applied metaanalysis in educational sciences]. ankara: pegem akademi. eser, m. t., yurtçu, m., & aksu, g. (2020). r programlama dili ve jamovi ile meta analiz uygulamaları [meta analysis applications with r programming language and jamovi]. ankara: pegem akademi yayıncılık. farwati, r., metafisika, k., sari, i., sitinjak, d. s., solikha, d. f., & solfarina, s. (2021). stem education implementation in indonesia: a scoping review. international journal of stem education for sustainability, 1(1), 11-32. http://dx.doi.org/10.53889/ijses.v1i1.2 güder, o., demir, m., & özden, m. (2022). the effect of stem education on science process skills of students in turkey: a metaanalysis study, international journal of eurasian education and culture, 7(17), 1104-1142. http://dx.doi.org/10.35826/ijoecc.575 güleryüz, h., dilber, r., & erdoğan, i̇. (2020). stem uygulamalarında öğretmen adaylarının kodlama eğitimi hakkındaki görüşleri [opinions of pre-service teachers about coding education in stem applications]. ağrı i̇brahim çeçen üniversitesi sosyal bilimler enstitüsü dergisi, 6(1), 71-83. https://doi.org/10.31463/aicusbed.610909 i̇zgi, s. (2020). the effect of science, technology, engineering and mathematics (stem) education activities based on 5e model and integrated in the topic of transformation of electrical energy in science courses on the academic achievement and scientific process skills of secondary school 7th grade students (master's https://doi.org/10.17755/esosder.625496 https://jurnal.unsil.ac.id/index.php/jarme/article/view/4844 https://doi.org/10.24036/jppf.v7i1.111634 https://doi.org/10.17522/balikesirnef.755111 https://doi.org/10.3102/0034654316670999 https://www.sri.com/wp-content/uploads/2021/12/simulations-for-stem-learning-brief.pdf https://www.sri.com/wp-content/uploads/2021/12/simulations-for-stem-learning-brief.pdf https://www.sri.com/wp-content/uploads/2021/12/simulations-for-stem-learning-brief.pdf http://dx.doi.org/10.53889/ijses.v1i1.2 http://dx.doi.org/10.35826/ijoecc.575 https://doi.org/10.31463/aicusbed.610909 journal of science learning article doi: 10.17509/jsl.v6i2.52316 141 j.sci.learn.2023.6(2).136-142 thesis). databases of national thesis center of the council of higher education. (no. 620030) izzah, n., asrizal, a., & festiyed, f. (2021). meta analisis effect size pengaruh bahan ajar ipa dan fisika berbasis stem terhadap hasil belajar siswa. jurnal pendidikan fisika, 9(1), 114-130. http://dx.doi.org/10.24127/jpf.v9i1.3495 jannah, r., lufri, l., asrizal, a., & putra, s. p. (2022). effect size: stem pada pembelajaran ipa terhadap keterampilan abad 21. natural science: jurnal penelitian bidang ipa dan pendidikan ipa, 8(2), 79-88. https://ejournal.uinib.ac.id/jurnal/index.php/naturalscience/arti cle/view/4227/pdf jeong, h., hmelo-silver, c. e., & jo, k. (2019). ten years of computersupported collaborative learning: a meta-analysis of cscl in stem education during 2005–2014. educational research review. https://doi.org/10.1016/j.edurev.2019.100284 kağnıcı, a. (2019). the effects of stem activity enriched instruction on eleventh grade students' achievement and conceptions of learning (master's thesis). databases of national thesis center of the council of higher education. (no. 572881) kanematsu, h., & barry, d. m. (2016). stem and ict education in intelligent environments. switzerland: springer international publishing. karaşah-çakıcı, ş., kol, ö., & yaman, s. (2021). the effects of stem education on students’ academic achievement in science courses: a meta-analysis. journal of theoretical educational science, 14(2), 264-290. https://doi.org/10.30831/akukeg.810989 kasuma, s. a., asrizal, a., & usmeldi, u. (2022). meta analisis efek stem dalam pembelajaran sains terhadap keterampilan abad 21. jurnal penelitian pembelajaran fisika, 8(2), 122-132. http://ejournal.unp.ac.id/index.php/jppf/article/view/115863/1 06892 kazu, i. y., & kurtoglu yalcin, c. (2021). the effect of stem education on academic performance: a meta-analysis study. turkish online journal of educational technology-tojet, 20(4), 101-116. https://files.eric.ed.gov/fulltext/ej1313488.pdf kurniawan, h., & susanti, e. (2021). pembelajaran matematika dengan stem (science, technology, engineering, mathematic) [learning mathematics with stem (science, technology, engineering, mathematic)]. yogyakarta: deepublish. lo, c. k. (2021). design principles for effective teacher professional development in integrated stem education. educational technology & society, 24(4), 136-152. https://www.jets.net/collection/published-issues/24_4 national science foundation (2001). nsf initiates massive effort to rebuild teaching leadership in science and mathematics. https://www.nsf.gov/od/lpa/news/press/01/pr0180.htm nugroho, o. f., permanasari, a., & firman, h. (2019). program belajar berbasis stem untuk pembelajaran ipa [stem-based learning program for science learning]. jurnal eksakta pendidikan (jep), 3(2), 117-125. https://doi.org/10.24036/jep/vol3-iss2/328 nugroho, o. f., permanasari, a., firman, h., & riandi, r. (2021). the urgency of stem education in indonesia. jurnal penelitian dan pembelajaran ipa, 7(2), 260-279. http://dx.doi.org/10.30870/jppi.v7i2.5979 nuraeni, f., malagola, y., pratomo, s., & putri, h. e. (2020). trends of science technology engineering mathematics (stem) based learning elementary school in indonesia. premiere educandum: jurnal pendidikan dasar dan pembelajaran, 11(2), 87-103. http://ejournal.unipma.ac.id/index.php/pe/article/view/8805/pdf nurwahyunani, a. (2021). literature review: a stem approach to improving the quality of science learning in indonesia. journal for the education of gifted young scientists, 9(5) special issue, 11-17. http://dx.doi.org/10.17478/jegys.853203 öçal, m. f. (2022). stem eğitimi yaklaşımında matematik eğitimi [mathematics education in stem education approach]. m. akarsu, n. okur akçay, & r. elmas (ed.), stem eğitimi yaklaşımı (pp. 99120). ankara: pegem akademi. ormanci, ü. (2020). thematic content analysis of doctoral theses in stem education: turkey context. journal of turkish science education, 17(1), 126-146. http://dx.doi.org/10.36681/tused.2020.17 sanders, m. (2009). stem, stem education, stemmania. the technology teacher, 68(4), 20-26. https://vtechworks.lib.vt.edu/bitstream/handle/10919/51616/s temmania.pdf?s santosa, t. a., razak, a., lufri, l., zulyusri, z., fradila, e., & arsih, f. (2021). meta-analisis: pengaruh bahan ajar berbasis pendekatan stem pada pembelajaran ekologi [meta-analysis: the effect of teaching materials based on the stem approach on ecological learning]. journal of digital learning and education, 1(1), 1-9. https://doi.org/10.52562/jdle.v1i01.24 saputri, v., & herman, t. (2022). integrasi stem dalam pembelajaran matematika: dampak terhadap kompetensi matematika abad 21 [stem integration in mathematics learning: impact on 21st century mathematics competence]. jpmi (jurnal pembelajaran matematika inovatif), 5(1), 247-260. https://journal.ikipsiliwangi.ac.id/index.php/jpmi/article/view/9 916/3010 saraç, h. (2018). the effect of science, technology, engineering and mathematics-stem educational practices on students' learning outcomes: a meta-analysis study. turkish online journal of educational technology-tojet, 17(2), 125-142. https://files.eric.ed.gov/fulltext/ej1176176.pdf sarıca, r. (2020). analysis of postgraduate theses related to stem education in turkey: a meta-synthesis study. acta didactica napocensia, 13(2), 1-29, https://doi.org/10.24193/adn.13.2.1 sawilowsky, s. s. (2009). new effect size rules of thumb. journal of modern applied statistical methods, 8(2), 597-599. https://digitalcommons.wayne.edu/cgi/viewcontent.cgi?article=1 536&context=jmasm şen, s., & yıldırım, i̇. (2020). cma ile meta-analiz uygulamaları [metaanalysis applications with cma]. ankara: anı yayıncılık. setyaningsih, e., ahmad, c. n. c., adnan, m., & anif, s. (2021). literature review: development of stem learning in indonesia based on variation of subjects, media, and strategy of study from 2015 to 2019. review of international geographical education online, 11(4), 1023-1033. https://web.p.ebscohost.com/ehost/pdfviewer/pdfviewer?vid=0 &sid=a82bdba0-60d5-4933-985c-d012c0709e41%40redis taşdemir, f. (2022). examination of the effect of stem education on academic achievement: a meta-analysis study. education quarterly reviews, 5(2), 282-298. http://dx.doi.org/10.31014/aior.1993.05.02.489 triani, f., asrizal, a., & usmeldi, u. (2022). meta analisis pengaruh penerapan stem terhadap hasil belajar fisika peserta didik [metaanalysis of the effect of stem implementation on students' physics learning outcomes]. jurnal inovasi dan pembelajaran fisika, 9(1), 99107. https://doi.org/10.36706/jipf.v9i1.16507 wang, l. h., chen, b., hwang, g. j., guan, j. q., & wang, y. q. (2022). effects of digital game-based stem education on students’ learning achievement: a meta-analysis. international journal of stem education, 9(1), 1-13. https://doi.org/10.1186/s40594-022-003440 wardani, r. p., & ardhyantama, v. (2021). kajian literature: stem dalam pembelajaran sekolah dasar [literature review: stem in elementary school learning]. jpp: jurnal penelitian pendidikan, 13(1), 1793-1805. https://repository.unej.ac.id/handle/123456789/106217 white, d. w. (2014). what is stem education and why is it important?. florida association of teacher educators journal, 1(14), 1-9. http://www.fate1.org/journals/2014/white.pdf widya, w., rifandi, r., & rahmi, y. l. (2019). stem education to fulfil the 21st century demand: a literature review. journal of physics: conf. series, 1317, 012208. http://dx.doi.org/10.1088/17426596/1317/1/012208 http://dx.doi.org/10.24127/jpf.v9i1.3495 https://ejournal.uinib.ac.id/jurnal/index.php/naturalscience/article/view/4227/pdf https://ejournal.uinib.ac.id/jurnal/index.php/naturalscience/article/view/4227/pdf https://doi.org/10.1016/j.edurev.2019.100284 https://doi.org/10.30831/akukeg.810989 http://ejournal.unp.ac.id/index.php/jppf/article/view/115863/106892 http://ejournal.unp.ac.id/index.php/jppf/article/view/115863/106892 https://files.eric.ed.gov/fulltext/ej1313488.pdf https://www.j-ets.net/collection/published-issues/24_4 https://www.j-ets.net/collection/published-issues/24_4 https://www.nsf.gov/od/lpa/news/press/01/pr0180.htm https://doi.org/10.24036/jep/vol3-iss2/328 http://dx.doi.org/10.30870/jppi.v7i2.5979 http://e-journal.unipma.ac.id/index.php/pe/article/view/8805/pdf http://e-journal.unipma.ac.id/index.php/pe/article/view/8805/pdf http://dx.doi.org/10.17478/jegys.853203 http://dx.doi.org/10.36681/tused.2020.17 https://vtechworks.lib.vt.edu/bitstream/handle/10919/51616/stemmania.pdf?s https://vtechworks.lib.vt.edu/bitstream/handle/10919/51616/stemmania.pdf?s https://doi.org/10.52562/jdle.v1i01.24 https://journal.ikipsiliwangi.ac.id/index.php/jpmi/article/view/9916/3010 https://journal.ikipsiliwangi.ac.id/index.php/jpmi/article/view/9916/3010 https://files.eric.ed.gov/fulltext/ej1176176.pdf https://doi.org/10.24193/adn.13.2.1 https://digitalcommons.wayne.edu/cgi/viewcontent.cgi?article=1536&context=jmasm https://digitalcommons.wayne.edu/cgi/viewcontent.cgi?article=1536&context=jmasm https://web.p.ebscohost.com/ehost/pdfviewer/pdfviewer?vid=0&sid=a82bdba0-60d5-4933-985c-d012c0709e41%40redis https://web.p.ebscohost.com/ehost/pdfviewer/pdfviewer?vid=0&sid=a82bdba0-60d5-4933-985c-d012c0709e41%40redis http://dx.doi.org/10.31014/aior.1993.05.02.489 https://doi.org/10.36706/jipf.v9i1.16507 https://doi.org/10.1186/s40594-022-00344-0 https://doi.org/10.1186/s40594-022-00344-0 https://repository.unej.ac.id/handle/123456789/106217 http://www.fate1.org/journals/2014/white.pdf http://dx.doi.org/10.1088/1742-6596/1317/1/012208 http://dx.doi.org/10.1088/1742-6596/1317/1/012208 journal of science learning article doi: 10.17509/jsl.v6i2.52316 142 j.sci.learn.2023.6(2).136-142 yılmaz, c. n. (2019). the effect of stem education on 10th grade students' academic success, their attitude towards stem and physics (master's thesis). databases of national thesis center of the council of higher education. (no. 566538) yücelyiğit, s., & toker, z. (2021). a meta-analysis on stem studies in early childhood education. turkish journal of education, 10(1), 23-36. https://doi.org/10.19128/turje.783724 zhong, b., liu, x., xia, l., & sun, w. (2022). a proposed taxonomy of teaching models in stem education: robotics as an example. sage open, 12(2), 1-15. https://doi.org/10.1177/21582440221099525 zulaikha, d. f., jumadi, j., mardiani, a., & lutfia, b. a. (2021). the development of physics learning research with stem approach in indonesia: a content analysis. edusains, 13(2), 138-152. http://dx.doi.org/10.15408/es.v13i2.18766 https://doi.org/10.19128/turje.783724 https://doi.org/10.1177/21582440221099525 http://dx.doi.org/10.15408/es.v13i2.18766 a © 2021 indonesian society for science educator 288 j.sci.learn.2021.4(3).288-297 received: 24 november 2020 revised: 28 march 2021 published: 05 july 2021 the effect of stem-based robotic applications on the creativity and attitude of students aydın tiryaki1*, sibel adıgüzel2 1department of education, istanbul university-cerrahpaşa, i̇stanbul, turkey 2department of education, marmara university, i̇stanbul, turkey *corresponding author: tiryaki0402@gmail.com abstract in the present study, the effects of stem-based robotics applications on students' creativity and scientific attitudes in the electricity unit of 7th grade have been investigated using the mixed method's nested pattern. sixty students, 30 of whom are the experimental group and the other 30 constitute the control group, attending a post-school course in istanbul in the 2018-2019 academic year, participated in two weeks of pre-applications and four weeks of applications. tosra measured attitude towards science, and the "torrance creative thinking test" to measure creativity was applied as pre and post-test. the data gained from the tests were analyzed with spss 21. in addition, semi-structured interviews' data were analyzed by using content analysis. as a result, it was observed that stem-based robotics applications significantly increased students' creativity and attitudes towards science. interview findings show that students enjoy using stem applications that contain applications instead of theoretical knowledge. using robotic and complex software materials to solve daily life problems, they felt like scientists during the practices, and the applications affected their future career choices. keywords stem, creativity, robotic application, attitude, tosra 1. introduction science, born with the need to understand the universe's phenomena, has shown rapid development from past to present. so much so that, due to the national security concerns emerging after world war ii and the launch of satellite sputnik in 1957, the role of science education became a priority among countries (deboer, 2000). in the 1970s, the idea of focusing on understanding the relationship between science and technology by deepening the science education provided emerged. in the following years, policies aiming to fully or partially combine these fields with engineering and mathematics were followed (kasım & ahmed, 2018). while providing the desired positive changes in education systems, these policies laid the philosophical foundations of stem education in the 1990s (adiguzel & cakiroglu, 2019). stem represents a relationship where science, mathematics, technology, and engineering are intertwined and support each other. the purpose of stem education is to raise individuals who are willing to examine stemrelated issues, use the scientific method in doing so, and recognize the cultural, contemporary, and financial environment created by each discipline in stem, and integrate all of them by understanding the characteristics of stem disciplines (roostika, setiawan, utami, julie, & panuluh, 2020). for this purpose, it can be said that stem education has a high potential to increase individuals' interests, achievements, and motivations and contribute to learning (anikarnisia & wilujeng, 2020). compared to previous years, it is observed that, since the 2000s, education environments have been enriched (diversified) to gain competencies such as design, creation, and invention in new generations in the light of technology and engineering. robotics applications enabling the integration of stem education into classes are the most prominent of these technologies. stem-based robotics applications include those that require creativity in nature, such as design and problemsolving. for example, students have to design a system that includes the construction and programming of a robot to work on robotic exercises. often, students encounter unexpected problems that they must solve during robotic applications. mailto:tiryaki0402@gmail.com journal of science learning article doi: 10.17509/jsl.v4i3.29683 289 j.sci.learn.2021.4(3).288-297 solving such problems also requires creativity. within the context of robotics applications, creative design and problem-solving can also contribute to other engineering and computing activities. in this way, robotic learning makes it possible to create a more comprehensive creative application among stem disciplines. as a result, design and problem solving that is the essence of robotic applications have a strong potential to improve student creativity (sullivan, 2017). robotic applications based on stem improve working together, problem-solving, and computational and creative thinking (romero & dupont, 2016). moreover, since creativity tends to produce ideas and alternatives that can be beneficial in problem-solving and getting into with others, this education also contributes to students be more innovative and creative (tiryaki, çakıroğlu, & yaman, 2019). also, it can be seen that it allows students to acquire programming logic, engineering, and design skills and positively affect their active participation in the course (comek & avci, 2016). it is thought that the desire to participate actively, that is, the student's attitude towards the course, is the determinant of future career choices (osborne, simon, & collins, 2003). developing positive attitudes towards science can motivate students' interest in science education and science-related careers because their attitudes significantly influence students' thoughts and actions (alam, 2017). when students find a science course boring and do not understand its benefits in daily life, they tend not to take the course seriously. therefore, innovative approaches should be adopted to increase students' attitudes towards science. it contains many abstract and complex topics due to its scientific structure. electricity is one of the science subjects that create confusion for students with its abstract structure. this structure, which causes the formation of many alternative concepts, makes it possible to be explained depending on models, analogies, and metaphors (mulhall, mckittrick, & gunstone, 2001). stem-based robotic applications, on the other hand, allow students to gain substantial experience in electricity. moreover, it contributes to the better learning of stem subjects by providing constructive learning environments suitable for a better understanding of scientific and non-scientific issues (khanlari, 2013). when the literature is analyzed, it is seen that a wide range of studies involving the effects of robotic applications within the scope of stem education mainly on academic success (özdoğru, 2013; wahab, azahari, & tajuddin, 2015; yilmaz, gulgun, & caglar, 2017; kozcu, cakir, & guven, 2019), attitude and interest towards stem (weinberg, pettibone, thomas, stephen, & stein, 2007; apedoe, reynolds, ellefson, & schunn, 2008; martín-ramos, da silva, lopes, & silva, 2016; damar, durmaz, & onder, 2018; zainal et al. 2018; yasin, prima, & sholihin, 2018; prima, oktaviani, & sholihin, 2018), and problem-solving skills (gaudiello, zibetti, & carrignon, 2010; kabátová & pekárová, 2010; cankaya, durak, & yünkül, 2017; apriyani, ramalis, & suwarma, 2019) have been carried out. on the other hand, it is remarkable that the studies within the extent of stem education on the effects of robotics applications on attitude towards science and creativity are pretty limited. the present study, it is aimed to determine the effect of stem-based robotics applications on 7th-grade students' creativity and science attitudes within the scope of the "electricity" unit to overcome this shortcoming in the literature. in the mixed method used in the research, answers to 3 research questions were sought for the predetermined purpose. the research question could be to answer quantitative and qualitative questions separately and relate these answers in the later stages of the research (teddlie & tashakkori, 2009). figure 1 research pattern journal of science learning article doi: 10.17509/jsl.v4i3.29683 290 j.sci.learn.2021.4(3).288-297 do stem-based robotic applications affect students' attitudes towards science? what are the effects of stem-based robotics applications on students' attitudes towards science? do stem-based robotic applications have an impact on students' creativity? 2. method 2.1 research design the method of the research has been determined as a mixed method. determining the pattern, determining the priority, the level of interaction, the timing, and how and where to combine quantitative and qualitative stages were considered (creswell, 2014). according to this; the level of interaction between quantitative and qualitative stages is "related", the timing of qualitative and quantitative stages is "concurrent", the priority of quantitative and qualitative data is "qualified priority," and finally, qualitative and quantitative data are combined in the "interpretation" stage. for this reason, it is decided to use the intertwined research pattern in the research. the intertwined pattern is based on collecting and analyzing the secondary dataset to support the more dominant dataset in a study. thus, the researcher embeds qualitative research into a quantitative experiment to support the experimental elements or reveal the differences. qualitative data can be collected before, during, or after the application. in studies in which the quantitative pattern is dominant, the reason for using the secondary qualitative pattern, which is supportive, is to determine whether the results of the experimental process is meaningful and the fact that the results obtained from the experimental process can be both related and different from it (teddlie & tashakkori, 2009). the research pattern is given in figure 1. 2.2 participants the research participants consisted of 7th-grade students in a post-school private education course, istanbul, in the 2018-2019 academic year. in the study, experimental and control groups were formed by using the one-to-one matching method by random-neutral assignment according to torrance creative thinking test and tosra scores. sixty students, experimental (n = 30) and control groups (n = 30), participated in the study. the age of participants ranged from 12 to 13. the control group consists of 13 girls and 17 boys, and the experimental group consists of 11 girls and 19 boys. 2.3 data collection instrument tosra the tosra (test of science-related attitudes) scale consisting of 70 items measures science's attitude in 7 subdimensions of 10 items. in the study, four of the 7 subdimensions were used: adopting scientific attitude, enjoying science lesson, science as leisure time interest, and science as career preference. the scale is likert-type. confirmatory factor analysis of the original (fraser, 1981) and turkish version (telli, 2006) of the 40-item tosra test consisting of four sub-dimensions were performed, and the scope and construct validity were examined. the original reliability value of the test was determined as .82. torrance creative thinking test the torrance creative thinking test developed in 1974 by e. paul torrance used in the research consists of form a and form b. the forms consist of "verbal" and "formal" parts. aslan (2001) used the internal consistency method for the reliability of the test translated into turkish and found their collations with cronbach alpha, guttmann, and sperman brown techniques for analysis. in internal consistency analyzes, correlation coefficients were obtained between (r = 0.38) and (r = 0.89). it compared the wechsler adults form and wonderlic personnel test in the criterion validity title within the scope of validity. the scale adapted to turkish consists of 4 sub-dimensions: flexibility, fluency, detailing, and originality as in the original (baptista et al. 2015; torrance, 1979). semi-structured robotic education interview form the robotics education interview form, created after the relevant literature review and received expert opinion, consists of 4 questions and 4 sub-dimensions. the four sub-dimensions created while developing the interview form were prepared based on tosra's "adopting scientific attitude", "enjoying the science lesson", "science as a leisure interest", and "science as a career" (table 1). 2.4 groups application steps the application process in the experimental group consists of two parts, pre-application, and application. the application consists of 4 activities, including stem-based robotics studies. general information about the activities table 1 sub-dimensions of robotic education interview form and questions measuring these dimensions tosra sub-dimensions interview questions enjoying science lesson q1. have you had fun doing stem-based robotic activities? why? science as career preference q2. did stem-based robotics applications have an impact on their thoughts on a career choice? science as leisure time interest q3. do you think that stem-based robotics applications have brought science to an important place in daily life? adopting scientific attitude q4. did you feel like a scientist during stem-based robotics applications? journal of science learning article doi: 10.17509/jsl.v4i3.29683 291 j.sci.learn.2021.4(3).288-297 and the process applied to the experimental group is in table 2. in the research, activities involving stem-based robotics applications took six weeks. the preliminary application period to the experimental group was two weeks (8 lessons hours), and the application process was four weeks (16 lessons hours). in the control group, whose activities were prepared according to the inquiry-based learning model, the process lasted four weeks (16 lesson hours). 2.5 data analysis the analysis of quantitative data obtained from the tosra and torrance creative thinking test was done by the spss 21.00 statistical program. the kolmogorov smirnov test was used because the experiment and control groups were 60 people in total. as a result of the kolmogorov smirnov test, it was understood that the pvalue was higher than .05, and the scores were normally distributed (p = .200, p>.05 for tosra and p = .058, p>.05 for torrance creative thinking test). paired samples t-test and independent samples t-test, which are parametric tests, were used for analysis. "torrance creative thinking test", one of the quantitative data collection tools, was made by a specialist with an evaluation certificate. qualitative data obtained from the semistructured robotic education interview form were analyzed by following the content analysis. reliability among coders was found 78.5% using miles & huberman (1994) reliability formula (reliability = consensus/(consensus + dissensus) x 100). the analysis of the qualitative data in this research was made based on the content analysis steps below. organizing the data: raw data started to be analyzed, and small notes were taken. coding of data: the data were analyzed in detail, separated according to small information codes, and what each code represented conceptually was found. determining the themes: to create a common idea in line with the determined purpose, themes were created by combining the codes. arrangement of data according to codes and themes: the codes created in line with the data were arranged according to themes, and it was ensured that they were transferred to the reader as descriptively and clearly as table 2 groups application process week lesson hour test experimental group 1. 2 as a pre-test tosra and torrance creative thinking test 2, 3. 8 pre-application introduction of arduino interface program and parts and demonstration of sample codes application process stem fields science mathematics engineering technology 4, 5, 6, 7. 16 1. installing serial and parallel circuits 2. setting up mixed circuits 3. current-voltageresistance relationship and bulb brightness in electrical circuits and measurement of current and voltage values in leds with digital voltmeter and ammeter s/he discovered series and parallel bonding in an electrical circuit and drew a mixed electrical circuit diagram. s/he found the difference in brightness when the bulbs are connected in series and parallel as resistance. s/he saw the relationship between the circuit element's voltage and the current flowing through it using a multimeter. s/he calculated the different resistance values according to the voltage that the light bulbs can handle in the circuits s/he created. s/he revealed the similarities and differences between circuit elements by measuring the mathematical values of the amount of current and voltage in the circuit with a digital multimeter. s/he built the serial, parallel, and mixed electrical circuits using essential engineering tools (instead of ordinary batteries, conductive wires, light bulbs, uno cards, different value resistors, breadboard, and sensors like photocell on the arduino platform). s/he made the software of the circuits using the arduino interface program. before the circuit was established, s/he created her/his circuits on the digital platform using drawing programs such as fritzing and tinkercad. 4. efficient use of electricity and elimination of light pollution s/he discussed the importance of the conscious and economic use of electrical energy in family and country economies. s/he calculated and used the appropriate threshold value of photocell, which causes the light bulbs in the circuit to turn on or off according to the light level in the environment in arduino. 8. 2 as a post-test; tosra and torrance creativethinking test 9. robotic education semi-structured interviews journal of science learning article doi: 10.17509/jsl.v4i3.29683 292 j.sci.learn.2021.4(3).288-297 possible. interpretation of the data: an understanding of the research has been revealed based on the codes and themes. presenting and visualizing data: data was presented with the help of visual tools such as tables or flowcharts. 3. result and discussion the paired samples and independent samples t-test results on the experimental and control group's pre-test and post-test science attitude scores are given in table 3. in consequence of the independent samples t-test conducted to determine whether there is a significant difference between the science attitude scores of the students in the experimental and control groups, a statistically significant difference between the pre-test scores [t(58) = -.702, p = .475] was not reached. however, in favor of the experimental group posttest, a significant difference was achieved [t(58) = 7.301, p = .000]. as a result of paired samples t-test conducted to determine whether there is a significant difference between students' science attitudes pretest-posttest mean scores, both in the experimental group [t(29) = -.32.138, p = .000] and in the control group [t(29) = -.21.787, p = .000] a statistically significant difference was found on behalf of the post-test. results obtained from the robotic education interview form on the students' attitude towards science; results related to tosra's "enjoying the science lesson" subdimension is in table 4. the related question (q1) was examined under the "yes" sub-theme within the scope of the "enjoying the science" theme. one hundred percent of the students stated that they had fun during the activities. they stated that they learned new and complex information about the subject, wrote the codes of the created circuits, programmed them, made applications, created circuits, and obtained theoretical information about the subject. results related to tosra's "science as a career preference" sub-dimension are in table 5. the interview question (q2) is gathered under two subthemes: "yes" and "no" within the scope of the "science as a career" theme. 83.4 percent of the students said that stem-based robotics applications impacted their career choice in the future. this is because they have learned that they can make different inventions in the future and realize that they can do technological studies in science, thanks to the training. on the other hand, 16.6 percent of the students stated that stem-based robotics applications would not affect their choice of profession in the future. they stated the reason for this as having chosen the profession to do before. results related to tosra's "science as leisure timeinterest" sub-dimension are in table 6. table 2 groups application process (continued) week lesson hour test control group 1. 2 as a pre-test; tosra and torrance creative thinking test application process explanation 2, 3, 4, 5. 16 1. installing serial and parallel circuits 2. setting up mixed circuits 3. comparison of bulb brightness 4. measuring current and voltage values in bulbs via a multimeter structured questioning was used in the activities held every week. s/he sought an answer to the practical problem that the teacher gave for his research. s/he established series and parallel circuits with the materials (light bulbs, conductor cables, switch, battery, battery bed) given to her/him by the teacher. s/he made inferences observing the light bulbs' brightness in the cases in which the bulbs are connected in series and parallel. s/he discovered the relationships between current and voltage by making multimeter measurements. 6 2 as a post-test; tosra and torrance creativethinking test table 3 t-test results related to the experimental and control group' pre-test and post-test science attitude scores groups application n mean standard deviation t sd p experimental pre-test 30 99.533 9.786 -.702 58 .475 control pre-test 30 101.600 12.310 experi-mental pre-test 30 99.533 9.786 -32.138 29 .000* post-test 30 184.370 8.572 control pre-test 30 101.600 12.310 -21.787 29 .000* post-test 30 167.200 9.611 experimental post-test 30 184.370 8.572 7.301 58 .000* control post-test 30 167.200 9.611 *p<0.005 journal of science learning article doi: 10.17509/jsl.v4i3.29683 293 j.sci.learn.2021.4(3).288-297 the interview question (q3) examined within the scope of the theme of "science as leisure time interest" is examined under the sub-theme of "yes" 100 percent of the students state that stem-based robotic applications have brought science to an im portant place in their daily life. students stated that the application enables them to understand the operation of the electronic devices used, increase their interest in technological developments, and examine the events in their environment more carefully. results regarding tosra's "adopting scientific attitude" sub-dimensions are given in table 7 below. in consequence of the content analysis of the data gained from the interview question (q4), the results were gathered under two sub-themes: "yes" and "no" within the scope of the "adopting scientific attitude" theme. 83.4 percent of the students stated that stem-based robotic applications make them feel like scientists. they cited that their ability to identify, investigate and solve problems, and work in robotics made them feel this way. on the other hand, 16.6 percent stated that stem-based robotics applications did not make them feel like scientists because they did things simpler than scientists did. the paired samples and independent samples t-test results on the groups' pre-test and post-test creativity scores are in table 8. table 4 results concerning enjoying science lesson theme subtheme explained code percent enjoying science lesson yes ● besides making theoretical information about the subject, making applications and creating circuits ● create a circuit ● to practice ● light the led ● connecting cable ● coding ● robotics ● computer ● programming ● new information ● learning complex information %100 ● programming the circuits created by writing their codes ● learning new and complex information about the subject table 5 results about science as a career preference theme subtheme explained code percent science as career preference yes ● to learn that make different inventions ● making new tools ● to discover ● inventing 83.4 % ● to think that work in the field of robotics and software ● writing code ● making a robot ● software ● computer programs ● to realize technological studies in the field of science ● science area ● studies in the field of science ● developing science concepts ● science technology no ● having previously chosen the profession to do ● deciding the profession in advance 16.6 % table 6 results concerning science as leisure time interest theme subtheme explained code percent science as leisure time interest yes ● having an idea about the operation of the electronic devices used ● trying to repair ● electronic devices %100 ● increased interest in technological developments ● technological developments ● track new devices ● examining the events around more carefully ● observation ● daily life ● surveying around journal of science learning article doi: 10.17509/jsl.v4i3.29683 294 j.sci.learn.2021.4(3).288-297 in consequence of the independent, t-test conducted to determine whether there is a significant difference between the creativity scores of the students in the experimental and control groups, no statistically significant difference was observed between the pre-test scores [t(58) = 1.757, p = .084]. however, it was found that statistically significant difference between the post-test scores in favor of the experimental group [t(58) = 8.602, p = .000]. furthermore, in consequence of the paired samples t-test conducted to determine whether there is a significant difference between the creativity pretest-posttest mean scores, both in the experimental group [t(29) = -21.630, p = .000] and control group [t(29) = -15.900, p = .000] a statistically significant difference was found on behalf of the post-test. in addition, some circuits and fritzing drawings made by the students in the experimental group process activities were shown in figure 2. when quantitative data about attitude are analyzed, it is understood that experiment and control groups' attitudes towards science were at the same level before application. when the control group's pre and post-test attitude scores are examined, it is determined that inquiry-based learning positively affects students' attitudes towards science. when the pretest-posttest science attitude scores of the experimental group are examined, it is found that stembased robotic applications positively affect the students' attitudes towards science. considering the post-test science attitude scores of the experimental and control groups, it is concluded that stem-based robotic applications effectively increase students' attitudes towards science than the inquiry-based learning model (table 3). similarly, özdoğru (2013) found that the activities carried out using robotic applications had a positive attitude towards students' science lessons and impacted their professional choices. yilmaz et al. (2017) realized that stem activities that include robotics positively improved students' attitudes towards science. the students stated that they were happy to obtain a product and that the activities they found intriguing made the lesson enjoyable and provided permanent learning. acar, korkmaz, erdoğmuş, & çakır (2018) found that students 'attitudes towards science were positively influenced by studying educational robot activities' effects on students' stem skill levels and attitudes towards science. karisan & yurdakul (2017) reported that microprocessor supported stem practices positively impact students' attitudes towards stem fields. the studies mentioned above and the results of this study are similar. looking at the qualitative data on attitude, it is concluded that stem-based robotics applications enable students to enjoy science lessons because they could make applications and create circuits related to the subject, write the circuits they created and learn complex information (table 4). dönmez (2017) also determined that students found robot kits fun, functional and exciting, and their motivation and interest in scientific studies increased. yıldız (2018), considering the opinions of students who made at least one project in robotic coding, concluded that robotic sets and computer use could increase motivation and interest in students positively. in his study, sırakaya table 7 results regarding adopting the scientific attitude theme subtheme explained code percent adopting scientific attitude yes ● identifying, investigating, and solving problems ● problem-solving ● identifying the problem ● setting up circuits ● research 83.4 % ● working in the field of robotics ● writing code ● making a robot no ● doing things simpler than what scientists do ● simple level circuits ● less complex ● simpler materials 16.6 % table 8 independent t-test results on groups pre-test creativity groups application n mean standart deviation t sd p experimental pre-test 30 48.100 8.297 1.757 58 .084 control pre-test 30 44.500 7.560 experimental pre-test 30 48.100 8.297 -21.630 29 .000* post-test 30 84.966 4.867 control pre-test 30 44.500 7.560 -15.900 29 .000* post-test 30 72.833 6.000 experimental post-test 30 84.966 4.867 8.602 58 .000* control post-test 30 72.833 6.000 *p<0.05 journal of science learning article doi: 10.17509/jsl.v4i3.29683 295 j.sci.learn.2021.4(3).288-297 (2018) stated that students are delighted to receive coding training and found education attractive and enjoyable. yıldız (2018) reported that students generally find activities challenging in robotics and programming challenging but fun and support the activities' collaborative activities. rubio, hierro, & pablo (2013) realized that students enjoyed working with arduino and their learning motivation increased. the studies mentioned in the literature and the results obtained in this study coincide. stem-based robotic applications are found to impact the student's future professional selection because the students stated that they learned that they could make different inventions, do studies in the field of robotics and software, and realized that they could make technological studies in robotics science (table 5). these results coincide with the results of the studies conducted by weinberg et al. (2007), who concluded that robotic activities increase female students' expectations in science/mathematics and their positive attitudes towards engineering/science careers. stem-based robotic applications have been found to bring science to an important place in students' lives because they have provided students with an idea about electronic devices' operation, increased their interest in technological developments, and positively influenced students to examine the events around them more carefully (table 6). in the study in which damar et al. (2018) conducted robotic projects, it was concluded that the students found the activities exciting and found themselves very popular because they made scientific research and produced projects. moreover, the students stated that they wanted lessons to be longer and coding changed their lives a lot to be excited to participate in the activities. it has been determined that stem-based robotic applications have made most students feel like scientists since it has enabled them to identify and solve problems and work in robotics (table 7). when students' attitudes towards science are analyzed, it is concluded that the quantitative and qualitative data collected about their attitudes could support each other. when quantitative data on creativity are examined, experimental and control groups' creativity is found to be at the same level as before application. when the pretestposttest creativity scores of the control group are examined, it is seen that inquiry-based learning has a positive effect on students' creativity. when the pretestposttest creativity scores of the experimental group are examined, it is found that stem-based robotic applications positively affect the students' creativity. when post-test creativity scores of the experimental and control groups are analyzed, it is seen that stem-based robotic applications are more effective in increasing students' creativity than the inquiry-based learning model (table 8). cavas et al. (2012) found that robotic after-school activities could positively increase students' scientific creativity and change their perception of robots, human beings, and society. similarly, soophung & seokju (2015), masril et al. (2019), noh & lee (2020), guven, cakir, sulun, cetin, & guven (2020), and adeleh (2019) have concluded that robotic applications increase the creativity of students. cankaya et al. (2017) found that students who program with robots have a significant relationship between their creative problem-solving skills and their performance scores. in addition, students generally have a positive attitude. it turns out that the education provided is motivating and amusing and contributes to students learning to program. the results of this research and the results of the studies mentioned above support each other. 4. conclusion as a result, it has been determined that stem-based robotics applications positively affect students' creativity and attitudes towards science. besides, it was stated that the students enjoyed making stem-based robotic applications rather than theoretical lectures and using robotics and complex software materials to solve daily life problems. it was observed that they felt like scientists while making stem applications, and stem applications affected their future career choices. future studies can be carried out at different educational levels (5th, 6th, or 8th grade) and in different units (force and motion, the structure of matter). it should not be overlooked that performing robotic applications in a laboratory equipped for this area, rather than in classroom environments, is vital for the lesson's efficiency and time. researchers are advised to keep spare parts during robotic applications to prevent the parts from working or malfunctioning. references acar, b., korkmaz, ö., erdoğmuş, f., & çakır, e. (2018). the effect of educational robot training and science and technology lesson simple machines on the root levels of middle school students and their attitudes towards the lesson. in 1st national contemporary education and social sciences symposium. adeleh, a. (2019). the effects of robotics training on students' creativity and learning in physics. education and information technologies, 1-13. figure 2 students' circuit and fritzing drawing examples https://link.springer.com/journal/10639 journal of science learning article doi: 10.17509/jsl.v4i3.29683 296 j.sci.learn.2021.4(3).288-297 adiguzel, s., & cakiroglu, o. (2019). association of mechanical clock elements with stem disciplines and energy conversion. international journal of science and research, 8(3). alam, q. (2017). impact of the school outreach tour program of citizens archive of pakistan on students' perceptions and attitudes. international journal of instruction, 10(1), 289-306. anikarnisia, n. m., & wilujeng, i. (2020). need assessment of stem education-based based on local wisdom in junior high school. in journal of physics: conference series, 1440(1). apedoe, x. s., reynolds, b., ellefson, m. r., & schunn, c. d. (2008). bringing engineering design into high school science classrooms: the heating/cooling unit. journal of science education and technology, 17(5), 454-465. apriyani, r., ramalis, t. r., & suwarma, i. r. (2019). analyzing students' problem solving abilities of direct current electricity in stem-based learning. journal of science learning, 2(3), 85-91. aslan, e. (2001). turkish version of torrance creative thinking test, m.ü. atatürk faculty of education journal of educational sciences, 14, 19-40. baptista, a., frick, l., holley k., remmik, m., tesch, j., & âkerlind, g. (2015). the doctorate as an original contribution to knowledge: considering relationships between originality, creativity, and innovation. frontline learning research, 3(3). cakir, n. k., & guven, g. (2019). arduino-assisted robotic and coding applications in science teaching: pulsimeter activity in compliance with the 5e learning model. science activities, 56(2), 42-51. cavas, b., kesercioglu, t., holbrook, j., rannikmae, m., ozdogru, e., & gokler, f. (2012). the effects of robotics club on the students' performance on science process & scientific creativity skills and perceptions on robots, human and society. in proceedings of 3rd international workshop teaching robotics, 40-50. cankaya, s., durak, g., & yünkül, e. (2017). education on programming with robots: examining students' experiences and views. turkish online journal of qualitative inquiry, 8(4), 428-445. comek, a., & avci, b. (2016). teachers' views on robotics applications in science education. in international new trends congress in higher education. creswell, j. w. (2014). a concise introduction to mixed methods research. sage publications. damar, a., durmaz, c., & önder, i̇. (2018). secondary school students' attitudes towards fetem practices and their views on these practices. journal of multidisciplinary studies in education, 1(1), 47-65. deboer, g. e. (2000). scientific literacy: another look at its historical and contemporary meanings and its relationship to science education reform. journal of research in science teaching, 37(6), 582601. dönmez, i̇. (2017). views of student and team coaches about robotic tournaments within the framework of stem education (science heroes meet example). journal of education science and technology research, 2(1), 25-42. fraser, b. j. (1981). test of science-related attitudes handbook (tosra). melbourne: australian council for educational research. gaudiello, i., zibetti, e., & carrignon, s. (2010). representations to go: learning robotics, learning by robotics. in workshop proceedings of intl. conf. on simulation, modeling and programming for autonomous robots, 484-493. guven, g., cakir, n. k., sulun, y., cetin, g., & guven, e. (2020). arduino-assisted robotics coding applications integrated into the 5e learning model in science teaching. journal of research on technology in education. doi: 10.1080/15391523.2020.1812136. kabátová, m., & pekárová, j. (2010). lessons learnt with lego mindstorms: from beginner to teaching robotics. group, 10(12), 16. karışan, d., & yurdakul, (2017). the effect of microprocessor supported science-technology-engineering mathematics applications on 6th grade students' attitudes towards these areas. journal of educational science, 8(1), 37-52. kasim, n. h., & ahmad, c. n. c. (2018). pro-stem module: the development and validation. international journal of academic research in business and social sciences, 8(1), 728-739. khanlari, a. (2013). effects of educational robots on learning stem and on students' attitude toward stem. in 5th conference on engineering education. martín-ramos, p., da silva, m. m. l., lopes, m. j., & silva, m. r. (2016). student2student: arduino project-based learning. in fourth international conference on technological ecosystems for enhancing multiculturality, 79-84. masril, m., hendrik, b., fikri, h. t., hazidar, a., priambodo, b., naf’an, e., putra, z. c., & nseaf, a. k. (2019). the effect of lego mindstorms as an ınnovative educational tool to develop students' creativity skills for a creative society. in international conference computer science and engineering. miles, m. b., & huberman, a. m. (1994). qualitative data analysis. ca: sage. mulhall, p., mckittrick, b., & gunstone, r. (2001). a perspective on the resolution of confusions in the teaching of electricity. research in science education, 31(4). noh, j. & lee, j. (2020). effects of robotics programming on the computational thinking and creativity of elementary school students. educational technology research and development, (68), 463– 484. osborne, j., simon, s., & collins, s. (2003). attitudes towards science: a review of the literature and its implications. international journal of science education, 25(9). özdoğru, e. (2013). the effect of lego program-based science and technology education on students' academic achievement, scientific process skills and attitudes towards science and technology lesson for physical events learning area (doctoral dissertation). deü eğitim bilimleri enstitüsü. prima, e. c., oktaviani, t. d., & sholihin, h. (2018). stem learning on electricity using arduino-phet based experiment to improve 8th grade students' stem literacy. in journal of physics: conference series, 1013(1), 12-30. romero, m., & dupont, y. (2016). educational robotics: from procedural learning to co-creative project oriented challenges with lego wedo. in 8th conference on education and new learning technology. roostika, m., setiawan, i. n., utami, s., julie, h., & panuluh, a. h. (2020). hypothetical learning trajectory for classification of animals and sets by using the stem approach. in 7th south east asia design research international conference, 114-117. rubio, m. a., hierro, c. m., & pablo, a. p. d. m. (2013). using arduino to enhance computer programming courses in science and engineering. in proceedings of edulearn conference. sırakaya, m. (2018). student views on coding education. journal of ondokuz mayıs university faculty of education, 37(2), 79-90. soophung, c., & seokju, c. (2015). the effects of steam-based programming education with robot on creativity and character of elementary school students. journal of the korean association of information education, 19(2), 159 – 166. sullivan, f., r. (2017). the creative nature of robotics activity: design and problem solving. switzerland: springer. telli, s. (2006). students perceptions' of their science teachers' interpersonal behaviour in two countries. turkey and the netherlands (doctoral dissertation). ankara: middle east technical university. tiryaki, a., çakıroğlu, o., & yaman, y. (2019). the effects of the program including differentiated stem applications based on the parallel curriculum model on the critical thinking skills, creativity and attitudes of gifted and talented students. international journal of science and research, 8(4), 1226-1230. teddlie, c., & tashakkori, a. (2009). foundations of mixed methods research. london: sage publications. torrance, e., p. (1979). the search for satori and creativity. new york: creative education. wahab, a. f. a., azahari, m. h., & tajuddin, r. m. (2015). an evaluation of robotic education scale in enhancing science https://doi.org/10.1080/15391523.2020.1812136 https://iopscience.iop.org/issue/1742-6596/1339/1 https://iopscience.iop.org/issue/1742-6596/1339/1 https://link.springer.com/journal/11423 http://www.koreascience.or.kr/journal/jbgobt.page http://www.koreascience.or.kr/journal/jbgobt.page journal of science learning article doi: 10.17509/jsl.v4i3.29683 297 j.sci.learn.2021.4(3).288-297 achievement. in international symposium on agents, multi-agent systems and robotics. weinberg, j. b., pettibone, j. c., thomas, s. l., stephen, m. l., & stein, c. (2007). the impact of robot projects on girls' attitudes toward science and engineering. in workshop on research in robots for education. yasin, a. i., prima, e. c., & sholihin, h. (2018). learning electricity using arduino-android based game to improve stem literacy. journal of science learning, 1(3), 77-94. yıldız, h. (2018). robotic coding trainings in secondary schools: evaluating the opinions of administrators, teachers and students. in international necatibey education and social sciences research congress. yılmaz, a., gülgün, c., & çağlar, a. (2017). teaching with stem applications for 7th class students unit of "force and energy": let's make a parachute, water jet, catapult, intelligent curtain and hydraulic work machine (bucket machine) activities. journal of current researches on educational studies, 7(1), 97-116. zainal, n. f. a., din, r., nasrudin, m. f., abdullah, s., rahman, a. h. a., abdullah, s. n. h. s., & majid, n. a. a. (2018). robotic prototype and module specification for increasing the interest of malaysian students in stem education. int. j. eng. technol, 7(25), 286-290. https://ieeexplore.ieee.org/xpl/conhome/7371321/proceeding https://ieeexplore.ieee.org/xpl/conhome/7371321/proceeding a © 2021 indonesian society for science educator 267 j.sci.learn.2021.4(3).267-274 received: 19 december 2020 revised: 01 april 2021 published: 05 july 2021 meta-analysis of the use of augmented reality applications in science teaching zeynel abidin yilmaz1, veli batdi2* 1department of mathematics and science education, faculty of muallim rıfat, kilis 7 aralık university, turkey 2faculty of education, gaziantep university, turkey *corresponding author: veb_27@hotmail.com abstract this study aims to reach the meta-analysis data by analyzing the augmented reality (ar) applications used in science education. the search was conducted on specific databases considering the studies carried out between 2000 and 2019 years. while collecting data, inclusion criteria were considered. after searching the related databases, 24 studies were reached. based on the random-effects model, the study results revealed that the es value was g = 0.602, which means a medium size in the meta-analysis process according to thalheimer and cook's (2002) calculation level. furthermore, z-test calculations conducted to reveal the statistical significance were found to be z = 4.989, which showed that ar applications had a significant effect on science learning. in this context, it is thought that this technological design, which offers a positive contribution to science education, allows interacting with animations or simulations simultaneously without breaking away from the real world and will serve as a model for future studies. keywords applications in subject areas, augmented reality, improving classroom teaching, meta-analysis, science 1. introduction there has been a tremendous increase in technological developments in recent years. depending on these developments, new methods are employed in educational environments. therefore, there are several applications that attract students' attention, trigger their curiosity, and make learning enjoyable and long-lasting. one of these applications is the augmented reality (ar) application we have encountered in many fields in our daily life recently. one of the biggest reasons for students' failures in especially sciences courses is that the abstracts terms are materialized. it is understood from the studies conducted in this regard that the new technologies which have been developed in educational environments have a significant role in overcoming these difficulties and enable students to understand the issues in science course and influence their attitudes positively. one of these technologies is ar applications which have entered our lives lately. the term ar first appeared in our lives in 1990, together with the ongoing technological developments, and reached a significant mass thanks to the preparation of flash-based ar applications in 2008 (johnson, levine, smith, & stone, 2010). ar instruments have been used in engineering, advertisement, marketing, architecture, construction, entertainment, health, and military (azuma, 1997). ar applications have begun to be employed in educational environments depending on the increase of mobile devices (desktop computers, laptops, smartphones, etc.) lately (billinghurst, 2002; johnson et al., 2010). ar environments provide several benefits such as cost reductions, directing educational studies, ensuring track plans, facilitating applications, creating cooperative environments, facilitating learning, enabling new learning environments, developing socialization, and supporting the researchoriented learning environments (abdüsselam and karal, 2015). it is envisaged that ar in education will be more effective in teaching tiny objects and events that are impossible to be seen with eyes, materialize the abstract concepts, and present sophisticated knowledge (walczak, wojciechowski, & cellary, 2006). abdüsselam and karal (2012) observed in their studies that students have shorter periods of attention in a traditional class environment, and their interests can be easily distracted. on the other hand, in an ar environment, their attention period increases through the highly technological device, materializing the abstract concepts and facilitating comprehension. mailto:veb_27@hotmail.com journal of science learning article doi: 10.17509/jsl.v4i3.30570 268 j.sci.learn.2021.4(3).267-274 the studies indicated that ar technologies increase students' learning levels compared to traditional class environments (freitas & campos, 2008; kerawalla, luckin, seljeflot, & woolard, 2006). ar instruction environments constitute profound learning by ensuring the objects appear from different perspectives under challenging subjects by attracting students' interests and attention. therefore, it is necessary to use ar technologies in science education where students have difficulty understanding subjects. when the related literature is reviewed, it is seen that the use of ar applications in education positively influences the learning process (kerawalla et al., 2006). students can have opportunities to see the cases in the modeled shapes in real environments, which they cannot perceive through their sense organs, and their interests and attentions are more attracted thanks to the differences added in the books (billinghurst, kato, & poupyrev, 2001). therefore, it is understood that the related material will constitute an effective process and a productive environment. while the literature was searched, many ar applications were used in almost every field in our daily lives besides education. for instance, there is an application called touch surgery which specializes in the surgical simulation used in the health field. it gives users the possibility to launch an ar platform to practice surgeries on virtual patients. in addition, many innovative projects created for medical education and training include microsoft hololens, mixed reality smart glasses. in figure 1, the holoanatomy app is seen, which was developed by case western reserve university and the cleveland clinic. this high-level technological app helps medical students learn anatomy. thus, surgeons do not need to leaf through books or medical journals. as holoanatomy app provides them to learn the process of a surgical procedure in an almost real-life scenario, there is no need to go over the books. it is possible to view the three-dimensional drawings of the structure planned to be built, created with cad programs, by a device with an augmented reality application (figure 2). thus, the designer and the user can have the opportunity to gain experience about some details such as the front view of the building, the material to be used, and design feedback in a more realistic way than it appears on the computer screen (kılıç, 2016). as can be seen, the phenomenon of ar, which has evolved towards mixed reality by blending with virtual reality, provides an opportunity for the transformation and reconstruction of information by changing the perception world of individuals (anıl ve batdı, 2020). ar applications whose usage area is expanding with technology-based development in mobile vehicles have a general usage area from architecture to medicine, from military applications to museums, from tourism to education. in the literature, it has been found that some other studies on ar have been carried out. however, it can be stated that these studies are different in purpose and scope from our current research. in detail, some studies examined the features, advantages, and effectiveness of ar in general, rather than examining it through a field or discipline (chen, liu, cheng, huang, 2017). some examined through meta-analysis, where they revealed advantages of ar in terms of educational purposes in general (bacca, baldiris, fabregat, graf, & kinshuk, 2014; saidin, halim & yahaya, 2015). another one made a systematic review on the application of the ar regarding both the advantages of ar in the general scope and problems encountered in its application (akçayır & akçayır, 2017). furthermore, in a systematic review from a conference paper (swensen, 2016), the contribution of ar to science teaching is evaluated in the context of specific topics. as a result of the examination of the studies reached from particular journals determined by the researcher, it was seen that qualitative results were acquired by composing some theme titles that were thought to affect science teaching. however, it can be stated that in the context of metaanalysis, our current study examines the studies carried out on ar applications in the field of science and covers the period between 2000-2017. thus, different from other studies, it is carried out to reflect the results of ar studies in different periods specific to the science field. figure 1 the human anatomy atlas app (https://www.visiblebody.com/anatomy-and-physiologyapps/human-anatomy-atlas) figure 2 augmented reality application used in architecture journal of science learning article doi: 10.17509/jsl.v4i3.30570 269 j.sci.learn.2021.4(3).267-274 1.1 the aim of the research this research aims to provide an investigation by conducting a meta-analysis of ag applications. it is known that the use of ar technologies, particularly on the difficult issues both in other fields (ibanez et al., 2020; rashevska, semerikov, zinonos, tkachuk, & shyshkina, 2020) and in science teaching, will positively influence students' achievements by making learning processes enjoyable. in addition, ar learning environments allow students to see two-dimensional objects in three dimensions and examine these objects from different angles by learning, doing, and living. so, more effective and long-lasting learning occurs (dunleavy, dede, & mitchell, 2009; wojciechowski & cellary, 2013; yen, tsai & wu, 2013). on the other hand, the comparative studies conducted concerning ar and traditional class applications revealed that ar technologies increase the students' learning (freitas & campos, 2008). in this regard, the results of ar applications in science teaching through a meta-analysis based on the document analysis in the literature review were aimed to be included for a detailed review. however, when the ar studies in the literature are examined, it is seen that these studies are generally conducted in one dimension that was whether on education in general (garzon, pavon, baldris, 2019; garzon & acevedo, 2019) or science field but in a qualitative aspect (swensen, 2016), thus the current research is thought to be different at this point. therefore, this study is planned to conduct a meta-analysis of studies on ar. in this way, it is expected that a general conclusion can be drawn by seeing what aspects are conducted and what the level of its effect is in the scientific field related to ar. thus, the main aim of this study is to see “what is the effect size of ar applications on academic success in science through meta-analytic examinations concerning document analysis?” 1.2 the significance of the research educational practices are at the top of the academic studies that have been improved by using ar technology. these applications can be used as educational materials at all levels, from primary education to university. when looking at the studies conducted within the framework of ar applications in education, they are generally related to the research about examining the effectiveness of ar in the educational context (garzon, pavon, baldris, 2019; garzon & acevedo, 2019). some of these studies reached some results, including participants' views on ar in science teaching (timur & özdemir, 2018; durak & yılmaz, 2019) or reflected the use and effectiveness of ar applications in chemistry and biology lessons (abriata, 2018). however, the reason for this research is that there are not so many studies conducted on the meta-analysis study, including ar applications in the field of science. therefore, it is thought that the meta-analysis of ar use between 2000 and 2017 in the science field and sharing the results obtained will contribute to the integration of findings and the literature. 2. method 2.1 research design in this research, the effectiveness of ag technology was determined by employing meta-analysis. for this purpose, the meta-analysis method that can be defined as a statistical analysis method to compound findings of independent studies implemented on a similar subject (crombie & davies, 2009) or as an analysis technique that combines the results of many small individual studies using one or more statistical methods to give more information (hedges ve olkin, 1985) was made use of considering the inclusion criteria of the present research. 2.2 data collection and inclusion criteria in the context of meta-analysis, sciencedirect, web of science, taylor & francis online, ebscohost, proquest dissertation & theses, council of higher education dissertation center, and google scholar search engines were scanned the years between 2000 and 2019. keywords such as "augmented reality in science, science, and augmented reality applications" were used as search terms. a total of 238 studies involving ar applications concerning the post-test scores of academic success were reached. in the context of the inclusion criteria, 26 data were derived from 24 studies. the selection of 24 studies out of 238 ones was made in the context of the inclusion criteria. inclusion criteria are stated as follows: an experimental or quasi-experimental study conducted for the use of ar in the field of science, carried out at many levels from primary school to university, examining the effect of ar on academic achievement in science, written in turkish or english, including enough data to allow calculation of effect size in meta-analysis [sample size belonging to the experimental and control groups (n), arithmetic average (x), standard deviation (sd)], carried out in the period between 2000-2017. 2.3 exclusion criteria in the literature review, the congress studies and the abstracts presented in the abstract booklet regarding ar in science teaching were excluded because they do not contain sufficient data for meta-analysis. lipsey and wilson (2001) stated that studies included in a meta-analysis should be related to the research subject and should contain statistical data necessary for analysis. in addition, among the studies obtained through the literature review, those that were not suitable for the scope of the research, included only qualitative findings, and did not comply with the inclusion criteria were excluded from the meta-analysis study. journal of science learning article doi: 10.17509/jsl.v4i3.30570 270 j.sci.learn.2021.4(3).267-274 2.4 analyzing the data a summary table of studies, including the qualities of the studies, was created to use them in the process as it was planned. the codes, names, types, and publication years of the studies, the author names, and the statistical data derived from the experimental and control groups (n, , sd) were displayed in this table. the dependent variable was determined as the effect size concerning the effect of ar on academic success. comprehensive meta-analysis (version 2.0) package program was used in the metaanalysis of the data recorded from the studies included in the study. effect sizes were interpreted by taking into account thalheimer and cook’s level classification (2002). in order to ensure the coding reliability of the research, the coding processes were examined separately by at least two independent coders (card, 2012). in case of any inconsistency, the analysis continued until a definite agreement was reached between the coders by restarting the coding processes. in this way, it was ensured that two different coders examined the data set and worked independently. as a result, the inter-rater reliability was found to be 0.86 according to cohen’s (1960) consistency values. 2.5 effect size and model selection meta-analysis requires a representation of scientific studies in terms of effect sizes. effect size can be expressed as the frequency of existence of a phenomenon in a population (cohen, 1988) or as a standardized value for different scaling instruments used in every study (turner & bernard, 2006). the use of different effect size indices (cohen's d, hedge's g, or glass's δ) is essential in obtaining standardized values and presenting correct findings and correct interpretation of the study. in the present study, hedges' g effect size index, which gives the pooled and standardized means difference between groups, was used (hedges, 1981). in determining the effect sizes during meta-analytical processes, analyzes are made according to the fixed effects model (fem) and random effects (rem) model. there are observed effect size and real effect size concepts in model selection. schmidt, oh, and hayes (2009) stated that the conditions under which fem would be suitable are minimal. therefore, many researchers recommend using rem instead of fem for meta-analysis studies (field, 2003). as studies included in this research showed diversity in terms of study design and variables, thus being heterogeneous, the random effect model was chosen as the most appropriate model (borenstein, hedges, higgins, & rothstein, 2010). the effect sizes of all studies included in the current research were presented at the end (appendix1). the i2 test, developed as a complement to the homogeneity test in the meta-analysis, reveals a more evident result regarding heterogeneity. unlike the q statistics, this value is not affected by the number of studies. while interpreting, 25% indicates a low level of heterogeneity, 50% moderate heterogeneity, and 75% high level of heterogeneity (cooper et al., 2009). in the present study, the rem was used in selecting the model because the i2 value showed a high level of heterogeneity with 86.37. 3. results the section of this study presents meta-analysis results regard to ar. the meta-analysis of all the studies concerning ar in teaching science was conducted herein, and the analysis is displayed in table 1. when table 1 is examined, it was calculated as a result of the analyses conducted according to random effects model that the standard data 0.121; the upper limit (0.839) and lower limit (0.366) of 95% confidence interval with average effect size g = 0.602. these values indicate that academic success is more effective in favor of ar applications than the traditional approach. when the effect size is considered, this value was accepted to be at a medium level according to the classifications made by thalheimer and cook (2002). as a result of z-test calculations, which were conducted to reveal the statistical significance, this value was z = 4.989. based on this result (p = 0.00), it can be stated that the analysis is statistically significant. as a result of the homogeneous test, q statistical value was calculated as 24.053. the value was 37.652 at a 95% significance level with 25 degrees of freedom in the x2 table. it was seen that q statistical value (24.053) did not exceed the chi-squared critical value (χ2(0.95) = 24.053). the average effect sizes of 26 data included in the meta-analysis according to the effect models are indicated in table 1. ar applications on academic success were found in the fixed-effect model as g = 0.516 and random-effects model as g = 0.602, meaning that the applications have a positive effect on academic success. table 1 homogeneous distribution values in effect models concerning academic success score of the studies included meta-analysis, average effect size, and confidence intervals model type n z p q es i2 % 95 confidence interval lower limit lower limit fem 26 11.434 0.00 170.842 0.516 86.37 0.427 0.604 rem 26 4.989 0.00 24.053 0.602 0.366 0.839 df:25 journal of science learning article doi: 10.17509/jsl.v4i3.30570 271 j.sci.learn.2021.4(3).267-274 3.1 results of publication bias publication bias means a tendency for studies with positive and statistical significance to be published compared to studies that are negative and without statistical significance. publication bias above a certain level affects the average effect size to be calculated and makes it higher than it should be (borenstein, hedges, higgins, & rothstein, 2009). the publication bias in the present study was calculated in two ways, one of which is considering rosenthal's (1979) classic fail-safe number, and the other one is the funnel plot. figure 1 shows the results of the funnel plot. in the funnel plot chart, the standard error value of the study is seen on the y-axis while the effect size value on the x-axis. thus, studies with small standard error values are collected towards the upper part of the funnel chart (borenstein et al., 2009). as seen in figure 3, the effect size of the studies included in this study was symmetrically located on both sides of the vertical line, in the upper and middle regions. therefore, this position is interpreted as no publication bias (borenstein et al., 2009). on the other hand, rosenthal's classic fail-safe n calculation was also checked. classic fail-safe n specifies the number of studies required to make the effect size “ineffective” by calculating the number of studies that may be missing in a meta-analysis study (borenstein et al., 2009). therefore, it can be stated that the effect size value calculated with the current meta-analysis research is 977 (appendix-2). however, considering the study's inclusion criteria and 24 studies included, this number can be considered as all of the studies that could be reached. therefore, it can be accepted that there is no publication bias in the current meta-analysis since there is no possibility to reach 977 more studies other than these studies, and this number is very large. 4. discussion and recommendation in this research, the effect of ar applications on science teaching was aimed to be evaluated with regard to meta-analysis. as a result of scanning the studies concerning ar applications in science teaching, a metaanalysis was conducted to analyze the obtained studies in the research. according to the results of 26 data, the effect of ar applications in the instructional process on academic success was found to be at a medium level. this result overlaps with the ones of several studies conducted in national (yen, tsai & wu, 2013; yılmaz, 2016; chang, chung & huang, 2016; yılmaz & batdı, 2016) and international arena (dunleavy et al., 2009; wojciechowski & cellary, 2013; yen et al., 2013). similar to the results that showed a positive impact of ar on science academic success, this new technology was also seen to increase the satisfaction and motivation of students (fonseca, martí, redondo, navarro, & sánchez, 2014). besides, the fact that the number of studies dealing with ar in teaching atomic models subject in science teaching at the university level is quite a few took attention of the authors. in this way, for future research, in one sense, it can be suggested to conduct research related to this subject in the science field. moreover, when the levels of subjects were considered, it was seen that fewer studies were conducted at the university level on the related subject of atomic models. thus, it is suggested to use ar on teaching atomic models to university students. figure 3 funnel plot of standard error by hedge’s g -3 -2 -1 0 1 2 3 0,0 0,1 0,2 0,3 0,4 s ta n d a rd e rr o r hedges's g funnel plot of standard error by hedges's g journal of science learning article doi: 10.17509/jsl.v4i3.30570 272 j.sci.learn.2021.4(3).267-274 5. conclusion in this study, using ar in science teaching on academic achievement was examined through the meta-analysis process. for this purpose, the effect sizes of the studies accessed through the databases searched from the context of the study's inclusion criteria were calculated, and the average effect size of 26 data in 24 studies was found to be at the medium level (g = .602) and significant. within the context of meta-analysis in this study, it was figured out from the included studies that while teaching and learning abstract subjects in science, presenting the subjects more concretely and straightforwardly using ar can provide more effective and permanent learning. this result means that ar applications positively affect academic achievement in science teaching. on the other hand, the values obtained from the funnel plot and fail-safe n calculations conducted for the reliability of the research in the meta-analysis process revealed that there is no publication bias in the study. the fact that the current research consists of only the science field does not mean that ar is only effective in this scope. at this point, arbased research encountered in literature reviews and studies in the data collection process has shown that ar has an application and effect dimension in many different areas such as health, economy, security, tourism, the industry as well as education. especially in the literature review of the study, it has been found that more studies have been conducted on ar recently. this situation may arise from technological developments and changes. in addition, the pandemic process has led the whole world to distance learning, online education, and virtual environments by moving the world away from real learning environments. this orientation has increased the interest and need for technologies that improve and facilitate the effectiveness of the learning process. for instance, it is concluded from the research results that ar technologies make business and operations in the tourism sector easy and reliable in the pandemic process (mohanty, hassan & ekis, 2020). therefore, it is stated in their research that there will be an excellent demand for ar both to ensure tourist safety and to create unique, accessible, personalized, and memorable experiences. in addition, another study resulted that the integration of ar with mobile-based technology increased the value and reach of ar (michele, michele & fabio, 2013). as this technology is hoped to ensure so many facilities, educational institutions are thought to explore the possibilities of virtual trips as a substitute for real-life trips (yung & khoo-lattimore, 2019), of the contents of courses in distance education. there is much more to say about the impact of ar on many subjects and its use in many fields. therefore, future research is highly recommended to carry out different ar effects on different fields and disciplines. 6. limitations and implications while searching the studies for the meta-analysis process within the current research, specific databases were used. however, it can be stated that researchers can keep their scope wider by using different databases other than this. in addition, the effectiveness of ar applications in different subject areas/ disciplines can be evaluated. for example, berryman (2012) stated that ar technology covers various subjects, target groups, and academic levels. moreover, alternative studies can be conducted by making differences in the inclusion criteria for the analysis, such as the research design/type, publication years (period), and moderator variables. it is remarkable and essential that this research is carried out with ar applications, which can be considered one of the most significant developments in this technology age. performing a meta-analysis process, which includes combining the current study results in the literature, can generalize the results on the relevant subject. as can be understood from the research results in the literature and the current research results, the effectiveness of ar applications in different dimensions can be evaluated, which can provide permanent and meaningful learning at a reasonable level. acknowledgments the present study was not supported by any foundations, institutions, or others. therefore, there is no funding to report for this submission. we confirm that all co-authors contributed equally to the study. to the best of our knowledge, there is no conflict of interest. references (the references marked with an asterisk (*) are the ones included in the analysis.) *abdüsselam, m. s. (2014). development and evaluation of an instructional material for physic lesson magnetism subject based on augmented reality environment (unpublished doctoral dissertation). karadeniz technical university, trabzon, turkey. abdüsselam, m. s., & karal, h. (2012). the effect of mixed reality environments on the students’ academic achievement in physics education: 11th grade magnetism topic example. journal of research in education and teaching, 1(4), 170-181. abdüsselam, m. s., & karal, h. (2015). artırılmış gerçeklik [augmented reality]. in b. akkoyunlu, a. i̇şman, h. f. odabaşı (eds.), eğitim teknolojileri okumaları [educational technology readings], 2015 (pp. 149-171). ankara: tojet. isbn: 9786053181262. abriata, l. a. (2018). towards commodity, web-based augmented reality applications for research and education in chemistry and structural biology. arxiv preprint arxiv:1806.08332. akçayır, m. & akçayır, g. (2017). advantages and challenges associated with augmented reality for education: a systematic review of the literature. educational research review, 20, 1-11. *akçayır, m., akçayır, g., pektaş, h. m, & ocak, m. a. (2016). augmented reality in science laboratories: the effects of augmented reality on university students’ laboratory skills and attitudes toward science laboratories. computers in human behavior, 57, 334-342. *aktamış, h. & arıcı, v. a. (2013). the effects of using virtual reality software in teaching astronomy subjects on academic achievement https://www.sciencedirect.com/science/journal/1747938x journal of science learning article doi: 10.17509/jsl.v4i3.30570 273 j.sci.learn.2021.4(3).267-274 and retention. mersin university journal of the faculty of education, 9(2), 58-70. *altıntaş, g. (2018) . the effect of augmented reality applications on teacher candidates’ scientific epistemological beliefs and mısconceptıons: global warming (unpublished doctoral dissertation). mehmet akif ersoy university, burdur, turkey. anıl, ö & batdı, v. (2020). artırılmış gerçeklik: eğitimdeki uygulamalar üzerine kuramsal bir çalışma [augmented reality: a theoretical study on applications in education]. in paksoy, s. (ed.). sosyal bilimlerde özgün çalışmalar-1 [original studies in social sciences-1]. ankara: iksad publications. azuma, r. (1997). a survey of virtual reality. presence: teleoperators and virtual environments, 6(4), 355–385. *babur, a. (2016). the effects of usıng of augmented realıty, sımulatıon and real object on learnıng achıevements, motıvatıon and psychomotor performance (unpublished doctoral dissertation). sakarya university, sakarya, turkey. bacca, j., baldiris, s., fabregat, r., graf, s., & kinshuk. (2014). augmented reality trends in education: a systematic review of research and applications. educational technology & society, 17(4), 133–149. berryman, d. r. (2012). augmented reality: a review. medical reference services quarterly, 31(2), 212-218. doi: https://doi.org/10.1080/02763869.2012.670604 billinghurst, m. (2002). augmented reality in education. new horizons for learning, 12(5), 1-5. billinghurst, m., kato, h., & poupyrev, i. (2001). the magicbookmoving seamlessly between reality and virtuality. computer graphics and applications, ieee, 21(3), 6-8. borenstein, m., hedges, l. v., higgins, j. p. t., & rothstein, h. r. (2009). introduction to meta-analysis (1st ed.), uk: john wiley & sons, ltd. borenstein, m., hedges, l. v., higgins, j. p. t., & rothstein, h. r. (2010). a basic introduction to fixed-effect and random effect models for meta-analysis. research synthesis methods, 1(1), 97-111. *buluş kırıkkaya, e. & şentürk, m. (2018). the impact of using augmented reality technology in the solar system and beyond unit on the academic achievement of the students. kastamonu education journal, 26(1), 181-189. *cankaya, b. & girgin, s. ( 2018). the effect of augmented realty technology on the academic success of science course. journal of social and humanities sciences research, 5(30), 4283-4290. card, n. a. (2012). applied meta-analysis for social science research. new york: the guilford press. *chang, r. c., chung, l. y., & huang, y. m. (2016). developing an interactive augmented reality system as a complement to plant education and comparing its effectiveness with video learning. interactive learning environments, 24(6), 1245-1264. doi:10.1080/10494820.2014.982131. chen p., liu x., cheng w., & huang r. (2017) a review of using augmented reality in education from 2011 to 2016. in popescu, e. et al. (eds.) innovations in smart learning. lecture notes in educational technology. springer, singapore. https://doi.org/10.1007/978-981-10-2419-1_2 *chiu, j. l., dejaegher, c. j. & chao, j. (2015). the effects of augmented virtual science laboratories on middle school students' understanding of gas properties. computers & education, 85, 59-73. *civelek, t., ucar, e., ustunel, h., & aydın, m. k. (2014). effects of a haptic augmented simulation on k-12 students’ achievement and their attitudes towards physics. eurasia journal of mathematics, science & technology education, 10(6), 565-574. cohen, j. (1960). coefficient of agreement for nominal scales. educational and psychological measurement, 20(1), 36-46. cohen, j. (1988). statistical power analysis for the behavioural sciences. new york: erlbaum. cooper, h., hedges, l. v., & valentine, j. c. (eds.). (2009). the handbook of research synthesis and meta-analysis (2nd edition). new york: russell sage publication. crombie, i. k. & davies, h. t. (2009). what is meta-analysis. what is series? (2nd ed.). retrieved april, 15, 2019 from http://www.bandolier.org.uk/painres/download/whatis/metaan.pdf *demirel, t. (2017). the effect of augmented realty activities supported by argumentation approach on academic achievement, critical thinking skills, motivation towards science and technology course and argumentation skill (unpublished doctoral dissertation). çukurova university, adana, turkey. dunleavy, m., dede, c., & mitchell, r. (2009). affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. journal of science education and technology, 18(1), 7-22. durak, a., & yilmaz, f. g. k. (2019). artirilmiş gerçekliğin eğitsel uygulamalari üzerine ortaokul öğrencilerinin görüşleri [opinions of secondary school students on the educational applications of augmented reality]. abant i̇zzet baysal üniversitesi eğitim fakültesi dergisi, 19(2), 468-481. *erbaş, ç. (2016). the effects of mobile augmented reality applications on students' academic achievement and motivation (master’s thesis). süleyman demirel university, isparta, turkey. *eroğlu, b. (2018). evaluation of teaching astronomy concept with augmented reality application for secondary student (master’s thesis). karadeniz technical university, trabzon, turkey. field, a. p. (2003). the problem in using fixed-effects models of metaanalysis on real world data. understanding statistics, 2, 77-96. fonseca, d., martí, n., redondo, e., navarro, i., & sánchez, a. (2014). relationship between student profile, tool use, participation, and academic performance with the use of augmented reality technology for visualized architecture models. computers in human behavior, 31, 434-445. freitas, r., & campos, p. (2008). smart: a system of augmented reality for teaching 2nd grade students. in proceedings of the 22nd british hci group annual conference on people and computers: culture, creativity, interaction-volume 2 (pp. 27-30). british computer society. garzon, j., & acevedo, j. (2019). meta-analysis of the impact of augmented reality on students’ learning gains. educational research review, 27, 244-260. garzon, j., pavon, j., & baldiris, s. (2019). systematic review and metaanalysis of augmented reality in educational settings. virtual reality, 23(4), 447-459. *golden, s. a. (2016). augmented 3d holograms in higher education, increasing students’ learning outcome scores: a mixed methods study (doctoral dissertation). retrieved from proquest dissertations and theses database. (umi no. 10256536). hedges, l. (1981). distribution theory for glass’s estimator of effect size and related estimates. journal of educational statistics, 6, 107-112. hedges, l. v., & olkin, i. (1985). statistical methods for meta-analysis. new york: academic press. *hsiao, k. f., chen, n. s. & huang, s. y. (2012). learning while exercising for science education in augmented reality among adolescents. interactive learning environments, 20(4), 331-349. doi:10.1080/10494820.2010.486682 ibanez, m. b., portillo, a. u., cabada, r. z., & barron, m. l. (2020). impact of augmented reality technology on academic achievement and motivation of students from public and private mexican schools. a case study in a middle-school geometry course. computers & education, 145, 103734. https://doi.org/10.1016/j.compedu.2019.103734 *ibanez, m. b., serio, a. d., villaran, d. & kloos, c. d. (2014). experimenting with electromagnetism using augmented reality: impact on flow student experience and educational effectiveness. computer & education, 71, 1-13. https://doi.org/10.1080/02763869.2012.670604 https://doi.org/10.1007/978-981-10-2419-1_2 http://www.bandolier.org.uk/painres/download/whatis/meta-an.pdf http://www.bandolier.org.uk/painres/download/whatis/meta-an.pdf https://doi.org/10.1016/j.compedu.2019.103734 journal of science learning article doi: 10.17509/jsl.v4i3.30570 274 j.sci.learn.2021.4(3).267-274 johnson, l., levine, a., smith, r., & stone, s. (2010). the 2010 horizon report. new media consortium. 6101 west courtyard drive building one suite 100, austin, tx 78730. kerawalla, l., luckin, r., seljeflot, s., & woolard, a. (2006). making it real: exploring the potential of augmented reality for teaching primary school science. virtual reality, 10(3-4), 163-174. kılıç, t. (2016). artırılmış gerçeklik teknolojisinin iç mekân tasarım sürecinde kullanılması [the use of augmented reality technology in the interior design process] (master’s thesis). mimar sinan fine arts university, institute of science, department of interior architecture, istanbul. *küçük, s. (2015). effects of learning anatomy via mobile augmented reality on medical students' academic achievement, cognitive load, and views toward implementation (doctoral dissertation). atatürk university, erzurum, turkey. lipsey, m. w., & wilson, d. b. (2001). practical meta-analysis. london, new delhi: sage. michele, g., michele, d.d. & fabio, s. (2013). visitar: a mobile application for tourism using ar. siggraph asia 2013 symposium on mobile graphics and interactive applications, pp. 1-6. mohanty, p., hassan, a., & ekis, e. (2020). augmented reality for relaunching tourism post-covid-19: socially distant, virtually connected. worldwide hospitality and tourism themes. 12(6), 753-760. rashevska, n. v., semerikov, s. o., zinonos, n. o., tkachuk, v. v., & shyshkina, m. p. (2020). using augmented reality tools in the teaching of two-dimensional plane geometry. ceur workshop proceedings, 2731, 79–90. rosenthal, r.(1979). the “file drawer problem” and tolerance for null results. psychol bull, 86, 638–41. *sahin, d.,(2017). effect of scıence teachıng wıth the augmented realıty technology on secondary school students’ achıevement and theır attıtude towards the course (unpublished master’s thesis). atatürk university, erzurum, turkey. saidin, n. f., halim, n. d. a., & yahaya, n. (2015). a review of research on augmented reality in education: advantages and applications. international education studies, 8(13), 1-8. *sarıkaya, m. (2015). effects of augmented reality applications on students’ achievement, misconceptions and course engagement (doctoral dissertation). gazi university, ankara, turkey. schmidt, f. l., oh, i.-s., & hayes, t. l. (2009). fixedversus randomeffects models in meta-analysis: model properties and an empirical comparison of differences in results. british journal of mathematical and statistical psychology, 62, 97-128. *sentürk, m. (2018). the investigation with solomon four-group design on the effect of using mobile augmented reality(ar) applications in the unit titled solar system and beyond in the seventh-grade on the students' academic success, motivation, science and technology attitude (master’s thesis). kocaeli university, kocaeli, turkey. swensen, h. (2016). potential of augmented reality in sciences education a literature review. a paper form proceedings of iceri2016 conference, 14th-16th november 2016, seville, spain. thalheimer, w., & cook, s. (2002). how to calculate effect sizes from published research articles: a simplified methodology. retrieved october 20, 2017 from http://education.gsu.edu/coshima/eprs8530/effect_sizes_pdf 4.pdf timur, b., & özdemir, m. (2018). fen eğitiminde artirilmiş gerçeklik ortamlarinin kullanimina ilişkin öğretmen görüşleri [teachers' views on the use of augmented reality environments in science education]. uluslararası türk eğitim bilimleri dergisi, 10, 62-75. *tosik gün, e., & atasoy, b. (2017). the effects of augmented reality on elementary school students’ spatial ability and academic achievement. education and science, 42(191), 31-51. turner, h. m. i., & bernard, r. m. (2006). calculating and synthesizing effect sizes. contemporary issues in communication science and disorders, 33, 42-55. walczak, k., wojciechowski, r., & cellary, w. (2006). dynamic interactive vr network services for education. in proceedings of the acm symposium on virtual reality software and technology (pp. 277-286). acm. wojciechowski, r., & cellary, w. (2013). evaluation of learners’ attitude toward learning in aries augmented reality environments. computers & education, 68, 570-585. *wu, p. h., hwang, g. j., yang, m. l., & chen, c. h. (2018). impacts of integrating the repertory grid into an augmented reality-based learning design on students’ learning achievements, cognitive load and degree of satisfaction. interactive learning environments, 26(2), 221-234. *yen, j. c., tsai, c. h., & wu, m. (2013). augmented reality in the higher education: students' science concept learning and academic achievement in astronomy. procedia social and behavioral sciences, 103, 165 – 173. *yıldırım, p. (2018). the impact of science teaching made with mobile augmented reality technology on science and technology attitudes and academic achievement of secondary school students (master’s thesis). fırat university, elaziğ, turkey. yılmaz, r. m. (2016). educational magic toys developed with augmented reality technology for early childhood education. computers in human behavior, 54, 240-248. yung, r. & khoo-lattimore, c. (2019). new realities: a systematic literature review on virtual reality and augmented reality in tourism research. current issues in tourism, 22(17), 2056-2081. http://education.gsu.edu/coshima/eprs8530/effect_sizes_pdf4.pdf http://education.gsu.edu/coshima/eprs8530/effect_sizes_pdf4.pdf microsoft word published_applying pre and post role-plays supported by stellarium virtual observatory to improve students’ un a doi: 10.1021/xxx.xxxx.xxxxxx © 2017 indonesian society for science educator 1 j.sci.learn.2017.1(1).1-7 received: 28 october 2017 revised: 29 november 2017 published: 30 november 2017 applying pre and post role-plays supported by stellarium virtual observatory to improve students’ understanding on learning solar system eka c prima1*, chika l putri1, fransisca sudargo2 1international program on science education, faculty of mathematics and science education, universitas pendidikan indonesia 2department of biology education, faculty of mathematics and science education, universitas pendidikan indonesia *corresponding author. ekacahyaprima@upi.edu abstract solar system is an abstract phenomenon that needs to understand by junior high school students. the phenomenon cannot be observed directly in the classroom. in this research, the role-play model is first proposed to provide analogical phenomenon combined by stellarium as an astrophysical virtual laboratory to give better visualization of solar system model. role-play is a part of psychodrama used as a learning method to help students understand some aspects of science. however, the best strategy on this approach needs to be further investigated. the research attempts to improve students’ understanding on learning solar system. the role-play model is applied before or after using stellarium virtual observatory as pre and post role-play learning designs, respectively. this research employed quasi-experimental method. the method used in this research is experimental with pretest-posttest design. there are three types of instrument used in this research. there are objective test, rubric, and questionnaire. the quantitative data of this research was collected by objective test (pretest-posttest), while the qualitative data is used to describe students’ performance and students’ responds in role-play. the result shows that there are statistically significant between pre role-play and post role-play. the n-gain of students’ understanding improvement on post role-play was 0.40 and pre role-play was 0.18. the result is confirmed by students’ performance and responds implying that students need learning concept previously so that they can perform role-play effectively. keywords role-play, stellarium, students’ understanding, solar system 1. introduction curriculum development in indonesia causing some of changes of its learning objectives, its contents and its implementations. according on curriculum 2013, the student are required to be more active, and seeking for knowledge from various sources, not only teacher. therefore, learning methods involve students more are needed. one of the learning method that can be used is role-play. role-play is a part of psychodrama that could be used as a learning method to help students understand the more specific aspects of literature, social studies, and even some aspects of science or mathematics. according to craciun (2010), stated that learning using role-play implies students to be more actively involved, more creative, more confident, more cooperative in-group work and more cooperative to solve problem. role-play really contributes in developing the creativity, communication skills and leadership skills’ aspects. role-play is a method that can be implemented to improve students’ understanding. according to works conducted by duveen & solomon (1994), aubusson, fogwill, barr, & perkovic (1997), craciun (2010), eilks, belova, & feierabend (2015), role gave positive effect to students understanding, creativity, empathy, cooperative group work, communication skills, and leadership skills. role-play is often used in social studies, historical, politics, and even science and mathematics as a teaching methodology. however, role-play have some disadvantages such as it spends more time to preparation rather than the classical methods, and not all of the topics in science is appropriate for role-play methods.. the topic in science that can be taught using role-play should be abstract phenomena, which occur in students’ daily life and need analogical analysis. solar system is chosen as a topic for this journal of science learning article 2 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).1-7 research because it is abstract phenomena happen in daily life and it is hard to observe directly that representative teaching of solar system is needed. by using role-play, a teacher encourages students to be intellectually and physically involved in the lesson content and his understanding on difficult concepts. role-play is often used in the end of learning processes. in line with the study conducted by duveen & solomon (1994), they stated that students need to have already been taught the concept first because without prior knowledge, the role-play is much less effective. using this method, the teacher encouraged the student to be intellectually and physically involved in the lesson content and that facility his understanding of difficult concepts (craciun, 2010). however, there is no study investigating the best strategy to put role-play as pre or post learning. in this research, students’ understanding treated using role play conducted in the beginning learning process (pre role-play) compared to role play in the end of learning process (post role-play) will be investigated. the stellarium software will be used as a virtual astrophysical laboratory providing better visualization of solar system. the previous works reported the use of stellarium as a teaching media such as hughes (2008), baleisis, dokter, & magee (2007). one of the indicators to measure the differences between pre role-play and post role-play is the students’ understanding. according to the research conducted by gibbs & simpson (2005), students’ understanding is one of the important aspects to measure the students’ improvement. assessing students’ understanding might be the most complex task for an educator or academic institution. unfortunately, the professional development gives a low attention to develop the qualified assessments, and assessment training. the challenge of assessment is not only about figuring out what a learner knows but also where they need to go next. the students’ performance in role-play should also be assessed because it is one of the indicator to measure the learning objective has been achieved or not achieved by students. 2. method this research used quasi-experimental method, according to creswell (2014), quasi experiment includes the assignments, but not random assignment of participants to group because the experimenter cannot artificially create groups for the creswell (2014). the researcher fully conducted the pre role-play and post roleplay treatments. class a conducted post role-play activities while class b conducted pre role-play activities. the experiment design is shown in table 1. the location of this research was held in one international school in bandung barat in the school period of 2015/2016. the school used cambridge curriculum 2013. the population in this research was 8th grade students. the participants are 50 students at 8th grade from two different classes in an international junior high school, bandung. the sampling technique was simple random sampling. fraenkel & wallen (2003) stated that simple random sampling is one in which each and every member of the population has an equal and independent chance being selected. in this research, the concept of solar system is limited based on indonesian curricula 2013 by core competence no. 3 and no. 4, basic competence no 3.13, 3.14, and 4.12 as attached in badan standar nasional pendidikan (2013). the analysis of curriculum about core competence and basic competence indicates the subtopics that will be investigated by students such as (1) characteristics of solar system component, (2) the movement of planets in the solar system, (3) moon phase, (4) the effect of earth rotation and revolution, (5) climate change in the earth surface. the terms of solar system in this research refer to the sun and all of celestial objects traveling around. there are planets, natural satellites such as our moon, asteroid belts, comets, and meteoroids. our solar system is part of a spiral galaxy known as the milky way. the sun, the center of our solar system, holds eight planets and countless smaller objects in its orbit. our solar system formed about 4.6 billion years ago. the four planets closest to the sun mercury, venus, earth, and mars called as the terrestrial planets because they have solid, rocky surfaces. there are three types of instrument used in this research. there are objective test, rubric, and questionnaire. first, the objective test is conducted to describe cognitive ability of students in mastering the concept. objective test consist of two sections that is pretest, and post-test. multiple-choice question consist of cognitive domain c2 which is about students’ comprehension including summarize, convert, defend, paraphrase, interpret, give examples, classify, infer, compare, explain, paraphrase, and discuss (anderson & krathwohl, 2001). cognitive paper test firstly consist of fifteen questions before passing judgment by experts. it is used to look students’ comprehension. after judged by the expert the objective is only ten questions as a representative for each learning indicators. then, test was distribute to students in grade 8 as a limited test. the next step after conducting limited test to x grade students is analyzing this objective test using anates to measure the validity, reliability, difficulty level, discriminating power and distractor. second, observation sheets has been created so that teaching staff can take the opportunity to observe and table 1 experiment design class test stage i stage ii test action a pretest virtual lab role play posttest post role play b pretest role play virtual lab posttest pre roleplay journal of science learning article 3 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).1-7 reflect on the particular positive teaching and behavior management strategies employed. a range of different observation sheets is provided, so that teachers can choose the ones most appropriate to their observations focus. in this research the observation sheet is based on craciun (2010) to evaluate the students’ performance and to differentiate from sessions pre and post role-playing activities, there are 8 aspects that observed used a scale from 1(lowest) to 5 (highest). the aspects are active presence, cooperative group work, creative performance, scientific knowledge, students’ confidence, students’ communication skills, students’ responsibility, and students’ leadership skills. third, questionnaire is an instrument, which is distribute to the students the purpose, is for investigate and describe students’ perspective of roleplay as a learning method. students are asked to assess their learning in terms of course content, teaching or research skills and technology use. they reflect on the course learning objectives as well as the advantages or limitations of the teaching method. the results shows quantitative and qualitative data. the pre-test and the post-test are conducted to determine the students’ understanding before and after treatments. qualitative analysis will describe the students’ performance and impression during learning solar system using roleplay. 3. result and discussion the results shows quantitative and qualitative data. the pre-test and the post-test are conducted to determine the students’ understanding before and after treatments. qualitative analysis will describe the students’ performance and impression during learning solar system using roleplay. 3.1 students’ understanding pretest was conducted in the beginning of the learning to investigate the students’ understanding before this lesson. pretest with the same question was given to both class a and class b. class a performed post role-play while class b performed pre-role play. statistic result shows that the data of normality test with significance 0.166 and 0.251 for post and pre role-plays, respectively, is normal with criteria sign. ≥ 0.05 as normal and homogeneity test with significance 0.684 as homogeny. the class a showed the highest score 80.00 and the lowest score 30.00 with the average score 58.80 and standard deviation 13.329. class b showed the highest score 80.00 and the lowest score 20.00 with the average 50.80 and the standard deviation 14.978. the analysis of pre-test resulted that there is no statistically significant between pre-test in class a and class b. this result indicates that prior knowledge of students’ understanding is rather similar. the average of students’ understanding pre-test in class a is 58.80 and class b 2 is 50.80. these results are categorized as medium because solar system concept have been taught in the primary school, but not as deep as discussed in junior high school. some students also might forget this concept. the result of pre-test is use as the reference for a teacher to investigate students’ prior knowledge in learning solar system using role-play. by giving the pre-test about solar system concept, students do not have enough knowledge about this subject to accomplish the standard as required by indonesian curriculum 2013 to meet minimum score 70. post-test given in the end of the learning process is to know students’ understanding after conducting this lesson. post-test are also given to both class a performing post role-play and class b conducting pre role-play. statistic result shows that the data of normality test with significance 0.121 and 0.160 for post and pre role-plays, respectively, is normal with criteria sign. ≥ 0.05 as normal and homogeneity test with significance 0.711 as homogeny. the class a showed the highest score 100.00 and the lowest score 50.00 with the average score 72.50 and standard deviation 13.266. class b showed the highest score 90.00 and the lowest score 40.00 with the average 61.60 and the standard deviation 13.127. the analysis of post-test resulted that there is no statistically significant between pre-test in class a and class b. according to the students’ average score, post role-play supported to better students’ understanding improvement than pre role-play. moreover, average students have successfully accomplished the standard as required by indonesian curriculum 2013 to meet minimum score 70. these results indicate that role-play can improve students’ understanding supporting previous research conducted by aubusson (1997). it showed that role-play was able to make students more understand deeper concept and happy with this method. the work implied that students in class a have already taught the solar system concept visualized by stellarium, so they can perform the role-play more effective. consequently, the result of average post-test score is better than class b. according to our direct observation in class b, some students do not know the concept previously, so figure 1 average n-gains of post and pre role plays 0.405 0.181 0.0 0.1 0.2 0.3 0.4 0.5 n -g a in post-role play pre-role play journal of science learning article 4 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).1-7 they feel confuse to perform role-play. in contrast, the students in class a are more enthusiast to perform roleplay. some students also focused on learning solar system concept. the pre role-play treatment make students more active and noisy, so the learning material might not be understood effectively. another analysis is performed to investigate students’ understanding improvement using role-play. analysis of ngain as seen in figure 1 is conducted to investigate the statistically significant different of students’ understanding improvement between pre and post role-plays. the average n-gain result is tabulated in table 2. class a performing post role-play resulted the average n-gain 0.404 while class b performing pre role-play showed the average n-gain 0.181. statistic result shows that the data of normality test with significance 0.166 and 0.160 for post and pre roleplays, respectively, is normal with criteria sign. ≥ 0.05 as normal and homogeneity test with significance 0.384 as homogeny. the hypothesis test showed the value of 0.011. it indicates that the value 0.011 < 0.05 with criteria of sig. (2 – tailed) ≥ 0.05 = accepted h0. therefore, h0 is rejected. the result indicated that there are statistical differences of average n-gain class a from class b. this result also confirms duveen’s (1994) and aubusson’s (1997) works stating that students have already been taught the concept before doing the role play, without prior knowledge role play is much less effective. objective test of students’ understanding is given to the students. the test includes several sub-indicators. table 3 shows the percentage of correct answers by students for each indicator in pre-test and post-test performing post and pre role play. based on n-gain, post role-play shows better improvement on several indicators than pre role-play such as (1) describing movement of the moon towards the sun, (2) explaining the component of the solar system, (3) describing various impacts of earth rotation and revolution, and (4) describing the earth movement towards the sun. on the other hand, the pre role-play has successfully improved other indicators about (1) describing the earth revolution, (2) explaining the earth rotation phenomena, (3) mentioning the impact of earth rotation and revolution. according to the curricula 2013 minimum standard, both pre and post role plays have achieved 5 indicators accomplished by students during learning solar system. the indicator about describing various impacts of earth rotation and revolution cannot be achieved by average students. students’ analysis might be needed to improve this understanding so that students can fulfill this indicator. the role-play is also suggested to describe how student can act as a model to visualize the impact of earth revolution and rotation with another appropriate supporting media. the visualization of stellarium describing this impact might be needed to give better understanding about the effect of table 2 result analysis of n-gain component post role-play pre role-play number of students 25 25 highest score 1 0.67 lowest score -0.25 -0.67 average 0.404 0.180 standard deviation 0.276 0.318 normality test sig. ≥ 0.05 = normal sig. 0.166 0.160 conclusion normal normal homogeneity test sig. ≥ 0.05 = homogeny sig. 0.384 conclusion homogeny hypothesis test sig. (2 – tailed) ≥ 0,05 = h0 accepted sig. (2-tailed) 0.011 conclusion h0 rejected table 3 average of students’ understanding for each indicator indicator of students’ understanding post role play pre role play pre-test (%) posttest (%) ngain pre-test (%) posttest (%) ngain describe movement of the moon towards the sun 64 84 0.56 68 72 0.13 describe the earth revolution 92 96 0.50 64 92 0.78 explain the component of the solar system 52 80 0.58 60 80 0.50 describe various impacts of earth rotation and revolution 40 58 0.30 36 41 0.08 describe the earth movement towards the sun 48 80 0.62 64 76 0.33 explain the earth rotation phenomena 88 92 0.33 64 92 0.78 mention the impact of earth rotation and revolution 88 96 0.67 80 96 0.80 table 4 average of students’ performance in class a and b no aspect post roleplay pre roleplay 1 active presence 4.64 3.88 2 cooperative group work 4.96 3.92 3 creative performance 4.52 3.56 4 scientific knowledge 4.60 3.64 5 students confidence 4.52 3.88 6 students communications skil 4.56 3.64 7 students responsibility 4.92 3.80 8 students leadership skills 4.84 3.96 average 4.69 3.78 journal of science learning article 5 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).1-7 earth rotation and revolution. therefore, students are able to make a creative media supporting role-play. 3.2 students’ performance rubric in this experiment is used to investigate students’ performance during preparation and role-play performance. this rubric contains students’ performance based on criteria previously defined. the score range around 1-5. score 1 is categorized as very bad performance contrasted to 5 as excellent performance. the student’s performance in class a and b is described in table 4. based on table 4, the students’ performance average in post role-play 4.69 is better than pre role-play 3.78. it can be concluded that the students’ performance on post roleplay mainly contributes to higher students’ understanding improvement in post role-play as previously described. the previous solar system concept presented using stellarium virtual observatory supports students’ interest to be involved in post role-play. consequently, their cooperation of group work shows the highest achievement of students’ performance aspect. 3.3 students’ responds on pre and post role-plays students are asked to assess their learning in terms of course content, teaching or research skills and technology use. they reflect to the course learning objectives as well as the advantages or limitations of the teaching method (craciun, 2010). anonymous questionnaire is given to the end at the learning process. it is used to know students’ response toward learning using role-play. students responds is important to evaluate this learning method applied and to investigate students’ impression during this learning process. the result can reflect the teacher awareness about students’ emotion. the result of students’ responds is shown in table 5. based on table 4 and figure 2, the students’ performance average in class a (post role-play) of 4.69 is higher than class b of 3.78. the average performance of post role-play can be categorized as good, while average performance in pre role-play is categorized as enough. in all aspects of students’ performance, students performing post role-play show higher score than students performing pre role-play. the highest aspect of class a is cooperative group work and its lowest aspect is creative performance and students’ confidence. moreover, the highest aspects of class b is students’ leadership skill and its lowest aspects is creative performance. for both class a and class b, creative performance results the lowest aspects. in this anonymous questionnaire, all of the questions are positive statement or questions indicating that students like role-play as a teaching method for science education. role-play is also helpful method in learning science. based on table 5, students in class a are 15% strongly agree, 38% agree, 40% not sure, 9% disagree, and 1% strongly disagree that role-play is an effective method to learn solar system concept. while students in class b are 29% strongly agree, 20% agree, 40% not sure, 25 disagree and 10% strongly disagree that role-play is an effective method to learn solar system concept. based on the table 5, almost all aspects of questionnaire are positively responded by the students in class a and b. however, some aspects are not positively responded by the students. for example, 60% students in the class b are not sure performing role-play during learning science. more than half students from both class a and b prepare the role-play, so the preparation of roleplay activity does not show maximum performance. students do not fully welcome to perform role-play activity. it might happen due to time limitation for preparation and performance. table 5 recapitulation of students’ responds regarding the role-play n o question post role-play pre role-play sa (%) a (%) ns (%) d (%) sd (%) sa (%) a (%) ns (%) d (%) sd (%) 1 did you like creative role-playing activities? 20 47 13 20 0 20 27 20 23 7 2 do you prefer science learning and teaching with or without role-playing? 20 20 74 6 6 14 14 60 26 0 3 do you like science learning and teaching more after these activities? 6 60 34 0 0 0 20 53 27 0 4 did you adopt some of the role-playing techniques in your future teaching activities? 7 33 53 7 0 0 20 34 26 20 5 do you spend more time to design a roleplaying scenario? 13 47 27 13 0 14 7 46 26 7 6 do you consider that these activities are easier or harder to control? 13 34 47 6 0 0 26 40 26 7 7 do you consider that these activities are generally valuable or must be adapted to the audience? 27 27 33 13 0 14 26 33 20 7 average 15 38 40 9 1 9 20 40 25 10 notice: sa = strongly agree, a = agree, ns = not sure, d = disagree, sd = strongly disagree journal of science learning article 6 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).1-7 students answer the questionnaire to express their own opinion about role-play. the result shows that almost all students give positive impression about their learning science using role-play. almost all students feel that roleplay is a funny activity. students can also explore their talent and knowledge through role-play. some students think that role-play can be used to refresh their mind in school because students feel easy to learn solar system concept by implementing role-play. moreover, some students does not prefer to learn science using role-play because role-play takes more time to prepare the materials. other students are not confidence to perform role-play in front of the class. they also feels stressful to conduct roleplay. students’ performance is an important aspect in roleplay. students’ understanding can also be improved using role-play as well as the improvement of students affective and psychomotor. based on craciun (2010), by implementing this method, we are able to develop our skills and our abilities such as responsibility and leadership in learning, peer learning/teaching, group work, or creative problem solving. these achievements would be difficult to developed using other teaching techniques without roleplay. however, students stated that not all topics in science are suitable to implement role-play. the abstract topics such as planet positions in solar system or current flow in electricity are suggested to perform role-play. most of students like learning science more after conducting roleplay because they previously known that learning science using lecturing method is boring, difficult and creepy. by conducting role-play, students feel fun and joyful. most of students performing post role-play are agreed and not sure that they will adopt some role-play activities in the future, role-play spend more time than other method, so some students think that it is hard to prepare role-play. students spend 1-3 days in average to prepare role-play. many students complain this preparation. most of students stated that this role-play is valuable. therefore, the audience must adapt this activity. aubusson (1997) stated that role-play give positive impact to students’ behavior because students can more fun and enjoy in classroom. role-play also make student understand deeper concept. 4. conclusion role-play is a useful method to learn certain topic in science based on teacher and students perspective. roleplay can improve students’ understanding as proven by the result of post-test for both class a and b, which are higher than the pre-test. students need to learn the concept before performing role-play. the visualization of stellarium is needed to support better understanding on solar system. by showing this media, students are able to make a creative media supporting role-play. moreover, without this prior knowledge, role-play is ineffective. some students who do not know the concept previously, they feel confused to perform role-play. the pre role-play treatment make students more active and noisy, so the learning material might not be understood effectively. in contrast, the figure 2 comparison between students performing post and pre role-plays 4.64 4.96 4.52 4.6 4.52 4.56 4.92 4.84 3.88 3.92 3.56 3.64 3.88 3.64 3.8 3.96 0 1 2 3 4 5 6 p er fo rm a n ce i n d ex role play aspects post-role play pre-role play journal of science learning article 7 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).1-7 students in class a performing post role-play are more enthusiast to implement role-play. in this case, some students also focused on learning solar system concept. the average performance of post role-play can be categorized as good, while average performance in pre roleplay is categorized as enough. finally, most of students like learning science more after conducting role-play because they previously known that learning science using lecturing method is boring, difficult and creepy. by conducting roleplay, students feel fun and joyful. acknowledgment the authors acknowledge mrs. rika rafikah agustin for stimulating discussion about educational assessment. references anderson, l. w. & krathwohl, d. (2001). taxonomy for learning, teaching, and assessing: a revision of bloom’s taxonomy of educational objectives. new york, ny: longman. aubusson, p., fogwill, s., barr, r., & perkovic, l. (1997). what happens when students do simulation-role-play in science?. research in science education, 27(4), 565-579. badan standar nasional pendidikan (2013). standar proses pendidikan dasar dan menengah. jakarta, indonesia: kemendikbud. baleisis, a., dokter, e., & magee, m. (2007). programming the universe: stellarium scripting as an inquiry tool in introductory college astronomy. bulletin of the american astronomical society, 39, 737. craciun, d. (2010). role–playing as a creative method in science education. journal of science and arts, 1(12), 175-182. creswell, j. w. (2014). a concise introduction to mixed methods research. california, us: sage publications. duveen, j., & solomon, j. (1994). the great evolution trial: use of role� play in the classroom. journal of research in science teaching, 31(5), 575582. eilks, i., belova, n., & feierabend, t. (2015). the evaluation of roleplaying in the context of teaching climate change. international journal of science and mathematics education, 13(1), 165-190. fraenkel, j. r., & wallen, n. e. (2003). how to design and evaluate research in education. new york, us: mcgraw-hill higher education. gibbs, g., & simpson, c. (2005). conditions under which assessment supports students’ learning. learning and teaching in higher education, (1), 3-31. hughes, s. w. (2008). stellarium–a valuable resource for teaching astronomy in the classroom and beyond. science education news (sen), 57(2), 83-86 a © 2021 indonesian society for science educator 385 j.sci.learn.2021.4(4).385-393 received: 22 december 2020 revised: 02 may 2021 published: 19 september 2021 effect of science education provided with digital and in-class games on the scientific process skills of preschool children selin yıldız1*, raşit zengin2 1institue of education science, firat university , elazig , turkey 2department of mathematics and science education, faculty of education, firat university, elazig, turkey *corresponding author slnylddz@hotmail.com abstract in this research the effects of educational digital games and in-class educational games on scientific process skills of 6 years old children were investigated. the research was carried out with 70 preschool children who were educated in a primary school in turkey. in the research, quasi-experimental design was used. within the scope of the research, there are 3 groups including 2 experimental and 1 control groups. the educational digital games with the children in the experiment i group were performed with in-class educational games with the children in the experiment ii group. the research covered a 10-week period. in order to test the hypotheses of the research, t test for the related samples, wilcoxon signed sequence test, single factor anova analysis for unrelated samples, kruskal wallis h-test and man whitney u-test were performed. according to the analysis results of the scientific process skill test scores of the groups after the application, it was found that there was no significant difference between experiment i and experiment ii.. it was determined that there was a significant difference between the experiment i and experiment ii groups and the control group, and this difference was in favor of the experimental groups. keywords in-class educational games, digital educational games, scientific process skills 1. introduction the early childhood period covers the first six years of life from birth. children want to study everything that is around them with the urge to research and wonder about innate learning in this period. children begin to learn the concepts of science with this urge to wonder and research. science activities are very important as they help preschool children make sense of the connections between events and objects (demiriz. & ulutaş, 2001). for children, the purpose of science allows them to make sense of and realize the environment in which they live (tu & hsiao, 2008). science, which is part of everyday life, includes experiences by all individuals, including young children (saçkes et al, 2011). the foundation of children's knowledge and skills is laid (avcı & dere, 2002), preschool age is important for children to develop scientific examination, research and observation skills to learn scientific thinking (gürdal, et al, 1993). science education given at an early age enables children to notice the events in their environment and nature, perceive relationships, observe, interpret information, gain problem solving and scientific process skills (avcı & dere, 2002; howe, 1996; kuru, 2015; ravanis, 2017; şahin, 1999). scientific process skills into two types basic and integrated (american association for the advancement of science [aaas], 1993; padilla, 1990). it is important to develop basic scientific process skills in early childhood. because in order for children to master integrated scientific process skills in the future, they must first have basic skills (turiman, et al, 2012). in the preschool education program of the ministry of national education [mone] (2013), scientific process skills are also emphasizes. from this point of view, it is known that science education affects children's development in many aspects in early childhood and it is recommended that researchers start from the first years of school life (ayvacı, 2010; eshach & fried, 2005; howes, 2008; spektor-et al, 2013; tu & hsiao, 2008; watters, et al, 2001) children examine the environment and nature and want to learn, the game comes into play. game, due to the nature of science are consistent with many features. it also includes opportunities that will be the basis for learning science. osborne & brady (2001), science identified as journal of science learning article doi: 10.17509/jsl.v4i4.30620 386 j.sci.learn.2021.4(4).385-393 playful, multi-faceted, multi-disciplinary, experimental and developing. children develop their skills such as reasoning with play, making choices, focusing attention, establishing a cause-effect relationship, directing themselves to a goal. the game allows them to recognize objects, to name them and to say their functions. in addition, cognitive processes such as thinking, analysis, perception, inference, evaluation, problem solving, sorting, classification are accelerated with the game. the game is also an important tool in reflecting the knowledge and experience gained through the game to the real life. although the game is an educational activity in itself, many concepts can be gained to the child through games. the child establishes a relationship between world and the environment through play (strengthening the vocational education and training system project, 2016). the child tries to perceive the events in his environment and reflects them to the games. they learn without being aware of the game environment (paino, 2001). according to çakı (2008), since games can address more than one development area, they provided multidimensional development of children. thus, the multi-faceted development of the child can be achieved. it also provides children with skills such as questioning, problem solving, judgment, analysis, synthesis and critical thinking. in this direction, educational games prepare appropriate environments for individuals to gain cognitive skills such as being able to group, formulate hypotheses, solve problems, analyze and synthesize (çamlıyer & çamlıyer, 1997; öncü & özbay 2010). therefore, increasing the importance of educational games. with the technological developments in recent years, it is seen that the games have been transferred to the technological platform and the understanding of entertainment has changed significantly. children started to play digital games with computers, tablets and mobile phones instead of classic games therefore, the concept of digital educational game comes across in education and science education. educational digital games can be defined as games that are prepared with the help of technological tools and that provide learning for certain goals with cognitive social, behavioral or emotional aspects (aksoy, 2014;samur, 2016). almost all of the educational digital games force the learner to perform mental processing because of time constraints or racing over time. therefore, it can contribute to the cognitive development of learners. briefly games and science is a part of children's daily life. therefore, it is important to investigate their effects. in the literature, there are studies that examine the effects of different methods of pre-school children science process skills (ayvacı, 2010; büyüktaşkapu, 2010; büyüktaşkapu et al, 2012; can et al, 2017; dilek, et al, 2020; elkeey 2017; şahin, et al, 2011; tekerci & kandir, 2017; van schijndel, singer, & raijmakers, 2008). however, there weren’t studies examining the effect of science-based digital and in-class games on scientific process skills. this research aimed to examine the effect of science education performed with digital and in-class educational games on the scientific process skills of preschool children. for this purpose, the following hypotheses have been tested: h01. the scientific process skills of preschool children do not differ significantly according to the method used (educational digital games-in-class educational gamestraditional teaching method). h02. preschool children scientific process skills test sub-dimension pretest and posttest scores do not show significant difference according to the method used (educational digital games-in-class educational gamestraditional teaching method). 2. method 2.1. general background the research was used quasi-experimental design. real experimental designs are studies in while individuals are randomly placed in groups according to the levels of the independent variable (büyüköztürk, at al,2017 ). the difference between the real experimental design and the quasi-experimental design is that participants cannot be randomly assigned to groups (creswell & plano clark, 2011). in this case, it is decided randomly to be the experimental and control groups among the previously formed groups. the research was carried out with three groups (two experimental and one control group) as seen in table 1. in this research, a quasi-experimental design was used because three of the ready groups were determined as experimental and control groups. 2.2 sample the research group of the research was in a primary school located in elazığ, turkey. the research used purposeful sampling procedures. a purposeful sampling technique was used to determine the school where the participants were located. use of this sampling technique is table 1 experimental design showed used in research groups pre-tests intervention post-test e1 o1 x o4 e2 o2 x o5 c o3 o6 e1; experiment group where digital educational games and science teaching practices are applied e2; experiment group where in class educational games and science teaching practices are applied c; control group, o1-o4; pre-test and post-test measurements of experiment i group, o2-o5; pre-test and post-test measurements of experiment ii group o3-o6; pre-test and post-test measurements of control group x: it shows the independent variables (experimental variables) applied to the subjects in the experimental group. journal of science learning article doi: 10.17509/jsl.v4i4.30620 387 j.sci.learn.2021.4(4).385-393 preferred in exceptional cases that have specific criteria and characteristics (büyüköztürk, et al, 2017). two experiment and one control groups were included in the research. science education was conducted with educational digital games for the children in experiment i group. science education was conducted with in-class educational games for the children in experiment ii group. in the selection of the experimental groups, children scientific process skills test was applied to the groups. there was no statistically significant difference between preschool children scientific process skills pre-test results and the pre-test results of three groups (experiment i, experiment ii and control), f=0.53; p>.05. this finding showed that the experimental and control groups were equivalent. for this reason, the experimental and control groups were randomly determined. the study group consisted of 70 children aged 60-72 months. information on the study group is presented in table 2. it was seen that the experimental group i and the experiment group ii consisted of 23 pre-school children each. it was observed that the control group consisted of 24 children it is seen that the number of boy children participating in the study is higher than the number of girl children. it was observed that the number of children in the control group was higher than the number of children in the experimental group. however, it was observed that there wasn’t big difference in percentage. 2.3 data collection tool the scientific process skill test (spst) developed by sağırekmekçi (2016) used to measure the scientific process skills of preschool children. this test consists of a total of 20 items, with 3 items with 3 options and 17 items with 2 options. determined as this test reliability coefficient 0.93 by sağırekmekçi. the scientific process skill test aims to measure a total of 5 scientific process skills, including observation, classification, measurement, prediction and inference. while applying spst to children, a noise-free environment without interesting objects was selected and a seating arrangement was created where the researcher and the child could sit face to face, and especially the chairs where the child could sit comfortably were preferred. in the application environment, the child and the researcher alone applied test. the children were shown the items in the test and asked to make choices. the researcher made markings according to their choices. 2.4 preparation of educational games within the scope of the research, the researcher designed educational digital games through the scratch program for used in the experimental group i. the researcher designed in-class educational games to be used in the experiment ii group. digital and in-class plays designed by the researcher; balanced nutrition, state of matter, magnet gravitational force, sensory organs, natural /unnatural substances, organs in our body, living/inanimate beings and buoyancy force of water take as basis. a total of two games, educational digital and inclass educational games, have been designed for each topic by researchers. each game; concepts consisted of scientific process skills and learning process subheadings. plays; it was presented to expert (two pre-school education and three science education) opinion for its suitability to scientific process skills, materials, suitability of concepts and target words, organization of the educational environment and examination of the learning process. sixteen game (eight digital educational game and eight inclass educational game) designs completed by making the recommended and necessary corrections. in figures 1 and figure 1 in-class educational game material figure 2 digital educational game material table 2 frequency and percentage distributions of the experimental and control group children by gender groups girl boy total f % f % f % experiment group i 13 38.2 10 27.8 23 32.8 experiment group ii 9 26.5 14 38.9 23 32.8 control 12 35.3 12 33.3 24 34.4 total 34 100 36 100 70 100 journal of science learning article doi: 10.17509/jsl.v4i4.30620 388 j.sci.learn.2021.4(4).385-393 2, there are in-class and digital educational games designed for the sensory organs. 2.5 process of using digital educational games and inclass educational games the researcher spent time with the children in the research group in the classrooms of the children before starting the research. because the researcher wanted to meet and communicate with children and get to know the developmental characteristics of children before the pretest applied. in the research, scientific process skill test applied to the experimental and control groups as pre-test and post-test. the research covered period of 10 weeks period. two weeks of this period were reserved for pre-test and post-test applications. after the pre-test applications, educational digital games were used in the experimental group i and in-class educational games in the experiment ii group. science subjects transferred to the control group with the traditional teaching method by researcher. the implementation process carried out once a week, with 40 minutes for each session. in the experiment group i, game activities carried out on the computer. in the experiment ii group, the researcher prepared the necessary materials in advance and children played games. in some games the class divided into two large groups, in some games the table 3 descriptive statistics of pre-test and post-test scores of scientific process skills test of preschool students test& sub-dimension group n �̅� ss skew ness kurtosis ra nge min max shapirowilk spstpre experiment i23 11.82 3.09 -.143 -.912 10 6 17 .666 experiment ii 23 11.56 2.67 -.061 -.289 11 7 17 .592 control 24 11.62 2.76 -.221 -.643 10 6 16 .688 observation-pre experimenti23 4.30 1.25 -.336 -.369 4 2 6 .008 experiment ii 23 4.0 1.12 .000 -.654 2 2 6 .090 control 24 4.37 1.09 -.193 -490 4 2 6 .053 classificationpre experiment i 23 .91 .73 1.39 -1.00 2 2 6 .001 experiment ii 23 .56 .58 .454 -.616 2 2 6 .000 control 24 .45 .58 .873 -.114 2 2 6 .000 measurementpre experiment i23 1.91 .79 .162 -1.34 1 2 6 .000 experiment ii 23 1.91 .94 -862 .217 3 2 6 .001 control 24 1.79 .97 -.461 -.605 3 2 6 .005 guesingpre experiment i23 2.21 .95 .565 -.336 3 2 6 .003 experiment ii 23 2.26 1.00 -.288 -.185 4 2 6 .052 control 24 2.25 .84 -.527 1.224 4 2 6 .004 inferencepre experiment i23 2.56 1.16 -.173 .163 5 2 6 .164 experiment ii 23 2.82 1.33 -.279 654 5 2 6 .195 control 24 2.75 1.29 -.542 -.035 5 2 6 .039 spstpost experiment i23 17.78 1.41 -.320 -.147 5 20 15 .082 experiment ii 23 17.91 1.31 -.223 -.128 5 20 15 .178 control 24 12.58 2.46 -.783 .569 10 6 16 .148 observation-post experiment i23 5.78 .51 -2.46 5.85 2 4 6 .000 experiment ii 23 5.86 .45 -3.71 13.69 2 4 6 .000 control 24 4.83 1.00 -.196 -1.170 3 3 6 .003 classification-post experiment i23 1.60 .58 -1.21 .684 2 0 2 .000 experiment ii 23 2.00 .00 0 2 2 .000 control 24 .79 .77 .395 -1.196 2 0 2 .000 measurement-post experiment i23 2.52 .79 -1.89 3.47 3 0 3 .000 experiment ii 23 2.78 .51 -2.46 5.85 2 1 3 .000 control 24 1.91 1.01 -.631 -.578 3 0 3 .002 guesing-post experiment i23 3.78 .42 -1.46 .161 1 3 4 .000 experiment ii 23 3.78 .51 -2.46 5.85 2 2 4 .000 control 24 2.29 .80 -.062 -.500 3 1 4 .004 inference-post experiment i23 4.08 1.16 -1.12 .609 4 1 5 .000 experiment ii 23 3.47 1.23 -.41 -.461 2 2 4 .017 control 24 2.75 .98 -.919 1.328 4 0 4 .004 journal of science learning article doi: 10.17509/jsl.v4i4.30620 389 j.sci.learn.2021.4(4).385-393 games divided into smaller groups and games conducted. in this process, the first games were played to explore the concepts of science, so the winning groups were not determined. in the later games, the winning groups identified and the losing group or groups encouraged to win the next games in order to ensure their motivation. the reasons for losing groups analyzed and the groups given feedback at the end of the game and provided to complete their deficiencies. in the games, children tried to focus on learning and having fun rather than winning. 2.6 data analysis spss 22.0 statistical package program was used for all statistical analyzes to analyze the data obtained during the research. prior to the analysis, the author checked whether the pre-and post-test data applied were entered correctly into the computer. descriptive statistical analysis was conducted among variables for the experimental and control group. arithmetic means, standard deviation, kurtosis coefficient, skewness coefficient, minimum and maximum values of the data belonging to the groups were calculated. descriptive statistical results were used to have an opinion on the data and to check the assumptions before analysis. in the literature, the application of the shapiro-wilks test is recommended if the group size is less than 50 to examine the normality of the scores (büyüköztürk, 2015; rovai et al, 2014). in this direction, the shapiro-wilks test was conducted to see if the scores were normal. data on variables in experiment i, experiment ii and control groups were analyzed using single factor anova for unrelated samples, kruskal wallis h-test and man whitney u-test for unrelated measurements. 3. result and discussion two titles were created to test research hypotheses; "descriptive statistics" and "inferential statistics." 3.1 descriptive statistics the scientific process test pre-test (spstpre) scores applied to preschool children who constitute the study group of the study and the average distributions of each sub-dimension of the test were tested. the mean test scores for the group, the number of children in the groups, the standard deviation of the groups, skewness and kurtosis values, minimum and maximum values and shapiro-wilk values are given in table 3. it shows that the skewness and kurtosis values of spstpre and spstpost scores remain between the normal distribution limits (+ 1.5, -1.5). however, it was determined that the test sub-dimensions were not within the normal distribution limits (+1.5, -1.5). in addition, when shapiro-wilk values were examined, it was found that the spstpre and spstpost scores were normally distributed, but the sub-dimensions of the test did not show normal distribution (p <0.05). 3.2 inferential statistics h01. the scientific process skills of preschool children do not differ significantly according to the method used (educational digital games-in-class educational gamestraditional teaching method). one-way anova was applied to the data to investigate whether children's scientific process skills differ according to the method used and the findings are given in table 4. it is observed that there is a statistically significant difference between the average post-spst scores of preschool children according to the method used (f=38.722; p<.05; η2=.59). the result obtained for the post-test scores also supports the eta-square value. when the eta-square values obtained (η^2=.59) are taken into consideration, it can be said that digital games and in-class educational games have a broad effect on post-spst scores. the dunnett c test was applied to the data as a post hoc test in order to investigate the source of this difference between the programs as a result of the analyses. dunnett c test results are given in table 5. according to the dunnett c test result, postspst average scores showed no significant difference between experiment and experiment ii. it was found that there was a significant difference between experiment i and control group in favor of experiment i group, and there was a table 4 anova results of preschool students' bsbt post-test scores according to the method used source of variance sum of squares sd mean square f p 𝜼𝟐 between groups 1106.989 5 221.398 38.722 .000 .59 within groups 766.154 134 5.718 total 1873.143 139 table 5 dunnett c test results group(i) group(j) mean difference ( i-j) experiment i post-spst experiment ii post-spst -.13043 control post-spst 5.19928* experiment ii post-spst experiment i post-spst .13043 control post-spst 5.32971* control post-spst experiment i post-spst 5.19928* experiment ii post-spst 5.32971* journal of science learning article doi: 10.17509/jsl.v4i4.30620 390 j.sci.learn.2021.4(4).385-393 significant difference between experiment ii and control group in favor of experiment ii group. h02. preschool children' scientific process skills test sub-dimension pre-test and posttest scores do not show significant difference according to the method used (educational digital games-in-clas s educational gamestraditional teaching method). kruskal wallis test was used to examine whether the children's scientific process skills differed according to the method used and the findings are given in table 6. when table 6 is examined, it is observed that the scientific process skill test lower dimension scores of the children participating in the practice differ significantly (p< .05). this finding shows that the three methods used have different effects on children's scientific process skills. the man whitney u-test was performed to determine the source of the difference between the groups. the results of the man whitney u-test are given in table 7. when table 7 is examined, post-spst subscale mean scores showed significant difference in favor of the experimental group ii in the classification sub-dimension between the experimental i and ii groups (u=172.50, p<.05). however, observation (u = 242.50, p <.05), measurement (u = 494.00, p <.05), guesing (u = 255.00, p <.05), inference (u = 255.50, p <.05) does not show significant differences in the sub-dimensions. observation (u = 127.50, p <.05), classification (u = 123.00, p <.05), measurement (u = 176.50, p <.05), guesing (u = 36.50) between experimental i and control group, p <.05), inference (u = 104.00, p <.05) sub-dimensions in favor of the experimental group i have a significant difference. there is a significant difference between experimental group i and control group in terms of observing (u=113.00, p<.05), classification (u=57.50, p<.05), measuring (u=132.50, p<.05), guesing (u=41.00, p<.05) and estima inference (u=179.50, p<.05) in favor of experimental group i. the present study aimed to determine the effect of science teaching, which is carried out with educational digital games and in-clas educational games, on the scientific process skills of preschool children. according to the results of the research, it was seen that both types of games (educational digital games and in-class educational games) were effective in developing scientific process skills. when the differences between the groups of the scientific process skills sub-dimensions were examined, it was found that there was a significant difference between the experimental group i (educational digital games) and the experimental group ii (in-class educational games) in the classification sub-dimension in favor of experiment ii, while the other sub-dimensions did not show significant difference. in other words, it was seen that both types of games were effective on observation, classification, measurement, guessing, inferential skills. however, it is seen that the effect of in-class educational games on classification skill is greater. it was concluded that the scientific process skills of the children in the experimental group i and ii were significantly higher than the children in the control group. it was concluded that the scientific process skill test sub-dimensions differed significantly between experimental and control groups in favor of experimental groups. this may be an indication that science education with educational digital games and inclass educational games is effective in increasing scientific process skills. when the literature was examined, no researches examining the effects of educational digital games and inclas educational games on scientific process skills in preschool period were found. there are many studies in the literature that examine the impact of science activities on scientific process skills (ayvacı, 2010; büyüktaşkapu, 2010; büyüktaşkapu et al, 2012; can et al, 2017; dilek, et al, 2020;. elkeey 2017; kalemkuş, et al2021; ping, et al,2019;. soléllussà, et al, 2020; şahin, güven, & yurdatapan, 2011; tekerci & kandir, 2017; uludağ, & erkan, 2020; van schijndel, singer, & raijmakers, 2008). in their research, van schijndel, singer, & raijmakers (2008) examined the impact of a science program on the research skills of twotable 6 results of kruskal wallis test of post-spst sub-dimensions according to applied method sub-dimension group n rank average sd χ2 p significant difference observation experiment i 23 41.02 2 22.44 .000 .000 experiment ii 23 43.54 2 control 24 22.50 2 classification experiment i 23 38.15 2 32.41 .000 .000 experiment ii 23 49.00 2 control 24 20.02 2 measurement experiment i 23 37.80 2 13.08 .001 .001 experiment ii 23 43.76 2 control 24 25.38 2 guesing experiment i 23 45.52 2 41.95 .000 .000 experiment ii 23 46.11 2 control 24 15.73 2 inference experiment i 23 46.35 2 14.82 .001 .001 experiment ii 23 36.33 2 control 24 24.31 2 journal of science learning article doi: 10.17509/jsl.v4i4.30620 391 j.sci.learn.2021.4(4).385-393 to three-year-old preschoolers. after applied the program, it was observed that there was an increase in the development of the research skills of the children in the experimental group and the related scientific process skills. ayvacı (2010) conducted a research that determined the impact of planning activities suitable for pre-school children on the development of children's scientific process skills. it has determined that children's ability to use scientific process skills can be improved with appropriate activities. mcfarlin (2011) conducted a qualitative study in which children attending a science-based kindergarten observed the use of scientific process skills in their daily activities in informal environments. as a result of the research, it was observed by the researcher that the children used their basic scientific process skills during play activities. sola, & oladayo, (2017) examined the scientific process skills that primary school children use while playing science games. as a result of the research, it was seen that primary school children used their scientific process skills while playing science-based games thus, science-based games can be said to be effective in improving the scientific process skills. in this context, these studies are in line with the current research results. at the high school level, limited studies have been found examining the impact of educational games and digital games on scientific process skills (yıldırım, 2018). the effects of physical activity based games and digital games method on the scientific process skills of 9th grade children were investigated. this research it has been observed that the scientific process skills of the children in the experimental groups are improved by using game-based methods, whether digital and physical activity or not. research data parallels the results of this research. considering the researches in the literature, it is seen that the methods and techniques used within the framework of science education have a significant effect on children's scientific process skills. in the research conducted, it is observed that the planned and systematic execution of the stimuli that children are exposed to in the tablo 7 u-test for post-spst sub-dimensions group sub-dimensions group rank mean rank total u p r e x p e ri m e n t i e x p e ri m e n t ii observation experiment i post -test 22.54 518.50 242.50 .408 .121 experiment ii post-test 22.46 562.50 classification experiment i post -test 19.50 448.50 172.50 .002 .453 experiment ii post-test 27.50 632.50 measurement experiment i post test 21.48 494.00 494.00 .183 .196 experiment ii post-test 25.52 587.00 guesing experiment i post -test 23.11 531.50 255.50 .774 .042 experiment ii post-test 23.89 549.50 inference experiment i post -test 26.87 618.00 187.00 .74 .263 experiment ii post-test 20.13 463.00 e x p e ri m e n t i c o n tr o l observation experiment i post-test 30.48 701.50 127.00 .000 .518 control post-test 17.79 427.00 classification experiment i post -test 30.65 705.00 123.00 .000 .509 control post test 17.63 423.00 measurement experiment i post-test 28.33 651.50 176.500 .022 .335 control post-test 19.58 476.50 guesing experiment i post-test 34.42 791.50 36.50 .000 .785 control post-test 14.01 336.50 inference experiment i post-test 31.48 724.00 104.00 .000 .552 control post-test 16.83 404.00 e x p e ri m e n t ii c o n tr o l l observation experiment ii post -test 31.09 715.00 113.00 .000 .582 control post-test 17.21 413.00 classification experiment ii post -test 33.50 770.50 57.50 .000 .772 control post-test 14.90 357.50 measurement experiment ii post -test 30.24 695.50 132.50 .000 .500 control post-test 18.02 432.50 guesing experiment ii post -test 34.22 787.00 41.00 .001 .772 control post-test 14.21 341.00 inference experiment ii post-test 28.20 648.50 179.50 .032 .312 control post-test 19.98 479.50 journal of science learning article doi: 10.17509/jsl.v4i4.30620 392 j.sci.learn.2021.4(4).385-393 early period increases the scientific process skills of children. what, how and how much a child can discover is related to how learning environments are offered to her/him (mone, 2013). among the findings obtained in the research and the research results presented above; it is seen that there is a parallelism in that science applications are effective in increasing the scientific process skills of children. 4. conclusion it is important that children who are adults of our future receive science education at an early age, have science literacy and scientific process skills. thus, children will be able to be more successful and more productive individuals in solving the problems they will encounter in their future lives. they can be people who investigate, investigate and question. this study showed that the use of in-clas and digital games in science education had an impact on children's scientific process skills. considering the results of the research, educators will be in the best interest of the children to integrate the games (digital, in-class) children enjoy with science. it has been observed that there are few studies in the literature examining the science-related effects of digital and in-clas games. the effects of sciencerelated digital and in-class games should be further investigated. acknowledgment this research was conducted from the related parts of master thesis entitled “the effects of science education provided with digital and in-class games on the scientific process skills and cognitive development levels of preschool students”. this research was supported by fırat university scientific research projects unit. references aksoy, n.c. (2014). the effect of digital game-based mathematics teaching on secondary school 6th grade students' achievement, motivation, self-efficacy and attitude characteristics. phd thesis, gazi university, institute of educational sciences, turkey. aaas (american association for the advancement of science) (1993). benchmarks for scientific literacy: a project 2061 report. new york: oxford university press. avcı, n. & dere, h. (2002, 16-18 april). preschool child and math. national science and mathematics education congress, ankara, turkey. ayvacı, h.ş. (2010). a pilot study to improve the ability of preschool children to use scientific process skills. necatibey faculty of education, electronic journal of science and mathematics education, 4(2), 1-24. büyüköztürk, s. (2015). data analysis manual for social sciences (21. edition). ankara: pegem academy. büyüköztürk, ş., çakmak, e.k., akgün, ö.e., karadeniz, ş., & demirel, f. (2017). scientific research methods. ankara: pegem academy. büyüktaşkapu, s. (2010). a constructivist approach to developing scientific process skills for children aged 6. doctora thesis, selçuk university, institute of social sciences, konya, turkey. büyüktaşkapu, s., çeliköz, n., & akman, b. (2012). [the impact of the constructivist science education program on the scientific process skills of 6-year-olds. education & science , 37(165), 275-292. https://doi.org/10.24106/kefdergi.2297 can, b., yıldız-demirtaş, v., & altun, e. (2017). the effect of projectbased science education programme on scientific process skills and conceptions of kindergarten students. journal of baltic science education, 16(3),395-413. çakı, e. (2008). drama technique as a method in fifth grade religious culture and ethics course. master's thesis, çanakkale on sekiz mart university, institute of social sciences, çanakkale, turkey. çamlıyer, h. & çamlıyer, h. (1997). children's motion education and play in educational integrity. can ofset publisher. demiriz, s. & ulutaş, i̇., (2001). determination of science and nature activities and related practises in preschool education ınstitutions, iv. congress of science education, proceeding book, 89-90. dilek, h., taşdemir, a., konca, a. s., & baltaci, s. (2020). preschool children’s science motivation and process skills during inquirybased stem activities. journal of education in science environment and health, 6(2), 92-104. elkeey, s. s. (2017). developing science process skills and some of accompanying skills through observation of life cycle of silkworm by kindergarten child. the online journal of new horizons in education, 7(1), 53-63. eshach, h. & fried, m. (2005). should science be taught in early childhood? journal of science education and technology, 14(3), 315-336. https://doi.org/10.1007/s10956-005-7198-9 gürdal, a., çağlar, a., şahin, f., ökçün, f., & macaroğlu, e. (1993). examples of pre-school science activities. 9. ya-pa pre-school education and non-formal seminar. howe, a.c. (1996). development of science concepts within a vygotskian framework. science education, 80(1), 35-51. https://doi.org/10.1002/(sici)1098237x(199601)80:1<35::aid-sce3>3.0.co;2-3 howes, e. v. (2008). educative experiences and early childhood science education: a deweyan perspective on learning to observe. teaching and teacher education, 24(3), 536-549. http://dx.doi.org/10.1016/j.tate.2007.03.006 kalemkuş, j., bayraktar, ş., & çiftçi, s. (2021) comparative effects of argumentation and laboratory experiments on metacognition, attitudes, and science process skills of primary school children. journal of science learning, 4(2).113-122. https://doi.org/10.17509/jsl.v4i2.27825 kuru, n. (2015). investigation of the relationship between scientific process skills and mathematics concepts of 48-66 month old children. unpublished master's thesis, hacettepe university, institute of educational sciences, ankara, turkey. ping, i. l. l., halim, l., & osman, k. (2019). the effects of explicit scientific argumentation instruction through practical work on science process skills. jurnal penelitian dan pembelajaran ipa, 5(2), 112-131. http://dx.doi.org/10.30870/jppi.v5i2.5931 mcfarlin, l. m. (2011). how children in a science-centered preschool use science process skills while engaged in play activities. unpublished doctoral dissertation, the university of texas at austin, united states ministry of national education-mone. preschool education program. ankara: ministry of national osborne, m.d. & brady, d.j. (2001). constructing a space for developing a rich understanding of science through play. journal of curriculum studies, 33(5), 511–524. https://doi.org/10.1080/00220270120863 öncü, e. ç., & özbay, e. (2010). game for early children. kök publisher. turiman, p., omar, j., daud, a. m., & osman, k. (2012). fostering the 21st century skills through scientific literacy and science process skills. procedia-social and behavioral sciences, 59, 110-116. https://doi.org/10.1016/j.sbspro.2012.09.253 padilla, m. (1990). the science process skills. paper 9004 in the series, science matters-to the science teacher, published by the national association for research in science teaching. retrieved apr 25, 2021, from https://narst.org/research-matters/science-processskills. patrick, h., mantzicopoulos, p., & samarapungavan, a. (2009). motivation for learning science in kindergarten: is there a gender journal of science learning article doi: 10.17509/jsl.v4i4.30620 393 j.sci.learn.2021.4(4).385-393 gap and does integrated inquiry and literacy instruction make a difference. journal of research in science teaching, 46(2), 166–191. https://doi.org/10.1002/tea.20276 paino, p. (2001). games students play. science teacher, 68 (4), 28-30. ravanis, k. (2017). early childhood science education: state of the art and perspectives. journal of baltic science education, 16(3), 284-288 rovai, a.p, baker, j.d., & ponton, m.k. (2014). social sci. research design and statistics: a practitioner’s guide to research methods and ibm spss analysis. watertree press llc. samur, y. (2016). digital game design. pusula publisher. saçkes, m., trundle, c.k., bell, r.l., & o’connell, a.a. (2011). the ınfluence of early science experience in kindergarten on children’s ımmediate and later science achievement: evidence from the early childhood longitudinal study. journal of research in science teaching, 48(2), 217-235. https://doi.org/10.1002/tea.20395 sağırekmekçi, h. (2016). the effect of science and nature activities based on prediction-observationunpublished master's thesis, mustafa kemal university, institute of sciences, hatay, turkey. solé-llussà, a., aguilar, d., & ibáñez, m. (2020). video-worked examples to support the development of elementary students’ science process skills: a case study in an inquiry activity on electrical circuits. research in science & technological education, 68(6), 34253448.. https://doi.org/10.1080/02635143.2020.1786361 sola, a. o., & oladayo, a. (2017). assessment of science process skills inherent in the play activities of primary school pupils in osun state, nigeria. international journal of arts & sciences, 10(2), 125-135. spektor-levy, o., baruch, y.k., & mevarech, z. (2013). science and scientific curiosity in pre-school -the teacher's point of view. international journal of science education, 35(13), 2226-2253. https://doi.org/10.1080/09500693.2011.631608 svet. (2016). strengthening the vocational education and training system project network site. ministry of national education, ankara şahin, f. (1999). examples of scientific thinking and activity in preschool marmara university kindergarten/kindergarten teacher handbook, i̇stanbul, ya-pa publications. şahin, f., güven, i̇., & yurdatapan, m. (2011). the effect of projectbased educational practices on the development of scientific process skills in preschool children. marmara university atatürk education faculty journal of educational sciences, 33, 157-176. tekerci, h., & kandir, a. (2017). effects of the sense-based science education program on scientific process skills of children aged 60–66 months. eurasian journal of educational research, 17(68), 239254. tu, t. & hsiao, w. (2008). teacher–child verbal interactions in preschool science teaching. electronic journal of science education,12(2), 1-23. van schijndel, t.j.p., singer, e., van der maas, h., & raijmakers, m. (2010). a sciencing programme and young children’s exploratory play in the sandpit. european journal of developmental psychology, 7(5), 603–617. https://doi.org/10.1080/17405620903412344 watters, j. j., diezmann, c. m., grieshaber, s. j., & davis, j. m. (2001). enhancing science education for young children: a contemporary initiative. australian journal of early childhood, 26(2), 1–7. yıldırım, z. (2018). comparison of the effects of physical activity-based games and computer games on 9th grade students' success in physics (force, laws of motion of newton and friction force) and their scientific process skills. master thesis, dicle university institute of educational sciences, diyarbakır, turkey. a © 2021 indonesian society for science educator 151 j.sci.learn.2021.4(2).151-159 received: 21 october 2020 revised: 22 february 2021 published: 24 march 2021 the effect of inquiry-based collaborative learning and inquiry-based online collaborative learning on success and permanent learning of students nurullah korkman1*, mustafa metin2 1republic middle school, yozgat, turkey 2department of mathematics and science education, faculty of education, erciyes university, kayseri, turkey *corresponding author. nkorkman@yahoo.com abstract this study aims to specify the effect of inquiry-based collaborative learning and inquiry-based online collaborative learning methods on students' success and permanent learning. in this quantitative study, a pre-test post-test non-equalized control group model of quasi-experimental design has been used. the study's sample comprises 64 students, 32 in the experimental and 32 in the control group, determined by the stratified sampling method. the research has taken place in four stages. firstly, an achievement test for chemical bonds subject has been applied as a pre-test to both experimental and control groups. the researchers have developed the test; it consists of 33 questions. secondly, chemical bonds have been taught to control groups with inquiry-based collaborative learning methods and experimental groups with inquiry-based online collaborative learning methods. the achievement test has been applied to both groups as a post-test in the third stage. the same test has been applied as a retention test six weeks later. in the analysis of tests, dependent and independent samples t-test have been used in p=.05 significance level. the research results show that the inquiry-based online collaborative learning method is more effective on students' success and permanent learning than other methods. keywords inquiry-based learning, online learning, collaborative learning, inquiry-based collaborative learning, inquiry-based online collaborative learning, permanent learning, chemical bonds 1. introduction inquiry-based learning, one of the main science education methods, aims to solve a problem they encounter by doing the necessary research (wood, 2013). therefore, inquiry-based learning contributes to students developing and using higher-order thinking skills such as search, query, criticism, correlate, and analysis (bybee, 2000; perry & richardson, 2001; hofstein & lunetta, 2004; duban, 2008). besides, it is reported in the literature that inquiry-based learning makes a significant contribution to students' academic success and higher-order cognitive features such as scientific process skills and self-efficacy (wilder & shuttleworth, 2005, duban, 2008; seyhan, 2008; akben, 2011; ulu, 2011; kocagül, 2013). besides intending to have students solve a daily-life problem by doing research (jorgenson, cleveland & vanosdall, 2004; wilder & shuttleworth, 2005), inquiry-based learning allow students to learn collaboratively with their peers by engaging in learning activities with them, and it contributes to students to develop some features like listening to the others and being open to different opinions. in addition to these advantages, inquiry-based learning has some limitations, and there are some issues to regard while implementing it. class management is one of the issues that must be considered during the implementation of inquiry-based learning (bayram, 2015). teachers should manage the inquiry process well. if the management is not effective, the teacher may not have the expected return from the students. moreover, if the teacher cannot prepare the questions that s/he will use during the inquiry, this will cause the process to go on differently from the purpose (campbell, zhang & neilson, 2011). furthermore, implementing inquiry-based learning in crowded classrooms may lead the process to get out of control and hinder each student's participation in the process (kocagül, 2013). some dominant students may also prevent others journal of science learning article doi: 10.17509/jsl.v4i2.29038 152 j.sci.learn.2021.4(2).151-159 from participating in the inquiry process, and as a result, they may display negative behavior against the course (campbell, zhang & neilson, 2011). therefore, the inquiry process should be planned and managed well. in planning and managing, the online learning method may be beneficial (günbatar, 2014). online learning is referred to in the literature as elearning, web-based learning, and distance learning. while online learning is named differently, all these concepts have a learning environment supported by a computer network infrastructure. during instructional activities in online learning, both student-student and student-teacher interaction may be seen (çalışkan, 1999; çalışkan, 2001). the online learning method extinguishes the physical environment's dependence during instructional activities and moves student-student and student-teacher interaction into online learning environments. thus, computer and online environments become a part of educationalinstructional activities (de wever, schellens, valcke & van keer, 2006; caspi & blau, 2008). wang (2008), cited in akgün (2012) states, thanks to computer-assisted/online learning, students can learn new things and share their experiences by communicating with their teachers and peers online or offline. online learning is widely used in science education as it makes searching the information more accessible and increases the chance for concrete experiences while learning. despite many benefits, online learning also has some limitations. online learning is limited for the reliability of the assessment and evaluation process, and it cannot reveal some positive outcomes of face-to-face interaction. also, establishing an effective student-teacher interaction in online learning takes longer than traditional learning, and students need to motivate themselves and study regularly (kaya, 2002). therefore, online learning may be used with collaborative learning, which increases face-to-face interaction to decrease its limitations and have a more effective educational-instructional process. in collaborative learning, students work in heterogeneous groups for a common purpose and are responsible for each other's success (demirel, 2006). contributing to students' socialization, developing their communication skills, and forming an in-group interaction are essential characteristics of collaborative learning (demirel, 2006; ünlüsoy, 2006). besides, collaborative learning increases student motivation and develops their features such as discussion, debate, criticizing ideas, respecting others' ideas, tolerating and shared decision making by providing a social environment (serrano & pons 2007). using collaborative learning leads to positive changes in students' characteristics like success and attitude (yapıcı, hevedanlı & oral, 2009; topsakal, 2010). thanks to these traits, collaborative learning is widely used in science education and inquiry-based learning, and online learning (bakanlığı, 2005; meb, & başkanlığı, 2006). apart from providing students many positive changes, using inquiry-based learning, collaborative learning, and online learning in science education helps students have concrete experiences. because science education consists of abstract concepts, concretizing these abstract concepts are rather crucial in science education. the chemical bond is one of the abstract subjects in science education. chemical bonds are among the chemistry course's fundamental subjects, and their great importance in learning many scientific concepts at the high school and university levels (ritter, 2007). this subject consists of abstract concepts such as atom and subatomic particles like proton, neutron, and electron, which are difficult to observe in the classroom environment. also, ionic bond, which is based on the exchange of electrons, and covalent bond, which occurs with the everyday use of electrons between the atoms that are prone to take an electron, are considered as a challenging subject to understand by both teachers and students (şen & yılmaz, 2013). while instructing this subject, abstract concepts like electron exchange and electron cooperation need to be concretized. therefore, inquiry-based learning, collaborative learning, and online learning may be used while teaching chemical bonds to concretize the abstract concepts and increase students' success. recently, there is a tendency to use inquiry-based learning, collaborative learning, and online learning together in science education because of their limitations and advantages. that is seen in the relevant literature. it is seen in the literature that the studies carried out with inquiry-based learning methods analyzed the variables like success, self-efficacy, attitude, and scientific process skills. gül (2011) found that inquiry-based learning affects students' success positively. gençtürk & türkmen (2007), ulu (2011), sağlam (2012), atar & atar (2012), yazgan (2013) and sarı & güven (2013) also had similar results. likewise, akben (2011) investigated the effect of inquirybased learning on self-efficacy and concluded that inquirybased learning had positive impacts on students' selfefficacy. kocagül (2013) and gezer (2014) found similar effects of inquiry-based learning on self-efficacy. some studies examine the effect of inquiry-based learning on students' attitudes. duban (2008) asserts that inquiry-based learning implemented in fifth grade positively affected their attitudes towards science courses. yaşar & duban (2009), akben & köseoğlu (2010), akben (2011) and yazgan (2013) report similar results. in addition to these, ulu (2011), akben (2011), kocagül (2013), and gezer (2014) investigated the effect of inquiry-based learning on scientific process skills, and they found positive results. therefore, it is possible to say that inquiry-based learning contributes significantly to success, self-efficacy, attitude towards the course, and scientific process skills. the studies carried out with online learning show that it increased students' success (bodur, 2010; coşkun, 2013; journal of science learning article doi: 10.17509/jsl.v4i2.29038 153 j.sci.learn.2021.4(2).151-159 başarmak, 2013; sarı, 2013), and made a positive contribution to their attitudes towards the course (bodur, 2010; coşkun, 2013; yılmaz & top, 2015). moreover, some studies use online learning and inquiry-based learning, and collaborative learning and examine learning effects. for instance, günbatar (2014) used online learning and inquiry-based learning together in the study and found that online inquiry-based learning had a statistically significant effect on motivation and learning strategies. the studies in which online learning was used and collaborative learning reported that students' success increased (korkmaz, 2013; esgin & saraç, 2015), and students had positive attitudes (korkmaz, 2013). it is seen in the literature that collaborative learning had positive impacts on success (bozkurt, orhan, keskin & mazi, 2008; gök & sılay, 2008; gök & sılay, 2009; yapıcı, hevedanlı & oral, 2009; doğan, uygur, doymuş & karaçöp, 2010; özdilek, erkol, doğan, doymuş & karaçöp, 2010; topsakal, 2010; aksoy & doymuş, 2011; aktaş, 2013; güngör & özkan, 2013). it also makes a statistically significant contribution to the attitudes of students (yapıcı, hevedanlı & oral, 2009; topsakal, 2010; güngör & özkan, 2011). in addition to these, collaborative learning had positive effects on laboratory skills (aksoy & doymuş, 2011), retention (özdilek, erkol, doğan, doymuş, & karaçöp, 2010), and problem-solving skills and motivation for success (gök & sılay, 2009). some studies use collaborative learning with inquiry-based learning. they show that the sample group displayed a scientific approach to determining environmental consciousness (gülin, 2010), and the reading comprehension skills of students developed (yılmaz & top, 2015). although there exist some studies about the effect of inquiry-based learning (gençtürk & türkmen 2007; ulu, 2011; sağlam, 2012; atar & atar 2012; yazgan, 2013; sarı & güven, 2013) and inquiry-based collaborative learning (gülin, 2010; yılmaz & top, 2015) on success, inquirybased online collaborative learning studies are limited (chang, sung & lee 2003; salovaara 2005; abdelraheem & asan 2006; linn, lee, tinker, husic, & chiu, 2006; kollar, fischer, & slotta, 2007; mäkitalo-siegl, kohnle, & fischer, 2011; sun, looi & xie, 2014; sinha, rogat, adamswiggins & hmelo-silver, 2015) in the literature. also, these studies do not entirely reveal the effect of implementations on success and permanent learning. they do not aim to determine whether inquiry-based collaborative learning or inquiry-based online collaborative learning is more effective on success and permanent learning. this study will compare inquiry-based collaborative learning and inquiry-based online collaborative learning and reveal which one is more effective on success and permanent learning. because the studies that use different methods and compare the effectiveness of those methods are rare in the literature, this study will substantially contribute to the literature. moreover, the study is different from other studies carried out in similar subjects as it implements the three frequently used methods in science education. this study also presents an example for the following studies by designing and implementing inquiry-based online collaborative learning. moreover, this study will guide the researchers who will use new learning methods in science education together. the fact that the studies that jointly investigate the effects on success and permanent learning are rare shows that this study will significantly contribute to the literature. in this context, the problem of this study is "what are the effects of inquiry-based collaborative learning and inquiry-based online collaborative learning, which was implemented in chemical bonds subject of structure and features of matter unit in 7th grade, on students' success and retention of learning?" the subproblems are: (a) is there a significant difference between the success of students who used inquiry-based collaborative learning and inquiry-based online collaborative learning? (b) is there a significant difference between the retention of students who used inquiry-based collaborative learning and inquirybased online collaborative learning? this study's general aim is to determine the effect of inquiry inquiry-based collaborative learning and inquiry-based online collaborative learning on students' success and retention of learning in chemical bonds subject of structure and features of matter unit in 7th grade. 2. method the study aims to determine the effect of inquiry-based collaborative learning and inquiry-based online collaborative learning on students' success and permanent learning. this study employs a quantitative research approach, and because dependent and independent variables have been used, the experimental method has been thought to be appropriate for the study. using a systematic method, experimental research is carried out to see how an individual intervention effectively solves a particular problem under controlled conditions (özmen, 2014, creswell, 2012). therefore, this is an experimental study because an intervention has been performed to see the effect of inquiry-based collaborative learning and inquiry-based online collaborative learning on students' success. besides, a quasi-experimental design has been used because the study has been carried out in an educational setting, and it is impossible to define the groups randomly at schools. quasi-experimental designs are subordinate to true experimental designs in terms of scientific value, and they can be preferred by paying attention to their limitations when true experimental designs are impossible to use (büyüköztürk, 2011; creswell, 2012). because the study aims to determine the students' level before the intervention and the change in their success after the intervention process, a pre-test post-test non-equalized control group model of quasi-experimental design has been used. journal of science learning article doi: 10.17509/jsl.v4i2.29038 154 j.sci.learn.2021.4(2).151-159 2.1. population and sample the population of the study is comprised of 7th-grade students of 14 secondary schools in yozgat. the sample has been chosen from the population by using a simple sampling method. in a simple sampling method, each individual or object in the population has an equal chance to be chosen. this method is accepted as moderate-good for the study's validity and reliability, and it is regarded as the simplest, the easiest, and the most reliable method (akarsu, 2016). firstly, a school has been randomly chosen from the population of the study. this school has four 7th grade classes and 128 students in them. one of these classes was randomly assigned as an experimental group and another as the control group. the study sample consists of 64 students, 32 in the experimental group and 32 in the control group. there are 20 females and 12 males in the control group and 17 males and 15 females in the experimental group. 2.2. implementation process implementation process of control group: there are 20 female and 12 male students in the control group. they have been separated into seven heterogeneous collaborative groups in the study's preparation phase according to their science and technology grades by using a stratified sampling method. the students have been informed about the practices of collaborative learning and inquiry-based learning. while performing the practices during the research process, inquiry-based learning activities have been integrated with collaborative learning activities. before the instructional process, the achievement test was implemented to the control group as the pre-test. then, students have settled in the class with their groups. the learning environment of the control group was shown in figure 1. after having students involved in groups for collaborative learning, inquiry-based learning activities have been performed. the students have tried to solve six worksheets as a group appropriate for inquiry-based learning and are about the "chemical bonds" subject in the "structure and features of matter" unit. in the first and second sheets called "salt of the meal" and "incombustible dress," ionic bonds are thought to students. in the other two sheets, "let's make a better one" and "find the antidote," covalent bonds are thought. the last two sheets, "solve the relationship" and "who is a friend to who," consist of students' activities to distinguish ionic and covalent bonds. each of these worksheets has been given to students in different courses. they have studied them collaboratively with the researcher's facilitation for skills such as communication, interaction, and exchange of ideas between students. group members have chosen ahead of the group, s/he has led the in-group work distribution during the study of sheets and ensured that each member had had the opportunity to speak. after taking the opinions of all the members, the group decision has been written on the worksheets. during the activities, one of the researchers has guided and helped students in problematic situations and provided the study to go on its standard procedure. after solving the worksheets, the researcher has gathered them. this process has taken six weeks. after the instruction of chemical bonds, the achievement test has been implemented as a post-test to the control group. six weeks after the post-test, the same test has been implemented as a retention test to see the method's effect on permanent learning. implementation process of control group: there are 15 female and 17 male students in the experimental group. they have been separated into nine heterogeneous collaborative groups in the study's preparation phase using a stratified sampling method. before group works, an online learning environment has been introduced to students, and students have practiced it. after eliminating the troubles, preliminary preparation has been completed. moreover, study hours have been determined with group members before the implementation. as the implementation is online, a class design has not been formed. therefore groups and group members have not been in the same environment. after having an agreement on every issue, the implementation process has begun. firstly, an achievement test has been applied to the experimental group as a pre-test before the implementation. students have then signed into the moodle system in pre-determined hours with their usernames and passwords given by the researcher. students have entered the course page of that day and accessed the private chat rooms for their moodle system group. all the group members have studied inquiry-based learning sheets together. for the control groups, experimental groups have also had six worksheets, and they studied one of them online and collaboratively each week. the researcher has guided and helped the groups in every phase of the implementation. students have tried to solve the sheets by figure 1 learning environment of control group journal of science learning article doi: 10.17509/jsl.v4i2.29038 155 j.sci.learn.2021.4(2).151-159 discussing with group members and searching on the internet. the head of the groups has ensured that each member has participated in the study and had an opportunity to present his/her idea. the communication between group members during the implementation has been performed in chat rooms. the students have been informed about the practices of collaborative learning and inquiry-based learning. while performing the practices during the research process, inquiry-based learning activities have been integrated with collaborative learning activities. before the instructional process, the achievement test was implemented to the control group as the pre-test. then, students have settled in the class with their groups. after having students involved in groups for collaborative learning, inquiry-based learning activities have been performed. after taking all the members' opinions, the group decision has been written on the chat screen. the group members' activities have been written on worksheets and given to the researcher by the groups' heads. during the implementation, heads of the groups have led the groups, and they have tried to prevent group members from doing different activities. they have allowed each member to state opinions and ensure the group work to go on in its standard procedure. the researcher has been online during group works, visited chat rooms, promoted and encouraged students for group work, and motivated them for collaborative study. online group works have been in four sessions and taken six-course hours. after the instructional process, the achievement test has been implemented as a post-test to the experimental group. six weeks after the post-test, the same test has been implemented as a retention test to see the effect of inquiry-based online collaborative learning on permanent learning. 2.3. data collection tool the researchers have prepared the chemical bonds achievement test used in this study according to the procedures metin (2016) stated for the achievement test development process. in the first phase, the aim of the achievement test has been determined. in this study, the test has been used before the implementation, right after the completion of implementation, after passing a particular time to determine students' readiness, learning levels, and permanent learning. in the second phase, the test's content has been designated. a table of specifications has been prepared by examining the chemical bonds subject's objectives in the 7th-grade science and technology curriculum. in the table of specifications, there have been a total of 40 questions for the achievement test; seven questions in remembering level, 12 questions in understanding level, 11 questions in applying level, and six questions in analysis level. in the third phase, 40 multiple choice test items have been analyzed by experts for validity, reliability, comprehensibility, grammar, spelling errors, scientific convenience of the test, and test items for the level of the students. after the expert opinions, the final test has had 36 items. in the fourth phase, the test has been implemented to 100 students for item analysis. in this process, true answers were coded as "1", false and unanswered items were coded as "0". the results have been ordered from the highest to lowest, and 27% of the highest scores were specified as a supergroup and 27% of the lowest ones as a subgroup. following criterion have been used in item analysis: the items that have "0" or negative distinctiveness are excluded from the test; the item is considered as very well and does not need correction if its distinctiveness is 0.40 and higher; the item is considered as good and does not need correction between 0.40 and 0.30; the item can be used without correction or by correcting between 0.30 and 0.20, and the item should be prepared again or excluded if its distinctiveness is lower than 0.20. for item difficulty index; 0.00-0.20 is considered as very difficult, 0.21-0.40 as difficult, 0.41-0.60 as moderate difficult, 0.61-0.80 as easy, and 0.81-1.00 as very easy (metin, 2016). after the item analysis, the 17th, 22nd, and 23rd items were excluded from the test as their difficulty and distinctiveness indexes were not acceptable. in the last phase, the test's reliability was calculated using kr20 reliability co-efficient, and this value was found to be 0.88. this result shows that the achievement test is reliable. after the validity and reliability processes, the final achievement test had 33 items. 2.4. data analysis the data obtained using the achievement test as a pretest, post-test, and retention tests have been analyzed using the spss program. because the groups have been randomly assigned before the implementation and the data collection tool is an interval scale, a t-test has been used in data analysis. the kolmogorov-smirnov test was applied to determine whether the data had a normal distribution, and results showed that they presented a normal distribution (p>0,05). levene test was used for testing the homogeneity of variances of experimental and control groups. because the levene test result was higher than 0.05, it was specified that both groups were equal. independent samples t-test was used to compare the experimental and control groups pre-test, post-test, and retention test results. paired samples t-test was used to analyze the pre-test post-test and posttest retention test average point differences of each group's test results. the level of significance was considered as p=.05. to comment on a test result, considering only the significance level is not sufficient. the result may be meaningful, but its effect may below. effect size is journal of science learning article doi: 10.17509/jsl.v4i2.29038 156 j.sci.learn.2021.4(2).151-159 calculated in different ways in different tests, and cohen's d was calculated in this study. effect size is considered as; low between 0-0.2, moderate around 0.5, and high 0.,8, and above. the data obtained in this study have been interpreted by considering correlation, mean, standard deviation, p-value, and cohen's d. 3. result and discussion this study compares the effect of inquiry-based collaborative learning and inquiry-based online collaborative learning methods on students' success and permanent learning in chemical bonds subject. therefore, pre-test, post-test, and retention test results between groups and the results of pre-test and post-test in-groups have been compared. the independent samples t-test applied to pre-tests of experimental and control groups have been given in table 1. according to the levene test, table 1 shows no difference between the pre-test variances of experimental and control groups. however, the control group's pre-test mean is higher than the experimental group's (x ̄control= 14.88; x ̄experimental = 14.72). there is no statistically significant difference between pre-test means of experimental and control groups (t = 0.192; p> 0.05). therefore, it is possible to say that both groups were equal before the implementation of inquiry-based collaborative learning and inquiry-based online collaborative learning. the independent samples t-test applied to post-tests of experimental and control groups was given in table 2. according to the levene test, table 2 shows no difference between the post-test variances of experimental and control groups. however, the experimental group's posttest mean is higher than the control group's (xexperimetal= 22.59; xcontrol = 20.38). there is no statistically significant difference between post-test means of experimental and control groups (t = 1.277; p> 0.05). therefore, it is possible to say that the implementation of both inquiry-based collaborative learning and inquiry-based online collaborative learning has similar effects on students' success. the paired samples t-test applied to pre-tests and posttests of experimental and control groups has been given in table 3. the table shows that post-test mean of experimental group (x ̄experimental= 22.59; ss = 6.116) is higher than its pre-test means (x ̄experimental= 14.72; ss = 2.466). it has been found that the experimental group has a significant increase in post-test (texperimental = 5.896; p< 0.05), and the effect size (dexperimental= 1.79) of this increase is at a high level. the table also shows post-test mean of control group (x ̄control= 20.38; ss = 7.691) is higher than its pre-test means (x ̄control= 14.88; ss = 3.892). it has been found that control group has a significant increase in posttest (tcontrol = 4.987; p< 0.05) and the effect size (dcontrol= 1.11) of this increase is in high level. therefore, both groups have shown a significant increase after the implementation; however, post-test means show that inquiry based online collaborative learning has affected the students' success more than inquiry-based collaborative learning. table 1 independent samples t-test results of experimental and control groups pre-test pre-test results control group experimental group t p levene's test x ̄ ss x ̄ ss achievement test 14.88 3.892 14.72 2.466 0.192 0.848 0.09 table 2 independent samples t-test results of experimental and control group's post-test post-test results control group experimental group t p levene's test x ̄ ss x ̄ ss başarı testi 20.38 7.691 22.59 6.116 1.277 0.206 0.059 table 3 paired samples t-test results of experimental and control group's pre-test and post-test groups tests achievement test d x ̄ ss t p experimental group pre-test 14.72 2.466 5.896 0.00 1.79 post-test 22,59 6.116 control group pre-test 14.88 3.892 4.987 0.00 1.11 post-test 20.38 7.691 table 4 paired samples t-test results of experimental and control group's post-test and retention test groups tests achievement test d x ̄ ss t p experimental group post-test 22.59 6.116 0.896 0.210 retention test 22.13 4.696 control group post-test 20.38 7.691 1.404 0.09 retention test 19.91 7.818 journal of science learning article doi: 10.17509/jsl.v4i2.29038 157 j.sci.learn.2021.4(2).151-159 the paired samples t-test applied to post-tests, and retention tests of experimental and control groups have been given in table 4. post-test mean of experimental group (x ̄experimental= 22.59; ss = 6.116) is higher than its retention test mean (x ̄experimental= 22.13; ss = 4.696). however, a significant difference has not been found (texperimental= 0.896; p>0.05) between post-test and retention test. the table also shows post-test mean of control group (x ̄control= 20.38; ss = 7.691) is higher than its retention test mean (x ̄control= 19.91; ss = 7.818). these results show that both experimental and control groups have decreased retention tests, but it is not significant in statistical terms. in recent years, there is a tendency to use several teaching methods to bring about more meaningful and permanent learning. different methods are decided to be used together by considering their limitations and advantages. by eliminating the limitation of a method with another's superiority, more quality instructional activities are performed. this study aims to present the effects of inquiry-based learning, collaborative learning, and online learning on students' success and learning by using them together. in this context, this study aims to determine the effect of inquiry-based collaborative learning and inquirybased online collaborative learning on students' success and retention of learning. the findings show that the inquiry-based collaborative learning group and inquiry-based online collaborative learning group have homogeneous distribution. there is no significant difference between the pre-test results. this means both groups had similar features before the implementation process. according to pre-tests after the instruction of the chemical bonds subject, there has been a significant increase in both groups' post-test results. this shows that both inquiry-based collaborative learning and inquiry-based online collaborative learning methods effectively affect students' success. the literature supports these results. for instance, gülin (2010) found that inquiry based collaborative learning had positive effects on students' learning and awareness. moreover, schwarzve gwekwerer, 2007; sun & looi (2013), and sun, looi & xie (2014) presented in their studies that inquiry based collaborative learning was effective on cognitive development of students and understanding the scientific concepts. similarly, some studies show inquiry-based online collaborative learning has positive effects on students' cognitive development, success, and learning (chang, sung & lee, 2003; salovaara, 2005; abdelraheem & asan, 2006; linn, lee, tinker, husic & chiu, 2006; sun, looi & xie, 2014; sinha, rogat, adams-wiggins & hmelosilver, 2015). because both groups' post-test results are significantly higher than pre-test results, it is not clear which method is more effective. effect sizes of groups have been analyzed, and it has been found that both methods have had a highlevel effect on students. however, the effect size of inquiry-based online collaborative learning is larger than inquiry-based collaborative learning. therefore, it is possible to say that inquiry based online collaborative learning is more effective than inquiry-based collaborative learning on students' success. this result may be that in inquiry-based online collaborative learning, students have extra opportunity to search on the internet, discuss and get feedback from friends, and communicate more thanks to the online environment. in inquiry-based collaborative learning, these opportunities are limited for students, resulting in a lower effect of instruction on students. many studies (moore & kearsley, 1996; jonassen & kwon, 2001; heckman & annabi, 2005; wang & woo, 2007; birişçi, 2013;) report that online play a more active role in providing a cognitive contribution to group works. it is also stated that student discussions designed in online environments enable developing common information with different individuals' participation (olaniran, savage & sorenson, 1996; marjanovic, 1999; mcalister, ravenscroft & scanlon, 2004; watson, 2004; balaji & chakrabarti, 2010). moreover, studies report that online learning environments help students; learn autonomously on the internet or computer network, or by interacting with an instructor, extinguish the dependency on time and place concepts in reaching the information and learning, interact with their peers and instructors during instructional activities, have concrete learning experiences by providing them all the visual and audial environments that computer supplies (çalışkan, 1999; çalışkan, 2001; salovaara 2005; abdelraheem & asan 2006; gümüş, 2007; yücel, 2013). it is a fact that the opportunities that online environments provide will positively affect students' learning. for this reason, inquiry-based online collaborative learning will have more effect on students than inquiry-based collaborative learning. 4. conclusion it has been found that both methods have had a highlevel effect on students. however, the effect size of inquiry-based online collaborative learning is larger than inquiry-based collaborative learning. therefore, it is possible to say that inquiry based online collaborative learning is more effective than inquiry-based collaborative learning on students' success. there has been a decrease in both groups' retention tests; however, this is not significantly different in statistical terms. therefore, it is possible to say that both methods have similar effects on remembering knowledge. yalın (2015) states that more permanent learning occurs when students become active, and they are provided multi learning environments and when more senses are addressed journal of science learning article doi: 10.17509/jsl.v4i2.29038 158 j.sci.learn.2021.4(2).151-159 references abdelraheem, a., & asan, a. (2006). the effectiveness of inquiry-based technology enhanced collaborative learning environment. international journal of technology in teaching and learning, 2(2), 65-87. akben, n. (2011). öğretmen adayları için bilimsel sorgulama destekli laboratuvar dersi geliştirilmesi. yayınlanmamış doktora tezi, gazi üniversitesi, ankara. akben, n., & köseoglu, f. (2010). ilköğretim 5. sinif yoğunluk konusunda bilimsel sorgulamaya dayali laboratuvar etkinlik örneği. education sciences, 5(3), 1281-1289. akgün, e. (2012). durum temelli bilgisayar destekli işbirlikli öğrenme ortamına öğrencilerin katılım ve katkısı. yayınlanmamış yüksek lisans tezi, hacettepe üniversitesi, ankara. aksoy, g. & doymuş, k. (2011). fen ve teknoloji dersi uygulamalarında işbirlikli okuma yazma-uygulama tekniğinin etkisi. gazi eğitim fakültesi dergisi, 31(2), 381-397. aktaş, m. (2013). 5e öğrenme modeli ve işbirlikli öğrenme yönteminin biyoloji dersi başarısına etkisi. ahi evran üniversitesi kırşehir eğitim fakültesi dergisi (kefad), 14(3), 37-58, aralık. akarsu, b. (2016). hipotezlerin, değişkenlerin ve örneklemin belirlenmesi, m. metin (edit) kuramdan uygulamaya eğitimde bilimsel araştırma yöntemleri, s. 22-43, ankara: pegem a yayıncılık, atar, h., y., & atar, b. (2012). türk eğitim reformunun öğrencilerin timss 2007 fen başarılarına etkisinin incelenmesi. kuram ve uygulamada eğitim bilimleri, 12(4), 2621-2636. yücel, ü. a. (2013). çevrimiçi işbirlikli öğrenme ortamlarında gerçekleşen bilgi yapılandırma süreçlerinde etkileşim ve katılım. yayınlamamış doktora tezi, hacettepe üniversitesi, ankara. balaji, m.s. & chakrabarti, d. (2010). student interactions in online discussion forum: empirical research from 'media richness theory' perspective. journal of interactive online learning, 9(1),1-22 http://www.ncolr.org/jiol/issues/pdf/9.1.1.pdf adresinden 24.02.2019 tarihinde erişilmiştir. başarmak, u. (2013). karikatür animasyonuna dayalı çevrim içi öğrenme ortamının öğrencilerin başarısına, fen öğrenmeye yönelik motivasyonuna ve mizaha yönelik tutumuna etkisi. yayınlanmamış doktora tezi, gazi üniversitesi, ankara. bayram, z. (2015). öğretmen adaylarının rehberli sorgulamaya dayalı fen etkinlikleri tasarlarken karşılaştıkları zorlukların incelenmesi. hacettepe üniversitesi eğitim fakültesi dergisi, 30(2), 15-29. birişçi, s. (2013). çevrimiçi ve sınıf ortamlarında grup çalışmasına dayalı problem çözme süreçlerinin incelenmesi, yayınlanmamış doktora tezi, karadeniz teknik üniversitesi, trabzon bodur, e. (2010). ağ günlüklerinin (blogların) i̇şbirliği fen ve teknoloji dersindeki başarı ve tutumlara etkisi: i̇lköğretim ii. kademe öğrenceleri üzerine bir uygulama. yayınlanmamış yüksek lisans tezi, ege üniversitesi, i̇zmir. bozkurt, o., orhan, a., t., keskin, a., & mazi, a. (2008). fen ve teknoloji dersinde işbirlikli öğrenme yönteminin akademik başarıya etkisi. türkiye sosyal araştırmalar dergisi (tsa), 2(12), 63-78. büyüköztürk, ş. (2011). sosyal bilimler i̇çin veri analizi el kitabı, ankara: pegem akademi. bybee, r., w. (2000). "teaching science as i̇nquiry". (ed. jim. minstrell, emily. h. van zee) inquiry into inquiry learning and teaching in science. s. 20-4, american associationforthe advancement of science, washington. campbell, t., zhang,d., & neilson, d. (2011). model based ınquiry ın the high school physics classroom: an explanatory study of ımplementations and outcomes. journal science education technology, 20,(3) 258-269. caspi, a., & blau, i. (2008). social presence in online discussion groups: testing three conceptions and their relations to perceived learning. social psychology of education, 11(3), 323-346. doi: 10.1007/s11218008-9054-2. chang, k, sung, y., &. lee, c. (2003). web-based collaborative inquiry learning. journal of computer assisted learning, 19, 56-69. coşkun, e., p. (2013). bilgisayar destekli eğitimde dyned programının ilköğretim ingilizce dersini desteklemesi hakkındaki öğretmen görüşleri (kırşehir ili örneği). yayınlanmamış yüksek lisans tezi, ahi evran üniversitesi, kırşehir, türkiye. creswell, j. w. (2012). educational research: planning, conducting, and evaluating quantitative and qualitative research (4th ed.). boston, ma: pearson education, inc. çalışkan, h. (1999). bilgisayar destekli öğretimin tasarımlanmasında öğrenme bağlamı. kurgu: anadolu üniversitesi i̇letişim bilimleri fakültesi dergisi, 16, 243-255. çalışkan, h. (2001). onlıne (çevrimiçi) eğitim ve işbirliği takımlarına dayalı öğrenme. kurgu: anadolu üniversitesi i̇letişim bilimleri fakültesi dergisi, 18, 187-198. demirel, ö. (2006). öğretimde planlama ve değerlendirme (öğretme sanatı). ankara: pegem a yayıncılık. de wever, b., schellens, t., valcke, m., & van keer, h. (2006). content analysis schemes to analyze transcripts of online asynchronous discussion groups: a review. computers & education, 46(1), 6-28. doğan, a., uygur, e., doymuş, k., & karaçöp, a. (2010). i̇lköğretim 7. sınıf fen ve teknoloji dersinde jigsaw tekniğinin uygulanması ve bu teknik hakkındaki öğrenci görüşleri. erzincan eğitim fakültesi dergisi, 12(1), 75-90. duban, n. (2008). i̇lköğretim fen ve teknoloji dersinin sorgulamaya dayalı öğrenme yaklaşımına göre işlenmesi: bir eylem araştırması. yayınlanmamış doktora tezi, anadolu üniversitesi, eskişehir. esgin, e., & saraç, a. (2015). çevrimiçi ve sınıf içi grup çalışmalarının akademik başarı, tutum ve sosyal etkileşime istekliliğe etkisinin incelenmesi, sdu international journal of educational studies, 2(1), 3852. gençtürk, h., a., & türkmen, l. (2007). i̇lköğretim 4. sınıf fen bilgisi dersinde sorgulama yöntemi ve etkinliği üzerine bir çalışma. gazi üniversitesi gazi eğitim fakültesi dergisi, 27(1), 277-292. sarı, f. g. (2013). çevrimiçi i̇şbirliği uygulamaları: google dokümanlar örneği. yayınlanmamış yüksek lisans tezi, marmara üniversitesi, i̇stanbul. gök, t., & sılay, i̇. (2008). fizik eğitiminde işbirlikli öğrenme gruplarında problem çözme stratejilerinin öğrenci başarısı üzerindeki etkileri. hacettepe üniversitesi eğitim fakültesi dergisi, 34(34), 116-126. gök, t., & sılay, i̇. (2009). i̇şbirlikli öğrenme gruplarında problem çözme stratejileri öğretiminin öğrencilerin başarı güdüsü üzerindeki etkileri. kastamonu eğitim dergisi, 17(3), 821-834. gül, z. (2011). sorgulamaya dayalı öğrenme sürecinde alternatif bir araç “t diyagramı”: enzimler ve enzimlerin çalışmasına etki eden faktörler üzerinde örnek bir uygulama. yayınlanmamış yüksek lisans tezi, gazi üniversitesi, ankara. gülin, c., c. (2010). fen bilgisi öğretmen adaylarının alan gezileri ve işbirlikçi sorgulama ortamında çevre bilgisini yapılandırma ve kavramalarını araştırma, yayınlanmamış yüksek lisans tezi, ortadoğu teknik üniversitesi, ankara. gümüş, s. (2007). çevrimiçi işbirliği ekiplerinde öğrenenlerin sorun çözerek öğrenmeyle ilgili tutum ve görüşleri. yayınlanmamış yüksek lisans tezi, anadolu üniversitesi, eskişehir. günbatar, m., s. (2014). çevrimiçi ortamlarda sorgulama tipine göre farklılaşan öğrenme süreçlerinin bazı değişkenlere göre karşılaştırılması. yayınlanmamış doktora tezi, gazi üniversitesi, ankara. güngör, s., n., & özkan, m. (2012). i̇lköğretim 7. sınıf fen ve teknoloji dersindeki insan ve çevre ünitesinin işbirlikli öğrenme yöntemiyle işlenmesinin öğrenci başarısı üzerine etkisi. uludağ üniversitesi eğitim fakültesi dergisi, 25 (1), 249-258. heckman, r., & annabi, h. (2005). a content analytic comparision of learning process in online and face-to-face case study discussions. journal of computer mediated communication, 10(2). http://jcmc.indiana.edu/ vol10/issue2/heckman.html. hofstein, a., & lunetta, v. (2004). the laboratory in science education: foundation for the 21st century. science education, 88, s. 28-54. jonassen, d., h. & kwon, h., i. (2001). communication patterns in computer-mediated vs. face-to-face group problem solving. educational technology: research and development, 49 (1), 35-51. journal of science learning article doi: 10.17509/jsl.v4i2.29038 159 j.sci.learn.2021.4(2).151-159 jorgenson, o., cleveland, j., & vanosdall, r. (2004). doing good science in middle school: a practical guide to inquiry-based instruction. national science teachers association, all rights reserved, printed in the united states of america. kaya, z. (2002). uzaktan eğitim. ankara: pegema yayıncılık. kocagül, m. (2013). sorgulamaya dayalı mesleki gelişim etkinliklerinin i̇lköğretim fen ve teknoloji öğretmenlerinin bilimsel süreç becerilerine, özyeterlik ve sorgulamaya dayalı öğretime i̇lişkin i̇nançlarına etkisi. yayınlanmamış yüksek lisans tezi, dokuz eylül üniversitesi, i̇zmir. kollar, i., fischer, f., & slotta, j. d. (2007). internal and external scripts in computer-supported collaborative inquiry learning. learning and instruction, 17(6), 708-721. korkmaz, ö. (2013). böte öğretmen adaylarının çevrimiçi işbirlikli öğrenmeye dönük tutumları ve görüşleri. i̇lköğretim online, 12(1), 283-294. linn, m., c., lee, h., s., tinker, r., husic, f., & chiu, j., l., (2006). teaching and assessing knowledge integration in science. science, 313(5790), 1049-1050. mäkitalo-siegl, k., kohnle, c., & fischer, f. (2011). computersupported collaborative inquiry learning and classroom scripts: effects on help seeking processes and learning outcomes. learning and instruction, 21(2), 257–266. marjanovic, o. (1999). learning and teaching in synchronous collaborative environments. journal of computer assisted learning, 15(2), 129-138. mcalister, s., ravenscroft, a. & scanlon, e. (2004). combining interaction and context design to support collaborative argumentation using a tool for synchronous cmc. journal of computer assisted learning (special issue: developing dialogue for learning), 20(3), 194-204. moore, m.g. & kearsley, g. (1996). distance education: a systems view wadsworth, belmont, ca. olaniran, b., savage, g. &sorenson, r. (1996). experimental and experiential approaches to teaching face-to-face and computermediated group discussion. communication education, 45(3). bakanlığı, m. e. (2005). i̇lköğretim fen ve teknoloji dersi (4-5. sınıflar) öğretim programı. ankara: devlet kitapları müdürlüğü.. meb, t., & başkanlığı, t. g. k. (2006). i̇lköğretim fen ve teknoloji dersi (6.-7.-8. sınıflar) öğretim programı. meb, ankara.. metin, m. (2016). nicel veri toplama araçları m. metin (edt.). kuramdan uygulamaya eğitimde bilimsel araştırma yöntemleri, s. 376-410, ankara: pegem a yayıncılık. özdilek, k., erkol, m., doğan, a., doymuş, k., & karaçöp, a. (2010). fen ve teknoloji dersinin öğretiminde jigsaw tekniğinin etkisi ve bu teknik hakkındaki öğrenci görüşleri. erzincan eğitim fakültesi dergisi, 12(2), 209-225. özmen, h. (2014). nicel araştırma yaklaşımına dayalı yöntemler metin, m. (ed) kuramdan uygulamaya eğitimde bilimsel araştırma yöntemleri, s. 47-76, ankara: pegem a yayıncılık. perry, v. r., & richardson, c. p. (2001, october). the new mexico tech master of science teaching program: an exemplary model of inquiry-based learning. in 31st annual frontiers in education conference. impact on engineering and science education. conference proceedings (cat. no. 01ch37193) (vol. 1, pp. t3e-1). ieee. ritter, s. k. (2007). the chemical bond-whether it's sextuple bonds or bonds involving no shared electrons, chemists chase down new modes of bonding. chemical & engineering news, 85(5), 37–40. salovaara, h. (2005). an exploration of students' strategy use in inquirybased computer-supported collaborative learning. journal of computer assisted learning, 21, 39–52. sağlam, s. (2012). lisans öğrencilerinin rna teknolojileri konusundaki bilgi seviyeleri ve sorgulamaya dayalı öğrenme yaklaşımıyla sunulan materyalin etkisi. yayınlanmamış yüksek lisan tezi, gazi üniversitesi, ankara. yazgan, b.s. (2013). araştırmaya dayalı sınıf dışı laboratuar etkinliklerinin öğrencilerin araştırma sorgulama becerilerine ve çevreye karşı tutumlarına etkisi. yayınlanmamış doktora tezi, marmara üniversitesi, i̇stanbul. sarı, u., & güven, g., b. (2013). etkileşimli tahta destekli sorgulamaya dayalı fizik öğretiminin başarı ve motivasyona etkisi ve öğretmen adaylarının öğretime yönelik görüşleri. necatibey eğitim fakültesi elektronik fen ve matematik eğitimi dergisi (efmed), 7(2), 110143. serrano, j., & pons, r. (2007). cooperative learning: we can also do it without task structure. intercultural education, 18, 215-230, seyhan, h., g. (2008). kimya eğitiminde sorgulamaya dayalı öğrenci deneylerinin geliştirilmesi ve sonuçlarının tartışılması. yayınlanmamış doktora tezi, hacettepe üniversitesi, ankara. sinha, s., rogat, t.k., adams-wiggins, k.r. & hmelo-silver, c. e. (2015). collaborative group engagement in a computer-supported inquiry learning environment. international journal of computer supported collabrotive learning, 10(3), 273–307 sun, d., & looi, c.-k (2013). designing a web-based science learning environment for model-based collaborative inquiry. journal of science education and technology, 22(1), 73-89. sun, d., looi, c.-k., & xie, w. (2014). collaborative ınquiry with a webbased science learning environment: when teachers enact ıt differently. educational technology & society, 17 (4), 390–403. şen, ş. & yılmaz, a. (2013). kimyasal bağlarla ilgili fenomenografik bir çalışma. necatibey eğitim fakültesi elektronik fen ve matematik eğitimi dergisi (efmed), 7(2), 144-177. topsakal, ü. (2010). 8. sınıf ‘canlılar için madde ve enerji’ ünitesi öğretiminde işbirlikli öğrenme yönteminin öğrenci başarısına ve tutumuna etkisi. ahi evran üniversitesi eğitim fakültesi dergisi, 11(1), 91-104. ulu, c. (2011). fen öğretiminde araştırma sorgulamaya dayalı bilim yazma aracı kullanımının kavramsal anlama, bilimsel süreç ve üstbiliş becerilerine etkisi. yayınlanmamış doktora tezi, marmara üniversitesi, i̇stanbul. gezer, s.u. (2014). yansıtıcı sorgulamaya dayalı genel biyoloji laboratuvarı etkinliklerinin fen bilgisi öğretmen adaylarının laboratuvar kullanımı özyeterlik algıları, eleştirel düşünme eğilimleri ve bilimsel süreç becerileri üzerine etkisi. yayınlanmamış doktora tezi, marmara üniversitesi, i̇stanbul. ünlüsoy, m. (2006). orta öğretim fizik müfredat konularından “impuls ve momentum” konularındaki kavram yanılgılarının tespiti ve düzeltilmesinde işbirlikli yaklaşımın etkisi. yayımlanmış yüksek lisans tezi, gazi üniversitesi, ankara. wang, s., k. (2008). the effects of a synchronous communication tool (yahoo messenger) on online learners' sense of community and their multimedia authoring skills. journal of interactive online learning, 7(1), 59-74. wang, q. y. & woo, h. l. (2007). comparing asynchronous online discussions and face-to-face discussions in a classroom setting. british journal of educational technology, 38(2), 272-286. watson, c. a. (2004). exploring student collaboration: a comparison of online and face-toface groups. unpublished doctoral dissertation, indiana university, indiana. wilder, m. & shuttleworth, p. (2005). cell ınquiry: a 5e learning cycle lesson. science activities. 41(4), 37-43. wood, w.b. (2003). inquiry-based undergraduate teaching in life sciences at large research universities: a perspective on the boyer commision report. cell biology education, 2(2), 112-116. yalın, h. (2015). öğretim teknolojileri ve materyal geliştirme (28. bs.). ankara: nobel yayın evi. yaşar, ş. & duban, n. (2009). sorgulamaya dayalı öğrenme yaklaşımına yönelik öğrenci görüşleri. i̇lköğretim online, 8(2), 457-475. yapıcı, i̇., ü., hevedanlı, m., & oral, b. (2009). i̇şbirlikli öğrenme ve geleneksel öğretim yöntemlerinin tohumlu bitkiler sistematiği laboratuvarı dersine yönelik tutum ve başarıya etkisi. pamukkale üniversitesi eğitim fakültesi dergisi, 26, 63-69. yılmaz, m., & top, m., b. (2015). i̇şbirlikli tartışma sorgulama (i̇ts) stratejisinin ilkokul 4. sınıf öğrencilerinin okuduğunu anlama başarılarına etkisi. mustafa kemal üniversitesi sosyal bilimler enstitüsü dergisi, 12(30), 78-97. a doi: 10.1021/xxx.xxxx.xxxxxx © 2017 indonesian society for science educator 28 j.sci.learn.2017.1(1).28-35 received: 28 october 2017 revised: 29 november 2017 published: 30 november 2017 using inquiry-based laboratory activities in lights and optics topic to improve students’ understanding about nature of science (nos) tiara b wardani,1,2*, nanang winarno2 1putra pertiwi kindergarten and learning courses, indonesia 2international program on science education, universitas pendidikan indonesia, indonesia *corresponding author. tiara.budi.wardani@student.upi.edu abstract the aim of this study was to determine the effect of inquiry-based laboratory activities on the students’ understanding of the nature of science (nos) in learning lights and optics topic. the method used in this research is quasi experiment. sampling technique using random sampling to class and the samples were taken from grade 8 in one of junior high school in bandung. the sample was 45 students, consisting of the experimental class (n = 24 students) and the control class (n=21 students). the experimental class is taught by inquiry-based laboratory activities, while the control class is taught by noninquiry laboratory activities. the result of this research shows that the experimental class got n-gain of 0.60, while the control class got n-gain of 0.44, and both classes proven to have statistically significant different improvement. keywords inquiry based laboratory activities, nature of science, lights and optics 1. introduction indonesia national curriculum of 2013 states that teaching learning process at school should guide the students to experience five learning aspects which are being emphasized in this curriculum, they are observing, questioning, experimenting, associating, and communicating (brookes publishing, 2012). those five learning experiences which are the stages of inquiry are used by the government to support indonesia in reaching the education objectives. the mission and orientation of indonesia national curriculum of 2013 is translated in educational practice with the specific purpose so that learners or students have the necessary competences to the lives of society today and in the future. the competences are included: (1) foster religious attitude and high social ethics in the life of society, nation and state; (2) the acquisition of knowledge; (3) have the skills or the ability to apply knowledge in order to conduct scientific inquiry, problem solving, and the making of creative works related to everyday life (kementerianpendidikandankebudayaan, 2015). the ideal condition that government expected has not yet been reached. it can be seen from international comparative survey such as pisa (program for international student assessment), the score of indonesia in science is only 403, while the average score is 497 (oecd data, 2015). and the students’ skills in applying the knowledge in order to conduct scientific inquiry, problem solving, and the making of creative works related to everyday life is still also low. the poor performance of students attests to the fact that the teaching and learning have not been effective enough (olubu, 2015). it depicts lack of acquisition of the required skills which may be as a result of inadequate exposure of learners to inquiry laboratory activities. this condition is an indication of a gap in the system of teaching and learning of science in junior high school which require investigation and remediation (olubu, 2015). the above condition may occur because natural science subject is one of the subjects that has many difficulties for many students included for junior high school students (bbc news, 2005). these difficulties then could lead students to have low motivation in learning science, although actually in this school level they should have a good concept understanding about natural science in order to be their foundation for learning natural science in the next school level. it is in line with what stated by anderman mailto:tiara.budi.wardani@student.upi.edu journal of science learning article 29 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).28-35 and sinatra (2008) that the state of science education for teenagers is at an important crossroad. as one of the main part of science subject, light and optics topic is also considered as difficult topic. light and optics is a rapidly developing and often encountered its technological practices in everyday lives (tural, g., 2015). however teaching and learning the subject of lights and optics is challenging for teachers and students (tural, g., 2015). teaching the teenagers or adolescents will also emerge unique challenges for science educators, it is because in the adolescent phase students are facing the transition phase means that students are in the process of trying to understand the abstract concepts where in the previous school level or child phase they only learn the concrete one (anderman and sinatra, 2008). to solve those problems, educators should provide a form of teaching and learning activity which can attract the interest of students (tural, g., 2015). teacher could facilitate students with learning activity which engage students to think deeply about the learning material in appropriate level. the conceptual mastery of the students can be enhanced when the students were provided with “learn-by-doing” or kinesthetic modes of knowledge acquisition (olubu, 2015). learning science including physics is not only the acquisition in the form of facts, concept, principles or theory but learning will be more meaningful if the students experience or observe it directly (tural, g., 2015). a focus on meaningful learning is consistent with the view of learning as knowledge construction, in which students seek to make sense of their experiences (ausubel, 1969). science is better to be taught using experimentation, it should be taught through activity-based approach in a well-equipped laboratory learning environment (azer et al., 2013). inquiry-based learning is one approach using more student-directed, interactive methods of learning and focusing on learning how to learn (wolf and fraser, 2007). the laboratory can help the teachers to demonstrate practically some of the principles taught in theory. however, based on a survey done by the researcher in 2016, it showed that in indonesia there is still lack inquiry laboratory activity conducted to support the learning process. so, inquiry based laboratory activity is expected to have positive effects toward junior high school students’ conceptual mastery. previously, there are several researches already conducted to test the effectiveness of inquiry based laboratory method, such as a research conducted by azer et al. (2013), this research investigating the students’ conceptual mastery and attitude after treated with laboratory learning (olubu, 2015). also a research conducted by bünning (2013) which evaluate empirically the effects of experimental learning on learning achievement (bunning, 2013) and the research that conducted by olubu (2015) also investigated the effects of laboratory learning environment on students’ learning outcomes in secondary school chemistry, almost all of them showed a positive effect. such research could further suggest changes to educational standards and practices (olubu, 2015). if inquiry-based learning can improve student outcomes in physical science, then similar strategies could work in other subject areas and for other age groups. this research was initiated to evaluate the effectiveness of using inquiry-based laboratory activities among 8th grade students in terms of students’ understanding about nature of science (nos) in lights and optics topic. as well, this research also examined the differential effectiveness of inquiry activities for male and female students. 2. method the research method used in this research is quasi experiment. quasi experiment includes assignment, but not random assignment of participants to groups (creswell, 2012). creswell (2012) stated that it can be used when the researcher cannot artificially create groups for the experiment. this method is appropriate with the purpose of the research which is investigating the effect of inquiry laboratory activity by comparing the inquiry and noninquiry laboratory activity in terms of students’ understanding about nature of science (nos) among junior high school students in the topic of lights and optics. the design that will be used in this research is pre-test and post-test design. the researcher assigns intact groups of the experimental and control groups, administers a pre-test to both groups, conducts experimental treatment activities with the experimental group only, and then administers a post-test to assess the differences between the two groups (creswell, 2012). this research conducted in one of junior high school in bandung which implements the indonesia national curriculum of 2013. the participants of this research is 44 students from 8th grade. the samples are two classes in eighth grade, one of the classes will act as a control group, table 1 pre-test and post-test design select control class pre-test non-inquiry based laboratory activity post-test select experimental class pre-test inquiry based laboratory activity post-test table 2 the percentage of students’ gender gender experiment class control class number of students percen tage number of students percen tage male 13 54,2% 12 57,1% female 11 45,8% 9 42.9% total 24 100% 21 100% journal of science learning article 30 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).28-35 while the other one will be as experimental group. there are 22 students in each groups, the experimental group consist of 10 females and 12 males while the control group consist of 13 females and 9 males. their ages ranged between 14 to 16 years old. the sampling technique that used in this research was simple random sampling to class. in this research, one class was randomly chosen as experimental class while the other as the control class (fraenkel, j. r., 2007). then, after the random lottery has been conducted, one class consisted of 24 students was taken as experimental class, while the other class consisted of 21 students taken as control class. both group generally consisted of almost half males and half females with the age around 14 years old. 3. result and discussion 3.1 the effect of inquiry based laboratory activity on students’ understandingabout nature of science (nos) students’ understanding about nature of science (nos) was measured using sussi (student understanding of science and scientific inquiry) questionnaire which was developed by liang et al. (2008) sussi was a questionnaire in form of likert-type items to assess students’ understandings on the nature of science (nos) in terms of six aspects which are observations and inferences, tentativeness, scientific theories and laws, social and cultural embeddedness, creativity and imagination, and scientific methods. sussi questionnaire allows the use of inferential statistics to determine the effect of any instructional interventions in small or large scale study. therefore, the statistical test was done in order to calculate the difference of students’ understanding about nature of science (nos) between the experimental and control class. the table below is the recapitulation of statistical test result of the experiment class and control class. sussi questionnaire can be used flexibly and the result can be processed statistically (liang et al., 2008). then this research employed the spss to test the hypothesis. but before it, the normality and homogeneity of the data have to be calculated in order to decide which statistic method was appropriate, whether parametric or non-parametric. the normality test result of pretest and posttest from experiment and control groups shows that both data are normally distributed, because the result shows the values which are more than 0,05. the same with normality test result, the homogeneity test result of pretest and posttest from both groups also shows a value which are more than 0,05, so it can be said that the data is homogeny. then, because the data were normal and homogeny, than it decided to compare the mean, it used parametric statistics which was independent sample t-test. the table 3 shows that the sig. value of independent sample t-test from pretest is less than 0,05. it means that the pretest score of both groups was statistically not significant different. it can be interpreted that the students’ prior understanding about nature of science in both classes was not really different. the students from both classes have not read or taught about the theory of nature table 3 the recapitulation of statistical test result of nature of science (nos) component pretest posttest experiment class control class experiment class control class n 24 21 24 21 average score -15,71 -15,95 22,17 12,76 standard deviation 11,63 9,79 9,49 6,83 maximum score 9 10 45 32 minimum score -33 -32 2 3 normality test signification (sig α=0,05) 0,639 0,551 0,526 0,480 information normally distributed normally distributed normally distributed normally distributed homogeneity test signification (sig α=0,05) 0,325 0,232 information both data are homogeny. both data are homogeny. hypothesis test signification (sig α=0,05) independent sample t-test independent sample t-test 0,940 0,001 information h0 is accepted h0 is rejected conclusion there is no significant different there is significant different journal of science learning article 31 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).28-35 of science, and that is why they got low score of pretest, which is around -15, moreover the students have not familiar yet with the form of the test items. nevertheless, the average score of posttest of both classes were proven to be significantly different. it can be seen from the hypothesis test of posttest score. this could be happend because of the effect of the different treatments implemented to both groups. the experimental group which implemented inquiry based laboratory activity got better score than the control one which implemented non inquiry based laboratory activity. this result is in line with the result of previous research that this model can improve the understanding about nature of science (nos) better than the control one (ozgelen,tuzun, and hanuscin ,2012). inquiry-based laboratory activity improve students’ understanding about nature of science better by involving students in scientific atmosphere where they were allowed to try how the scientist works. they are stimulated to pose scientifically oriented questions, collect and analyse data, form hypotheses, design and conduct scientific investigations, formulate and revise scientific explanations, and communicate arguments (hofstein, a. and lunneta v.n., 2004), and it is cohere with the scientific method. in the control class with conventional laboratory settings, students are usually only follow step-by-step instructions to complete an experiment. hofstein and lunetta (2004) stated that the students generally only concentrate on the completion of individual steps but did not really concentrate to the knowledge behind the experiment itself so they often do not have a deep understanding of the scientific experiment design and did not open their views or understanding about nature of science (nos) (hofstein, a. and lunneta v.n., 2004). to see the clearer comparison between the improvement of students understanding about nature of science (nos), the normalized gain (n-gain) have to be calculated, the experiment class got n-gain of 0,60 and it was categorized as high improvement based on hake (1999). while the control class got 0,44 which categorized as medium improvement. the comparison of n-gain from both classes is provided as the figure 1. this study shows that both learning with inquiry and non inquiry based laboratory activity, both can improve the view or understanding of the nature of science (nos). the improvement of students’ understanding about nature of science (nos) in both classes is cohere with the mentioned literature review that laboratory activities have long had a distinctive and central role in science and educators have suggested that many benefits accrue from engaging students in science laboratory activities (garnett and hackling 2005; hofstein, a. and lunneta v.n., 2004;hofstein et al. 2004; lunetta 2008; tobin 1998) and the america’s lab report (2006) also stated that laboratory experience may enhance the students’ understanding of specific scientific facts and concepts and of the way which these facts and concepts are organized in the scientific disciplines, it develops the scientific reasoning by promoting the students’ ability in identifying the questions and concept. the students’ understanding about nature of science (nos) is measured by using sussi questionnaire in a form of likert-scale questionnaire. those statements in the questionnaire represent the aspects of students’ understanding about nature of science (nos). as what has been explained before that in this research the aspects that involved were observations and inferences, tentativeness, scientific theories and laws, social and cultural embededdness, creativity and imagination, and scientific methods. 3.2 the effect of inquiry based laboratory activity in every aspect of nature of science (nos) the students’ improvement of understanding about nature of science (nos) are also supported by the result of statistical tests showing that the implementation of inquiry based laboratory activity can improve more students’ understanding about nature of science (nos) in every aspects compared to conventional laboratory activity. the improvement of students’ understanding about nature of science (nos) using inquiry based laboratory activity in general is evident from the increase in the average value of each aspects. below is the table of the score of nos in each aspect. based on the sussi questionnaire by liang et al. (2008) the maximum score that can be gained by the students in each aspect with 4 statements is 8, while the minimum score is -8. according to the result of the research, the detail of students’ achievement on six aspect of nature of science (nos) can be described. the score of each aspects shows different result of students’ acquisition either in pretest or posttest. in the pretest, from both classes generally the students obtained the high score in observation and inferences aspect. it is might be because the questions in this subtopic were the basic knowledge about science and figure 1 n-gain comparison of experiment and control class 0 0.5 1 experiment class control class 0.6 0.44n -g a in class journal of science learning article 32 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).28-35 they might have learned about it in previous grade or in the primary school. in addition, the lowest pretest score obtained by both classes is in the aspect of scientific theories and law. many of the students think that scientific laws cannot be replaced by the new laws, since they only know several science laws. they have not known or taught about the laws which have been replaced. 3.3 the categorization of students understanding about nature of science (nos) beside considering the effect between both treatments through the improvement chart, it also needs to categorize the students’ views or understanding about nature of science (nos) into three categories which are informed, intermediary and naїve views (liang et al., 2008) and khishfe and laderman, 2006) in every aspects involved in the questionnaire (sussi questionnaire) which are observations and inferences, tentativeness, scientific theories and laws, social and cultural embededdness, creativity and imagination, and scientific methods. the students’ views were categorized as naive views if none of four responses received a score >3 within each theme; the students views were categorized as intermediate views if several responses received a score >3 within each theme; the students views were categorized as informed views if all four responses received a score >3 within each theme (khishfe and laderman, 2006, and liang et al., 2008. the categorization result of students' views based on the views of the informed, and naive categories of the nature of science are presented in table 5. developing students' informed views on the nature of science is one of the main goals in science education (khishfe and laderman, 2006). the increased percentage of informed views indicates an increase in students' view of the nature of science in a better direction and vice versa, the naive view of students decreases. while the intermediary view can not be seen directly whether it increases or decreases, it depends on its influence on the percentage of the other category (informed and naive). table 5 shows the percentage of categorized results of the nature of science that has been categorized into informed, intermediary and naive views, according to the criteria described by khishfe and laderman (2006). in general, it can be seen that the categories of science views in both classes do have differences in every aspect, both pretest and posttest. more details, the following will be presented the findings and discussion in the next section. 3.3.1 observations and inferences the pretest results of the two classes (see table 5.) show that before the treatment given, over 70% of the students had a naive view of observation and inferences, with the experimental class superior to the informed view over the control class. so in general, in both classes, the average student is in the naive category, whereas the informed and intermediary categories are few. the result of the experimental class posttest shows that there is an increasing percentage of students who have informed views on the observation and inferences aspect of 62,5%. this increase was followed by a decrease in the percentage of naive and intermediary, as the percentages switched to informed during posttest. while the result of posttest of control class showed that there was also an increase of percentage of students with informed views about the observation and inferences aspect although smaller compared to the experimental class, that is as much as 45,8%. 3.3.2 tentativeness science is tentative, there is not any exact theory and law of science because nature is always changing (buxer, 2014). many of the students did not know that science, especially physics is developing. science tentativeness describes that science is never absolute or certain although it is reliable and durable. furthermore, science tentativeness is also subject to change shows that the knowledge is based on the result of new evidence and innovation. hence, the previous theories and laws can be changed. the pre-test results of the two classes in this aspects shows that before the treatment given, over 60% of the students had a naive view of science tentativeness, with the experimental class superior to the 8informed view over the control class. so in general, in both classes, the average student is in the naive category, whereas the informed and intermediary categories are few. the result of the experimental class posttest shows that there is an increasing percentage of students who have informed views on the science tentativeness aspect of 70,8%. this increase was followed by a decrease in the percentage of naive and intermediary, as the percentages switched to informed during posttest. while the result of posttest of control class showed that there was also an increase of percentage of students with informed views about the science tentativeness aspect although smaller compared to the experimental class, that is as much as 45,8%. future, the students who get good understanding of science tentativeness expectedly will be able to compete with others globally. knowledge and technology are the things made up science, as long as there is a new evidence, new cases solved and new observation held. science and technology will always develop and give human opportunities and innovation. the atmosphere of the school which research was conducted may affect to the improvement of the students’ views or understanding about science tentativeness. many of the students are interact with new technologies, new reality and new social life in daily, that interaction support journal of science learning article 33 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).28-35 their new understanding and make them more aware and got the better score in this aspects of nature of science (nos). 3.2.3 scientific theories and laws it is proven due to the result of the research that this aspect is the least students are categorized as the informed views where there is less than quarter of the students or participants (0%) of the students are categorized as informed views and the control class superior to the experiment class (4,2%). in this aspect the pretest results from both classes (see table 5.) shows that before the treatment, all of students had a naive view of scientific theories and laws, with both classes having no informed views at all. so in general, in both classes, almost all students are in the naive category, whereas the informed category is absent and the intermediary is small. the result of the experimental class posttest shows that there is high increasing percentage of students who have informed views on the scientific method aspect as much as 79.2% or 15% superior if compared with the control class. this increase was followed by a decrease in the percentage of naive and intermediary, as the percentages switched to informed during posttest. while the result of posttest of control class showed that there was also an increase of percentage of students with informed views about scientific theories and laws aspect although smaller compared to experiment class, that is 54,2%. 3.2.4 social and cultural embededdness the aspects of nature of science (nos) relates to some feature in human life which are the cultural, social, political, ethical, commercial, cognitive, structural and phsylogical (mattehews, 2012). in the result of the pretest, in this table 4.the students’ score in each aspect of nos class component the aspect of nos observations and inferences tentativenes s scientific theories and laws social and cultural embededdness creativity and imagination scientific methods e x p e ri m e n t pretest -1,67 -2,54 -3,38 -2,79 -2,29 -2,79 posttest 3,71 3,75 3,67 3,54 3,75 3,75 gain 5,38 6,29 7,04 6,33 6,04 6,54 normalized gain 0,55 0,58 0,61 0,57 0,58 0,60 category medium medium high medium medium high c o n tr o l pretest -2,33 -2,62 -2,95 -2,67 -2,38 -3 posttest 1,952 2,143 2,286 2,095 1,857 2,429 gain 4,286 4,762 5,238 4,81 4,19 5,429 normalized gain 0,377 0,433 0,449 0,429 0,363 0,467 category medium medium medium medium medium medium figure 2 n-gain in each aspects of nos between experiment and control class 0.55 0.58 0.61 0.57 0.58 0.6 0.377 0.433 0.449 0.429 0.363 0.467 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 observation and inferences tentativeness scientific theories and laws social and cultural embededness creativity and imagination scientific method n -g a in nos aspects experiment class control class journal of science learning article 34 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).28-35 aspect all the students are categorized also categorized as naive views where there is none of the students or participants (0%) of the students are categorized as informed views. in this aspect the pretest results from both classes (see table 4) shows that before the treatment, all of students had a naive view of scientific theories and laws, with both classes having no informed views at all. so in general, in both classes, almost all students are in the naive category, whereas the informed category is absent and the intermediary is small. the result of the experimental class posttest shows that there is a high increasing percentage of students who have informed views on the social and cultural embededdness aspect as much as 83,3% or 33,3% superior if compared with the control class. this increase was followed by a decrease in the percentage of naive and intermediary, as the percentages switched to informed during posttest. while the result of posttest of control class showed that there was also an increase of percentage of students with informed views about scientific theories and laws aspect although smaller compared to experiment class that is 50%. 3.2.5 creativity and imagination creativity and imagination is source of inspiration in science which is used along with logic and prior knowledge (liang et al., 2008). science involves the invention of theoritical and explanation requires a great deal of creativity. hence, creativity and science cannot be apart because it is involved in all stages of scientific investigation and particularly relevant to generate and interpret the data starting from the beginning, during and following the collection of data (lederman et al., 2012; chen, 2006; abdel-khalick, 2012). creative and imagination support science becomes unique and innovative. a good scientist must be creative in designing a good experiment and also shall be imaginative in coming up with a theory, but shall use scientific method to stay objective (akerson and hanuscin, 2007). coming up to the result of this research that creativity and imagination aspects. the experimental class posttest shows that there is a high increasing percentage of students who have informed views on creativity and imagination aspect as much as 79% or 29,3% superior if compared with the control class. this increase was followed by a decrease in the percentage of naive and intermediary, as the percentages switched to informed during posttest. while the result of posttest of control class showed that there was also an increase of percentage of students with informed views about scientific theories and laws aspect although smaller compared to experiment class, that is 45,8 %. 3.2.6 scientific methods the pretest results from both classes (see table 5.) show that prior to treatment, over 70% of students had a naive view of scientific method, with both classes having no informed views at all. so in general, in both classes, almost all students are in the naive category, whereas the informed category is absent and the intermediary is small. table 5.the categorization result of students’ understanding about nature of science (nos) test aspects experiment class control class naїve views (%) intermediary views (%) informed views (%) naїve views (%) intermediary views (%) informed views (%) p re te s t observations and inferences 75 16,7 8,3 79,2 16,6 4,2 tentativeness 66,7 29,1 4,2 66,7 33,3 0 scientific theories and laws 95,8 4,2 0 87,5 8,3 4,2 social and cultural embededness 87,5 12,5 0 91,6 8,4 0 creativity and imagination 70,8 25 4,2 58,3 33,4 8,3 scientific method 83,3 16,7 0 75 25 0 p o s tt e s t observations and inferences 16,7 12,5 70,8 29,2 20,8 50 tentativeness 4,16 25 70,8 16,7 37,5 45,8 scientific theories and laws 12,5 8,3 79,2 25 20,8 54,2 social and cultural embededness 12,5 4,16 83,3 33,3 16,7 50 creativity and imagination 8,3 8,3 83,4 29,2 16,7 54,1 scientific method 4,16 16,7 79,1 25 8,3 66,7 journal of science learning article 35 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).28-35 the result of the experimental class posttest shows that there is an increasing percentage of students who have informed views on the scientific method aspect as much as 79.1% or 12.4% superior when compared with the control class. this increase was followed by a decrease in the percentage of naive and intermediary, as the percentages switched to informed during post-test. while the result of posttest of control class showed that there was also an increase of percentage of students with informed views about the observation and inferences aspect although smaller compared to experiment class, that is 66,7%. 4. conclusion be rooted in data analysis, the inquiry based laboratory activities appears to have more benefited to the students in terms of understanding about nature of science (nos). the students in the inquiry class scored slightly better than did students in the non-inquiry group, they perceived a statistically significantly bigger gain than did students in the non-inquiry class. so, it can be concluded that the implementation of inquiry based laboratory activity in learning light and optics can improve students’ understanding about nature of science (nos). based on the findings of the research that has been conducted and concluded, there are several recommendation that necessary to be conveyed by the researcher, some of them are, first before grouping the students in this research, it is important to analyze students’ ability. the group members should be consist of high and low achievement students. teacher also should make sure that each group is working collaboratively. second, teacher supervision is really necessary in order to make sure that every group member active in giving idea and discussion in their group. while, in the learning process, the students need to be triggered more to formulate the critical questions and relate the laboratory activity with what scientist do in reality. then, the time allocation to implements the experimental treatment should be considered and determined properly since there are several phases, so the students can finish the laboratory activity optimally. references anderman & sinatra. (2008). the challenges of teaching and learning about science in the 21st century: exploring. las vegas: national academy of education. ausubel, david p. (1969). school learning:an introduction to education psychology. new york: holt, rinehart and winston, inc. azer, s.a., hasanato,r., al-nassar,s., somily, a.,&alsaadi, muslimm. (2013). introducing integrated laboratory classes in a pbl curriculum: impact on student’s learning and satisfaction.bmc medical education, 13 (71), 112. bbc news. (2005, june 19). science dull and hard, pupils say. retrieved october 17, 2016, from bbc news: http://news.bbc.co.uk/2/hi/uk_news/education. brookes publishing. (2012). 5 tips for getting all your students engaged in learning. retrieved october 18, 2016, from: http://archive.brookespublishing.com/articles/ed-article0212.htm. bunning, frank. (2013). effects of experimental learning outcomes of an empirical study in the vocational field of structural engineering. international journal of training research, 11 (1), 44-55. cresswel, john w. (2012). educational research: planning, conducting, and evaluating quantitative and qualitative research. unated states of america. pearson. fraenkel, j. r., wallen, norman e., & hyun helen h. (2007). how to design and evaluate research in education. new york: mcgraw-hill, inc. hofstein, a. and lunneta v.n. (2004). the laboratory in science education: foundations for the twenty-first century. science education. 88. 28-54. kementerian pendidikan dan kebudayaan. (2015). kurikulum 2013 kompetensi dasar. jakarta: kemdikbud. liang, l., chen, s., chen, x., kaya, o.n., adams, a.d., macklin, m., and ebenezer, j. (2008). assessing preservice elementary teachers' views on the nature of nature of scientific knowledge. asia-pacific forum on science learning and teaching. 9, 1. oecd data. (2015). science performance pisa. retrieved december 12, 2016, from oecd data:https://data.oecd.org/pisa/scienceperformance-pisa.htm. olubu, odutuyi musili (2015). effects of laboratory learning environment on students’ learning outcomes in secondary school chemistry. international journal of arts & sciences, 8 (2), 507525. ozgelen, s., tuzun, o.y.,andhanuscin, d.l. (2012). exploring the development of preservice science teachers’ views on the nature of science in inquiry-based laboratory instruction. science and business media dordrecht. 43, 1551–1570. tural, g. (2015). cross-grade comparison of students’ conceptual understanding with lenses in geometric optics. science education international journal, 26 (3), 325-346. wang, jing-ru., wang, yuh-chao., tai, hsin-jung. & chen, wen-ju. (2010). investigating the effectiveness of inquiry-based instruction on students with different prior knowledge and reading abilities. international journal of science and mathematics education. 8, 801-820. wolf, s.j.,& fraser, b.j. (2007). learning environment, attitudes and achievement among middle-school science students using inquiry-based laboratory activities. springer science + business media b.v. 38, 321–341. journal of science learning article 36 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).28-35 a © 2023 indonesian society for science educator 194 j.sci.learn.2023.6(2).194-203 received: 21 march 2023 revised: 19 may 2023 published: 30 june 2023 the effectiveness of the problem-based flipped classroom learning model to improve conceptual understanding of physics teacher candidates on crystal structure material heni rusnayati1*, wawan ruswandi1, tiara husnul khotimah2 1department of physics education, faculty of mathematics and science education, universitas pendidikan indonesia, bandung, indonesia 2department of mathematics, faculty of science and technology, universitas al-azhar indonesia, jakarta, indonesia *corresponding author: heni@upi.edu abstract the limited time allocation for lectures in the classroom is an obstacle to presenting student-centered learning. this is one of the factors that prospective physics teacher students experience conceptual difficulties, especially in crystal structure material. so the purpose of this study was to determine the effectiveness of the problem flipped classroom learning model on understanding the concepts of prospective physics teacher students on crystal structure material. this study used a quasi-experimental method with a one-group pretest-posttest research design. the subjects of this study were 16 students who took solid-state physics courses. data analysis techniques were carried out using normality tests, homogeneity tests, and pretest and posttest average difference tests. then test the n-gain to see the increase in the pretest and posttest results and continue with the effect size test using the effect size. the instrument used was a test of mastery of the concept of crystal structure material. the results of the paired sample t-test analysis show that the problem based flipped classroom significantly influences learning outcomes with a t value of 11.439 with a significance of 0.000. students' understanding of crystal structure material has increased with an n-gain of 0.75, which is in the high category. this means that the pretest and posttest scores have a high increase. while the results of the effect size test obtained a score of d = 2.86, which means that learning with the problem based flipped classroom has a strong effect on student learning outcomes on crystal structure material. keywords crystal structure, problem based flipped classroom, concept understanding 1. introduction in general, prospective physics teachers are required to master physics content, and how to teach it (wenning, 2006). mastery of content is very important because it is part of professional competence. mastery of good content will impact prospective teachers with an innovative attitude toward learning (rollnick, 2017). the various innovative prospective teachers in question can be seen from their ability to determine the order of learning and themes from various concepts to guide students in the inquiry process in class (purwaningsih, r, 2021). based on this, understanding the concept is very important for prospective teachers because it is part of improving their quality as teachers and the basis for improving the quality of education. understanding the concept of prospective physics teachers is facilitated through lectures or learning in class. for example, in the physics education study program at the indonesian university of education (upi), one of the courses presented to prospective physics teachers is solid matter physics. solid substance physics is one of upi's elective courses in physics education. one of the materials in solid substance physics is studying crystal structures. this material is important to learn as a basis for broadening and, at the same time, supporting the knowledge and understanding of prospective physics teachers towards their school material, besides playing a role in their professionalism as a physics teacher in high school (parno, 2012) field conditions have not supported the importance of this crystal structure material. based on research conducted by ardhuha et al. (2019), students generally experience some difficulties in learning this material, including students having difficulty understanding terms in teaching resources, difficulties analyzing a crystal lattice, difficulty mailto:heni@upi.edu journal of science learning article doi: 10.17509/jsl.v6i2.56316 195 j.sci.learn.2023.6(2).194-203 solving the problems contained in the problem, difficulty determining the miller index of a crystal field. the findings during learning show that many students cannot achieve the expected learning outcomes, so they experience conceptual difficulties. this is obtained based on a review of the results of tests or exams, which shows that the percentage level of passing in this material only reaches 30%. the difficulties experienced by students are common. even though they have completed many questions in the learning material during lectures, they still need help with conceptual difficulties for students (byun. t, 2014). the difficulties experienced by students in understanding the concept of the crystal structure are caused by a need for more reading sources, such as textbooks, that can help strengthen understanding so that innovation is needed in learning. (novia, h. 2013). the learning process in the classroom generally begins with providing knowledge information directly, either through discussions, powerpoint presentations, or presentation activities carried out by students. it ends with examples of questions and exercises to be completed at home. the learning model lecturers use in lectures greatly influences student learning processes and outcomes. findings in the field show that generally, learning is carried out through providing information directly to students or using certain methods only or not varying, which tend not to give birth to student-centered learning. as a result, students tend to experience difficulties in building their knowledge independently (sinaga, 2017). the occurrence of learning does not vary, considering that the time allocation for lectures in the classroom is very limited, so the limited time allocation becomes one of the factors that hinder lecturers from presenting student-centered learning, namely time allocation (sinaga, 2017). so, learning is needed to facilitate flexibility in student study time and build student understanding. one learning model that can support student study time allocation factors is flipped classroom. the advantages of using this learning model include student learning resources presented in the form of videos that are studied before lectures begin as a basis for knowledge so that classes can focus on building discussion processes and solving problems students raise. this learning model has no limitations in teaching space (akçayir, 2018). such a learning process can facilitate students to build their knowledge. it can also be used as a form of preparation before attending lectures so that lectures will effectively build conceptual understanding and training skills in solving problems (subali b et al., 2015). in addition, this learning model also makes students push themselves to become independent learners (o'flaherty & phillips, 2015). flipped classroom was introduced as a learning model by bergmann and sams (2012) to minimize the shortcomings of traditional learning methods, which are considered ineffective and sometimes fail to engage students during class learning. flipped classroom is a learning activity by minimizing instruction (in the form of providing knowledge information) directly and maximizes each student's interaction with learning resources (johnson, 2012). the learning resources are reading material or learning videos (alamri, 2019; damayanti et al., 2016), carried out before lectures. so that students can focus on starting discussions, exchanging knowledge, and solving a problem in class. thus flipped classroom can facilitate student-centered learning (aşıksoy & ozdamli, 2016; nolam et al., 2021), has the potential to provide learning that is more structured and involves active participation from students in learning activities (sterlan et al., 2020; aguileraruiz et al., 2017; aprianto, ritonga, et al., 2020). in addition, learning must also be able to train students to find concepts and solve problems, so the problem based flipped classroom learning model can potentially support these learning conditions. based on this, this study aims to examine the application of the problem based flipped classroom to improve students' conceptual understanding of physics teacher candidates on crystal structure material. 2. method this study uses a quasi-experimental method with a one-group pretest-posttest research design. in this research design, students were given a pretest regarding conceptual understanding before being given treatment to research subjects to determine the initial conditions of the class of students involved. after the treatment, students were given a posttest to measure the level of students understanding of concepts related to crystal structure material. the following research design used is shown in table 1 in general, the implementation of the research is divided into three stages, namely the preparation stage, the implementation stage, and the final stage. each stage is outlined in the research procedure, as shown in figure 1. based on figure 1, the initial stage of learning using the problem flipped classroom model begins with preparing lecture material that refers to the rps for the solid matter physics course. the prepared lecture material includes learning videos, teaching materials in the form of learning table 1 research design one group pretest-posttest design. o x o pre-test (concept understanding test) treatment (problembased flipped classroom learning model) post-test (concept understanding test) journal of science learning article doi: 10.17509/jsl.v6i2.56316 196 j.sci.learn.2023.6(2).194-203 modules and resume slides with examples of practice questions. in the next stage, the learning tools that have been prepared are delivered to students via whatsapp, which is uploaded before the lecture day is carried out so that students have more time to prepare material that can be discussed during lectures. an overview of the learning module and learning video material slides are shown in figure 2 and figure 3 the next activity carried out outside the classroom is that students carry out independent learning of the learning tools that have been distributed, with the aim of students practicing their ability to remember and understand by figure 1 research procedure table 2 implementation of lectures using the problem-based flipped classroom learning model stages flipped classroom learning model lecturer activity student activity out-ofclass practice (students) before class lecturers prepare 1) material in the form of ppt and video, 2) practice questions, and 3) make assignments. prepare learning devices in the form of gadgets or laptops inform students of the problem-based flipped classroom learning model. lecturers inform students via whatsapp about flipped classroom learning. at this stage, the lecturer provides an overview of the learning that students must carry out students listen to online information submitted by lecturers regarding implementing the learning system using a flipped classroom. explain to students how to access learning materials and learning videos. 1. the lecturer informs students that learning materials and learning videos can be accessed via a link sent via the whatsapp group. 2. lecturers give assignments to students to understand the material that is listened to through learning videos and provide solutions to the problems that have been presented students listen to, and study learning material delivered via ppt and video, then analyze problems related to crystal structure material provided by lecturers delivered via video and look for alternative solutions. classroom practice (teacher and students) lecturers explore the initial knowledge that students have acquired during the learning stages outside the classroom the lecturer collects and directs various questions from students without giving comments first. students explain the concepts they have learned during learning outside the classroom group discussion tosolve on the problem given in the video. the lecturer acts as a facilitator and does not provide explanations. lecturers guide so that students understand what their friends convey. lecturers interact with students actively to analyze, evaluate and construct the concepts learned. students ask each other questions between groups and within groups and respond to the questions asked. students must also verify the discussion results through presentations in their respective groups. create a learning outcomes assessment system lecturers conduct student assessments through post-test activities related to understanding concepts. students work on the posttest. journal of science learning article doi: 10.17509/jsl.v6i2.56316 197 j.sci.learn.2023.6(2).194-203 increasing their interaction with teaching materials. while the activities are carried out in the class that the lecturer only acts as a facilitator in the lecture process, the lecturer also provides topics of discussion that can be discussed with the whole class on each concept. the following is an overview of the activities in the lecture process with the flipped classroom model. the implementation of this lecture is carried out based on the syntax of the flipped classroom learning model, which is based on the theory of bergmann & sams (2012), except that the lecturer directs it to solve problems related to crystal structure material as shown in table 2 sampling in this study uses a purposive sampling technique. meanwhile, the research subjects involved were 16 students in the seventh semester of the physics education program who took the solid substance physics course in the 2021/2022 academic year. the research instrument used was in the form of tests of students' mastery of the concept of solids physics material in the crystal structure chapter with several concepts including lattice, basis, crystal structure, conventional cells, primitive cells, describing primitive cells using weigner seist, number of lattice points of conventional and primitive cells for hexagonal crystals, body center cube (bcc) and face center cube (fcc) and determined the index between the two planes. the instrument used is shown in figure 4 based on figure 4. the question consists of 5 numbers, but for number 1, it consists of 3 questions, namely explaining the concepts of lattice, basis, and crystal structure; for number 2, there are two questions, namely explaining the concept of conventional cells and primitive figure 2 crystal structure learning module figure 3 material in the video for the flipped classroom figure 4 research instrument for understanding the concept of crystal structure material journal of science learning article doi: 10.17509/jsl.v6i2.56316 198 j.sci.learn.2023.6(2).194-203 cells, number 3 only has 1 question is describing primitive cells using weigner seist, number 4 has two questions, namely determining the number of lattice points of conventional cells and primitive cells and for number 5 there is 1 question, namely regarding the field index on the fcc cube. based on this, it was concluded that this instrument consisted of 5 numbers with nine questions. before using this instrument, a content validity and reliability test were carried out, where the validity test used the product-moment correlation coefficient. in contrast, the reliability test used the cronbach alpha formula to obtain a valid and reliable instrument. the results of calculating the validity and reliability tests can be seen in table 3. based on table 3, from the nine questions tested, it can be concluded that eight questions are valid. in contrast, 1 question is not valid, but overall the instrument used has a reliability value of 0.830 with a reference value of 0.79, so cronbach's alpha value> 0.70, which concluded that this instrument is reliable. based on the pre-test and post-test results, it can be analyzed whether there is an increase in students' conceptual understanding by comparing the average achievement of pre-test and post-test scores. at the same time, the data analysis technique uses the kolmogorovsmirnov test for the normality test and the levene test for the test. homogeneity. given the relatively small number of samples of 16 students, the kolmogorov-smirnov test was carried out using the exact method to determine if the data obtained were normally distributed (mehta & patel, 2013). once it is known that the research data are normally distributed, then a hypothesis test is carried out using the paired sample t-test. the research hypothesis is formulated as follows: ho: there is no increase in understanding of concepts through problem-based flipped classrooms ha: there is an increased understanding of the concept through a problem based flipped classroom. guidelines for decision-making in the paired sample ttest based on the significance value (sig.) proposed by santoso (2014), namely 1) if the value of sig. (2-tailed) <0.05, then ho is rejected and ha is accepted; 2) if the sig. (2-tailed)> 0.05, then ho is accepted, and ha is rejected. to see how much an increase in conceptual understanding using the problem based flipped classroom learning model is obtained based on the n-gain test (hake, 1999) from the pretest and posttest scores using the following equation: 〈𝑔〉 = 〈𝑆𝑝𝑜𝑠𝑡 〉 − 〈𝑆𝑝𝑟𝑒 〉 100% − 〈𝑆𝑝𝑟𝑒 〉 (1) the information for the above formula is 〈g〉= normalized gain score, 𝑆𝑝𝑜𝑠𝑡 = post-test score, 𝑆𝑝𝑟𝑒 = pretest score. meanwhile, the magnitude of the g factor can be interpreted as shown in table 4 the effect size test is used to see how much the effectiveness of learning is using the problem based flipped classroom model. this test is used to measure the scale of the effectiveness of the learning model that has been carried out (lakens, 2013). 𝑑 = �̅�𝑡 − �̅�𝑐 𝑆𝑝𝑜𝑜𝑙𝑒𝑑 (2) with, 𝑆𝑝𝑜𝑜𝑙𝑒𝑑 = √ (𝑛𝑡 − 1)𝑠𝑡 2 + (𝑛𝑐 − 1)𝑠𝑐 2 𝑛𝑡 + 𝑛𝑐 (3) the explanation for the formula above is 𝑑 = cohen's effect size; �̅�= mean; 𝑠𝑡 = posttest standard deviation; 𝑠𝑐 = pretest standard deviation; n = number of samples. from the acquisition of the effect size test score, it can be interpreted based on the adaptation of cohen (1988), as shown in table 5. after the statistical test draws, table 3 recapitulation of validity and reliability test results no question topic 𝒓𝒄𝒐𝒖𝒏𝒕 𝒓𝒕𝒂𝒃𝒍𝒆 information reliability 1 (1) lattice 0.901 0.497 valid 0,830 (reliable) (2) base 0.674 0.497 valid (3) crystal structure 0.746 0.497 valid 2 (4) conventional cell 0.616 0.497 valid (5) primitive cell 0.752 0.497 valid 3 (6) describe primitive cells using weigner seist 0.352 0.497 invalid 4 (7) determine the number of conventional cell lattice points 0.498 0.497 valid (8) determine the number of primitive cell lattice points. 0.610 0.497 valid 5 (9) field index on the fcc cube 0.733 0.497 valid table 4 interpretation of the average n-gain score score criteria interpretation n-gain > 0,7 high 0,3 ≤ n-gain ≤ 0,7 medium n-gain < 0,3 low journal of science learning article doi: 10.17509/jsl.v6i2.56316 199 j.sci.learn.2023.6(2).194-203 conclusions are based on the problems and research objectives. 3. result and discussion in this study, problem-based flipped classroom learning was used, divided into two stages, namely the stages outside and in the classroom, carried out on different days. prior to the implementation of learning, a pretest is carried out. at the stage outside the classroom, the lecturer arranges the material in the form of ppt, modules, and learning videos which are then given to students online to be listened to simultaneously by informing them of the implementation of problem-based flipped classroom learning activities, which will be carried out at the next meeting in class. meanwhile, during the stages in the class, the lecturer directs students to ask as many questions as possible related to crystal structure material. when learning in the classroom, the lecturer only facilitates student discussions and directs various questions that arise to find answers together. the answers formed in the discussions can build on each concept learned. after learning, the lecturer creates a scoring system, including conducting a pretest on crystal structure material. the activity pattern, which is divided into two learning stages, is based on the theory of bergman & sams (2012), reidsema, kavanagh, hadgraft, & smith (2017), and baytiyeh (2017) only in every implementation of learning directed to solve problems. based on the acquisition of data, the results of the pretest and posttest are shown in figure 5 figure 5 illustrates that the results of the pretest scores of 16 students on crystal structure material have the lowest score of 15 and the highest score of 50. no student achieves a minimum understanding ability with a score of 70. while the post-test results show that the lowest student score is 40 and the highest score is 100, it can be said that the number of students who achieve the standard of comprehension ability is 14 people while two others are still below the minimum standard. based on the student test scores obtained, a normality test was carried out using the kolmogorov-smirnov test using ibm spss 25. given the relatively small number of samples, the kolmogorov-smirnov test was carried out using the exact method to determine whether the data obtained was normally distributed. (mehta & patel, 2013). based on this, the normality test results obtained in this study are shown in table 6. table 6 shows that the sig. the exact pretest was 0.177, while the posttest score was 0.688. both values were known to be greater than 0.05, meaning that at a significance level of 5%, the normalized gain pretest and posttest scores of learning outcomes in crystal structure material come from normally distributed populations. under normal conditions, the number of research subjects described is limited to the kolmogorov-smirnov test with the exact or monte carlo method due to the relatively small number of samples that affect the variance condition. the homogeneity test is carried out using the levene test. furthermore, the homogeneity test in this study is shown in table 7 table 7 shows that the significance of homogeneity is 0.197, which means that at a significance level of 5%, the research subject data obtained has a homogeneous table 5 interpretation of effect size scores score criteria interpretation 0 – 0,20 very weak effect 0.,21 – 0,50 weak effect 0,51 – 1,00 moderate effect >1,00 strong effect figure 5 student pre-test and post-test scores table 6 kolmogorov-smirnov one sample normality test n sig.pretest sig. posttest significance description 16 0.177 0.688 0.05 normal table 7 levene's test homogeneity levene statistic df1 df2 sig 1.696 1 0,7544 0,197 journal of science learning article doi: 10.17509/jsl.v6i2.56316 200 j.sci.learn.2023.6(2).194-203 condition with a statistical level of 1.696. by knowing that the samples obtained are normally distributed and have homogeneous conditions. then a different test or hypothesis testing can be carried out. the researcher tested the hypothesis using the paired sample t test. this result test was used to find out whether there was a difference in the average of two paired samples. the results of the t-test are described in the following three output tables table 8 or output one above summarizes descriptive statistics based on pre-test and post-test data. it was found that the average pre-test value was 23.4375 while the posttest average value was 80.6250. with a total of 16 students (n). the std. the deviation score on the pre-test was 9.61228, and the post-test was 20.23817. as for the value of std. the mean error for the pre-test is 2.40307, and for the post-test, it is 5.0595. the results of output 1 show the average value of students' understanding of crystal structure concepts at pre-test 23.4375 < post-test 80.6250, meaning that descriptively there is a difference in average students' understanding of concepts between pre-test and post-test. table 9 or output two above is the result of the paired sample t-test (paired t-test) is the result of the correlation or relationship between the two data, namely the pre-test, and post-test. the correlation referred to in this case is the pearson product-moment. based on the table above, the correlation coefficient (correlation) is 0.262 with a sig. 0.326 because of the value of sig. 0.326> 0.05, it can be concluded that there is no relationship between the pre-test and post-test. table 10 or output three above is data regarding whether there is a difference or an increase in student's conceptual understanding of crystal structure material before and after lectures with the problem based flipped classroom model. based on the paired samples test table, the sig is known. (2-tailed) is 0.000 <0.005, then 𝐻𝑜 is rejected, and 𝐻𝑎 is accepted. so there are differences in the average understanding of concepts, or other words, there is an increase in students' conceptual understanding skills through problem based flipped classroom learning. table 10 also explains information regarding the mean paired difference value of -57,187. this value indicates the difference in the average understanding of students' concepts in the pre-test with the average post-test understanding or 23.438-80.625 = -57.18 and between 67.843 to -46.531 (95% confidence interval of the difference lower and upper ). furthermore, it is known that the t count is negative, namely -11,439. this negative sign is because the average value of students' understanding of crystal structure concepts at the pre-test is lower than the average students' understanding of the concept of the crystal structure at the time of the post-test. in cases like this, a negative t-count value can have a positive meaning so that the t-count value becomes 11.439. after carrying out statistical tests, based on the pretest and posttest value data shown in figure 5, it can be analyzed to increase understanding of the concept of crystal structure material which is determined by n-gain analysis of the understanding of the concept. by using spss, the n-gain score is obtained as follows. table 11 shows increased student conceptual understanding of crystal structure material using the problem based flipped classroom model. the table shows that the n-gain score results are obtained with an average value of 0.75 which is in the high category, meaning that there is an increase in conceptual understanding through the learning model problem based flipped classroom. meanwhile, to determine the effectiveness of the problembased flipped classroom model, it can be seen from the learning outcomes in crystal structure material by conducting a statistical effect size (d) test. the result of the effect size test in this study was 2.86. if viewed based on the category of effective magnitude, the score d = 2.86 can be interpreted as a strong effect. this means that the table 8 paired sample t-test results (output 1) paired samples correlation mean n std. deviation std. error mean pair 1 pretest 23.438 16 9.61228 2.40307 postest 80.625 16 20.23817 5.05954 table 9 paired sample t test results (output 2) paired samples correlation n correlation sig. pair 1 pretest postest 16 0.262 0.326 table 10 paired sample t test results (output 3) paired differences 95% confidence interval of the difference mean std. deviation std.error mean lower upper t df sig.(2tailed) pair 1 pretest &postest -57.187 19,997 4.999 -67.843 -46.531 -11.439 15 .000 table 11 improved understanding of the concept reviewed through n-gain maximum n-gain minimum n-gain average n gain category 1.00 0.20 0,7544 high journal of science learning article doi: 10.17509/jsl.v6i2.56316 201 j.sci.learn.2023.6(2).194-203 problem-based flipped classroom model strongly affects student learning outcomes in crystal structure material. in terms of the individuality of each student (s), there were 16 students. it was found that student 1 (s1) had difficulty with question number 3 (q3) related to the concept of crystal structure, question number 4 (q4) related to the conventional cell concept and problem number 7 (q7) related to how to determine the number of conventional cell lattice points, even this difficulty is also experienced by other students which can be seen based on the acquisition score of each question that no student can answer perfectly for these three questions. efforts to overcome these problems include providing more practice questions and discussing the practice questions carried out within the scope of each student discussion group. based on this, it is known that by increasing the intensity of the discussion related to the problem of students' understanding difficulties within the scope of their study group, students become more active in showing good responses related to questions or responses from other students. as for s9 and s14 students who seemed to have difficulty studying crystal structure, grouping efforts were made by placing them in the most active discussion group so that the two students got the greater motivation to understand the concepts they considered difficult. the amount of intensity of discussion that is carried out is one of the factors in increasing the understanding of concepts in each student in this lesson. this finding is related to research findings by basal (2015), herlindayana et al. (2017), and kozikoğlu (2019), which explain that flipped classrooms can make students more active through discussions in building an understanding of the concept. as for some descriptions of learning that are carried out while in class, as shown in figure 6 this improvement in learning outcomes is, of course, inseparable from the role of learning activities carried out with the problem-based flipped classroom model, which begins with giving orientation to students on a problem contained in crystal structure material, where students will study the problem to find a solution. in their learning, the lecturer displays the problems and concepts contained in the crystal structure material through a video that is watched before the lecture begins. various interesting phenomena shown in the video will motivate students to learn more deeply about the various concepts contained in the video (arends, 2008). based on the implementation of learning using the problem-based flipped classroom model, the findings of the factors that led to an increase in conceptual understanding in this study were a more flexible allocation of student learning time. students are required to interact with longer learning material. besides that, knowledge when learning in class is built by discussing a posed problem. this was also found in meilantari's research (2021), nouri (2016), and yulianti et al. (2021), who explained that learning with a flipped classroom has flexibility in time and learning space. the relationship between time allocation and the potential for increased opportunities for students to learn the material also follows the research results of havwini & wu (2019) and låg & sæle (2019). other findings in this study are related to student discussion activities which are another factor that causes an increase in student understanding. however, based on the data shown in figure 6, one student still needs help, or the result of understanding the concept of the crystal structure still needs to be below the minimum score. activities in class can happen because students need to get used to the learning being done, while 14 other students get scores above the average. this is considering that the discussion is carried out as a process between students exchanging ideas. at the same time, the lecturer provides direction in the discussion process so that all students get maximum understanding. this discussion process is an alternative that can be carried out in developing students' conceptual understanding, especially in the field of science (erduran, 2004). in addition, the discussion process in solving problems will also bring up various alternative solutions to figure 6. learning in the classroom journal of science learning article doi: 10.17509/jsl.v6i2.56316 202 j.sci.learn.2023.6(2).194-203 problems so that students' conceptual understanding can be formed in more depth (newton, 2010) from the advantages of the flexibility aspect of time allocation and discussion processes that occur in class, a student's conceptual understanding of crystal structure material can be obtained using the problem-based flipped classroom model. thus this model can provide changes in student thinking that can be seen during the lecture process. students become more active in conducting discussions because before lectures begin, students focus on studying the material first as initial preparation for lecture activities in class, either in the form of interactive conceptual discussions that occur in groups or between students and lecturers or in solving problems on crystal structure material. 4. conclusion the use of problem-based flipped classrooms in solid matter physics learning, especially in crystal structure material, has succeeded in increasing student learning outcomes, as indicated by an increase in the value of understanding the concept of crystal structure. this research provides an alternative to active and cooperative contextual learning compared to conventional learning, which tends to be informative. the problem-based flipped classroom, which focuses on aspects of understanding concepts, plays a significant role in encouraging students to prepare themselves before attending lectures. the impact is that when students learn in class with prior knowledge that has been prepared, and even students can bring up many problems related to crystal structure material, learning in class can fully become a place for discussion in solving these problems. understanding that is built through the process of discussion and problem solving with the preparation of prior knowledge will be formed in depth compared to understanding that is formed through information processes only. references aguilera-ruiz, a., manzano, l., martínez, m., lozano, s., & casiano, s. (2017). el modelo flipped classroom. international journal of developmental and educational psychology, 4(1), 261-266. https://doi.org/10.17060/ijodaep.2017.n1.v4.1055 akçayır, g., & akçayır, m. (2018). the flipped classroom: a review of its advantages and challenges. elsevier, computers & education, 126, 334-345. https://doi.org/10.1016/j.compedu.2018.07.02 alamri, m. m. (2019). students’ academic achievement performance and satisfaction in a flipped classroom in saudi arabia. int. j. technology enhanced learning, 11(1), 103-119. https://doi.org/10.1504/ijtel.2019.096786 aprianto., ritonga, m., marlius, y., & nusyur, r. (2020). speaking skill in madrasah diniyah takmiliyah awwaliyyah. izdihar : journal of arabic language teaching, linguistics, and literature, 3(2), 147-160. https://doi.org/10.22219/jiz.v3i2.12514 ardhuha, j., & doyan, a. (2019). pengembangan modul ajar pada mata kuliah pendahuluan fisika zat padat untuk meningkatkan penguasan konsep struktur kristal bagi calon guru. jurnal pendidikan fisika dan teknologi, 5(1), 40-49. doi: http://dx.doi.org/10.29303/jpft.v5i1.951 arends, r. (2008). learning to teach: belajar untuk mengajar (jilid 2) (edisi 7). yogyakarta: pustaka pelajar. aşıksoy, g., & özdamlı, f. (2016). flipped classroom adapted to the arcs model of motivation and applied to a physics course. eurasia journal of mathematics, science & technology education, 12(6), 1589-1603. https://doi.org/10.12973/eurasia.2016.1251a basal, a. (2015). the implementation of a flipped classroom in foreign language teaching. turkish online journal of distance education, 16(4), 28–37. https://doi.org/10.17718/tojde.72185 baytiyeh, h. (2017). the flipped classroom model: when technology enhances professional skills. the international journal of information and learning technology, 34(1), 51-62. https://doi.org/10.1108/ijilt-07-2016-0025. bergmann, j., & sams, a. (2012). flip your classroom: reach every student in every class every day. washington dc: international society for technology in education. from https://www.rcboe.org/cms/lib/ga01903614/centricity/domain /15451/flip_your_ classroom.pdf byun, t (2014). why students still can’t solve physics problems after solving over 2000. american journal of physics, 82(9), 906-913. https://doi.org/10.1119/1.4881606 cohen, j. (1988). statistical power analysis for the behavioural science (2nd edition). new york: lawrence erlbaum associates. retrieved from https://www.utstat.toronto.edu/~brunner/oldclass/378f16/read ings/cohenpower.pdf damayanti, n., & sutama. (2016). efektivitas flipped classroom terhadap sikap dan ketrampilan belajar matematika di smk. manajemen pendidikan. 11(1). https://doi.org/0.23917/jmp.v11i1.1799 erduran, s., simon, s., & osborne, j. (2004) tapping into argumentation: developments in the application of toulmin's argument pattern for studying science discourse. science education.88(6):915-933. https://doi.org/ 10.1002/sce.20012 hake, r.r. (1999). analyzing change/gain scores. dept.of physics indiana university. retrieved from http://www.physics.indiana.edu hamdan, n., mcknight, p., mcknight, k., & arfstrom. (2013). a review of flipped learning flipped learning network-george mason university, retrieved from https://www.researchgate.net/publication/338804273 havwini, t., & wu. y. t. (2019). the implementation of flipped classroom in efl class: a taiwan case study. journal of digital education, communication, and arts, 2(02), 79–88. https://doi.org/10.30871/deca.v2i02.153 herlindayana., sahlan., & alberth (2017). the effect of flipped classroom on students’ reading comprehension. journal of language education and educational technology, 2(1), 1–16. http://dx.doi.org/10.33772/jleet.v2i1.6699 johnson, l., adams, s., & cummins, m. (2012). nmc horizon report: 2012 k–12 education austin, tx: the new media consortium. retrieved from https://files.eric.ed.gov/fulltext/ed593595.pdf kozikoğlu, i. (2019). analysis of the studies concerning flipped learning model: a comparative meta-synthesis study. international journal of instruction, 12(1), 851–868. https://doi.org/10.29333/iji.2019.12155a låg, t., & sæle, r. g. (2019). does the flipped classroom improve student learning and satisfaction? a systematic review and metaanalysis. american educational research association journals (aera) open, 5(3), 1–17. https://doi.org/10.1177/2332858419870489 lakens, d. (2013). calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and anovas. frontiers in psychology. 4. 1-12. doi: https://doi.org/10.3389/fpsyg.2013.00863 mehta, c.r., & patel, n.r. (2012). ibm spss exact tests. ibm corporation. retrieved from https://www.ibm.com meilantari, n.l.g. (2021). penerapan flipped classroom pada pembelajaran daring bahasa jepang di kelas xii smk saraswati 3 denpasar, jurnal pendidikan bahasa jepang (jpbj). 7(1). 80-89. https://doi.org/10.23887/jpbj.v7i1.31100 https://doi.org/10.17060/ijodaep.2017.n1.v4.1055 https://doi.org/10.1016/j.compedu.2018.07.02 https://doi.org/10.1504/ijtel.2019.096786 https://doi.org/10.22219/jiz.v3i2.12514 http://dx.doi.org/10.29303/jpft.v5i1.951 https://doi.org/10.12973/eurasia.2016.1251a https://doi.org/10.17718/tojde.72185 https://doi.org/10.1108/ijilt-07-2016-0025 https://doi.org/10.1119/1.4881606 https://www.utstat.toronto.edu/~brunner/oldclass/378f16/readings/cohenpower.pdf https://www.utstat.toronto.edu/~brunner/oldclass/378f16/readings/cohenpower.pdf https://doi.org/0.23917/jmp.v11i1.1799 http://www.physics.indiana.edu/ https://www.researchgate.net/publication/338804273 https://doi.org/10.30871/deca.v2i02.153 http://dx.doi.org/10.33772/jleet.v2i1.6699 https://files.eric.ed.gov/fulltext/ed593595.pdf https://doi.org/10.29333/iji.2019.12155a https://doi.org/10.1177/2332858419870489 https://doi.org/10.3389/fpsyg.2013.00863 https://www.ibm.com/ https://doi.org/10.23887/jpbj.v7i1.31100 journal of science learning article doi: 10.17509/jsl.v6i2.56316 203 j.sci.learn.2023.6(2).194-203 newton, p., driver, r., & osborne, j. (2010) the place of argumentation in the pedagogy of school science international journal of science education. 21(5). 553-576. https://doi.org/10.1080/095006999290570 nouri, j. (2016). the flipped classroom: for active, effective and increased learning—especially for low achievers. international journal of educational technology in higher education. 13(33). https://doi.org/10.1186/s41239-016-0032-z novia, h., rusdiana, d., & kaniawati, i. (2013). analisis kesulitan konsep strutur kristal pada perkuliahan fisika zat padat bagi calon guru fisika. prosiding seminar nasional sains dan pendidikan sains viii, fakultas sains dan matematika, universitas kristen satya wacana institutional repository. retrieved from http://repository.uksw.edu/handle/123456789/3115 o’flaherty & phillips. (2015). the use of flipped classrooms in higher education: a scoping review. elsevier, internet and higher education.25. 85-95. https://doi.org/10.1016/j.iheduc.2015.02.002 parno. (2012). peningkatan prestasi belajar matakuliah pilihan fisika zat padat mahasiswa pendidikan fisika melalui model stad dan strategi self-explating. jurnal pendidikan fisika indonesia. 8(2). 115126. https://doi.org/10.15294/jpfi.v8i2.2151 purwaningsih, e. (2021). membangun profesionalisme dan identitas calon guru fisika. kementerian pendidikan kebudayaan riset dan teknologi universitas negeri malang. retrieved from http://repository.um.ac.id/1194/1/fullteks%203.pdf reidsema, c., kavanagh, l., hadgraft, r., & smith, n (eds). (2017). the flipped classroom. singapore: springer. https://doi.org/10.1007/978-981-10-3413-8 rollnick, m. (2017). learning about semimconductors for teaching— the role played by content knowledge in pedagogical content knowledge (pck) development. research in science education, 47(4) 833–868. https://doi.org/10.1007/s11165-016-9530-1 santoso, s. (2014). statistik parametrik edisi revisi. jakarta: elex media komputindo. subali, b., rusdiana, d., firman, h., & kaniawati, i. (2015). analisis kemampuan interpretasi grafik kinematika pada mahasiswa calon guru fisika. prosiding simposium nasional inovasi dan pembelajaran sains (institut teknologi bandung, bandung, indonesia). 268-272 sterlan, p., osborn, a., & palmer, e. (2020). student satisfaction with courses and instructors in a flipped classroom: a meta-analysis. journal of computer assisted learning. 36(20). 1-20. https://doi.org/10.1111/jcal.12421\ wenning, c.j. (2006). framework for teaching the nature of science. journal of physics teacher education online. 3(3). 3-10 yulianti, y.a & wulandari, d. (2021). flipped classroom: model pembelajaran untuk mencapai kecakapan abad 21 sesuai kurikulum 2013. jurnal kependidikan:jurnal hasil penelitian dan kajian kepustakaan di bidang pendidikan, pengajaran dan pembelajaran. 7(2). 372-384. https://doi.org/10.33394/jk.v7i2.3209 https://doi.org/10.1080/095006999290570 https://doi.org/10.1186/s41239-016-0032-z http://repository.uksw.edu/handle/123456789/3115 https://doi.org/10.1016/j.iheduc.2015.02.002 https://doi.org/10.15294/jpfi.v8i2.2151 http://repository.um.ac.id/1194/1/fullteks%203.pdf https://doi.org/10.1007/978-981-10-3413-8 https://doi.org/10.1007/s11165-016-9530-1 https://doi.org/10.1111/jcal.12421/ https://doi.org/10.33394/jk.v7i2.3209 microsoft word 71-76_the impact of problem-solving model on students’ concept mastery and motivation in learning heat based o a © 2018 indonesian society for science educator 71 j.sci.learn.2018.1(2).71-76 received: 31 january 2018 revised: 26 march 2018 published: 31 march 2018 the impact of problem-solving model on students’ concept mastery and motivation in learning heat based on gender annisa utami putri1, lilit rusyati1*, diana rochintaniawati1 1international program on science education, faculty of mathematics and science education, universitas pendidikan indonesia, indonesia *corresponding author. lilitrusyati@upi.edu abstract this study investigated the impact of the problem-solving model on students’ concept mastery and motivation in learning heat based on gender. the method which was used in this research was quasi-experiment with pretest-posttest design in girls and boys class. data are collected from girl class (n=16) and boys class (n=16) of a 7th grade in one of a bilingual boarding school in bandung. the quantitative data of this research was obtained through the objective test, while the qualitative data was gained through a questionnaire. students’ motivation is measured by attention, relevance, confidence, and satisfaction (arcs) model. on the other hand, students’ concept mastery is measured based on bloom’s taxonomy cognitive level through the objective test and being analyzed by using statistical software statistical package for social science (spss) version 20.0 for windows 8. the results show that overall concept mastery of girls and boys students have improved. however, girls students are higher achiever than boys. overall, the problem-solving model makes an improvement in students’ concept mastery in both girls and boys. thus, it indicates that the implementation of the problem-solving model in learning heat can improve students’ motivation and students’ conceptual understanding in secondary level. keywords problem-solving model, students’ concept mastery, students’ motivation, heat, gender 1. introduction the basic goal of science education system is bringing in skills to get information instead of transferring to literal at the present day in the information age. one of the purposes of science education reformation is to train up students who are interested in science actively. as a result of these rapid changes, the education systems need to be modified that they can enable the students to learn the ways to reach the knowledge, improve the skills of decisionmaking and to solve problems (lorsbach & tobin, 1992). much of the literature concerning the relative performance of girls and boys in science indicates that boys outperform girls in most areas of the science curriculum and that it is rare to identify areas of the science curriculum in which girls outperform boys (hola, 2005). student's attitudes towards science have been found in some studies to be linked to performance in the subject and have been found to be influenced by different teaching approaches (hola, 2005). students process of acquiring knowledge and achieving the goal in the class activity commonly done by their own effort (ajaja & eravwoke, 2010). it is a common problem for middle school students that school is boring and that they cannot relate to nor understand the material that is presented to them each day in class (king, 2009). students are neither passionate nor motivated to learn in class. this problem can be both challenge and opportunity for the teacher to deliver the material in an alternative way by engaging the students to learn in class. students’ engagement can be done by providing several activities which lead them to have experience and connect them immediately to the knowledge (umam, 2014). in another hand, application of interaction among students will help to reshape and develop students reasoning, critical thinking, and some others students’ ability. cooperative learning is one of the ways in teaching learning process which gives chance to students to interact each other in order implementing their learning ability. this interaction will eventually lead students to shape their values and perspective that can be useful to be used either inside or outside of class (ajaja & eravwoke, 2010). one of the fundamental achievements of education is to enable students to use their knowledge journal of science learning article 72 j.sci.learn.2018.1(2).71-76 in problem-solving. therefore, many researchers find that their students do not solve problems at the wanted level of proficiency. to help improve the teaching and learning of physics problem solving, studies were started in the 1970’s (gok and silay, 2010). two primary goals in teaching introductory physics are to help students learn major concepts and principles and to help students learn how to apply them to solve the problems. in traditionally taught courses we assign many problems with the assumption that solving the problems will help develop students in the understanding of concepts and principles, as well as an appreciation of the role they play in solving problems. the research on student’s concept has been being approved as a key concern for science learning especially after ausuble has developed the idea about the importance of prior knowledge to the learning of scientific knowledge (ausuble, 1963). students nowadays are lack of critical thinking and they are only good at memorizing. instead of giving readily prepared information, teaching students to learn how to learn, make comments, getting them to understand and apply the information is needed in science courses. furthermore, making them gain skills of problem solving, behaviors and helping them to gain a habit of scientific thinking should be taught. therefore, it is needed to improve students’ skills of problem-solving (altunçekiç, yaman & koray, 2005). motivation is the determining factor in learning since students who do not want to learn will not learn regardless of the caliber of the instructor, and students who do want to learn will. however, students' motivations may change, such that even those who do not want to learn will change their minds upon exposure to stimulating environments that capture their attention. 2. method the quasi-experimental method is used in this research to collect and obtain data in the field. researcher uses this experimental research to test the effect of the problemsolving model on students’ concept mastery. in this method, research carried out on two experimental groups of 7th-grade students which are girl class and boy class. pretest and post-test design definition from cresswell (2012) provides a measure on some attribute or characteristic that is assessed for participants in an experiment before they receive a treatment. post-test can be used to assess participant in experiment after a treatment. the experiment design is shown in table 1. the location of this research is taken in one of bilingual school in bandung. this school is one of the pasiad partner school which uses kurikulum 2013 along with zambak modular system. this school named as a bilingual school because it uses bahasa and english as its instructions language. this school is appropriate for this research because it has a different system of class division. the class of girl and boy students are separated. the population in this research is 7th-grade students at bilingual junior high school in bandung. the sample is taken from two classes of the 7th grades. there are 7 a and 7 b as different based on gender classes. 7 a is boys’ class and 7 b is girls’ class, both of classes are given the same treatment. a sampling is selected by purposive sampling technique according to fraenkel et.al., (2011). the sampling consideration is based on the specific purpose which compares the cognitive achievement of the girl and boy students, therefore researcher chooses a sample from based gender classes (fraenkel et.al., 2011). the total samplings are 32 students of a 7th grader, with 16 girl students and 16 boy students. in this research, the concept of heat topic is limited based on indonesian curricula 2013 by core competence no. 3, basic competence no 3.7 as attached in badan standar nasional pendidikan (2013). the analysis of curriculum about core competence and basic competence indicates the subtopics that will be investigated by students such as (1) temperature (2) heat and expansion (3) heat transfer (4) its application in the body to maintain the stability of body temperature in humans and animals in everyday life. the terms of heat in this research refer to a form of energy associated with the motion of atoms or molecules of a body and the subject encompasses temperature, thermometry, heat transfer, such as conduction, convection, and radiation. there are three types of instrument used in this research. there are an objective test, observational sheet, and questionnaire. first, the objective test is conducted to describe the cognitive ability of students in mastering the concept. the objective test consists of two sections which are pre-test and post-test. multiple choice questions consist of cognitive domain c1 (remembering), c2 (understanding), c3 (applying), c4 (analyzing) and c5 (evaluating) (anderson et al., 2001). the objective test consists of twenty-seven questions before passing judgment by experts. it is used to look students’ comprehension. after judging by the expert the objective is only twenty-five questions as a representative for each learning indicators. then, the test was distributed to students in grade 7 as a limited test. the next step after conducting a limited test to 7th-grade students, this objective test is analyzed using anates to measure the validity, reliability, difficulty level, discriminating power and table 1 experiment design class test stage i test action girl pretest problem-solving model posttest questionnaire boy pretest problem-solving model posttest questionnaire journal of science learning article 73 j.sci.learn.2018.1(2).71-76 distractor. second, observational sheets have been created so that teaching staff can take the opportunity to observe and reflect on the particular positive teaching and behavior management strategies employed. third, questionnaire is an instrument, which is distributed to the students to investigate students’ motivation towards this learning model which is consist of 15 questions that assess attention, relevance, confidence, and satisfaction towards this learning model. 3. result and discussion the results show quantitative and qualitative data. the pre-test and the post-test are conducted to determine the students’ concept mastery before and after treatment. qualitative analysis will describe the students’ motivation during this learning model. 3.1 students’ understanding the pretest was conducted at the beginning of the learning to investigate the students’ prior knowledge. a pretest with the same question was given to both girl and boy class. statistic result shows that the data of normality test with significance 0.520 for girl class’ pre-test with criteria sign. ≥ 0.05. it can be categorized as a normal distribution. it is the same with boy class, 0.772 for the pretest. meanwhile, for homogeneity test, the result of pre-test (both girl and boy class) is 0.585 with criteria and for posttest is 0.004. both of them can be categorized as homogeny. for pre-test, girl class shows 80 for the highest score with standard deviation 17.220 and 92 is the highest score in the girl class for post-test with the standard deviation 7.711. meanwhile, in boy class, it reaches 72 for pre-test with standard deviation 17.435 and 92 for post-test with standard deviation 16.360. the analysis of pre-test resulted that there is a difference between boy and girl class’ result. although the difference is not really significant, this result indicates that prior knowledge between girl and boy are quite similar but girls have a more comprehensive understanding. the average score of pre-test in girl class is 54 men meanwhile in boy class is 45. these results are categorized as a low result because this is the first time for the students to learn heat concept in a comprehensive way. chances are primary school have taught them the heated topic but it probably only the basic knowledge as it can be seen from the result that they have been already good in the basic theory of heat concept. post-test is given at the end of the learning process to know students’ concept mastery after receiving the treatment. post-test are also given to both girl and boy class. statistic result shows that the data of normality test with significance 0.591 for girl class and 0.397 for boy class. it is categorized as normal with criteria sign. ≥ 0.05. homogeneity test with significance 0.711 can be categorized as homogeny. girl class shows the highest score 92 and the lowest score 60 with the average score 84.5 and standard deviation 7.711. for boy class, the highest score is the same as the girl class, 92 but for the lowest score is 40 with the average score 71.7 and the standard deviation 16.360. the analysis of post-test showed that there is also a difference between girl and boy class post-test result. according to the students’ average score (84.5 for girl class and 71.7 for boy class), problem-solving model contributes to students’ concept mastery improvement. this result indicates that problem-solving model can improve students’ concept mastery supporting previous research conducted by aka, aydogdu, and guven (2010). it showed that problem-solving model was able to make students have a more comprehensive understanding. another analysis is performed to investigate students’ concept mastery improvement using problem-solving model. analysis of n-gain is conducted to investigate the statistically significant differences in students’ understanding improvement between pre and posttreatment. the average of n-gain result is tabulated in table 2. the result of the girl class is higher than resulted in the average. it achieved 0.83 n-gain for girl class and 0.62 for boy class. conceptual mastery in this study is also analyzed from the cognitive domain refer to bloom taxonomy revised of cognitive domain (anderson, 2001). the cognitive domain table 2 result analysis of n-gain component girl class boy class pretest posttest pretest posttest n 16 16 16 16 average 54 84,5 45 71,7 standard deviation 17.220 7.711 17.435 16.360 highest score 80 92 72 92 lowest score 16 60 16 40 gain score 30.50 26.75 n-gain score 0.83 0.62 n-gain n-gain interpretation result <g> ≥ 0.7 : high 15 6 0.7 < <g> ≥ 0.3 : medium 1 9 <g> < 0.3 : low 0 1 homogeneity test (levene statistics) pre-test post-test sig. 0.585 0.004 conclusion homogeny normality test (kolmogorov-smirnov) sig. 0.520 0.591 0.772 0.397 conclusion normal t-test sig. 0,007 conclusion ho rejected journal of science learning article 74 j.sci.learn.2018.1(2).71-76 which involves in this research is remembering (c1), understanding (c2), applying (c3), analyzing (c4) and evaluating (c5). the improvement in each cognitive domain also categorizes as a medium until high improvement for both girl and boy classes. but most of the high improvements are categorized in c3 and c4 cognitive domain. this result establishes that problem-solving model works well in higher order thinking level, especially c4 (analyzing), correlate the problem-solving model to cognitive taxonomy. constructing and learning by using this model factually improve the conceptual mastery in each cognitive domain, and better in the c3-c4 cognitive domain. c3 deals with applying concept meanwhile c4 deals with analyzing. this is in line with the hypothesis that the ability of information analysis & processing. thus, this learning model better improves the higher level of the cognitive domain. it proves that the problem-solving model support student to higher thinking ability as analyzing and applying. those phenomena show that problem-solving model works in helping the student to improve their conceptual mastery. besides that, the problem-solving model also facilitates student to involve most of all their senses, improve their critical thinking and stimulate their curiosity. another factor which is involved in the role of the problem-solving model in improving students’ level of cognitive is the involvement of critical thinking when students are learning. the problem-solving model here facilitates students’ imagination in the process of learning. learning physics, especially on the topic of heat transfer is not that easy for some students because there are a lot of things that could be leading into misconception. an objective test of students’ understanding is given to the students. the test includes four sub-indicators concepts. table 3 shows the comparison of the result of the objective test. based on n-gain, the problem-solving model is resulting good in students’ concept mastery in general knowledge heat concept. it has the highest score of both boys and girls classes. it also makes an improvement in several indicators such as temperature, heat transfer and heat concept application in daily life. it can be seen from the result that the students either boys and girl can differentiate easily the way of heat transfer at the end of the learning without misconception. according to the curricula 2013 minimum standard, both girls and boys classes’ result have achieved 5 indicators accomplished by students during the learning activity. the improvement in indicator about the application of heat concept on daily basis is not significant, but fortunately, overall result can make students pass this concept. students’ critical thinking might be needed to improve this understanding so that students can fulfill this indicator to the maximum score. problem-solving model is also suggested to improve students’ critical thinking and how a student can evaluate their own work. it is suitable to be used for heat concept because this concept of heat needs actual explanation or contextual explanation so they can imagine by themselves what it is like and what it relates to their life and they can easily understand the concept. 3.2 students’ motivation students’ motivation toward this learning model is obtained by questionnaire. this questionnaire consists of 15 questions that have positive and negative statements which represent the point of attention, relevance, confidence, and satisfaction. this questionnaire also consisted of three different indicators of the student's response toward instructions using the problem-solving model in learning heat. those statements in questionnaire also indicate interested in learning leisure in using problem-solving model, response toward the usefulness of the problemsolving model, and indicate students’ preference of learning method. this questionnaire is distributed to determine table 3 students’ concept mastery result based on gender class subcategory concepts questions number of students max score pre-test score post test score boys general heat concept 2, 4, 6, 9, 11, 15, 18, 19, 21, 24 16 160 70 150 heat transfer 1, 3, 5, 12, 22, 25 96 30 90 temperature 8, 14, 16 48 12 40 heat concept on daily life 7, 10, 13, 17, 20, 23 96 24 72 girls general heat concept 2, 4, 6, 9, 11, 15, 18, 19, 21, 24 16 160 50 100 heat transfer 1, 3, 5, 12, 22, 25 96 18 72 temperature 8, 14, 16 48 15 27 heat concept on daily life 7, 10, 13, 17, 20, 23 96 30 60 table 4 students’ response toward problem-solving model on their motivation in percentage score category numbers percentage boys girls boys girls 1.00–1.49 very unmotivated 0 0 1.50–2.49 unmotivated 0 0 2.50–3.49 sufficiently motivated 3 1 18.75 6.25 3.50–4.49 motivated 10 3 62.5 18.75 4.50–5.00 highly motivated 3 12 18.75 75 journal of science learning article 75 j.sci.learn.2018.1(2).71-76 students’ motivation and interests toward this learning model. summary of the result of students’ motivation towards this model represents in table 4. it is shown that girls have higher motivation about the instructions than boys. it can be seen from the value of the percentage. it is 75% of students in girl class are highly motivated toward this learning. meanwhile, it is only 62.5% of students in boys class are only motivated. however, students’ motivation toward this learning was appeared to be alright. it can be concluded that they have resulted in the motivated rate of learning heat using problem-solving model. even so, due to the needs of a deeper analysis of the arcs motivational categories which are attention, relevance, confidence, and satisfaction, which has negative and positive statements, the implementation of the problem-solving model is expected to gain the motivation of students to learn and understand the heat chapter. the arcs motivational questionnaire consisting of 15 statements resulted in various response and different rates of each category of attention, relevance, confidence, and satisfaction. from 32 students participated in this research that is consisted of 16 girl students and 16 boy students, there is 10 boy students are classified as motivated students. while other 3 students are sufficiently motivated and the other 3 student is highly motivated. corresponding to borphy (2011) at “motivating students to learn” stated that establishing the class with learning communication through emphasizing that learning goals are more important than performance goals so students are paying attention to self-improvement rather than comparison with classmates. this is what happening insufficiently motivated category students who are more importantly keep themselves motivated without worrying. girl class has a higher percentage of highly motivated students than boy class. it means that girl class tends to have a positive response in each indicator. in the first indicator which is attention, the girl’s class averagely have score 4.7 from the scale of 1-5. this can be interpreted that girl’s class is in highly motivated to have the leisure when they learn using problem-solving model. meanwhile, boy’s class tend to have a positive response but the score of boy class for this indicator is diverse and mostly in the motivated categorized which is 3,5 – 4,49. it can be interpreted that their response is not really good for the learning leisure when they learn using problem-solving model. a second indicator, both of girl and boy class also have the diverse contribution of frequency but girl’s class still in the lead in the motivated and highly motivated category. it can be interpreted that they tend to have positive response toward the relevance category. although mostly boy and girl students are categorized as motivated and highly motivated students, there are still some few people who are sufficiently motivated during this instruction. approximately 3 students in total both of classes according to table 5 are just only sufficiently motivated during the instructions, but still, boy students seem like not really motivated toward this leaning. this might happen because of the different style of different gender student, as it is stated in the previous explanation based on theory of kolb (1984) women tend to prefer concrete learning styles, and boy prefers to an abstract one. problem-solving model facilitates student to learn in a concrete way. thus, girl class is mostly like and feeling helpful and motivated when they learn using problem-solving model. hence from all of the discussion above it can be interpreted, generally most of girl student response positive toward this learning strategy, which means they all are generally feel satisfy, relatable and prefer this learning strategy than the conventional one. only around half of boy students give the positive response toward this learning strategy. this preference also gives any impact on the learning outcomes. girl students which show more positive attitudes toward this learning model result a higher improvement than the boy who less interested in this learning model. the preference and feeling of joy might increase the learning motivation. so that it gives any big impact on learning outcomes, this is in line with what ormrod (2008) stated in his book about the girl motivation in learning. it can be concluded from all of the discussion above that the girl class has a higher motivation than boy class. girl class gives more positive response than boy students in all of the category. there is learning leisure during the instructions using problem-solving model, feeling helpful while learning using problem-solving model, and the preference of student to learning using problem-solving model compare to conventional learning strategy. this positive response of girl class is given a positive impact on the achievement of girl class. girl’s class who give more positive response toward problem-solving learning model gain more achievement than boy students who do not really respond positively toward this learning model. 4. conclusion research about the impact of the problem-solving model on students’ concept mastery and motivation has been conducted systematically. according to the research table 5 recapitulation of arcs categorization of students’ motivation score number of students (f) attention relevance confidence satisfaction boys girls boys girls boys girls boys girls 1.00-1.49 1.50-2.49 1 2 1 1 2.50-3.49 1 1 1 3 3 3 3.50-4.49 9 4 7 6 7 1 7 4 4.50-5.00 5 11 6 10 5 12 5 12 journal of science learning article 76 j.sci.learn.2018.1(2).71-76 results, it is obtained some conclusions as follows. the implementation of the problem-solving model in learning heat chapter can improve students’ conceptual mastery. it is noticed and proven by the results of average n-gain of both boys’ and girls’ class. the average n-gain result obtained by girl’s class is 0.83 which is categorized as high, while boy’s class got 0.62 for an average n-gain result which can be categorized as a medium. the result indicates that the concept of heat can be understood better by the students after having instruction using problem-solving model. the improvement of students’ conceptual mastery is also supported by the acceptance of h1 which means that there is a significant effect of the problem-solving model towards students’ conceptual mastery. the implementation of the problem-solving model in learning heat chapter can improve students’ motivation. it is noticed and proven by processing data of boy students’ response percentage. the percentage of girl students’ motivation class is higher than the percentage of boy students’ motivation. the response of students’ motivation towards the implementation of the problem-solving model in learning heat chapter shows positive response in two indicators; leisure and usefulness. the highest score is obtained by the indicator of usefulness which means that the students agree that problem-solving learning is stimulating students’ curiosity. references ajaja, o. p., & eravwoke, o. u. (2010). effects of cooperative learning strategy on junior secondary school students achievement in integrated science. journal of science education. 4(1), 2-18. aka, e.i., guven, e., aydogdu, m. (2010). effect of problem solving method on science process skills and academic achievement. journal of science education. 17(6), 3-19. altunçekiç, a., yaman, s. & koray, ö. (2005). a study on levels of selfefficacy beliefs and problem-solving skills of teacher candidates. kastamonu education journal, 13(1), 93–102. anderson & krathwohl. (2001). a taxonomy for learning, teaching, and assessing: a revision of bloom’s taxonomy of educational objective. new york: longman publishing. ausuble, p. d. (1963). the psychology of meaningful verbal learning. new york: grune & stratton. bloom, b. s. taxonomy of educational objectives: the classification of educational goals (new york: david mckay, 1956). fraenkel, j. r., wallen, n. e., & hyun, h. h. (2012). how to design and evaluate research in education. new york: mcgraw-hill. gok, t. & silay, i. (2010). the effects of problem solving strategies on students’ achievement, attitude, and motivation. journal of science education, 28(2), 4-16. king, s. h., et.al, (2009). project-based learning: inspiring middle school students to engage in deep and active learning. nyc department of education. 7-10 kolb, d.a. (1984). experiential learning, experience as a source of learning and development. englewood cliffs nj: prentice-hall. lorsbach, a. & tobin, k. (1992). constructivism as a referent for science teaching. in lawrenz, f. research matters to the science teacher. monograph number 5. kansas state university: national association for research in science teaching. microsoft word published_the development and validation of science virtual test to assess 7th grade students’ critical thinki a doi: 10.1021/xxx.xxxx.xxxxxx © 2017 indonesian society for science educator 17 j.sci.learn.2017.1(1).17-27 received: 28 october 2017 revised: 29 november 2017 published: 30 november 2017 the development and validation of science virtual test to assess 7th grade students’ critical thinking on matter and heat topic yustika sya’bandari1*,harry firman2, lilit rusyati1 1international program on science education, faculty of mathematics and science education, universitas pendidikan indonesia 2department of chemistry education, faculty of mathematics and science education, universitas pendidikan indonesia *corresponding author. yustika.sya’bandari@student.upi.edu abstract an efficient way to improve the quality of education in critical thinking is developing the better tests. the test has been shifted towards the use of computer-based procedures. lack of specific topic of critical thinking tests in science and the advanced of technology made the researcher intended to develop and validate a test to measure students’ critical thinking in the matter and heat topic based on computer for seventh grade junior high school. the method that used in this research was descriptive. generally, the process of developing and validating the test consists of 5 steps: (1) content analysis; (2) constructing multiple choice items; (3) readability test and expert validation; (4) limited tryout; and (5) larger application. based on larger application, it is obtained the reliability value, difficulty level, discriminating power and distractor quality. the subject was 117 students of public junior high school in kabupaten bandung. the instrument validation resulted 30 items that represent 8 elements and 21 sub-elements to measure students’ critical thinking based on inch in matter and heat topic. the instrument is further called as science virtual test matter and heat (svt-mh). the alpha cronbach (α) is 0.642 which means that the instrument is sufficient to measure students’ critical thinking matter and heat topic. the result also shows the profile of students in critical thinking is 73% categorized as ‘moderate’, while students respond the positive impression towards the use of svt-mh. keywords critical thinking, matter and heat, reliability, science virtual test, students’ impression, validity. 1. introduction in the 21st century, there have been shifts in the way we live as well as in how we get and evaluate information. these shifts mean we need to look critically at critical thinking itself and determine what approaches are most effective to meet the challenges and opportunities (anne & kreitzberg, 2010). critical thinking is the skill to prevent people from making bad decisions and helps to solve the problems (inch, warnick & endres, 2006). critical thinking makes us think not just about the world around us but also about the thought process itself (harpen, 2003). consequently, teaching critical thinking skills is important for students, as they need to adjust to such change by actively and skillfully conceptualizing, applying, analyzing, synthesizing, evaluating the information gathered from, or generated by, observation, experience, reflection, reasoning, or communication (paul & elder, 2004). teaching critical thinking at schools is the main topics in the discussion regarding 21st century skills (greenhill, 2009). students can be taught to critically examine different viewpoints on issues concerning the impact of science and technology on everyday life and evaluate these issues through critical thinking (mapeala & siew, 2015). teaching critical thinking in science and technology also helps to develop students’ analytical skills, as well as their ability to make informed choices in their everyday lives (mapeala & siew, 2015). according to (unesco, 2000) measuring the improvements of critical thinking skills will essentially improve the education quality. if tests are understood to shape both the curriculum and teaching, an efficient way to improve the quality of education in critical thinking is developing the better tests (yeh, 2001). tiruneh et al. (2016) stated that critical thinking tests which have been developed such as the cornell critical thinking test–level z (cctt), the california critical thinking skills test (cctst), the ennis-weir critical thinking essay test, the watson-glaser critical thinking appraisal, and the halpern critical thinking assessment (hcta) were mostly general content-based and reviewed based on the following criteria: (a) is the test based on a clear journal of science learning article 18 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).17-27 definition/conception of critical thinking, (b) are the targeted critical thinking skills common across tests, and (c) do the test items appear to sufficiently measure the critical thinking skills targeted on a test. all the general content of critical thinking tests are diverse in terms of their formats, scope, and psychometric characteristics. recognizing the lack of specific content of critical thinking tests, it is required to develop critical thinking test in specific subject area. the accompanying expectation has been embedding critical thinking skills within a subject matter instruction in various specifics content will facilitate the acquisition of critical thinking skills that are applicable to a wide variety of thinking tasks within the content in question and that it will facilitate their transfer to other problems in everyday life (lawson, 2004). the present study addressed this concern by developing and validating a critical thinking test based on inch critical thinking in science content for junior high school student. the higher order thinking and complex information in the critical thinking test can be supported by the advanced technology in its development. the assessment also has been shifted towards the use of computer-based procedures (jurecka 2008). the computer based test (cbt) is a test that the presentation and selection are computerized which can simulate real-world problems (illstructured and complex in nature) (rusyati & firman, 2017). computer based-test enables teachers and researchers to collect different types of data. hence, researchers have the possibility to design meaningful and motivating real-life scenarios, where students can solve complex and interactive problems (greiff et al. 2013). the information in the computer based-test can be presented not only using picture, the item which is presented also can use the interactive video. even in the field of science education, there are different approaches to use computers in order to facilitate the work on complex problems (jonassen, 2004). computer based test is effective solution for education evaluation (flagbola, adigun & oke, 2013). in addition, to traditional scores on multiple-choice or constructed-response items, data on the time needed to perform a number of interactions and the sequence of operations are accessible (wirth, 2008). the major advantage of computer-based tests is in the assessment of new content areas and constructs (scherer, koppelt & tiemann, 2014). the use of computer based assessment is advantageous due to test economics, improvements in objectivity, and test reliability (scherer, koppelt & tiemann, 2014). the ministry of education and culture of indonesia also has launched the national examination based on computer in 2014 in smp indonesia singapura dan smp indonesia kuala lumpur (sikl). ujian nasional berbasis komputer (unbk) also called computer based test (cbt) is a national examination using the computer as a media. in the implementation, unbk is different with paper national examination or paper based test (pbt) which has been running out (kemendikbud, 2016). it means, developing a test should follow the current technology in order to have the better benefits. according to agah, ogbeche & okorie (2016), there was a positive and direct relationship between computer anxiety, operation skills, attitude and students’ preparedness for computer-based assessment. by using this developed test, students can practice before they conduct the national examination to prevent the influences of their computer anxiety, operation skills and attitude in using computer. the author intended to develop the instrument in the topic of matter and heat because in 2013 curriculum, this topic integrates each other which is connected from basic competence 3.3 and 3.4. based on rahayu & kita (2009) students have great difficulties with and hold some alternative conceptions of the concepts of matter (homogeneous mixtures, phase changes from solid to liquid, and phase changes from solid to gas). so, matter and heat topic is also important to be presented virtually because the content which is abstract for students is easier to be presented in the form of animation or interactive video. thus the present study, first, proposes to develop and validate test to assess 7th grade students’ critical thinking based on inch critical thinking on the topic of matter and heat. author defines critical proficiency as the ability to reasonably respond to critical thinking tasks that do not necessarily require specific content knowledge, but rather application of content knowledge of everyday life. the topic which is included is based on the basic competence of 2013 curriculum, matter and heat topic that consists of mixture, element, compound, physical and chemical properties, physical and chemical change, state of matter and its change, temperature, expansion, heat and heat transfer sub-topic. second, the design and characteristics of the assessment tool which use the computer based test are described. the design of critical thinking test using the computer based-test will be in the form of “science virtual test” which not only can assess students’ critical thinking but also can used as preparation for students’ before conducting another computer based examination. 2. method the research method that used in this research was descriptive research. according to mcmillan and schumacher (2001) descriptive research is concerned with the current or past status of something and simply describe achievements, attitudes, behaviors or other characteristics of a group of subject. gall, gall and borg (2003) stated that descriptive research is a research that involves making careful descriptions of educational phenomena. descriptive research involves gathering data that describe events as organizes, tabulates, illustrate and journal of science learning article 19 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).17-27 describe data collection. therefore, the research method in this research is descriptive because appropriate with the objectives of the research which is describing the development and validation of critical thinking instrument based on inch to describe 7th grade students’ critical thinking on matter and heat topic. the location of this research was public junior high school “x” in kabupaten bandung. the school uses bahasa indonesia in teaching learning process. 2013 curriculum is implemented for 7th and 8th grade, while 9th grade is still using ktsp curriculum. this school appropriate because the instrument intended for 7th grade which uses 2013 curriculum. this school has 540 students of 7th grade, 551 students of 8th grade and 533 students of 9th grade in 2015. this school was chosen because it has been accredited “a” by indonesia ministry of education and also supported by multimedia facilities e.g. personal computer. the subject was students from three classes of 7th grade students who have learnt about matter and heat topic. the average age is about 12-13 years old. for the descriptive research, the subject is better more than 100 students. in this research, researcher involved 117 seventh grade junior high school student consist of 60 female students and 57 male students. critical thinking was measured by science virtual test that has been developed based on inch elements of critical thinking. the elements are purpose, question at issue, assumption, point of view, information, concepts, interpretation and inference, implication and consequences. these eight elements also developed to be 26 sub elements. these elements can lead student the students to the level of critical thinking. the science virtual test (svt) is the test that use computer as media. it was developed using macromedia flash professional 8.0. the test is multiple choice questions with four option of critical thinking instrument. the content validity was measured by expert judgment through rubric assessment and students responds that eventually was analyzed through ibm statistic 23 and anates 4.1.0. there are two types of instrument used in this research. there are rubric and questionnaire. first, the rubric was used to validate the content of the instrument. the validation was divided into content, media and education which have its own aspects to be assessed. second, the questionnaire was used to know students’ impression after using the science virtual test. it consists of four aspects which are experience, technical, preference and media. students’ impression uses the likert scale analysis. in likert, subject reads every statement in the questionnaire and evaluates the question based on the categorized answer (suherman, 2003). there are 4 answers that can be chosen by students and it has its own score. for the favorable statement (positive statement), the strongly agree has 5 score, agree has 4 score, disagree has 2 score, strongly disagree has 1 score, and neutral score is 3. 3. result and discussion the result and discussion founded in this research is divided into 5 parts: (1) the development of science virtual test, (2) the validity of science virtual test, (3) the reliability of science virtual test, (4) the characteristics of science virtual test, (5) the profile of students’ critical thinking and students’ impression toward svt. 3.1 the development of science virtual test the importance of developing measurement for students to think critically especially on specific subject of science continues to grow. researchers need to have valid and reliable tests to evaluate the effectiveness of various instructional efforts. in this study, the author has developed the instrument to measure students’ critical thinking on the matter and heat topic which implements the computer based test. this instrument is further called ‘science virtual test matter and heat or svt-mh’. the development of science virtual test to measure students’ critical thinking on matter and heat topic consists of 5 steps: (1) content analysis of 26 elements inch critical thinking framework and the matter and heat topic; (2) constructing the multiple choice items; (3) readability test by 3 junior high school students and 2 science teacher, validity test by 2 education expert, 2 content expert and 2 media expert; (4) conducting limited tryout by 40 students of junior high school; (5) conducting a larger test by 117 students of junior high school. after the content was analyzed, the items were constructed, it resulted the preliminary design of science virtual test. the preliminary design tested its readability, validity and reliability (limited and larger test) result the final design of svt-mh. 3.1.1 the preliminary design of science virtual test  matter and heat  svt-mh consists of 41 items based on two basic competences of 2013 curriculum: (1) explain the concept of mixture and pure substance (element and compound), physical and chemical properties, physical and chemical changes in daily life; (2) analyze the concept of temperature, expansion, heat, heat transfer and its implementation in daily life including the mechanism in maintaining the stability of human and animal body temperature. these two basic competences interpreted into the items that represent 8 elements and 26 sub-elements of inch critical thinking. the story board developed into the virtual test and result the preliminary design of svt-mh. the topic of matter and heat consists of eight sub-topic which is matter (element, compound, mixture), physical and chemical properties, physical and chemical change, state of matter and its change, temperature, expansion and heat transfer. the researcher intended to develop 40 items journal of science learning article 20 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).17-27 where every topic has 5 items representation (14%). in constructing the item, the researcher collaborated the topic with 26 inch critical thinking sub-elements. it is resulted 41 item which consist of 7 items of matter (17%), 6 items of physical and chemical properties (15%), 8 items of physical and chemical change (19%), 4 items of state of matter and its change (10%), 8 items of temperature (19%), 3 items of expansion (7%) and 5 items of heat transfer (12%). 3.1.2 the final design of science virtual test matter  and heat  svt-mh has been revised from its preliminary design resulted the final design of the instrument that can measure students’ critical thinking on matter and heat topic. the item number 7, 11, 12, 20, 23, 25, 33, 35, 36, 38 and 40 were deleted. the blueprint of the items svt-mh can be seen in table 1. the final design and its item analysis specifically is in the form of item card, consists of 30 items that represent 8 elements and 21 sub-elements of inch critical thinking. in this research, author also find several constraints that occurs in developing and validating the svt-mh such as: (1) difficulties in selecting the video, image, comic, articles that provide the relevant information with inch critical thinking framework and the matter and heat topic; (2) in making the instrument, it’s hard to proportionally distribute the items to the 26 sub-elements of inch critical thinking and 8 sub-topic of matter and heat, and the result of expert validation will possibly resulted disproportional distribution of the items; (3) the low spec of computer which is used that made the program hang and loading while the svt-mh is installed; (4) the earphone is not available in the computer laboratory that make the video can’t be open in each computer because the music will disturb other students. 3.2 the validity of science virtual test before the instrument conducted its validity test, it was tested its readability by three 7th grade students and two science teachers of public junior high school. there are 5 aspects which should be assessed by students and teachers: (1) the description of the article/figure/comic/video/ table/graph was easily to comprehend, (2) the question was easily to comprehend, (3) the option was easily to comprehend, (4) there was correlations between the question and the answer, (5) there was no ambiguous word/term which made student find difficulties in comprehending the question. most of the items are easily to understand. the recommendation of readability test on multiple choice questions of svt-mh are: (1) the word "paraffin wax" elusive; (2) students can assume all the options are right, make the words “tujuan utama” in bold or italic; (3) for table 1 the blueprint of svt-mh critical thinking element matter physical and chemical properties physical and chemical change state of matter and its change tempe rature expansion heat and heat transfer number of item percentage item’s number purpose (4 sub-element) 1,5 3 6 2,4 8 7 23% question at issue (3 sub-element) 15 10,13 14 9 5 17% assumption (2 sub-element) 17 16 2 6.5% point of view (2 sub-element) 19 18 2 6.5% information (3 sub-element) 24,27 21,26 22 5 17% concepts (3 sub-element) 29,31 32 28 30 5 17% interpretation and inference (2 sub-element) 34 37 2 6.5% implication and consequences (2 sub-element) 41 40 2 6.5% total 6 5 5 4 6 2 2 30 percentage 20% 17% 17% 13% 20% 6.5% 6.5% 100 journal of science learning article 21 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).17-27 jhs students, there should be a common name for the chemical gas. example: co2 (carbon dioxide); (4) there is mistyping in several words. the validation of svt-mh used content validity by experts. the items judged by 2 experts of education to validate the critical thinking element, 2 experts of content to validate the matter and heat topic, and 2 experts of media to validate the virtual test. the analyses of the expert judgment result used the average congruency percentage by popham (1978). the item will be valid if the percentage is 90% or higher. based on the expert validation, there are 30 items that have been valid. the item number 7, 11, 12, 20, 23, 25, 33, 35, 36, 38 and 40 were deleted. the 30 items cover 8 elements and 21 sub-element of inch critical thinking. the difficulty level, discriminating power and distractor of svt-mh are analyzed using anates version 4.1.0. 3.2.1 discriminating power of svt‐mh  the discriminating power is a measurement to compare the number of people with high test scores who answered that item correctly with the number of people with low scores who answered the same item correctly. the higher the discriminating power, the better the item because such a value indicates that the item discriminates in favor of the upper group, which should get more items correct (hetzel, 1997). the item that categorized into poor discriminating power are the items number 2, 4, 7, 8, 9, 12, 15, 24 satisfactory discriminating power are the items number 1, 3, 11, 16, 17, 18, 20, 21, 26, 27, 29, and good discriminating power are the items number 5, 6, 10, 13, 14, 19, 22, 23, 25, 28, 30. from the result, the item has a good quality to differentiate between high and low achiever with 22 satisfactory and good discriminating power or 73% from the whole items. additional validation studies that involve a larger and diverse group of respondents representing the target population should be conducted to further strengthen the quantitative data set and related measures (tiruneh et al., 2016). 3.2.2 difficulty level of svt‐mh  difficulty level expresses the proportion or percentage of students who answered the item correctly. it one of the most useful, and most frequently reported, item analysis statistics (testing, 2006). the item which categorized into very easy level are item number 26, 30, easy level are items number 1, 3, 4, 13, 15, 16, 19, 20, 24, 27, 29, moderate level are items number 2, 5, 6, 7, 8, 9, 11, 14, 17, 18, 21, 22, 23, 25, 28, difficult level is item number 10 and very difficult is item number 12. the total number of very easy level is 2, easy level is 11, moderate level is 15, difficult level is 1 and very difficult level is 1. it appears that most of the svt-mh items were rather moderate to easy to our test participants. about 43% of the items were found to have easy level. 2 items included into difficult and very difficult which are item number 10 and 12. therefore, from the researcher analyses these two items included to the question at issue element and this critical thinking element had lowest achievement from students’ result. students are confused to the form of the questions where they have to raise the question based on the information presented. 3.2.3 distractor of svt‐mh  one important element in the quality of a multiple choice item is the quality of the item’s distractors. a distractor analysis addresses the performance of these incorrect response options. distractor is measured and analyzed to know whether it is plausible or not. distractors used in the items need to be plausible so that they attract at least 5% students (chavda, misra & duttaroy, 2015). there are 120 distractors from 30 items. in this research, the distractors are computed by anates version 4.1.0. the result shows that there are 18 distractors which are not usable because it is less than 5% chosen by total respondent. there should also be the revision for the usable distractor because some of them are included to the poor distractor. it has bigger respondent rather than its answer key such as the item number 10. 39 students chose distractor d, while the correct answer is distractor c which chosen by 32 students. most of students assumed that the answer key is d. so, we have to recheck the correct answer key or revise another distractors. in general, the distractor that should be revised is 30% from the total distractors. 3.3 the reliability of science virtual test after svt-mh is revised based on readability and validity test, the test continue to analyze its reliability by a group of student. the respondents were 7th grade students of public junior high school that consists of 40 students. the data obtained analyzed using statistical analyses by ibm spss statistics version 23. the result showed in the table 2 that reliability of the test of 40 students α = 0.638. it is common to see the journal articles where one or more scale alphas which the range was 0.60 0.69 (leech, barrett & morgan, 2005). it interpreted as ‘acceptable’. so, the instrument is reliable to measure students’ critical thinking on heat and matter topic. svt-mh that has been revised tested to the 7th grade public junior high school student. the respondent consists of 117 students. the table 2 shows that the cronbach’s table 2 reliability result of svt-mh n of student n of item cronbach's alpha 40 30 0.638 117 30 0.642 journal of science learning article 22 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).17-27 alpha 117 students, α = 0.642. the value is sufficient because it is common to see the journal articles where alpha were in the 0.60 – 0.69 range (leech, barrett & morgan, 2005). the interpretation of reliability coefficient of the test categorized as ‘acceptable’ to measure students’ critical thinking on heat and matter topic. the most commonly used type of internal consistency reliability is cronbach’s coefficient alpha. this measure indicates the consistency of a multiple item scale (leech, barrett & morgan, 2005). it means if the instrument is used for the next measurement, the result of the measurement will be quite the same with the previous result. however, it has to be noted that the coefficient alpha was not as large as expected. the relatively low alpha value can be explained by at least two factors: the svt-mh was intended to elicit students’ ability in eight elements of inch critical thinking. but according tiruneh et al. (2016) it is possible that cognitive processes required to respond to the items were multifaceted. for instance, a student who performed well in an item that focuses on purpose element, may not have done well on a different item that focuses on question at issue element as these two components slightly vary terms of the required cognitive process. the lower coefficient alpha may have to do with the composition of the test group and number of participants. responding accurately and consistently to the svt-mh items require students who have an adequate mastery of the matter and heat content. the present test takers clearly show the low average score and may have possibly responded to some of the randomly item. this may have influenced the internal consistency of the test. 3.4 the characteristics of science virtual test the characteristics of science virtual test found after the svt-mh was constructed. the item is in the form of multiple choice items and the information presented in each items is in the form of video, article, comic, figure or number of item 5 critical thinking element purpose critical thinking sub-element stating objectives on target topic mixture separation basic competence menjelaskan konsep campuran dan zat tunggal (unsur dan senyawa), sifat fisika dan kimia, perubahan fisika dan kimia dalam kehidupan sehari-hari difficulty level discriminating power 0.58 0.44 distractor a (12.8%) b (6.9%) **c (58.1%) d (22.2%) **answer key figure 1 item card represent video as the information journal of science learning article 23 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).17-27 graph. the information also related with daily activity. this instrument not only change the media form paper based test to computer based test but also change virtually the information to be interactive and the concept is applied in daily life. the examples of the item card that represents the format test with the video as information presented can be seen in figure 1. it shows that difficulty level of the item no.5 is categorized in ‘moderate’ level (0.58). the discriminating power is categorized into ‘good’ discriminating power (0.44). the item no.5 can differentiate the high and low achiever of the students. the distractor a is chosen by 12.8% students, b is chosen by 6.9% students and d is chosen by 22.2% students. it means that all distractor are usable. analyzing the item card, science virtual test mater and heat (svt-mh) resulted several features of svt-mh which make it different with other instrument. svt-mh is the instrument to measure students’ critical thinking skill which is developed in specific subject topic. the topic is based on the two basic competence of 2013 curriculum that integrates each other. the connected topic is matter and heat. similar with the research (tiruneh et al., 2016) which develops and validates the measurement of critical thinking skills in specific science domains. this research targeted content of an introductory electricity and magnetism course and resulted the design of ctem (critical thinking electricity and magnetism) test; the items are in the form of the multiple choices. multiple choice items are more objective rather than essay to analyze the profile of students because every single item of critical thinking sub-element has the exact answer. according to weimer (2015) multiple choice items are quick and easy to score by hand or electronically, can be written so that they test a wide range of higher-order thinking skills, can cover lots of content areas on a single exam and still be answered in a class period. similarly with the research mapeala and siew (2015) which has developed the multiple choice items measuring critical thinking ability among fifth graders in primary science the other commonly administered general critical thinking tests that used the multiple choice item format are cornell critical thinking test–level z (cctt), the california critical thinking skills test (cctst), the watson-glaser critical thinking appraisal (tiruneh et al., 2016). svt-mh uses the computer based test which follow the advanced of technology nowadays. the utilization of technology in educational assessment is aimed at the effectiveness and efficiency of the implementation of the test (chee and wong, 2003). svt-mh also can be used by students to practice their computer operational skill and anxiety for computer preparedness before they conduct the national examination that has been implemented computer based test. in the research of agah, ogbeche & okorie (2016) there was a positive and direct relationship between figure 2 frequency distribution of students’ critical thinking figure 3 the average score of each element 63.49 35.72 56.83 74.35 61.19 58.63 70.51 76.49 0 20 40 60 80 100 purpose question at issue assumption point of view information concept interpretation and inference implication and consequences average score c ri ti ca l t h in k in g e le m en t journal of science learning article 24 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).17-27 computer anxiety, operation skills, attitude and students’ preparedness for computer-based assessment. so, students need to practice in using computer based-assessment; svt-mh is paper less and easy to use. paperless make it effective and efficient in economic especially for the test with higher number of test taker. this benefit have been corroborated by scherer, koppelt & tiemann (2014), who stated that computer based-test is advantageous due to test economics, improvements in objectivity, and test reliability. the information presented in the item of svt-mh can be in the form of interesting animation, video with music compared with paper based test. it is due to the complexity of the critical thinking problem, so the virtual test (computer based-test) is needed in developing the instrument. the score also can be seen directly by students after they finish the test. similar with the research of scherer, koppelt & tiemann (2014) which stated that computer-based assessments is needed in instructional development due to the complexity of complex problemsolving processes in chemistry, multidimensionality of the construct could be assumed. 3.5 the profile of students’ critical thinking and students’ impression toward svt‐mh 3.5.1 students’ critical thinking on matter and heat  topic  svt-mh was used to measure students’ critical thinking on mater and heat topic. the data collected have been analyzed using ibm spss statistics 23. it computed the descriptive statistics test that resulted in the table 3. the result shows that the respondent consists of 117 students. the average score of students’ critical thinking is 59.29 in the scale 0-100. the standard deviation is 13.57. the range is 66.67 and other frequency distribution can be seen in the figure 2. it shows that data has normal distribution. the minimum score achieved by student is 26.67 while the maximum score achieved by student is 93.33. most students have a score 66.67 which are 16 students and the middle score of students is 60. from the 117 students, the authors categorized students’ achievement in critical thinking into low, moderate and high level based on its mean and standard deviation. table 4 shows that 14% students are categorized into low level of critical thinking, 13% students are categorized into high level and most of students are categorized into moderate level of critical thinking which is 73%. the author calculated the students’ score based on the eight elements of inch critical thinking. the data based on 117 students can be seen in the figure 3. it shows that the table 3 descriptive statistics test of students’ profile n mean range std. deviation value 117 59.29 66.67 13.57 table 4 students’ critical thinking level level number of student percentage low 17 14% moderate 85 73% high 15 13% figure 4 the average score of each topic 58.11 60.51 58.29 58.76 57.26 54.27 74.35 0 20 40 60 80 matter physical and chemical properties physica and chemical changes state of matter and its changes temperature expansion heat and heat transfer average score s u b -t o p ic o f m a tt er a n d h ea t journal of science learning article 25 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).17-27 highest average score of the element is implication and consequence which is 76.50. while, the lowest average scores of the element is question at issue which is 35.73. students are rare to get the questions to train their critical thinking. from 8 elements, the highest average score is 76.50 for implication and consequences element. it means that most of students can reason and think which carries with potential outcome from the process. while, the lowest average scores of the element is question at issue which is 35.73. most of the students can’t identify an issue or problem that needs to be addressed. besides calculating the average score based on the eight element of inch critical thinking, the authors also calculated the score based on the sub-topic of matter and heat. the result shown in the figure 4, tt shows the achievement of students in each topic. the highest average score is the topic of heat and heat transfer which is 74.36. while, the lowest average score is the topic of expansion which is 54.27. the matter and heat topic of the instrument consists of 7 subtopics that author analyzed its each score. the figure 4 shows that students’ achievement in each topic is almost the same. but, there is the high score which is 74.36 in the topic heat transfer. most of students already mastered this topic is very daily life topic that the example is easy to find by students. 3.5.2  students’  impression  after  using  science  virtual test  the students have been given the questionnaire to know their impression towards critical thinking. it consists of four aspects which are experience, technical, preference and media. each aspect has several statements that should be chosen by student through indicator strongly agree (5), agree (4), disagree (2) and strongly disagree (1). table 5 and figure 5 show the students’ impression in average value 1-5. the result shows that the value of every aspect is more that neutral score which means that it has positive (+) impression towards svt-mh. the highest impression is in preference and the lowest impression is in media aspect. students’ impression after using science virtual test is used to know students’ opinion in experiencing svt-mh, in using svt-mh technically, the preference and the evaluation to the media. the result shows that the students’ impression is good (positive) to science virtual test because figure 5 students’ impression after using svt-mh 3.85 3.9 3.95 4 4.05 4.1 4.15 4.2 experience tecnique preference media im p re ss io n v a lu e ( 1 ‐5 ) impression aspect table 5 students impression after using svt-mh aspect description average value (1-5) experience students’ opinion after experiencing science virtual test. 4.02 (+) technical students’ impression in accessing science virtual test technically. 4.10 (+) preference students’ preference between paper-based test and science virtual test. 4.17 (+) media students’ impression to the appearance of science virtual test in term of position and size of text, background, information (picture, video, table, graph), and navigation button. 3.98 (+) journal of science learning article 26 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).17-27 the average value for experience, technical, preference and media are more than 3 (neutral score) from the average range 1-5. the preference shows the highest value which is 4.17. it means that students prefer to use science virtual test rather than paper based test. the research is similar with joshua, joshua & ikiroma (2015) which stated that the level of readiness for using computer based-test is high and acceptability is moderate, with a relatively higher preference by students’ federal government owned and privately owned schools. 4. conclusion research in developing and validating the science virtual test in the topic of matter and heat has been done systematically. it produced 30 items that can assess 7th grade students’ critical thinking on matter and heat topic which is further called ‘science virtual test matter and heat (svt-mh)’. based on the result and analyses, it is acquired some conclusions as follows. 1. the development of science virtual test to measure students’ critical thinking on matter and heat topic consists of 5 steps: (1) content analysis of 26 elements inch critical thinking framework and the matter and heat topic; (2) constructing the multiple choice items; (3) readability test by 3 junior high school students and 2 science teacher, validity test by 2 education expert, 2 content expert and 2 media expert; (4) conducting limited tryout by 40 students of junior high school; (5) conducting a larger test by 117 students of junior high school. 2. in the content validity by experts that analyzed using the average congruency percentage, it is produced 30 valid items that covers eight elements and 21 sub-element of inch critical thinking. 3. based on the larger test with 117 junior high school students, it is acquired the reliability of the test where the coefficient alpha (α) is 0.642. it means that the instrument has sufficient reliability to measure students’ critical thinking on matter and heat topic. 4. the items of svt-mh are in the form of multiple choice item which has specific characteristic: (1) the information presented not only in the form of figure, article, graph but also in form of video and animation, (2) the information is related with daily activity, (3) the information of some items is in the form of interesting comic. 5. the profile of students’ critical thinking in public junior high school is 73% moderate level. the element that mastered by students is implication and consequences, while the element that less mastered by students is question and issue. the topic that mastered by students is heat transfer, while the topic that less mastered by students is expansion. most of students also respond the positive impression to the svt-mh in all aspects (experience, technical, preference and media). references agah, d. j., ogbeche, a. t. & okorie, e. u. (2016). computer anxiety, operation skills and attitude as correlate of students preparedness for computer based assessment. international journal of education and research, 4(2), 71-74. anne & kreitzberg, c. (2010). critical thinking for the twenty-first century: what it is and why it matters to you. [online]. retrieved from www.agilecriticalthinking.com. [accessed on november 15, 2016]. chavda, p., misra, s. & duttaroy, b. (2015). item analysis of multiple choice questions based undergraduate assessment in community medicine. south east asian journal of medical education, 9(1), 66-69. chee, t. s. & wong, a. f. (2003). teaching and learning with technology. singapore: prentice hall. flagbola, t. m., adigun, a. a. & oke, a. o. (2013). computer-based test (cbt) system for university academic enterprise examination. internatonal journal of scientific & technology research, 2(8), 1. gall, m., gall, j. p. & borg, w. r. (2003). educational research. boston, ma: pearson education. greenhill. (2009). p21 framework definitions document. [online]. retrieved from www.21stcenturyskills.org [accessed on october 20, 2016 ] greiff, s, holt, dv, wüstenberg, s, goldhammer, f, & funke, j. (2013). computer based assessment of complex problem solving: concept, implementation, and application. educational technology research & development, 61, 407-421. harpen. (2003). thought and knowledge: an introduction to critical thinking. mahwah, nj: lawrence erlbaum associates. hetzel, s. m. (1997). basic concepts in item and test analysis. clearinghouse on assessment and evaluation, 1-24. inch, e. s., warnick, b. & endres, d. (2006). critical thinking and communication the use of reason in argument. boston, ma: pearson. jonassen, dh. (2004). learning to solve problems: an instructional design guide. san francisco: john wiley & sons. joshua, m., t., joshua, a., m. & ikiroma, b. (2015). computer-based testing in nigeria’s university entrants’ matriculation examination: readiness and acceptability of critical stakeholders, 1-10. jurecka, a. (2008). introduction to the computer-based assessment of competencies. international journal hartig, e klieme, & d leutner (eds.), assessment of competencies in educational contexts (193– 214). cambridge/göttingen: hogrefe & huber publishers kemendikbud. (2016). kementerian pendidikan dan kebudayaan: ujian nasional berbasis komputer (unbk). [online]. retrieved from http://unbk.kemdikbud.go.id. [accessed october 20, 2016]. lawson, a. (2004). the nature and development of scientific reasoning: a synthetic view. international journal of science and mathematics education, 2(3), 307–338. leech, barrett & morgan. (2005). spss for intermediate statistics: use and interpretation. mahwah, nj: lawrence erlbaum associates mapeala, r. & siew, n. m. (2015). the development and validation of a test of science critical thinking for fifth graders. spinger plus, 4(741), 1-13. mcmillan, j. h. & schumacher, s. (2001). research in education a conceptual introduction. new york, ny: priscilla mcgeehon. paul, r. & elder, l. (2004). the nature and functions of critical and creative thinking. dillon beach: the foundation for critical thinking. popham, w.j. (1978). criterion-referenced measurement. englewood cliffs, nj: prentice hall. rahayu, s. & kita, m. (2009). an analysis of indonesian and japanese students’ understandings of macroscopic and submicroscopic levels of representing matter and its changes. international journal of science and mathematics education. 8(4), 667–688. journal of science learning article 27 doi: 10.1021/xxx.xxxx.xxxxxx j.sci.learn.2017.1(1).17-27 rusyati, l & firman, h. (2017). validation of science virtual test to assess 8th grade students’ critical thinking on living things and environmental sustainability theme. bandung: american institute of physics, 1-7. scherer, r., koppelt, j. m & tiemann, r. (2014). developing a computer-based assessment of complex problem solving in chemistry. international journal of stem education, 1(2), 1-6. suherman, e. (2003). evaluasi pemelajaan matematika. bandung: fpmipa upi testing, p. (2006). step 9. conduct the item analysis. [online]. retrieved from http://www.proftesting.com. [accessed on july 9, 2017] tiruneh, d. t., cock, m. d., weldeslassie, a. g., elen, j. & janssen, r. (2016). measuring critical thinking in physics: development and validation of a critical thinking test in electricity and magnetism. international journal of science and mathematics education, 1-20. unesco. (2000). dakar framework for action. education for all: meeting our collective commitments. in adopted by the world education forum, dakar. [online]. retrieved from http://unesdoc.unesco.org. [accessed on october 23, 2016] weimer, m. (2015). different type of test question. [online]. retrieved from http://www.facultyfocus.com/. [accessed on july 7, 2017] wirth, j. (2008). computer-based tests: alternatives for test and item design. international journal hartig, e klieme, & d leutner (eds.), assessment of competencies in educational contexts (235–252). cambridge/göttingen: hogrefe & huber publishers. yeh, s. s., (2001). tests worth teaching to: constructing state-mandated tests that emphasize critical thinking. sage journals, vol 30(9), 1217. a © 2022 indonesian society for science educator 509 j.sci.learn.2022.5(3).509-519 received: 18 april 2022 revised: 1 june 2022 published: 27 november 2022 the investigation of the research on out-of-school learning activities in turkey: a systematic review mustafa metin1*, emir oker1, aslı saylan kirmizigül1 1faculty of education, erciyes university, kayseri, turkey *corresponding author: mustafametin@erciyes.edu.tr abstract this research aims to analyze the studies on out-of-school learning activities in turkey between 2000 and 2020. for this purpose, 303 studies were selected, of which 211 articles, 73 master's thesis, and 19 doctoral dissertations, with the phrase "out-of-school learning" in the title and keywords. a systematic literature review was conducted in the research. the data were entered into the content analysis monitoring form, and the analyses were made on the microsoft excel program. in line with the findings obtained, it was determined that most of the studies on out-of-school learning activities were carried out in 2020. in these studies, it was seen that it was mainly aimed to determine the opinions of teachers or students about out-of-school learning activities, the sample was selected in the range of 21-40 people, and the most selected sample group was secondary school students. it was also determined that the studies were mostly done in the science discipline and that the case study from qualitative and experimental research design from quantitative methods was used. when all the findings were evaluated, since there are not many studies on the effect of out-of-school learning activities on anxiety, motivation, and interest, it is recommended to carry out studies in this area and to increase the studies to be done towards high school, preschool, primary school students. keywords out-of-school learning, systematic review, content analyses 1. introduction the age we live in has been defined as the age in which information is rapidly renewed and produced" and one of the characteristics that individuals should have been expressed as accessing information, using, and producing information (ministry of national education [mone], 2005). in this context, individuals are expected to realize the change, produce information by interpreting rather than accepting it as it is and prepare themselves for innovations and development. therefore, the education given to the students plays a vital role in achieving the typical characteristics of individuals. education continues throughout the individual's life and includes formal, non-formal, and informal learning activities (lockhart, 2016; unesco, undp, unfpa, unhcr, unicef, un women, …, ilo, 2015). formal learning takes place with the effort of acquiring the pre-determined knowledge and skills of the individual under the umbrella of an educational institution within a certain period and plan (güven, 2010). however, learning may not take place only in educational institutions. learning environments outside educational institutions are informal and non-formal (eshach, 2007). informal learning is lifelong learning that takes place randomly in daily life without being tied to a specific program and authority (cross, 2007; metin & özcan, 2015; öner & öztürk, 2019). informal environments are the areas that do not have institutional features and that we can visit at any time (tal & morag, 2009). these learning environments are unplanned and haphazard. here, the individual unwittingly acquires new information due to the situation s/he encounters and the interaction with the members of the group that s/he is part of (fidan, 2012). informal learning can occur when an individual discovers new things in the park, on the street, in the cinema hall, while watching tv, or chatting with friends (bozdoğan, 2007; metin & özcan, 2015). in addition, mobile devices, home environments, elearning, and web 2.0 applications are examples of informal learning environments (bell, lewenstein, shouse, & feder, 2009; eshach, 2007). non-formal learning is used by mailto:mustafametin@erciyes.edu.tr journal of science learning article doi: 10.17509/jsl.v5i3.45460 510 j.sci.learn.2022.5(3).509-519 different names such as out-of-doors learning, free-choice learning, out-of-school learning, out-of-class experiences, outdoor learning, and out-of-school experiences in the literature (ayotte-beaudet, potvin, lapierre, & glackin, 2017; strauss & terenzini, 2007; şimşek & kaymakçı, 2015). the most widely used definition is out-of-school learning (bozdoğan, 2007). out-of-school learning is systematic, planned, and structured learning activities outside formal learning environments (bilir, 2007). one of the most critical differences between out-of-school and informal learning is that informal learning is unplanned, while non-formal learning is carried out within a specific plan (bozdoğan, 2007; eshach, 2007; şen, 2019). therefore, out-of-school learning is located at the intersection of formal and informal learning and acts as a bridge between them. furthermore, it was stated that out-of-school learning consists of environmental education, outdoor activities, and personal and social development (higgins, loynes, & crowther, 1997). therefore, it can be said that out-ofschool learning is related to many disciplines (erten & taşçı, 2016). out-of-school learning, which is related to many disciplines, includes learning activities by visiting places such as a zoo, botanical garden, museum, science center, national park, planetarium, excursions and nature activities, interactive exhibitions, aquarium, and industrial enterprise within the framework of a specific plan (eshach, 2007; bostan sarıoğlan & küçüközer, 2017; bozdoğan, 2007; laçin şimşek, 2011). similarly, according to binbaşıoğlu (2000), all out-of-school learning activities, especially school trips, can be examples of out-of-school learning because they are planned and controlled by the teacher. the common point in the statements of many researchers is the emphasis that out-of-school learning is not just a trip but that these trips are carried out within a specific plan. in the literature, it was stated that out-of-school learning activities carried out within a specific plan increase the students' academic achievement, motivation, and attitudes towards the lessons, enable students to learn by having fun, develop their sense of responsibility and curiosity, and positively affect their communication with their friends and provide solutions to the daily life problems (bozdoğan, 2008; braund & reiss, 2006; dori & tal, 2000; eshach, 2007; higgins et al., 1997; melber & abraham, 1999; ramey-gassert, 1997; strauss & terenzini, 2007). moreover, it was stated that out-ofschool learning develops students' interpersonal, social, and leadership skills, provides students with individual and team learning skills, contributes to the self, social and moral development of the students, and allows them to make efficient use of their spare time and gain a democratic attitude (eshach, 2007; higgins et al., 1997; karademir, 2013). there are also some limitations of out-of-school learning. some limitations were revealed in the literature, such as the inability to achieve productive results with outof-school learning in cases where the student's prior knowledge and necessary readiness are not available, the limited participation of students with low socioeconomic status in such activities, and the teachers' unwillingness to organize out-of-school activities without financial support (ay, anagün, & demir, 2015; bozdoğan 2007; karademir 2013; karamustafaoğlu, ayvalı, & ocak 2018; türkmen, 2018). in addition, there are disadvantages such as; out-ofschool learning activities cost would be high if not carried out within a specific plan framework, it would take more time, and it would be challenging to ensure the students' safety. problems would also arise in concentrating the students' attention on the activities and in classroom control, making the desired learning quality unachieved (ay et al., 2015; bostan sarıoğlan & küçüközer, 2017; karamustafaoğlu et al., 2018; kubat, 2018; türkmen, 2015; 2018). the countries considering that the benefits of the outof-school learning environment are more than its limitations are trying to increase and diversify the number of applications in this field daily. in recent years, the number of environments such as museums, planetariums, zoos, botanical gardens, science centers, and aquariums has increased rapidly in our country, and it has come to the fore to use these environments for educational purposes, considering that these environments will create rich opportunities for students (laçin şimşek, 2011). in the out-of-school learning environments guide published by the ministry of national education in 2019, out-ofschool learning was defined, and detailed information was given on how to evaluate out-of-school learning environments. following the ministry of national education's guide, provincial directorates of national education also published guides introducing out-of-school learning environments in their provinces and organized inservice training for teachers. moreover, targets for out-ofschool learning environments were determined in the 2023 education vision of the ministry of national education (mone, 2018a). the 2023 educational vision emphasizes that the cooperation between schools, science centers, museums, art centers, technology centers, and universities will be increased. the "out-of-school learning environments" professional knowledge elective course was added to the program of education faculties in 2018 by the council of higher education. in addition, out-of-school learning environments course was added as compulsory to some departments, such as science and primary school mathematics teaching (council of higher education [cohe], 2018). the number of out-of-school learning environments in our country is increasing rapidly, and the studies of the ministry of national education on out-of-school learning journal of science learning article doi: 10.17509/jsl.v5i3.45460 511 j.sci.learn.2022.5(3).509-519 have begun to pay off. more and more students have started to visit out-of-school learning environments, and a significant increase has been observed in school trips to these environments. accordingly, there has been an increase in studies on out-of-school learning. when the literature is examined, it is seen that there are studies that investigate the opinions of students, preservice teachers and teachers about out-of-school learning (akman, özen altınkaynak, ertürk kara, & can gül, 2015; avcı & gümüş, 2019; ay et al., 2015; aydemir & toker gökçe, 2016; bakioğlu & karamustafaoğlu, 2020; batman, 2020; bozdoğan, 2017; büyükkaynak, ok, & aslan, 2016; güngör & göloğlu demir, 2022; uludag, 2021), and investigate the effects of out-of-school learning on students' academic achievement (akça, balkan kıyıcı, & yıldız, 2017; bolat, karamustafaoğlu, & karamustafaoğlu, 2020; bozdoğan & kavcı, 2016; bülbül, 2018; karakaş özür & şahin, 2017; küçük & yıldırım, 2021) and attitude (caner, 2019; küçük, 2021; özay köse & gül, 2019; seyhan & şimşek, 2017; soysal, 2019), and their experiences with these environments (can, 2019; çalışkan & çerkez, 2012; çiçek & saraç, 2017; çil & yanmaz, 2016; demir & öner armağan, 2018). when the research are examined, it is seen that the subject area, aims, method, data collection tools, and study groups of the studies are different from each other, and each of them should be analyzed separately. furthermore, these studies' simultaneous access, reading, interpretation, and analysis difficulties reveal the need for research based on holistic analysis in this field. although there are review studies on out-of-school learning for this need, it is seen that their number is quite limited. when the literature was examined, a meta-analysis (mutlu & çelik, 2019) and two content analysis studies (demircioğlu & aslan, 2018; saraç, 2017) were found. mutlu and çelik (2019) conducted a meta-analysis study on research conducted on out-of-school learning in science education in turkey between 2006 and 2019. in their research, two themes were focused on: perceptions of outof-school learning environments and the effect of out-ofschool learning environments on student outcomes. in their research, demircioğlu and aslan (2018) investigated postgraduate studies on out-of-school learning environments in turkey within the framework of the subject area, method, sampling, data collection tools, and data analysis methods using content analysis. in saraç's (2017) research, the national studies on out-of-school learning environments between 2007 and 2016 were subjected to content analysis. in recent years, the number and variety of out-of-school learning activities have increased; accordingly, studies on out-of-school learning have also increased. however, no comprehensive systematic review study has been carried out recently. the research results, which include holistic analyses of out-of-school learning activities, will contribute to summarizing current studies in the relevant field and facilitate access of researchers, teachers, and other stakeholders to research data related to the subject area. in addition, the holistic data presented on the content of the studies carried out on this subject will allow researchers to see new and different studies holistically and contribute to developing different perspectives. from this point of view, this research aims to make a systematic review of the research on "out-of-school learning activities" in turkey between 2000-2020. therefore, all articles, master's theses, and doctoral dissertations, which have the concept of out-of-school learning in the title and keywords, and published within the specified period, were examined according to different variables. therefore, this systematic review seeks answers to the following questions: how is the distribution of studies conducted on out-ofschool learning between 2000 and 2020 by: 1. publication types? 2. years? 3. subject areas? 4. educational stages of the samples? 5. sample sizes? 6. their aims? 7. out-of-school learning environments? 8. research approaches? 9. research methods? 10. data collection tools? 11. data analysis methods? 2. method 2.1 research design this research was carried out to determine the general trends of the studies on "out-of-school learning activities" in turkey between 2000-2020. for this purpose, a systematic review was applied to the theses and articles published in this field within 20 years. this systematic review was conducted following preferred reporting items for systematic reviews and meta-analyses (prisma) principles (liberati et al., 2009). the prisma principles with the checklist consisting of 27 items and a four-step flow chart ensure that the literature review studies are conducted transparently (liberati et al., 2009). search strategy and criteria for inclusion and exclusion of studies in the systematic review the inclusion criteria for the study-selection phase of the systematic review are: ⚫ the studies should be published between january 1, 2000, and december 31, 2020, and their full texts can be accessed, ⚫ the articles should be included in google scholar and/or tr index databases, and their full texts should be available, ⚫ master's theses and doctoral dissertations should be included in the council of higher education national journal of science learning article doi: 10.17509/jsl.v5i3.45460 512 j.sci.learn.2022.5(3).509-519 thesis center database, and their full texts should be available, ⚫ the studies should have been conducted in the field of education, ⚫ the studies should include the keywords “out-ofschool learning”, “out-of-course learning”, “out-ofclass learning”, “free-choice learning”, “learning outside” or “outdoor education”. when searching in turkish by limiting the years as 2000-2020, 1140 when "out-of-school learning" is written as a keyword in the google scholar database, 390 when "out-of-class learning" is written, 259 when "outdoor education" is written, 101 when "out-of-course learning" is written, 14 when "learning outside" is written, and 12 articles were found when "free-choice learning" is written. in addition, when these concepts were written and searched in the cohe national thesis center database, 92 theses were reached. as a result of the examination of the reached studies following the research purpose, it was decided to include 303 studies. totally 1465 studies related to out-of-school learning were found in databases. among these studies, 45 were excluded since their full texts are unavailable. then, 105 remaining studies were excluded since they did not focus on education. the remaining 303 studies are involved in this research. the prisma process is represented in figure 1. 2.2 data collection tool a research classification form was developed to evaluate the studies about out-of-school learning. the form was used as a data collection tool in the research context. while developing the form, the related literature was investigated (çiltaş, güler, & sözbilir, 2012; selçuk, palancı, kandemir, & dündar, 2014; sözbilir, kutu, & yaşar, 2012), and the features that should be found in scientific research were examined (büyüköztürk, akgün, karadeniz, demirel, & kılıç, 2016; çepni, 2014; metin, 2014). in the research classification form, there are nine sections: the publication type, year, purpose, subject area, out-of-school environment, research approach and method, sample group, sample size, and data collection tool. the categories in this classification form and the options given under these categories were presented in the opinion of two experts working in content analysis. the form was finalized in line with expert opinions. 2.3 data analysis 211 articles and 92 theses examined within the scope of the research were analyzed according to the research classification form developed by the researchers. in the research, researcher triangulation was done to ensure the validity of the data. in this framework, two researchers took part in all data collection, analysis, and interpretation processes (merriam, 2009). during the analysis of the studies, the researchers evaluated the determined studies separately, taking into account the categories determined in the form. the findings obtained from the studies were recorded in a microsoft excel file and categorized. after the analysis of the studies was completed, the coding reliability was checked to ensure the reliability of the research (miles & huberman, 2015). the analyses made by the two researchers were compared, and the analysis data were arranged in line with the common opinion. the results obtained are expressed in graphs and presented in the findings section. 3. findings within the scope of the research, a total of 303 studies were found. the findings obtained from the content analysis are presented under five headings: distribution of the studies according to 1) publication types, years, and subject areas, 2) educational stages and sample sizes, 3) aims and out-of-school learning environments, 4) research figure 1 the process of reaching the studies journal of science learning article doi: 10.17509/jsl.v5i3.45460 513 j.sci.learn.2022.5(3).509-519 approach and methods, 5) data collection tools and analysis methods. 3.1 distribution of the studies according to publication types, years, and subject areas of the 303 studies examined, 211 (69.6%) were articles, 73 (24.1%) were master's theses, and 19 (6.3%) were doctoral dissertations (figure 2). it was determined that outdoor learning studies conducted in line with the 303 studies analyzed have increased over the years (see figure 3). the number of studies, which was one in 2002, increased to 20 in 2016 and reached the maximum number in 2020. the number of studies published in 2020 is currently 52. according to the results, 131 studies (43.2%) were carried out in science, 53 studies (17.5%) were in social studies, and 31 (10.2%) were in other subject areas-four geography, four physical education, four physics, four special education, three biology, three life science, three turkish, two chemistry, one information technologies, one english, one religious culture and moral knowledge, and one technology and design. moreover, 12 studies (4.0%) were conducted in preschool education, 12 studies (4.0%) were in visual arts, six studies (2.0%) were in mathematics, and five studies (1.6%) were in history subject area. no subject area was specified in 53 studies (17.5%) (see figure 4). 3.2 distribution of the studies according to educational stages and sample sizes of the 303 studies analyzed, middle school students were included in 121 studies, teachers were in 68, preservice teachers were in 43, primary school students were in 14, high school students were in 13, and preschool students were included in nine studies. in addition, parents were included in five, museum educators were in two, and academicians were included in one research (see figure 5). in 31 studies, no educational stage was specified as they are grounded theory, meta-synthesis, and document analysis studies. figure 2 distribution of the studies according to the publication types 70% 24% 6% articles master's theses doctoral dissertations figure 3 number of articles published by years 52 49 34 33 20 18 14 16 18 7 12 8 8 6 2 2 2 1 1 0 10 20 30 40 50 60 n u m b e r o f th e s tu d ie s years figure 4 distribution of the studies according to the subject areas journal of science learning article doi: 10.17509/jsl.v5i3.45460 514 j.sci.learn.2022.5(3).509-519 the distribution of examined studies according to the sample sizes is presented in figure 6. accordingly, 59 studies have 1-20 participants, 73 studies have 21-40 participants, 36 studies have 41-60 participants, 28 studies have 61-80 participants, 35 studies have 81-160 participants, and 37 studies have 161 and more participants. in 35 studies, sample sizes were not specified. according to the results, most studies have 21-40 participants. 3.3 distribution of the studies according to the aims the distribution of examined studies according to their aims is presented in figure 7. accordingly, 123 studies aimed to investigate participants' views regarding out-ofschool learning environments, and 46 studies aimed to investigate the effects of out-of-school learning environments on the participants' academic achievement. moreover, out-of-school learning environments' effects on the participants' attitudes and skills (critical thinking skills, scientific process skills, decision-making skills, etc.) were investigated in 38 and 19 studies, respectively. in addition, participants' self-efficacy perceptions and perceptions regarding out-of-school learning environments were investigated in 17 studies. lastly, the effect of out-ofschool learning activities on middle school students' interest in science, motivation, and anxiety was examined in seven, six, and three studies, respectively. figure 5 distribution of the studies according to the educational stages of the samples figure 6 distribution of the studies according to the sample sizes 121 68 43 31 14 13 9 5 2 1 0 20 40 60 80 100 120 140 n u m b e r o f th e s tu d ie s educational stages 59 73 36 28 11 6 9 9 37 35 0 10 20 30 40 50 60 70 80 1-20 21-40 41-60 61-80 81-100 101-120 121-140 141-160 161 and above not specified n u m b e r o f th e s tu d ie s sample size figure 7 distribution of the studies according to the aims 123 46 38 19 17 7 6 3 0 50 100 150 view achievement attitude skills perception interest motivation anxiety number of the studies a im o f th e s tu d y journal of science learning article doi: 10.17509/jsl.v5i3.45460 515 j.sci.learn.2022.5(3).509-519 3.4 distribution of the studies according to the out-ofschool learning environments the distribution of examined studies according to the out-of-school learning environments is presented in figure 8. when figure 8 is examined, it is seen that the studies on out-of-school learning have been carried out in various environments. accordingly, it is observed that the majority of the studies have been conducted in museums and science centers. moreover, 16 studies were conducted in the planetarium, and 16 were carried out in different places such as canyon, conservatory, sports hall, and earthquake monitoring center. 3.5 distribution of the studies according to research approach and methods the distribution of the examined studies according to the research approach and method is given in figure 9. accordingly, six studies (one master's thesis and five articles) are scale development studies. the quantitative research method has been used in 93 studies, the qualitative research method in 141 studies, and the mixed research method in 45 studies. moreover, in the 18 studies, no research approach was specified. the results determined that the most preferred research methods in qualitative and quantitative studies are case studies and experimental research. however, it is seen that mixed-methods designs were generally not specified in the studies, and they are expressed only as "mixed method was used". 3.6 distribution of the studies according to data collection tools and analysis methods the distribution of the data collection tools used in examined studies is given in figure 10. accordingly, a total of 68 scales, 59 questionnaires, 151 forms, 51 tests, and 38 other data collection tools were included in the studies. it is seen that primarily attitude scales (38) were used in studies where the scales were used, and mainly interview forms (122) were used in studies where forms were used. also, mostly achievement tests (43) were preferred in the studies where the tests were used. figure 8 distribution of the studies according to the out-of-school learning environments figure 9 distribution of the studies according to research approach and methods journal of science learning article doi: 10.17509/jsl.v5i3.45460 516 j.sci.learn.2022.5(3).509-519 the distribution of the analysis methods of the examined studies is given in figure 11. accordingly, content analysis was conducted in 106 studies, and descriptive analysis was conducted in 71. moreover, t-tests were used in 68 studies, descriptive statistics were used in 67 studies, nonparametric tests were used in 38 studies, and anova was used in 34 studies. 4. discussion in the light of the findings obtained from the research, it is seen that the articles published on out-of-school learning environments outnumbered doctoral dissertations and master's theses. this finding parallels the findings obtained in saraç's (2017) study. in his research, saraç (2017) examined 133 studies conducted in the national field and published between 2007 and 2016 on out-of-school learning environments and concluded that 76 of them were articles. in addition, saraç (2017) and demircioğlu and aslan (2018) concluded that the number of master's theses published on out-of-school learning in turkey is much more than the doctoral dissertations. the findings of the present research are also in this direction. of the 303 studies examined, 211 were articles, 73 were master's theses, and only 19 were doctoral dissertations. according to the findings, the highest number of publications between 2000 and 2020 on out-of-school learning were made in 2020, and the studies have increased over the years. moreover, substantial increases in 2012, 2017, and 2019 were detected. the leap in 2012 was also detected in saraç's (2017) studies and demircioğlu and aslan's (2018) studies. in the increase in the number of studies in 2019, it is thought that the addition of the professional knowledge elective course called "out-ofschool learning environments" to the education faculties in 2018 by the council of higher education also played a role. in addition to this course, the out-of-school learning environments course was added as a compulsory course in some departments such as science teaching and primary school mathematics teaching in 2018 (council of higher education [cohe], 2018a, 2018b). this situation may have increased the interest and focus of researchers and educators on out-of-school learning after 2018. the results revealed that most of the reviewed studies were conducted in the science subject area. social studies course comes after science. on the other hand, the less common subject areas are found in visual arts, mathematics, and history. in these curricula, out-of-school learning is not emphasized. similar to the present research results, according to saraç’s (2017) and demircioğlu and aslan's (2018) studies, the two most common subject areas are science education and social studies, respectively. the research of ertuğrul and karamustafaoğlu (2020) also supports these results. figure 10 distribution of the studies according to data collection tools figure 11 distribution of the studies according to the analysis methods 106 71 68 67 38 34 12 8 6 0 50 100 150 content analysis descriptive analysis t-tests descriptive statistics nonparametric tests anova other ancova factor analysis number of the studies a n a ly s is m e th o d s journal of science learning article doi: 10.17509/jsl.v5i3.45460 517 j.sci.learn.2022.5(3).509-519 furthermore, the researchers investigated the views of classroom teachers towards out-of-school learning environments. their findings revealed that classroom teachers thought out-of-school learning environments should be visited within science and social studies lessons. it is thought that the ministry of national education stated in the 2013 and 2018 science curriculums that out-ofschool learning environments can also be used for students to learn knowledge meaningfully and permanently is thought to be effective in obtaining these findings (mone, 2013, 2018b). the results obtained from the analysis indicated that middle school students and teachers were included in most of the studies. on the other hand, preschool students, parents, museum educators, and academicians were included in only a few studies. parallel to this finding, demircioğlu and aslan (2018) also found that the studies are mostly carried out with middle school students and teachers. it is thought that the recent changes in both the science curriculum (ministry of national education [mone], 2013, 2018b) and teacher education program (cohe, 2018a, 2018b), whereby out-of-school learning environments have been included in teaching plans have brought along the need to work more frequently with that sample group. the results revealed that the number of sample groups studied is not much high. accordingly, most studies have 1-20 and 21-40 participants. similarly, in the study of demircioğlu and aslan (2018), most of the theses have 1050 participants. this preference is assumed to be linked to the recommendation that studies should be conducted with a smaller number of sample groups in order to make trips to out-of-school learning contexts more effective. however, this situation might also be due to the shortage of time and official procedures. furthermore, it is known that working with large sample groups in out-of-school learning environments can cause various problems such as a decrease in efficiency, ensuring safety, and ethical problems (karbeyaz & kurt, 2020; ocak & korkmaz, 2018). therefore, according to the findings obtained in the present research, the most frequently used data collection tool is the interview form. additionally, the findings regarding the distribution of the studies according to the aims indicated that of the 303 studies, 123 studies aimed to investigate participants’ views regarding out-of-school learning environments. therefore, it can be said that the most commonly used sample size matches the most frequently preferred data collection tools and aims of the studies. it was revealed that most studies were conducted in museums and science centers. this finding parallels the other studies (demircioğlu & aslan, 2018; saraç, 2017). since the studies are mainly conducted in science and social studies subject areas, selecting these environments makes sense. however, when the studies were examined in more detail, it was seen that the studies on science education were mostly carried out in science centers, and the studies on social studies were mainly carried out in museums. interview forms, questionnaires, and achievement tests are the most used data collection tools in the studies. in parallel with these findings, saraç’s (2017) research found that questionnaires and interview forms are the most used data collection tools in the investigated studies. in the research of demircioğlu and aslan (2018), two of the most commonly preferred data collection tools were interviews and achievement tests. concordant with these results, in the present research, most of the studies aimed to investigate participants' views regarding out-of-school learning environments and to investigate the effects of outof-school learning environments on the participants' academic achievement. on the other hand, the effect of out-of-school learning activities on students' interest in science, motivation, and anxiety was examined in only a few studies. as a result, the most commonly used data collection tools match the aims of the studies. the results indicated that the qualitative research method was used in most reviewed studies while the mixed method is relatively less preferred. this result is parallel with the literature (saraç, 2017). çiltaş et al. (2012) also stated that mixed studies are scarce in our country's field of education. the detailed analyses determined that the most preferred research methods in qualitative and quantitative studies are case studies and experimental research. concordantly, demircioğlu and aslan (2018) had precisely the same results in their research. in parallel with these findings, saraç (2017) also found that the reviewed qualitative studies were mostly carried out with case studies. consistent with the most used research methods; content analysis, descriptive analysis, t-tests, and descriptive statistics are the studies' most used four analysis methods. recommendation in the light of the findings, some suggestions are provided for researchers who may perform studies on similar subjects in the future: ⚫ remarkably, there are very few doctoral dissertations conducted in out-of-school learning environments. therefore, the number of doctoral dissertations should be increased. ⚫ the results indicated a high number of studies performed on science and social studies education. therefore, it is anticipated that new studies on out-ofschool learning environments in other branches such as history, mathematics, and preschool education will contribute to the literature. ⚫ according to the results, middle school students and teachers were included in most studies; however, preschool students and parents were included in only a few studies. since people of all ages visit out-ofjournal of science learning article doi: 10.17509/jsl.v5i3.45460 518 j.sci.learn.2022.5(3).509-519 school learning environments, choosing diverse sample groups for the studies would benefit the researchers. ⚫ the number of samples in the studies may be increased to reach more data and more accurate results. ⚫ out-of-school learning environments should not be limited to science centers and museums. visiting different learning environments would increase students’ motivation and interest. ⚫ it is considered that using the mixed method and the qualitative and quantitative methods will be effective in the studies carried out. ⚫ lastly, it is suggested to conduct studies that compare out-of-school learning in turkey and other countries. references akça, z., balkan kıyıcı, f., & yıldız, a. (2017). the effect of use of conservatories in the teaching of voice subject on the academic achievements and thinking skills of students. journal of current researches on educational studies, 7(2), 171-184. akman, b., özen altınkaynak, ş., ertürk kara, h. g., & can gül, ş. (2015). the views of preschool teachers on museum education. journal of uludag university of faculty of education, 28(1), 97-115. ayotte-beaudet, j. p., potvin, p., lapierre, h. g., & glackin, m. (2017). teaching and learning science outdoors in schools’ immediate surroundings at k-12 levels: a meta-synthesis. eurasia journal of mathematics science and technology education, 13(8), 5343-5363. http://dx.doi.org/10.12973/eurasia.2017.00833a avcı, g., & gümüş, n. (2019). views of students on outdooroutdoor education. turkish studies-educational sciences, 14(3), 351-377. ay, y., anagün, ş. s., & demir, z. m. (2015). preservice primary school teachers' opinions about out-of-school learning in science teaching. turkish studiesinternational periodical for the languages, literature and history of turkish or turkic, 10(15), 103-118. aydemir, i̇., & toker gökçe, a. (2016). perspectives of school administrators regarding informal learning/education settings. in proceedings of 3rd international eurasian educational research congress (pp. 630-638). bakioğlu, b., & karamustafaoğlu, o. (2020). pupil views on the use of out-of-school learning environments in teaching process. journal of research in informal environments, 5(1), 80-94. batman, d. (2020). investigation of physics teachers’ views about the out-of-school learning environments. journal of research in informal environments, 5(1), 59-79. bell, p., lewenstein, b., shouse, a.w., & feder, m.a. (eds.). (2009). learning science in informal environments. washington, dc: national research council. bilir, m. (2007). non-formal education implementations in turkey: issues and latest challenges. international journal of lifelong education, 26(6), 621-633. binbaşıoğlu, c., (2000). extracurricular activities at school. mone teacher’s book series, national education press, i̇stanbul. bolat, a., karamustafaoğlu, s., & karamustafaoğlu, o. (2020). the effect of outdooroutdoor school learning environment on student achievement in 5th grade 'world of living' unit: example of biodiversity museum. karaelmas journal of educational sciences, 8, 4254. bostan sarıoğlan, a., & küçüközer, h. (2017). investigation of preservice science teachers' opinions regarded to outdooroutdoor school learning environments. journal of research in informal environments, 2(1), 1-15. bozdoğan, a. e. (2007). role and importance of science and technology in education. (unpublished doctoral dissertation). gazi university, ankara. bozdoğan, a. e. (2008). planning and evaluation of field trips to informal learning envıronments: case of the ‘energy park’. journal of theory and practice in education, 4(2), 282-290. bozdoğan, a. e. (2017). preservice teachers' views about "informal learning environments in science education" course. international journal of turkish education sciences, 5(8), 1-17. bozdoğan, a. e., & kavcı, a. (2016). the effects of out of class teaching activities to secondary school student's academic achievement in science course. gazi journal of education sciences, 2(1), 13-30. braund, m., & reiss, m. (2006). towards a more authentic science curriculum: the contribution of out-of-school learning. international journal of science education, 28, 1373-1388. http://dx.doi.org/10.1080/09500690500498419 bülbül, m. (2018). the effect of non-school learning environments on the academic achievements of students: the example of the hydroelectric power plant trip. (unpublished master’s thesis). giresun university, giresun. büyükkaynak, e., ok, z., & aslan, o. (2016). science teachers’ views on out-of-school learning environments in science education. kafkas university journal of the institute of social sciences, 1, 43-60. büyüköztürk, ş., akgün, ö. e., karadeniz, ş., demirel, f., & kılıç, e. (2016). scientific research methods. ankara: pegem academy. can, n. s. (2019). the investigation of out-of-school activities of learning environments on recycling and environmental effects in primary school students in terms of different variables. (unpublished master’s thesis). erzincan binali yıldırım university, erzincan. caner, ö. (2019). teacher candidates’ attitudes towards the sustainable environmental education in out-of-school learning environments. (unpublished master’s thesis). akdeniz university, antalya. council of higher education [cohe]. (2018a). science education undergraduate program. retrieved from https://www.yok.gov.tr/documents/kurumsal/egitim_ogretim_ dairesi/yeni-ogretmen-yetistirme-lisansprogramlari/fen_bilgisi_ogretmenligi_lisans_programi.pdf cohe. (2018b). elementary mathematics teaching undergraduate program. retrieved from https://www.yok.gov.tr/documents/kurumsal/egitim_ogretim_ dairesi/yeni-ogretmenyetistirme-lisansprogramlari/ilkogretim_matematik_lisans_programi.pdf cross, j. (2007). informal learning: rediscovering the natural pathways that inspire innovation and performance. san francisco: pfeiffer. çalışkan, h., & çerkez, s. (2012). an evaluation of education with museum practices in social studies classes based on student views. international online journal of educational sciences, 4(1), 162-173. çepni, s. (2014). introduction to research and project work. ankara: pegem a publishing. çiçek, ö., & saraç, e. (2017). science teachers' opinions about experience in out of school learning environments. ahi evran university journal of kırşehir education faculty, 18(3), 504-522. çil, e., & yanmaz, d. (2016). primary education students’ museum visit experiences. in proceedings of 15th international primary teacher education symposium (pp. 860-869), muğla, turkey. çiltaş, a., güler, g., & sözbilir, m. (2012). mathematics education research in turkey: a content analysis study. educational sciences: theory & practice, 12(1), 574-578. demir, n., & öner armağan, f. (2018). science teachers’ views about informal learning environments: planetarium. journal of social and humanities sciences research, 5(30), 4241-4248. demircioğlu, g., & aslan, a. (2018). a review on turkish graduate studies performed on out-of-school learning environments. karadeniz technical university institute of social sciences journal of social sciences, 8(16), 379-402. dori, y. j., & tal, r. t. (2000). formal and informal collaborative projects: engaging in industry with environmental awareness. science education, 84(1), 95-113. http://dx.doi.org/10.12973/eurasia.2017.00833a http://dx.doi.org/10.1080/09500690500498419 https://www.yok.gov.tr/documents/kurumsal/egitim_ogretim_dairesi/yeni-ogretmen-yetistirme-lisans-programlari/fen_bilgisi_ogretmenligi_lisans_programi.pdf https://www.yok.gov.tr/documents/kurumsal/egitim_ogretim_dairesi/yeni-ogretmen-yetistirme-lisans-programlari/fen_bilgisi_ogretmenligi_lisans_programi.pdf https://www.yok.gov.tr/documents/kurumsal/egitim_ogretim_dairesi/yeni-ogretmen-yetistirme-lisans-programlari/fen_bilgisi_ogretmenligi_lisans_programi.pdf https://www.yok.gov.tr/documents/kurumsal/egitim_ogretim_dairesi/yeni-ogretmenyetistirme-lisans-programlari/ilkogretim_matematik_lisans_programi.pdf https://www.yok.gov.tr/documents/kurumsal/egitim_ogretim_dairesi/yeni-ogretmenyetistirme-lisans-programlari/ilkogretim_matematik_lisans_programi.pdf https://www.yok.gov.tr/documents/kurumsal/egitim_ogretim_dairesi/yeni-ogretmenyetistirme-lisans-programlari/ilkogretim_matematik_lisans_programi.pdf journal of science learning article doi: 10.17509/jsl.v5i3.45460 519 j.sci.learn.2022.5(3).509-519 erten, z., & taşçi, g. (2016). developing activities of out of the school learning environments for science classes, and analysing their effects on students’ scientific process skills. erzincan university journal of education faculty, 18(2), 638-657. ertuğrul, a., & karamustafaoğlu, o. (2020). views of classroom teachers about out-of-school learning environments: kayseri science center. social sciences research journal, 9(2), 107-116. eshach, h. (2007). bridging in-school and out-of-school learning: formal, non-formal, and informal education. journal of science education and technology, 16, 171-190. fidan, n. (2012). learning and teaching at school (3rd ed.). pegem academy, ankara. güngör, c., & göloğlu demir, c. (2022). analysis of preschool teachers' views on out of school learning activities. afyon kocatepe university journal of social sciences, 24(1), 15-30. güven, i̇. (2010). türk eğitim tarihi [turkish education history]. naturel yayıncılık. higgins, p., loynes, c., & crowther, n. (eds.). (1997). a guide for outdoor educators in scotland. snh: perth. karademir, e.(2013). determination of objectives realization at outdooroutdoor science education activities of teachers and preservice teachers by the theory of planned behavior within the scope of science and technology lesson. (unpublished doctoral dissertation). hacettepe university, ankara. karakaş özür, n., & şahin, s. (2017). the effect of out of classroom activities on student success in the social studies course. ahi evran university journal of kırşehir education faculty, 18(3), 324-347. karamustafaoğlu, s., ayvalı, l., & ocak, y. (2018). teachers' opinions on informal environments in preschool education. journal of research in informal environments, 3(2), 38-65. karbeyaz, a., & kurt, m. (2020). the views of classroom teachers for use of out-of-school learning environments in life studies course. international journal of new approaches in social studies, 4(1), 79-93. kubat, u. (2018). opinions of preservice science teachers about outdooroutdoor education. mehmet akif ersoy university journal of education faculty, 48, 111-135. küçük, a. (2021). investigation of the change towards scientific attitudes of students with out-of-school learning experience. opus international journal of society researches, 18(44), 7552-7580. küçük, a., & yıldırım, n. (2021). the effect of out-of-school learning environments on the academic achievement of the human and environment unit. fen bilimleri öğretimi dergisi, 9(2), 205-264. laçin şimşek, c. (2011). out-of-school learning environments and science education. in laçin şimşek (ed.), out-of-school learning environments in science teaching (pp. 1-23). pegem academy, ankara. liberati, a., altman, d. g., tetzlaff, j., mulrow, c., gøtzsche, p. c., ioannidis, j. p., ... & moher, d. (2009). the prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. journal of clinical epidemiology, 62(10), e1-e34. lockhart, a. s. (2016). non-formal and informal programs and activities that promote the acquisition of knowledge and skills in areas of global citizenship education (gced) and education for sustainable development (esd). unesdoc: unesco digital library. retrieved february 21, 2021 from https://unesdoc.unesco.org/ark:/48223/pf0000245625 melber, l. h., & abraham, l. m. (1999). beyond the classroom: linking with informal education. science activities, 36, 3-4. merriam, s. b. (2009). qualitative research: a guide to design and implementation (2nd ed.). san francisco, ca: jossey-bass. metin, m. (2014). nicel veri toplama araçları [quantitative data collection tools]. in m metin (ed.), kuramdan uygulamaya eğitimde bilimsel araştırma yöntemleri (pp.161-214), ankara: pegem a yayıncılık. metin, m., & özcan, a. (2015). basic concepts of education. in metin, m. & aytaç t. (eds.), introduction to educational science. pegem-a publishing, ankara. miles, m., & huberman, a. m. (2015). qualitative data analysis: a resource book challenging new methods. jakarta (id): ui press. ministry of national education [mone]. (2005). primary schools science and technology education program for grades 4 and 5. ankara. mone. (2013). elementary science course (4, 5, 6, 7 and 8th grades) curriculum. ankara: mone publishing. mone. (2018a). 2023 education vision program. ankara: mone publishing. mone. (2018b). science curriculum (primary and middle school 3, 4, 5, 6, 7 and 8th grades). ankara. mutlu, g., & çelik, m. (2019). research conducted about out-of-school learning environments in science education in turkey: a metasynthesis study. in proceedings of 1st international educational research conference (pp. 159-169), izmir, turkey. ocak, i̇., & korkmaz, ç. (2018). an examination of the views of science and preschool teachers on non-formal learning environments. international journal of field education, 4(1), 18-38. öner, g., & öztürk, m. (2019). science centres as outdooroutdoor teaching environments: experience of prospective social studies teachers. eskisehir osmangazi university journal of social sciences, 20, 1109-1135. özay köse, e., & gül, ş. (2019). investigation of high school students' attitudes about botanical gardens according to different variables. international journal of humanities and education, 5(12), 1019-1036. ramey-gassert, l. (1997). learning science beyond the classroom. the elementary school journal, 97(4), 433-450. saraç, h. (2017). researches related to outdooroutdoor learning environments in turkey: content analysis study. journal of education theory and practical research, 3(2), 60-81. selçuk, z., palancı, m., kandemir, m. & dündar, h. (2014). tendencies of the researches published in education and science journal: content analysis. education and science, 39(173), 430-453. seyhan, a., & şimşek, u. (2017). the effects of using cemeteries as an educational stage on students’ academic success and attitudes in a high school history course. turkish history education journal, 6(2), 343-362. soysal, e. (2019). the effects of out of school learning environment on attitude, interest and motivation toward science course of 7th grade students. (unpublished master’s thesis). gazi university, ankara. sözbilir, m., kutu, h., & yaşar, m. d. (2012). science education research in turkey: a content analysis of selected features of papers published. in j. dillon & d. jorde (eds). the world of science education: handbook of research in europe (pp.341-374). rotterdam: sense publishers. strauss, l., & terenzini, p. (2007). the effects of students' inand outof-class experiences on their analytical and group skills: a study of engineering education. research in higher education, 48(8), 967-992. şen, a. i̇. (2019). what is an out-of-school learning environment? in a. i̇. şen (ed.) out-of-school learning environments (pp.1-20). ankara: pegem academy publishing. şimşek, a., & kaymakçı, s. (2015). the purpose of out-of-school social studies education. in şimşek & kaymakçı (eds), out-of-school social studies education (pp. 1-11). ankara: pegem a. tal, t., & morag, o. (2009). reflective practice as a means for preparing to teach outdoors in an ecological garden. journal of science teacher education, 20, 245-262. türkmen, h. (2015). primary teachers' point of view about science teaching in outdooroutdoor learning environments. journal of european education, 5(2), 47-55. türkmen, h. (2018). perspectives of secondary school teachers about outdooroutdoor teaching. journal of ege social science, 1(1), 12-26. uludag, g. (2021). views of preschool teachers on using out-of-school learning environments in preschool education. international online journal of education and teaching, 8(2), 1225-1249. unesco, undp, unfpa, unhcr, unicef, un women, … ilo. (2015). education 2030: incheon declaration and framework for action for the implementation of sustainable development goal 4: ensure inclusive and equitable quality education and promote lifelong learning opportunities for all. retrieved february 21, 2021 from http://unesdoc.unesco.org/images/0024/002456/245656e.pdf 4 https://unesdoc.unesco.org/ark:/48223/pf0000245625 http://unesdoc.unesco.org/images/0024/002456/245656e.pdf%204 a © 2021 indonesian society for science educator 220 j.sci.learn.2021.4(3).220-229 received: 27 december 2020 revised: 03 march 2021 published: 03 july 2021 secondary school students' knowledge and views on laboratory safety belkız caymaz1* 1ministry of national education, turkey *corresponding author caymazbelkiz@gmail.com abstract laboratory studies are very important for science education but also involve various risks. it is possible to minimize these risks by providing a safe working environment. this study aims to determine the knowledge and views of secondary school students on laboratory safety and to identify their deficiencies in this regard. the survey method was used in the study. the study sample consists of 136 students from 6th, 7th, and 8th-grade students of a public secondary school in kastamonu province, turkey. an open-ended questionnaire was used to determine the students' knowledge and views on the subject, and the data obtained were analyzed by the content analysis method. as a result of the study, it was seen that although secondary school students have some general information about laboratory safety, they have deficiencies in terms of most of the information. it was determined that students do not know most of the behaviors that must/must not be made in the laboratory, and they are not aware of some of the safety symbols in the science textbooks. in addition, most of the students stated that they don't feel competent about laboratory safety and need training on this subject. keywords science lesson, laboratory safety, secondary school students 1. introduction laboratory studies are an integral part of science education and provide students with various opportunities to learn by doing and experiencing (hamurcu, 1998; yazıcı & özmen, 2015). well-designed laboratory studies encourage students to question, think scientifically and interpret the events around them (aydoğdu, 2018). it also contributes to developing students' psycho-motor skills and communication skills (hofstein & lunetta, 2004). despite its various benefits, the teachers do not want to do laboratory studies for various reasons (hackling, goodrum & rennie, 2001; yazıcı & özmen, 2015). insufficient course time (hackling, goodrum & rennie, 2001; hodson, 1990; yazıcı & özmen, 2015), crowded classes (aydoğdu, 2018; uluçınar, cansaran & karaca, 2004), labs being inadequate in terms of equipment (hofstein & lunetta, 2004; nakiboğlu & sarıkaya, 1999; yazıcı & özmen, 2015) are among the factors that hinder laboratory studies. böyük, demir & erol (2010) found in their study that secondary school science teachers did not know the laboratory materials sufficiently, could not use them and did not have enough knowledge of maintenance and repair of these materials. teachers feel insufficient in terms of knowledge and skills for laboratory use and have difficulty providing classroom management. for this reason, they prefer to teach science in the classroom, even if the physical conditions in the laboratory are suitable. another reason teachers avoid using the laboratory is the insecurity they feel about laboratory safety (nakiboğlu & sarıkaya, 1999; yazıcı & özmen, 2015). laboratory studies are required for effective science education; however, they must be planned very carefully due to the dangers they may contain. at this point, ensuring laboratory safety is the essential condition. laboratory safety is the process of taking precautions against accidents and dangers that may occur during laboratory studies, determining the problems that may occur in the laboratory, and solving the problems with scientific methods (bayraktar, erten & aydoğdu, 2006). in addition, laboratory safety can be defined as following specific laboratory rules to protect people and the environment during laboratory studies and utilize laboratory equipment in the most appropriate way (deniz & ercan-kalkan, 2013). laboratory accidents are inevitable when safety precautions are not observed. these accidents may result in simple injuries as well as very serious situations such as loss of life. the main causes of accidents include the incorrect design of laboratories, improper storage of materials and equipment, the absence or non-use journal of science learning article doi: 10.17509/jsl.v4i3.30752 221 j.sci.learn.2021.4(3).220-229 of protective equipment for safety. in addition, crowded classes, unwanted student behaviors, carelessness, tiredness, teachers and students not having enough information about the properties of chemicals are among the other reasons (aydoğdu & yardımcı, 2013; aydoğdu & pekbay, 2016; nwele, 2013; west, westerlund, stephenson, nelson & nyland, 2003). there have been many accidents due to these reasons. for example, methanol caught fire during the experiment, and one student had severe burns (gerlovich, parsa & jordan, 2004). a student who did not listen to the teacher's warnings added water over the acid in another accident. as a result of the laboratory explosion, both the careless student and his partner were injured (hill & finster, 2010). these accidents show that unconscious and careless work can lead to many dangerous situations. both teachers and students have significant responsibilities to carry out laboratory studies safely. according to america's lab report (singer, hilton & schweingruber, 2006), teachers' responsibilities are: (1) the duty of teaching (the teacher must predetermine possible risks, inform the students in advance of possible dangers, teach the laboratory rules), (2) the duty of supervision (the teacher must constantly watch the students, not ignore those who exhibit wrong behavior, increase supervision in more dangerous studies), (3) the duty of maintenance (the teacher must never use a damaged material, prepare a written report for the maintenance and repair of the materials, act following the procedures for regular checks of the safety equipment). eliminating the conditions that can cause injuries and accidents in the laboratory is one of the teacher's primary duties. however, students who will work in the laboratory should not forget that they are responsible for their safety and their people. students should get information about the possible risks of the experiment in advance, follow and apply the safety instructions, listen to the warnings of the teacher carefully, learn what to do in case of an emergency, and inform the teacher about possible dangerous situations (hasenekoğlu, 2003; topsakal, 2006). most accidents are caused by human errors (canel, 2002; kerimak-öner, 2020). for this reason, both teachers and students should have the necessary knowledge and skills to ensure laboratory safety. the most important thing to do is to identify the deficiencies of teachers and students in laboratory safety and provide the training they need. aydoğdu & yardimci (2013) analyzed the news about laboratory accidents in local and national newspapers. as a result of the examination, it was determined that there were accidents such as test tube explosion, chemical spreading, gas release, etc. these accidents' leading causes include having inadequate/incorrect information about chemicals, carelessness, and unauthorized use of experimental materials by students. when the studies on laboratory safety were examined in the literature, it was seen that most of the studies were carried out with teachers and teacher candidates. in demir's (2016) study, the "laboratory safety knowledge test" was applied to 74 science teachers, and it was observed that very few of them were successful in the test. muhammad (2017) conducted a study that measured laboratory-related skills of science teachers in nigeria, such as laboratory safety, using materials, experimenting, first aid. as a result of the study, it was reported that teachers have deficiencies in laboratory safety. gudyanga (2020) researched the levels of chemical laboratory safety (cls) awareness of physical sciences teachers. teacher cls awareness was generally suboptimal, especially concerning awareness of safe chemical storage and waste disposal and emergency laboratory safety procedures. the teacher candidates’ knowledge and views on laboratory safety were investigated (aydın, diken, yel & yılmaz, 2011; kırbaşlar, özsoy-güneş & derelioğlu, 2010). it was reported that teacher candidates have general knowledge about laboratory safety, but they had deficiencies and did not sufficiently know the laboratory safety symbols. in the studies conducted by anılan (2010), gökmen & atmaca (2019), it was determined that the teacher candidates' level of awareness about safety symbols was low. the state of the secondary schools having laboratory safety equipment (emendu, 2007; nwele, 2013) was also investigated. the results showed that the laboratory conditions were not suitable for safety and the safety equipment was insufficient. gerlovich, parsa & jordan (2004) found that most teachers were not sufficiently aware of their legal and professional responsibilities regarding laboratory safety, and they prepared a safety education program. the number of studies conducted with students on laboratory safety is very limited. in the study conducted by yılmaz (2005), a test including laboratory safety and the dangers of chemical substances was applied to high school students. it was observed that the students had the basic knowledge required for laboratory studies, but their safety knowledge on chemical substances was insufficient. alaimo, langenhan, tanner & ferrenberg (2010) prepared a training program on laboratory safety. stating that the students were bored of reading the papers on which the safety rules were written, they trained them for a year in remarkable ways such as safety games, puzzles, cartoons, and videos. ali et al. (2018) selected eight secondary schools to determine students' awareness of laboratory safety in pahang, malaysia. a likert-type questionnaire with five components (work procedure, safety equipment, handling experiments, chemical waste management, emergency response plan) was applied to the students from schools located in urban and rural areas. this study indicates that the overall level of students' awareness on laboratory safety is at medium-high level, and this has shown that the existing measures in secondary school to enhance awareness of laboratory safety are adequate. it might help minimize the accidents that may occur if the students act consciously during the laboratory studies, obey journal of science learning article doi: 10.17509/jsl.v4i3.30752 222 j.sci.learn.2021.4(3).220-229 the safety rules, and realize possible safety problems. when the studies were examined, there was no study investigating secondary school students' knowledge and views on laboratory safety. due to the scarcity of studies on laboratory safety, it is aimed that this study will contribute to the field. this study aims to determine the knowledge and views of secondary school students on laboratory safety and to identify their deficiencies in this regard. for this purpose, answers to the following questions were sought: (1) what is the knowledge of secondary school students about laboratory safety?. (2) what are the views of secondary school students on laboratory safety?. 2. method this research was carried out using the survey method, which is one of the quantitative research methods. the survey method aims to describe an existing situation as it is, and the individual or object is tried to be defined as it is in its conditions (karasar, 2004). 2.1 sampling the research was conducted with 6th, 7th, and 8thgrade students of a public secondary school in the fall semester of the 2019-2020 academic year in kastamonu province, turkey. a total of 136 students, including 39 students from the 6th grade, 54 students from the 7th grade, and 43 students from the 8th grade, voluntarily participated in the study. the criterion sampling method was used in determining the sample. it is the sampling method in which people, events, objects, or situations with predetermined qualities are selected (yıldırım & şimşek, 2006). the school has two science teachers and a laboratory. before the research, science teachers were interviewed, and information was obtained about their laboratory use. the most important criterion in choosing the school was that science teachers use the laboratory actively and alternately. 2.2 data collection tool the researcher developed an open-ended questionnaire to determine the students' knowledge and views on laboratory safety. first, a draft form consisting of eight questions was created. similar studies in the literature were examined (aydın, diken, yel & yılmaz, 2011; 2011; demir, 2016; kırbaşlar, özsoy-güneş & derelioğlu, 2010), and these studies were used to form some questions. for example, aydın, diken, yel & yılmaz (2011) asked the teacher candidates to write down the safety errors they saw in the picture. in this study, students were shown a picture and asked to write down the right and wrong behaviors they saw. the picture used by aydın, diken, yel & yılmaz, 2011 (2011) and the picture used in this study were different. the draft form was submitted for expert opinion (two faculty members, three science teachers, and two turkish teachers) to be examined in terms of language and content. necessary corrections were made in line with expert opinions, and two questions were removed from the questionnaire. then, five students selected from a different school were interviewed to check the questionnaire's comprehensibility. in line with the interviews, the students' expressions were not understood or perceived differently, and the questionnaire consisting of six questions was finalized. the questionnaire consists of two parts (appendix). in the first part, there are two questions for determining the students' knowledge: (1) the picture of students working in the laboratory was given. the correct and wrong behaviors of the students were asked. (2) 11 safety symbols were given, and they were asked what they mean. these are common symbols found in 6th, 7th, and 8th-grade science textbooks. in the second part, there are four questions for determining the views of the students. the students were asked: (1) do they have enough information about laboratory safety?, (2) what should be considered in terms of safety during laboratory studies, (3) what kind of problems may arise when safety precautions are not taken into consideration in laboratory studies. (4) what are their expectations from science teachers and school administrators regarding laboratory safety? the first and second parts of the questionnaire were applied on different days. one lesson hour (40 minutes) was given to students for both parts. there was a possibility that taking a long time to answer the questionnaire could cause students to get bored and carelessly answer the last questions. this possibility has been taken into consideration, and possible data loss has been tried to be prevented. 2.3 analysis of the data descriptive statistics such as frequency and percentage were used to analyze the first part and the first question in the second part of the questionnaire. content analysis was used to analyze the 2nd, 3rd, and 4th questions in the second part. there are two types of approaches in content analysis, namely "inductive" and "deductive." if there is not enough information about the subject under study, then the inductive approach will go from specific to general (hsieh & shannon, 2005). in this context, the inductive content analysis approach was used in this study. in content analysis, data are analyzed in four stages; (1) coding, (2) finding themes, (3) organizing codes and themes, (4) defining and interpreting the findings (çepni, 2012). the data obtained were first coded, and themes were created. then, themes and codes were tabulated, and the frequency and percentage value of each code were calculated. the questionnaires collected in the study were given to a science educator working at the state university's education faculty. the science educator randomly and equally selected and analyzed questionnaires from each grade level (25%). the codings made independently by the researcher and the science educator were compared. the reliability percentage was calculated with the formula suggested by miles & huberman (1994). as a result of the journal of science learning article doi: 10.17509/jsl.v4i3.30752 223 j.sci.learn.2021.4(3).220-229 calculation, a value of .80 and above indicates that the evaluation is reliable (miles & huberman, 1994; miles, huberman & saldana, 2014). in this study, the agreement percentage between coders was found to be .94. this result shows that the evaluation made is reliable. non-compliance points were reviewed together, and a common decision was reached. 3. result and discussion in this section, the data obtained from the questionnaire were arranged and interpreted according to each subproblem. the data were analyzed both in general (all students) and separately for each grade level. 3.1 secondary school students' knowledge about laboratory safety in the first part of the questionnaire, two questions were asked to the students. in the first question, the students were asked what the right and wrong behaviors were in the picture's laboratory environment. the frequency and percentage values of the obtained findings are given in table 1. when table 1 is examined, it is seen that the most determined right behaviors in the picture are "regularity of the materials on the shelf (46.3%), a student wearing lab glasses (38.2%) and a student wearing a lab coat (30.1%)". the most determined wrong behaviors are "some students eating and drinking somethings (88.9%), disorganized laboratory (79.4%), spilling the liquid in the broken glass beaker to the floor (56.6%)". all right and wrong behaviors in the picture have been identified. however, the number of students varies according to the grade level. "the regularity of the materials on the shelf" is the most emphasized right behavior at each grade level (48.7%57.4%-30.2%). "a student wearing lab glasses and wearing a lab coat" is the right behavior mainly detected by 7th graders (72.2%-44.4%). while the "the material cabine has fixed to the wall", which is very important in terms of laboratory safety, was determined as the right behavior by only two 6th grade students, it is noteworthy that no students detected this the 7th and 8th grades. students identified 19 wrong behaviors in the picture. "some students eating or drinking somethings" (97.4%-90.7%79%) and " disorganized laboratory" (84.6%-81.4%-72%) are the most defined wrong behaviors at every grade level. besides, "spilling the liquid in the broken glass beaker to the floor" (43.5%-68.5%-53.4%) and "the microscope not unplugged" (43.5%-38.8%-58.1%) was reported as wrong behaviors by most students. very few students mentioned wrong behaviors such as not using protective equipment table 1 the right and wrong behaviors seen in the laboratory environment themes codes 6th grade 7th grade 8th grade total f % f % f % f % right behaviors some chairs are in place 4 10.2 8 14.8 12 27.9 24 17.6 regularity of the materials on the shelf 19 48.7 31 57.4 13 30.2 63 46.3 the material cabine has fixed to the wall 2 5.1 2 1.4 a student sitting in place 4 10.2 5 11.6 9 6.6 a student wearing lab glasses 3 7.6 39 72.2 10 23.2 52 38.2 a student wearing a lab coat 4 10.2 24 44.4 13 30.2 41 30.1 there is no right behavior 15 38.4 8 14.8 11 25.5 34 25 wrong behaviors some students eating or drinking somethings 38 97.4 49 90.7 34 79 121 88.9 spilling the liquid in the broken glass beaker to the floor 17 43.5 37 68.5 23 53.4 77 56.6 disorganized laboratory (bag and chair thrown on the floor) 33 84.6 44 81.4 31 72 108 79.4 the microscope not unplugged 17 43.5 21 38.8 25 58.1 63 46.3 a student holding the test tube facing himself 5 12.8 4 7.4 5 11.6 14 10.2 putting food on the experimental table 4 10.2 3 5.5 10 23.2 17 12.5 two students playing games in the lab 6 15.3 1 1.8 1 2.3 8 5.8 most students are not wearing lab glasses 2 5.1 8 14.8 10 7.3 lab coat not worn 3 5.5 1 2.3 4 2.9 gloves not worn while doing the experiment 2 5.1 8 14.8 10 23.2 20 14.7 the lab is not clean 4 10.2 5 9.2 9 20.9 18 13.2 a student sniffs the gas he doesn't know 6 15.3 1 1.8 2 4.6 9 6.6 most students walking around 6 15.3 2 3.7 3 6.9 11 8.1 students touching materials they do not know 2 5.1 7 12.9 12 27.9 21 15.4 damaging to materials/stuffs in the laboratory 1 2.5 4 7.4 3 6.9 8 5.8 safety precautions not taken 3 7.6 2 3.7 8 18.6 13 9.5 the materials used has not removed in place 1 2.5 1 1.8 2 4.6 4 2.9 glass beaker on the table is about to fall to the floor 4 10.2 4 9.3 8 5.8 students experimenting on their own without teacher supervision 4 10.2 11 20.3 9 20.9 24 17.6 journal of science learning article doi: 10.17509/jsl.v4i3.30752 224 j.sci.learn.2021.4(3).220-229 (lab glasses, lab coat, gloves) and making experiments dangerously without teacher supervision. most of the students do not know precisely what is right and what is wrong during laboratory studies. this may be due to the teacher's attitude during laboratory studies. the teacher's warnings, the rules he/she cares about, the behaviors he/she pays attention to affect and shape the students' behavior. for example, if the teacher warns the students about the subject more often, their attention may be directed to that behavior. similarly, if the teacher does not use protective equipment such as gloves or lab glasses in laboratory studies, it may cause students to perceive this behavior as a normal situation and not see it as a deficiency. aydın, diken, yel & yılmaz, 2011 (2011) asked prospective science and biology teachers to explain the safety errors they saw in the picture. some errors were detected by most prospective teachers, while some errors were detected by a very small number of prospective teachers. in other studies in the literature, it was concluded that science teachers (demir, 2016; muhammad, 2017) and prospective teachers (kırbaşlar, özsoy-güneş & derelioğlu, 2010) had insufficient knowledge about laboratory safety. although the sample groups of these studies and the present study are different, their findings are similar. in the second question, the meanings of some laboratory safety symbols were asked. the frequency and percentage values of the students who gave the right answer are given in table 2. in table 2, it is seen that the eye safety (92.6%) symbol is the most known, while the symbols of plant safety (22.7%) and animal safety (22.1%) are the least known. it is also seen that most students know some safety symbols (eye safety, sharp and pointed objects, hand safety, chemical substance warning, hand cleaning warning, lab coat warning) at all grade levels. while most of the 7th and 8th graders were known while the heat safety symbol was known, the rate of correct answers was found to be low in 6th graders. while nearly half of the 7th graders (51.8%) explained the plant safety symbol correctly, only one student (2.5%) from the 6th grade and two students from the 8th grade (4.6%) explained it correctly. similarly, half of the 7th graders (53.7%) answered the animal safety symbol correctly, only one of the 8th graders (2.3%) gave the correct answer. it is noteworthy that no student from the 6th grade gave the right answer about the animal safety symbol. the most known and least known symbols differ by grade level. in the 6th grade, the symbol of "hand safety" is the most known. the symbol of "animal safety" is unknown. in the 7th grade, the symbol of "eye safety" is the most known, and the symbol of "plant safety" is the least known. in the 8th grade, the symbol of "eye safety" is the most known, and the symbol of "animal safety" is the least known. when the explanations of the students were examined, it was seen that the students made explanations according to the picture in the symbol. for example, the animal safety symbol has a mouse image representing the live animal. most students thought this symbol was only about the mouse because of the picture and explained this direction. similarly, the presence of a picture of glasses in the eye safety symbol may have facilitated the symbol's recognition. the study's findings show that the pictures on the symbols facilitate the recognition of the symbol but do not provide enough information for students. similar findings have been reached in studies conducted on different samples in the literature. aydın, diken, yel & yılmaz, 2011 (2011) found that some safety symbols were known by most prospective teachers, while a small number knew some of them. in other studies, it was reported that prospective science teachers (anılan, 2010), prospective classroom teachers (gökmen & atmaca, 2019) and prospective biology teachers (derman & çakmak, 2016) did not know the safety symbols. artdej (2012) investigated thai undergraduate students' scientific understanding of safety symbols. the findings indicated that most of the students experienced confusion in the meaning of chemical safety symbols ,and they did not pay attention to safety symbols labeled on containers during experiments. tepe and tekbıyık (2019) evaluated secondary school science textbooks in terms of experiment and activity safety. it has table 2 students' knowledge of safety symbols themes codes 6th grade 7th grade 8th grade total f % f % f % f % safety symbols eye safety 32 82 53 98.1 41 95.3 126 92.6 sharp and pointed objects 27 69.2 52 96.2 33 76.7 112 82.3 hand safety 35 89.7 47 87 40 93 122 89.7 heat safety 10 25.6 38 70.3 36 83.7 84 61.7 electrical safety 16 41 32 59.2 25 58.1 73 53.6 chemical substance warning 27 69.2 47 87 37 86 111 81.6 plant safety 1 2.5 28 51.8 2 4.6 31 22.7 animal safety 29 53.7 1 2.3 30 22.1 glass equipment warning 5 12.8 29 53.7 14 32.5 48 35.2 hand cleaning warning 31 79.4 36 66.6 29 67.4 96 70.5 lab coat warning 27 69.2 45 83.3 36 83.7 108 79.4 journal of science learning article doi: 10.17509/jsl.v4i3.30752 225 j.sci.learn.2021.4(3).220-229 been determined that the science textbooks mainly include activities requiring low-level security measures and few activities that require medium and high-level security measures. in addition, it was observed that safety symbols were used enough to take necessary precautions in the activities. it is necessary but not sufficient to include safety symbols in the science textbooks. to reduce the risk of accidents, students must recognize the safety symbols and act accordingly in laboratory studies. in this study, although the students were asked about the safety symbols in science textbooks, it was seen that the meaning of each symbol was not known enough. 3.2 secondary school students' views on laboratory safety in the second part of the questionnaire, four questions were asked to the students. first of all, the students were asked if they had enough information about laboratory safety. the frequency and percentage values of the answers are given in table 3 according to table 3, 34.5% of the students think that they have sufficient knowledge about laboratory safety, while 65.5% think that they do not have sufficient knowledge. while 33.3% of 6th graders, 37.1% of 7th graders, and 32.5% of 8th graders think that they have sufficient knowledge, 66.7% of 6th graders, 62.9% of 7th graders, and 67.5% of 8th graders think that they do not have sufficient knowledge about laboratory safety. in short, most of the students do not think they have sufficient knowledge about laboratory safety. in the second question, their views were asked about what should be considered in terms of safety during table 3 students' views on having sufficient knowledge about laboratory safety 6th grade 7th grade 8th grade total f % f % f % f % yes 13 33.3 20 37.1 14 32.5 47 34.5 no 26 66.7 34 62.9 29 67.5 89 65.5 table 4 student views on safe working environment themes codes 6th grade 7th grade 8th grade total f % f % f % f % use of chemicals storing the chemicals in a separate cabinet 1 2.5 2 3.7 3 2.2 storing flammable and volatile substances in closed bottles 1 2.3 1 0.7 not touching/not smelling chemicals 10 25.6 20 37 18 41.8 48 35.2 not mixing different chemicals 1 2.5 4 7.4 5 11.6 10 7.3 not using out-of-date materials 1 2.3 1 0.7 not exposing chemicals that could cause explosions 3 7.6 2 4.6 5 3.6 layout of the laboratory environment keeping the lab clean and tidy 23 58.9 41 75.8 5 11.5 69 50.7 checking around before leaving the laboratory (is the plug pulled out? is the spirit extinguished? etc.) 1 2.5 6 11.1 3 6.9 10 7.3 venting the lab 2 4.6 2 1.4 not leaving the windows open 2 5.1 3 5.5 1 2.3 6 4.4 unwanted student behavior using laboratory materials without permission 17 43.5 19 35.1 7 16.2 43 31.6 running in the lab 14 35.8 6 11.1 12 27.9 32 23.5 joking/fighting in the lab 5 12.8 3 5.5 1 2.3 9 6.6 using laboratory materials carelessly/ damage to materials 16 29.6 16 11.7 eating and drinking somethings 1 2.5 1 2.3 2 1.4 pouring water on the ground 2 3.7 2 1.4 general measures fixing cabinets to the wall 1 2.5 1 0.7 keeping a first aid kit 1 2.3 1 0.7 hanging signs with safety warnings 1 2.3 1 0.7 staying calm in adverse situations such as fire. etc. 2 5.1 3 6.9 5 3.6 not to panic in the event of an accident 3 6.9 3 2.2 working rules listening to the teacher/paying attention to warnings 11 28.2 8 14.8 6 13.9 25 18.3 seeking help from the teacher with dangerous experiments 1 2.3 1 0.7 using protective equipment such as gloves, lab glasses, lab coat during the experiment 4 10.2 17 31.4 31 72.1 52 38.2 doing experiment with security measures 5 12.8 5 9.2 8 18.6 18 13.2 not watching dangerous experiments closely 2 5.1 5 11.6 7 5.1 journal of science learning article doi: 10.17509/jsl.v4i3.30752 226 j.sci.learn.2021.4(3).220-229 laboratory studies. the frequency and percentage values of student views are given in table 4 when table 4 is examined, it is seen that students' views about a safe working environment are grouped under five themes. students mainly stated that chemicals should not be touched/smelled (35.2%). in the theme of the laboratory environment's layout, it was mostly stated that the laboratory should be kept clean and tidy (50.7%). most of the students emphasized the wrong behaviors that threaten the safe working environment at all grade levels. in the theme of unwanted student behavior, students mostly explained that behaviors such as "using laboratory materials without permission (31.6%)" and "running in the lab (23.5%)" are wrong. in the theme of general measures, it is noteworthy that 7th-grade students did not give any opinion, and very few students from the 6th and 8th grade made statements. in the theme of working rules, students mostly stated that behaviors such as "using protective equipment such as gloves, lab glasses, lab coat during the experiment (38.2%)", "listening to the teacher/paying attention to warnings (18.3%)", "doing an experiment with security measures (13.2%)" must be done. when the table is examined in general, it is noteworthy that the number of students explaining the behaviors that must/must not be done is low and that 8th graders have more views about the safe working environment. most of the behaviors were expressed by a small number of students, showing that they lack knowledge on this subject. most of the students do not know what must/must not be done during laboratory studies, and they have already expressed this view in the previous question. okebukola et al. (2020) investigated the awareness level and implementation of the best practices of chemical safety by senior secondary school chemistry students in nigerian. students in rural schools were found to have a lower level of awareness of chemical safety than the students in urban schools. it was stated that this situation was due to a low level of chemistry laboratory resourcing, inadequate chemical safety training of the teachers, inadequacies in safety tools, charts, and kits, and weak enforcement of safety regulations. in some studies, it was observed that students were not given sufficient information about laboratory safety (aydoğdu & pekbay, 2016) and students had deficiencies about the precautions to be taken (ateş & özarslan, 2013; yılmaz, 2005). however, specific rules must be followed in laboratory studies. students must be informed about these rules and raise awareness. teachers' lack of informing and raising students' awareness is an essential factor in the occurrence of problems in the laboratory (aydoğdu & pekbay, 2016). in the third question, it was asked what kind of problems may arise when safety precautions are not considered in laboratory studies. the frequency and percentage values of student views are given in table 5. according to table 5, students think that problems may arise in the laboratory environment or people working in this environment. students mostly think that problems such as "injury/disability (63.9%)", "explosion (36.7%)", "fire (30.1%)" may occur. the 6th and 8th graders mostly stated that laboratory-related problems such as "explosion (43.5%-37.2%)", "fire (38.4%-46.5%)" may occur; 7th graders mostly stated that laboratory-related problems such as "explosion (31.4%)" and "damage to lab materials (14.8%)" may occur. while "injury/disability (61.5%64.8%-65.1%)" was mostly stated as the people-related problem at each grade level, "electric shock" was reported only by 7th graders (22.2%). students may have made such inferences due to an accident they had or news they heard about it. this is in line with ateş & özarslan (2013) findings that students think that injury problems may occur mostly if safety precautions are not taken into account in laboratory studies. deniz & ercan-kalkan (2013) explained that "fire, poisoning, electric shock, explosions, chemical spills, toxic smoke formation, etc." problems may occur if safety measures are not considered. tekbıyık & tepe (2017) examined the laboratory accidents that occurred between the years 2001-2017 and concluded that students caused the most accidents and that the accidents mostly occurred in the form of poisoning/burning/injury. accidents in the laboratory are mostly caused by factors such as the ignorance of teachers and students, careless and unconscious behaviors of them, and not knowing how to intervene in possible accidents (aydoğdu & yardımcı, 2013; aydoğdu & pekbay, 2016). this can cause problems from simple injuries to fatal accidents in laboratory studies. in this context, both teachers and students must take the necessary measures for a safe working environment. table 5 problems that may arise in laboratory studies themes codes 6th grade 7th grade 8th grade total f % f % f % f % laboratory related fire 15 38.4 6 11.1 20 46.5 41 30.1 explosion 17 43.5 17 31.4 16 37.2 50 36.7 cabinets may fall over 3 7.6 1 1.8 4 2.9 lab materials may be damaged 5 12.8 8 14.8 9 20.9 22 16.1 people related injury/disability 24 61.5 35 64.8 28 65.1 87 63.9 fatal accident 6 15.3 6 11.1 9 20.9 21 15.4 poisoning 6 15.3 5 9.2 8 18.6 19 13.9 electric shock 12 22.2 12 8.8 journal of science learning article doi: 10.17509/jsl.v4i3.30752 227 j.sci.learn.2021.4(3).220-229 in the fourth question, the students were asked about their expectations from science teacher and school administrators regarding laboratory safety. the frequency and percentage values of student opinions are given in table 6 according to table 6, students expect science teachers and school administrators to take some laboratory safety precautions. at all grade levels, the science teacher is mainly expected to inform students about safety precautions (17.9%-18.5%-44.1%) and introduce the materials used in the experiment (23%-12.9%-18.6%). in addition, students expect most from the school administration to provide training on laboratory safety. it is also noteworthy that many students made irrelevant explanations and stated that they had no expectations from the science teacher (25.6%42.5%-23.2%) or school administrators (30.7%-29.6%11.6%). although other safety measures that are expected to be taken by the science teacher or the school administrators are also crucial, they were mentioned by very few students. these findings are in line with ateş & özarslan (2013) findings that the students stated that they must be educated about laboratory safety and that various techniques (such as presentation, film, animation) must be used in this training. the laboratories must always be kept ready for a safe education environment, and students must be reminded of safety rules frequently. stepenuck (2002) prepared a report describing the dangers of chemical materials and explained that the safety information specified in the report must be taught to students. mogopodi, paphane, & petros (2015) researched the chemical management practices and safety in junior secondary school laboratories in gaborone. it was seen that there is a lack of knowledge and awareness on sound chemical management, and there are no measures in place for disposal of out-of-date stock or expired chemicals. in addition, chemical containers were either not labeled or had fading labels, and incompatible chemicals were stored together, increasing the risks of chemical accidents. ezrailson (2013) stated that science teachers and school administrators have responsibilities such as checking whether there is safety equipment in the laboratory and taking necessary precautions. some studies determined that safety equipment in school laboratories was insufficient (demir, 2016; emendu, 2007; nwele, 2013). according to ritch & rank (2001), science laboratory safety at all levels must be taken seriously. accidents are commonly reported in science laboratories worldwide, which underlined the need for awareness and adequate information on the importance of prioritizing safety in science laboratories 4. conclusion as a result of the study, it was seen that although secondary school students have some general information table 6 expectations regarding laboratory safety themes codes 6th grade 7th grade 8th grade total f % f % f % f % expectations from the science teacher working more carefully and meticulously in the laboratory environment 3 7.6 4 7.4 4 9.3 11 8.1 locking the lab door when there is no course 2 5.1 2 3.7 4 2.9 putting laboratory materials in the right place (storing chemicals in a different cabinet) 2 5.1 4 7.4 6 4.4 providing information about the materials to be used in the experiment 9 23 7 12.9 8 18.6 24 17.6 informing us about safety measures 7 17.9 10 18.5 19 44.1 36 26.4 taking more precautions 1 2.5 1 1.8 2 4.6 4 2.9 helping us in dangerous experiments 2 5.1 4 7.4 1 2.3 7 5.1 acting exemplary by using protective equipment 1 2.5 2 4.6 3 2.2 preventing our contact with harmful substances such as chemicals. etc. 5 9.2 4 9.3 9 6.6 no expectation/unrelated response 10 25.6 23 42.5 10 23.2 43 31.6 expectations from school administrators keeping a fire extinguisher and first aid kit in the laboratory for emergencies 3 7.6 8 18.6 11 8.1 fixing the cabinets 5 12.8 3 5.5 8 5.8 training on laboratory safety 10 25.6 16 29.6 9 20.9 35 25.7 purchasing new and quality materials for the laboratory 4 10.2 1 1.8 3 6.9 8 5.8 detecting broken/non-working materials 1 2.5 1 1.8 10 23.2 12 8.8 putting warning signs on the walls/cupboards 1 2.5 3 5.5 6 13.9 10 7.3 providing safety equipment such as gloves/lab glasses 1 2.5 2 4.6 3 2.2 having special cabinets for chemicals 7 12.9 1 2.3 8 5.8 installing a camera in the laboratory and detecting students who commit security violations 2 3.7 2 1.4 doing safety drills 2 4.6 2 1.4 no expectation/unrelated response 12 30.7 16 29.6 5 11.6 33 24.2 journal of science learning article doi: 10.17509/jsl.v4i3.30752 228 j.sci.learn.2021.4(3).220-229 about laboratory safety, they have deficiencies in terms of most of the information. it was determined that students do not know most of the behaviors that must/must not be made in the laboratory, and they are not aware of some of the safety symbols in the science textbooks. besides, most of the students stated that they don't feel competent about laboratory safety and need training on this subject. the following suggestions can be made in line with the results of the study: • as a result of the study, it was observed that although secondary school students had general knowledge about laboratory safety, they did not have detailed information. therefore, students should be informed about some issues such as safety symbols, the purpose of using laboratory materials, what to do before/during/after the experiment, and how to behave in possible accidents. • although the students were asked about the safety symbols found in the textbooks, it was determined that they did not have any idea about some safety symbols. this shows that the presence of safety symbols in textbooks alone is not enough. teachers should show the necessary sensitivity in conveying this information to students. • the students explained the behaviors that should and should not be done in a safe working environment. long-term laboratory observations can be made to compare whether students' views and actions coincide. • students expect both science teachers and school administrators to provide information about laboratory safety. they expressed that they felt inadequate in this area. interviews can be done with school administrators and science teachers about this issue. "have students been trained on this subject before? if so, by whom and how often?" answers to such questions can be sought. besides, the laboratory safety knowledge test can be used to determine which subject students need information. • to properly educate the students about laboratory safety, he/she must first have the correct and sufficient knowledge. therefore, teachers' knowledge and views about laboratory safety can be investigated. • similar studies can be conducted on different samples in different schools. increasing the number of researches will provide more information on the subject and reveal whether there is consistency among the results.a quantitative data collection tool was used in this study. by using qualitative data collection tools, research can be done, and in-depth information can be accessed. for example, "are students taking enough personal responsibility or do they see it as their teacher's job? what are the responsibilities of students and teachers regarding laboratory safety?". answers to such questions can be sought by conducting interviews with students and teachers references alaimo, p. j., langenhan, j. m., tanner, m. j., & ferrenberg, s. m. (2010). safety teams: an approach to engage students in laboratory safety. journal of chemical education, 87(8), 856-861. ali, n. l., ta, g. c., zakaria, s. z. s., halim, s. a., mokhtar, m., ern, l. k., & alam, l. (2018). assessing awareness on laboratory safety: a case study in pahang, malaysia. jurnal pendidikan malaysia, 43(2), 73-80. anılan, b. (2010). the recognition level of the students of science education about the hazard symbols of chemicals (case of esogu, eskisehir). procedia social and behavioral sciences, 2(2), 40924097. artdej, r. (2012). investigating undergraduate students’ scientific understanding of laboratory safety. procedia social and behavioral sciences, 46, 5058-5062. ateş, i̇., & özarslan, m. (2013). üstün zekalı ve yetenekli öğrencilerin fen bilimleri laboratuvar çalışmalarındaki güvenlik önlemleri ile ilgili görüşleri [the opinions of gifted and talented students about security measures in science laboratory studies]. journal of educational science, 2(3), 42-49. aydın, s., diken, e. h., yel, m., & yılmaz, m. (2011). fen ve teknoloji ile biyoloji öğretmen adaylarının laboratuvar güvenliği hakkındaki bilgi düzeylerinin belirlenmesi [determining the science and technology and biology teacher candidates' knowledge levels about laboratory safety]. journal of gazi education faculty, 31(2), 583-604. aydoğdu, c., & yardımcı, e. (2013). i̇lköğretim fen laboratuvarlarında meydana gelen kazalar ve öğretmenlerin geliştirebilecekleri davranış tarzları [accidents that occur in primary education science laboratories and behaviors that teachers can develop]. hacettepe university journal of education faculty, 44, 52-60. aydoğdu, c., & pekbay, c. (2016). sınıf öğretmen adaylarının laboratuvarlarda yaşanan kazaların nedenlerine yönelik görüşleri [the opinions of the classroom teacher candidates about the causes of the accidents in the laboratories]. journal of education science and technology research, 1(2), 103-112. aydoğdu, c. (2018). historical development of laboratory in the elementary school science program in turkey. m. shelley & s. a. kıray (eds.), education research highlights in mathematics, science and technology (pp. 141-174). iowa: isres publishing. bayraktar, ş., erten, s., & aydoğdu, c. (2006). fen ve teknoloji öğretiminde laboratuvarın önemi ve deneyler [the importance of the laboratory and experiments in science and technology teaching]. m. bahar (editor), science and technology teaching (pp. 219248). ankara: pegem a publishing. böyük, u., demir, s., & erol, m. (2010). fen ve teknoloji dersi öğretmenlerinin laboratuvar çalışmalarına yönelik yeterlik görüşlerinin farklı değişkenlere göre incelenmesi [examination of science and technology lesson teachers' efficacy opinions on laboratory studies according to different variables]. tubav journal of science, 3(4), 342-349. canel, m. (2002). laboratuvar güvenliği [laboratory safety]. ankara university, faculty of science, revolving fund business publications, no:26, ankara. çepni, s. (2012). araştırma ve proje çalışmalarına giriş (6. baskı) [introduction to research and project work (6th edition)]. trabzon: celepler publishing. demir, e. (2016). fen laboratuvarlarının fiziki şartlarının ve fen bilimleri öğretmenlerinin laboratuvar güvenliği konusundaki bilgi düzeylerinin araştırılması [investigation of the physical conditions of science laboratories and science teachers' knowledge level about laboratory safety]. master thesis, gazi university, ankara. deniz, v., & ercan-kalkan, m. (2013). laboratuvar kazaları [laboratory accidents].https://www.isgturkiye.com/konu/laboratuvarkazalari.3551/ derman, m., & çakmak, m. (2016). biyoloji öğrencilerinin laboratuvar güvenliği konusundaki görüşlerinin incelenmesi [examination of https://www.isgturkiye.com/konu/laboratuvar-kazalari.3551/ https://www.isgturkiye.com/konu/laboratuvar-kazalari.3551/ journal of science learning article doi: 10.17509/jsl.v4i3.30752 229 j.sci.learn.2021.4(3).220-229 biology students' opinions on laboratory safety]. bartın university journal of education faculty, 5(1), 178-187. emendu, n. b. (2007). the status of safety precaution and adequacy of equipment in chemistry laboratories in senior secondary schools in anambra state. in the nigerian academic forum: a multidisciplinary journal, 12, 84-89. ezrailson, c. m. (2013). danger in the school science lab: are students at risk? proceedings of the south dakota academy of science, 92, 149-164. gerlovich, j., parsa, r., & jordan, l. (2004). the 2003 status of science safety in tennessee secondary schools. journal of the tennessee academy of science, 79(4), 83-90. gökmen, a., & atmaca, s. (2019). öğretmen adaylarının laboratuvar güvenlik işaretleri konusundaki farkındalıkları [pre-service teachers' awareness of laboratory safety signs]. journal of education and social research, 6(2), 426-442. gudyanga, r. (2020). probing physical sciences teachers' chemical laboratory safety awareness in some south african high schools. african journal of research in mathematics, science and technology education, 24(3), 423–434.hackling, m., goodrum, d., & rennie, l. (2001). the state of science in australian secondary schools. australian sciences teachers' journal, 47(4), 12-17. hamurcu, h. (1998). fen derslerinde güvenlik [safety in science classes]. hacettepe university journal of education faculty, 14(14), 29-32. hasenekoğlu, i̇. (2003). laboratuvar güvenliği (2-3) [laboratory safety (23)]. erzurum: kazım karabekir education faculty publications. hill, r. h., & finster, d. c. (2010). laboratory safety for chemistry students. new jersey: john wiley & sons publishing. hodson, d. (1990). a critical look at practical work in school science. school science review, 70, 3340. hofstein, a., & lunetta, v.n. (2004). the laboratory in science education: foundations for the twenty-first century. science education, 88, 28-54. hsieh, h. f., & shannon, s. e. (2005). three approaches to qualitative content analysis. qualitative health research, 15(9), 1277-1288. karasar, n. (2004). bilimsel araştırma yöntemi (13. baskı) [scientific research method (13th edition)]. ankara: nobel publishing. kerimak-öner, m. n. (2020). kimya eğitiminde laboratuvar güvenliği kültürünün yerleştirilmesi [placement of laboratory safety culture in chemistry education]. isg academic, 2(1), 15-25. kırbaşlar, f.g, özsoy-güneş, z., & derelioğlu, y. (2010). fen bilgisi öğretmen adaylarının laboratuvar güvenliği konusuna yönelik düşünce ve bilgi düzeylerinin araştırılması [investigation of preservice science teachers' level of thought and knowledge on laboratory safety.]. journal of gazi education faculty, 30(3), 801-818. miles, m. b., & huberman, a. m. (1994). qualitative data analysis: an expanded sourcebook. thousand oaks, ca: sage publications. miles, m., huberman, m., & saldana, j. (2014). qualitative data analysis: a methods sourcebook. european journal of science education. los angeles: sage puplication, thousand oaks. mogopodi, d., paphane, b., & petros, s. (2015). assessment of chemical management practices and safety in junior secondary school laboratories in gaborone. journal of chemical health & safety, 22(5), 17–27. muhammad, r. (2017). assessing the laboratory safety and security skills among science, technology and mathematics (stm) teachers in sokoto state, nigeria. american journal of education and learning, 1(2), 62-68. nakiboğlu c., & sarıkaya, ş. (1999). orta öğretim kurumlarında kimya derslerinde görevli öğretmenlerin laboratuvardan yararlanma durumlarının değerlendirilmesi [evaluation of the use of laboratory by teachers working in chemistry lessons in secondary education institutions]. journal of buca education faculty, 11, 395405. nwele, c. n. (2013). extent of availability of safety instruments/equipment in science laboratories in secondary schools in ebonyi state of nigeria. academic journal of interdisciplinary studies, 2(10), 131-136. okebukola, p. a., oladejo, a., onowugbeda, f., awaah, f., ademola, i., odekeye, t., & ajayi, o. a. (2020). investigating chemical safety awareness and practices in nigerian schools. journal of chemical education, 98(1), 105-112. ritch, d., & rank, j. (2001). laboratory safety in the biology lab. bioscene, 27(3), 17-22. singer, s. r., hilton, m. l., & schweingruber, h. a. (2006). america's lab report: investigations in high school science. committee on high school science laboratories: role and vision. washington d.c.: national research council, national academies press. stepenuck, s. (2002). material safety data sheets. neact journal, 21(1), 28-32. tekbıyık, a., & tepe, m. (2017). türkiye’de 2001-2017 yılları arasında yaşanan laboratuvar ve deney kazalarının değerlendirilmesi [evaluation of laboratory experiments and accidents that occurred between the years of 2001-2017 in turkey]. international journal of innovative approaches in education, 1(1), 11-20. tepe, m., & tekbıyık (2019). ortaokul fen bilimleri ders kitaplarının deney ve etkinlik güvenliği bakımından değerlendirilmesi [evaluation of secondary school science textbooks in terms of experiment and effectiveness safety]. journal of national education, 48(1), 223-240. topsakal, s. (2006). eğitim fakülteleri sınıf ve fen bilgisi bölümü öğrencileri ile sınıf ve fen bilgisi öğretmenleri için ilköğretim 6. 7. ve 8. sınıflar fen ve teknoloji öğretimi [primary 6th, 7th and 8th grades science and technology education for students of education faculties, classroom and science departments, and classroom and science teachers]. ankara: nobel publishing. uluçınar, ş., cansaran, a., & karaca, a. (2004). fen bilimleri laboratuvar uygulamalarının değerlendirilmesi [evaluation of science laboratory applications]. turkish journal of educational sciences, 2(4), 465475. west, s. s., westerlund, j. f., stephenson, a. l., nelson, n. c., & nyland, c. k. (2003). safety in science classrooms: what research and best practice say. in the educational forum, 67(2), 174-183. yazıcı, e.k., & özmen, h. (2015). fen ve teknoloji öğretim programında yer alan deney ve etkinliklerin uygulanabilirliğine ilişkin öğretmen görüşleri [teachers' views on the applicability of the experiments and activities in the science and technology curriculum]. amasya university journal of education faculty, 4(1), 92117. yıldırım, a., & şimşek, h. (2006). sosyal bilimlerde nitel araştırma yöntemleri [qualitative research methods in the social sciences]. ankara: seçkin publishing. yılmaz, a. (2005). lise 1 kimya ders kitabındaki bazı deneylerde kullanılan kimyasalların tehlikeli özelliklerine yönelik öğrencilerin bilgi düzeyleri ve öneriler [students' knowledge levels and recommendations regarding the dangerous properties of chemicals used in some experiments in the high school 1 chemistry textbook]. hacettepe university journal of education faculty, 28, 226235. a © 2022 indonesian society for science educator 520 j.sci.learn.2022.5(3).520-530 received: 6 march 2022 revised: 25 april 2022 published: 27 november 2022 difficulties encountered by a dyslexic secondary school student in learning science and suggestions for solutions hamiye tırıl, seda okumuş* science education program, department of mathematics and science education, ataturk university, turkey *corresponding author: seda.okumus@atauni.edu.tr abstract this study aims to reveal the problems faced by a dyslexic student in learning science and determine what can be considered an effective science teaching method for this kind of learner. the case study design was used, and the study was conducted with a 7th-grade dyslexic female student. this study used observations and a semi-structured interview form as data collection tools. according to the findings, the problems faced by the student with dyslexia in learning science were generally compatible with the literature and immediately affected science learning. it was deduced that the student's reading skills, writing skills, motor skills, attention, memory and comprehension, language skills, use of sensory organs, and math skills are effective in learning science and that the development of these skills is necessary for understanding and comprehending science. in order to develop these skills, it is predicted that technology-based studentcentered activities and individualized teaching may be beneficial, which is in line with the constructivist philosophy. in addition, it may be practical to give technology-supported reading extracts containing scientific texts, tasks involving social interaction, and writing tasks to improve students' reading, writing, and language skills. keywords dyslexia, learning, problems, secondary school student, science 1. introduction according to the world health organization [who] (2011), dyslexia is the unexpected and permanent discomfort in acquiring productive reading skills despite adequate intelligence, socio-cultural opportunities, and education applied to the individual. since dyslexia is a concept that is not fully understood by society, it is often perceived as an intelligence problem. however, to be diagnosed with dyslexia, the individual's intelligence level must be within normal limits or above. (er nas, gülay, pehlevan, & delimehmet dada, 2018). in the 1920s, neurologist samuel t. orton, one of the scientists who conducted the first studies on dyslexia, expressed the common features of dyslexia as follows: • difficulty learning and remembering words • skipping words while reading • difficulty in writing • reverse detection of the letters b and d, p and q, and the numbers 6 and 9; mixed perception of letters or numbers in words • mixing the sounds of syllables or displacing consonants, often making typos • delayed or inadequate speech • difficulty choosing the most appropriate word when speaking • problems perceiving direction (like up and down) and time (like before, after, yesterday and tomorrow) • problem establishing hand-arm coordination (korkmazlar, 2009; kolburan & erbay, 2015). in 2016, the concept of learning disability was redefined by the national joint committee on learning disabilities (njcld) in a way that supports and resembles previous definitions. according to this definition, dyslexia; is a neurological condition that affects a person's reading, writing, speaking, spelling, calculation and inference skills, attention and memory, coordination, social skills, and emotional maturity (njcld, 2014). science is complicated for some students because it includes a wide learning area (physics, chemistry, biology, environment, etc.) and is related to abstract topics (okumuş & doymuş, 2018a). in addition, students may mailto:seda.okumus@atauni.edu.tr journal of science learning article doi: 10.17509/jsl.v5i3.44559 521 j.sci.learn.2022.5(3).520-530 have difficulties learning science because they do not effectively associate science subjects with daily situations (kirman bilgin & yiğit, 2017; okumuş & doymuş, 2018b). since the problems experienced by dyslexic students in fundamental areas such as reading and writing affect their learning, these students may encounter some problems in learning science. dyslexic children often learn to read and write later than their peers. the literature states that this learning difficulty can be overcome with specific training, some exercises, and tests (bonifacci, montuschi, lami, & snowling, 2014). dyslexia is just a learning difference. dyslexic students need a personalized reading education that includes reading instructions more systematically and clearly than their peers, designed in line with their reading level. dyslexic students have difficulty learning due to reading difficulties (horton, lovitt, & bergerud, 1990; olson & platt, 2004; ormsbee & finson, 2000). it has been emphasized that reading education to be given to dyslexic students should include intervention strategies for the basic features of dyslexia. for example, studies such as multisensory approach sound awareness training, teaching lettersound relationships, and use of fluency strategies are necessary (bonifacci et al., 2014). a similar situation can be considered for writing skills. dyslexic students often have poor writing skills and difficulty expressing themselves (benedek-wood, mason, wood, hoffman, & mcguire, 2014: horton et al., 1990; olson & platt, 2004; ormsbee & finson, 2000). dyslexic students whose reading difficulties are resolved will be more successful in writing and science as their reading comprehension skills improve. although dyslexia is not an intelligence problem, dyslexic students struggle with reading, writing, and language skills. therefore, in the science teaching process, it would be beneficial for a teacher with a dyslexic student in his/her class to adapt his/her teaching method with regular students to the dyslexic student. dyslexic students have trouble understanding complex instructions. for this reason, it is essential to use instructions that they can understand during the lesson. for example, a dyslexic student can understand a concept in a science class experiment on his/her own. however, when that concept is used with other situations or concepts, it may not be understood by the student, or the student cannot generalize what he/she has learned to other situations (kaldenberg, watt, & therrien, 2014; melekoğlu & çakıroğlu, 2017). therefore, this situation will cause the student to lag behind his/her peers in the learning process. in light of this, teaching the process steps more slowly and repetitively would be appropriate, especially in processes that include steps such as experiments. in dyslexic students' science learning studies, students need additional learning (therrien, taylor, hosp, kaldenberg, & gorsh, 2011). also, it was determined that students could not associate abstract concepts with the natural world and their motivation toward science was low (laine, nygren, dirin, & suk, 2016). studies on science learning of students with learning disabilities are available in the literature (gebbels, evans, & murphy, 2010; turan & atila, 2021). for example, turan and atila (2021) determined that augmented reality technology positively affected the science learning of students with special learning difficulties. in their study, scruggs, mastropieri, and boon (1993) taught two science units named "magnetism and electricity" and "rocks and minerals" to four students with special learning difficulties in a special education class using activity-based, inquiry-based, or textbook-based teaching approaches. at the end of the study, although the students' vocabulary acquisition in both units was limited, they performed significantly higher in the immediate and delayed tests when the students learned with the inquirybased approach. if a dyslexic student receives supportive education, being in constant cooperation with the place where he/she receives support facilitates the process. in this way, keeping in touch helps the student to be better known by the teacher, and the science teaching process becomes effective. in the education of dyslexic students, it is crucial to close the gap between them and their peers by providing various support applications with the cooperation of the family and school in terms of the correct management of the process (karaer & melekoğlu, 2020). dyslexic students often have problems with math skills (american association for the advancement of science (aaas), 1993; national research council (nrc), 2000). considering that science and mathematics are two essential and interrelated fields, it can be said that the students' inadequacy in mathematics will also directly affect science learning. in this context, dyslexic students need help adapting to the process with different strategies and practices to improve their math skills. it should not be forgotten that dyslexic learners have attention deficits. they cannot concentrate on a subject or situation for long periods. for this reason, they also have difficulties in understanding subjects. in this process, the teacher must make interventions that facilitate learning. for example, since some dyslexic students write letters and numbers backward, it may help to focus students' attention if the teacher writes on the board more broadly and clearly and adjusts the tone so they can also hear clearly. science courses are also important in attracting students' attention because science is fundamental as it develops the skills of dyslexic students, such as observing and classifying ordinary events (karaer & melekoğlu, 2020). another common characteristic of dyslexic students is that they often remain silent in the lessons. a quiet student may have problems fully understanding the lesson as he/she cannot tell the teacher what he/she does not understand. for this reason, students' language skills may also be insufficient. the teacher can encourage students in this situation to say things they do not understand. the journal of science learning article doi: 10.17509/jsl.v5i3.44559 522 j.sci.learn.2022.5(3).520-530 science includes verbal subjects as well as numerical subjects. therefore, it would be appropriate for dyslexic students if the instructor gave plenty of examples so that everyone could understand the verbal parts of the lesson and put further instructions for the dyslexic student on the board. in this way, the student is encouraged. another way to increase the interest of dyslexic students in a science classroom setting can be to show animation with colorful and moving objects (karaer & melekoğlu, 2020). griffin, simmons, and kamenui (2006) stated in their study that graphic organizers make it easier for dyslexic students to understand various concepts. in addition, it is an effective way to give clues about the questions the dyslexic student could not solve during the lesson so that his/her interest in science does not diminish. this will contribute to the student's efforts to solve the question again and motivate him/her. motivation increases the perceptive power of the individual. strong motivation ensures that attention is focused on one point. in other words, the higher the individual's motivation, the higher his/her perception (vatansever bayraktar, 2015). maybe this is about creativity. there are not enough studies in the literature on teaching science to students with dyslexia (dilber, 2017; er nas, şenel çoruhlu, çalık, ergül, & gülay, 2019; i̇lik, 2009; ward, 2010) and most studies are generally in the field of classroom education (balcı & çayır, 2017; başar & göncü, 2018; çeliktürk sezgin & akyol, 2015; dündar & akyol, 2014; fırat & koçak, 2018). er nas et al. (2019) developed a science experiment guide for middle school students with learning difficulties and looked at its effectiveness. accordingly, the developed guide positively affected students' conceptual understanding. dilber (2017), in her study of science teachers, revealed that many teachers do not create or produce a special application for their students with learning difficulties; instead, they conduct the lessons the same way they do with other students. keeping dyslexic students in the background in the teaching process does not comply with forming all students as science literate. in this respect, it is necessary to reveal the difficulties these students face in learning science and perform studies on this to integrate them into society. for this purpose, this study aims to reveal the problems faced by a dyslexic student in learning science and determine what can be considered for effective science teaching. for this reason, this research will contribute to the literature. in this study, answers to the following research questions were sought. 1. what are the problems a secondary school dyslexic student faces in the 7th-grade learning science in turkey? 2. what are the points to be considered for effective science teaching to dyslexic students? 2. method in this study, the exploratory case study design was used as the aim was to reveal the problems faced by a 7th-grade secondary school dyslexic student in learning science and determine the points to be considered for effective science teaching. stake (1995) stated that “a case study is expected to catch the complexity of a single case … case study is the study of the particularity and complexity of a single case, coming to understand its activity within important circumstances.” yin (2003) emphasized the following for the case study: “in general, case studies are the preferred strategy when ‘how’ or ‘why’ questions are being posed, when the investigator has little control over events, and when the focus is on a contemporary phenomenon within some real-life context.” for all these reasons, the exploratory case study pattern was used. 2.1 participant the study's participant consists of a dyslexic female student studying in the 7th grade of a secondary school in turkey. the convenience sampling method was used in sample selection. the provincial directorate of national education provided a list of schools with dyslexic students, and a suitable student sample was then selected by contacting the school administration and parents and obtaining their permission. the selected student's parents cooperate with the school for their children to learn better. in addition, the student was informed about the process. the student welcomed the observation. other students in the class did not know that the dyslexic student was being observed. 2.2 data collection tool observations and semi-structured interviews were used as data collection tools in the study. accordingly, the student was observed in the science lesson with the observation form prepared in advance. since the student was observed in her natural environment, natural and participatory observations were used since the researcher personally participated in the process. the observation form used was titled dyslexic secondary school student observation form-dsssof. while creating the dsssof, the "performance determination form for individuals with special learning disabilities," created by the general directorate of special education guidance and counseling services of the ministry of national education in turkey, was used. in the form, reading skills, writing skills, motor skills, attention, memory and comprehension, language skills, use of sensory organs, and math skills were observed, and general observations were made. observations continued for four weeks. the statements in the form were arranged within the science course framework and reviewed by an expert in science education. a pilot study was conducted on a dyslexic student to ensure the reliability of the dsssof. in this study, a semi-structured interview was used because it allowed the interview to be planned and prolonged. for this purpose, a semi-structured journal of science learning article doi: 10.17509/jsl.v5i3.44559 523 j.sci.learn.2022.5(3).520-530 interview form (ssif) was developed. there are eight open-ended questions in the ssif. expert opinion was taken for the questions formed, and a pilot study of the ssif was conducted with the dyslexic student. 2.3 data analyses qualitative data analysis was performed, and content analysis was applied to analyze the data collected from the dssof and the ssif in depth. firstly, the themes, codes, and sub-codes for the dsssof were created based on the existing themes and codes in the literature regarding the features examined in the observation form. researchers created sub-codes by taking samples from previous studies and according to the frequency of the student's behavior. each observation was analyzed according to this theme, codes, and sub-codes. the relationship of the related skill with science learning was analyzed in the sub-codes. accordingly, the themes, codes, and sub-codes created for the dsssof are given in table 1. the content analysis of the dssof consists of eight themes, 24 codes, and 24 sub-codes. the data obtained from the ssif were analyzed in two parts the student's interest in science, the parts with the most difficulty, and how much of the subjects she could understand. accordingly, the themes and codes in the content analysis for the ssif are given in table 2. according to table 2, the content analysis of the ssif consists of three themes and nine codes. 2.4 process the dsssof and the ssif were primarily developed during the study process. the validity and reliability of the dsssof and the ssif were then provided. subsequently, observations were performed for four weeks and 16 hours. observations were collected during the four weeks when schools were open and the student attended school during the covid-19 pandemic. after the observations were completed, the ssif was conducted with the student. after the whole process was finished, the data analysis phase was initiated. table 1 themes, codes, and sub-codes generated for the content analysis of the dsssof theme code sub-code (related to science) reading skillsrs sufficients high level-hl partially sufficient ps i̇ntermediate levelil insufficient -i low level-ll writing skillsws sufficients high level-hl partially sufficient ps i̇ntermediate levelil insufficient -i low level-ll motor skillsms sufficients high level-hl partially sufficient ps i̇ntermediate levelil insufficient -i low level-ll attention, memory, comprehensionamc successfulsc high level-hl partially successful pss i̇ntermediate levelil unsuccessful-u low level-ll language skillsls sufficients high level-hl partially sufficient ps i̇ntermediate levelil insufficient -i low level-ll using the sense organs uso successfulsc high level-hl partially successful pss i̇ntermediate levelil unsuccessful-u low level-ll math skillsmas sufficients high level-hl partially sufficient ps i̇ntermediate levelil insufficient -i low level-ll general observation go high level-hl high level-hl i̇ntermediate levelil i̇ntermediate levelil low level-ll low level-ll table 2 themes and codes generated for the content analysis of the ssif theme code interest in science -is positive negative most difficult parts mdp unit teacher content understanding the topics ut focusing activity content journal of science learning article doi: 10.17509/jsl.v5i3.44559 524 j.sci.learn.2022.5(3).520-530 3. result and discussion 3.1 findings and discussion obtained from the dsssof this study aimed to reveal the most critical problems facing a dyslexic student in learning science. accordingly, the student's reading, writing, motor, attention, memory and comprehension, language skills, use of sensory organs, and math skills were observed. the findings obtained from the dsssof showing the change in student behavior for each theme in the study process are presented in table 3. according to table 3, the student is partially sufficient in the reading skills theme. according to the notes taken during the observations, she cannot understand the text for the rs theme every week if she does not read it herself, and she is reluctant to read because long texts are boring, and she does not want to read until the end of the topic. her ability to read texts in a book, notebook or 5in an activity description is partially sufficient. the fact that she gets bored quickly with the act of reading and cannot understand what someone else is reading causes her to reread the text. therefore, she experiences a loss of time during the course process and does not want to read the subject to the end. this situation negatively affects her science learning. also, she sometimes adds sounds and syllables to the text she is reading, misreads sounds and words, makes up the end of the word, and follows the text with her fingers while reading. these are common in dyslexia (korkmazlar, 2009; kolburan & erbay, 2014; taymaz sarı & biçer, 2020). in addition, she sometimes has difficulty answering the questions about the text she has read. furthermore, her reading skill has an intermediate effect on learning the science lesson. another noteworthy situation is that she constantly followed the writings with her fingers while reading. it can be deduced that all these behaviors make it difficult for her to understand the reading passage because it is observed that she sometimes has difficulty answering the questions about the text she has read. from the general observations, it is seen that the reading skill of the student affected the learning of the science lesson at an intermediate level. it is thought that developing her reading skills will enable her to understand science concepts more quickly. in this respect, it is recommended that the student regularly read texts on scientific subjects with traditional and technological tools to improve reading skills (tangkakarn & gampper, 2020). because the z generation has existed in technology, and some applications make life easier in many areas thanks to technology. she is partially sufficient in the writing skills theme. her writing is legible for the ws theme. the fact that she tries to write very quickly while taking notes is determined. this table 3 findings from the dsssof theme code sub-code week 1 week 2 week 3 week 4 rs s hl ps il x x x x i ll ws s hl ps il x x x x i ll ms s hl ps il x i il x x x ll amc sc hl pss il x u ll x x x ls s hl ps il x x x x i ll uso sc hl il x x x pss il x u ll mas s hl ps il x x x i ll x go hl hl il il x x x x ll ll journal of science learning article doi: 10.17509/jsl.v5i3.44559 525 j.sci.learn.2022.5(3).520-530 situation causes her handwriting to be sometimes illegible because it is more legible when she is not trying to write quickly. she sometimes also skips syllables and letters, adds letters or syllables, and does not use punctuation marks in the appropriate place. additionally, she sometimes writes the words aloud and cannot write her thoughts in their entirety. that is, it also pronounces the word while typing. therefore, she cannot put into writing what she says in language. another remarkable aspect is that she tries to take notes on everything that is said. this situation prevents the student from focusing on the lesson. it is a typical situation that one who cannot focus on the lesson will not be able to fully understand the subject because the focus is essential for learning. in addition, she sometimes has difficulty writing down what is written on the board in her notebook. this shows the weakness in writing skills. johnson (2013) determined that children who have difficulty reading also have problems in the field of writing, and he determined that systematic writing practices also accelerate reading (taymaz sarı & biçer, 2020). in this context, reading and writing skills are essential in improving the student's science achievement and should be considered as a whole. from the general observations, the student's writing skill affects learning the science lesson at an intermediate level. writing skills can be improved by giving the student various assignments that require writing regularly (in a way that does not cause boredom for the student) (rodriguezhernandez & silva-maceda, 2021). for example, the student can be given the task of writing a composition about where she can use the information she learned in the science lesson that day in daily life. alternatively, the student may be asked to write a list of the topics they understand best and have difficulty understanding in the science lesson that day. she is generally inadequate in the motor skills theme. it is noteworthy that she holds the pencil tightly as if it will fall out of her hand at any moment. her hand-arm coordination is weak. this situation causes the student to have difficulty doing assignments that require psychomotor skills. hand-eye coordination is sometimes poor. in addition, she has severe problems converting homework into the material. this may be due to not fully understanding the purpose of the assignment. the fact that she is not very successful in material assignments may also be due to the teacher's attitude because their critical comments in response to her homework disrupted her motivation. this situation causes the student to develop a negative attitude towards the other assignments she has to do and reduces her interest in the lesson. her weak motor skills and the teacher's attitude towards the materials they create negatively affect her science learning. she is generally unsuccessful in terms of attention, memory, and comprehension skills. this finding is consistent with the literature (başar & göncü, 2018; ekşi sınır, 2020; kesikçi & amado, 2005). she sometimes gives irrelevant answers to the questions asked and refers to examples that are not relevant to the subject. in addition, she uses gestures and facial expressions more than necessary. another excellent point about the student is her inability to plan the time correctly. for this reason, she cannot complete her work. she often finds it difficult to follow directions. also, she confuses the order of events, especially when telling a story, she has read or listened to. furthermore, she does not know where to start the tasks she needs to fulfill. confusing the order in a given situation is another critical issue. it is also known that individuals with dyslexia have 12-24% of attention deficit and hyperactivity disorder (balcı, 2019; shaywitz, fletcher, & shaywitz, 1994). this may be the reason for the student's behavior. the general observations show that the student's lack of attention, comprehension, and comprehension skills generally affects learning the science lesson negatively. concepts are essential for science education. in this respect, necessary to perform various studies to reveal and develop the comprehension skills of dyslexic students for effective science learning. she is partially sufficient in terms of language skills. she can generally express herself, sometimes use words with different meanings, and has difficulty finding the right word while speaking. it is also stated in the literature that the vocabulary of dyslexic students is limited (bishop, mcdonald, bird, & hayiou-thomas, 2009; delimehmet dada & ergül, 2020; seçkin yılmaz & yaşaroğlu, 2020). in addition, she is excited while talking in a crowd and has difficulty expressing her feelings and thoughts. dyslexic individuals get excited, stuck, and confused, especially when talking in public. while they are sometimes very selfconfident in verbally expressing their thoughts and feelings, they are sometimes introverted. while they can communicate more efficiently with their close friends, they avoid making eye contact with strangers and prefer not to talk. this was also the case with our student. the results obtained are compatible with the literature (seçkin yılmaz & şemşedinovksa, 2020), and this negatively affects the social relations of dyslexic individuals (conti-ramsden & botting, 2004; helland, lundervold, heiman, & posserud, 2014). in the literature, dyslexic individuals also experienced these problems in terms of language skills (seçkin yılmaz & sarı, 2020; seçkin yılmaz & şemşedinovksa, 2020; snowling, hayiou-thomas, nash, & hulme, 2019). in this respect, the results of this study are compatible with the literature. from the general observations, the student's language skills moderately affected the learning of the science lesson. her ability to use the language effectively will contribute to correctly learning science concepts. there is a relationship between language skills and science learning (delimehmet dada & ergül, 2020). doing oral activities that improve the journal of science learning article doi: 10.17509/jsl.v5i3.44559 526 j.sci.learn.2022.5(3).520-530 student's language skills and ensuring the student's participation will positively affect science learning. she is generally successful in using sensory organs (sense of sight, sense of hearing, and sense of touch) effectively. although she does not experience any problems in seeing, hearing, and feeling, the perception of distance, depth, and size is not very good. the literature states that dyslexic students generally have no problems using their sensory organs (korkmazlar, 2003). in this respect, the results of this study are compatible with the literature. she does not have difficulty distinguishing the objects. however, she sometimes cannot perceive one specific sound she hears simultaneously with other sounds. also, she may sometimes have problems in grouping and matching objects. therefore, it will be adequate to perform the applications in line with the experts' recommendations (special education specialists) so that she can use her sensory organs effectively. from the general observations, her ability to use her senses affects learning the science lesson at an intermediate level. considering that the sufficient psychomotor skills of the student will facilitate understanding and learning, it is predicted that activities that will ensure the physical and mental development of the student as a whole will be practical. her math skills are partially sufficient. however, she does not like mathematical operations (she said that in the semi-structured interview. these were discussed in the ssif analysis section.). considering that many students see mathematics as a complex and boring subject, it can be said that children with learning difficulties do not like mathematics. also, she confuses about math terms and concepts. it is not surprising that these behaviors, common in dyslexia (taymaz sarı & biçer, 2020), are also present in this student. complex symbols and terms reduced the student’s interest in the lesson. in addition, she has difficulty following the steps to be followed while solving the problem and sometimes performing the four operations. she gets bored of studying the units or subjects required by mathematical operations, and this affects science learning. the general observations show that the math skill of the student affects the learning of the science lesson at an intermediate level. many operations in science require mathematical processes. this is why students with weak mathematics have difficulty understanding science. in this respect, designing and implementing activities that improve the student's math skills will also increase the student's science achievement. in general, when the student's academic achievement is considered, it is at an intermediate level compared to her classmates and age. despite having a learning disability, she is not a student who does very poorly in the classroom. this shows that if a good education is provided, she has similar learning to her peers in the classroom. it has been observed that behaviors affect the student's learning. in this context, her acting without thinking, acting impatiently, and sometimes dropping out of assignments containing long texts can affect her learning. in addition, it can be deduced that the rapid decline of her self-confidence in unfamiliar environments or the presence of foreign people prevents the student from showing sufficient success in learning science. better recognizing dyslexic students and identifying their needs will effectively ensure effective science learning. 3.2 findings and discussion obtained from the ssif this section presents the findings obtained from the ssi conducted with the student after the observations were completed. in this context, the students' answers were analyzed by considering the themes and codes in the content analysis. findings from the ssif are given in table 4. according to table 4, she gives a positive answer to the question “do you like science? are you interested in the lesson? why?” for the interest in science theme. she enjoys research, learning, and discovering new things and sees the science lesson as a tool. however, she states that the verbal parts of the lesson do not attract her attention. she likes the science lesson but has difficulties with some parts. it can be said that she sees science as a way to learn and discover new things. the broad subject area of science is related to almost every field of the scientific world. in this respect, understanding science makes it easier to understand the world. it is gratifying that she is aware of table 4 findings from the ssif theme code student's statements is positive “science class interests me. because i enjoy researching, learning and discovering something new..” negative “doesn't attract my attention when verbal issues are predominant.” mdp unit “i had a very difficult time understanding the 'cell and divisions' unit. because it was very abstract and i had a lot of trouble grasping it. i was so bored with this unit that i could easily get distracted. this made me focus on the lesson.” teacher “sometimes i have a hard time understanding. i can’t understand because of the way our teacher explains the subject. i can’t understand the subject or forget it quickly when it is only verbally explained and passed..” content “i have more difficulty in understanding verbal content. because when someone else reads it, i can't understand the subject. i need to read that topic from beginning to end so that i can understand it. otherwise, i cannot comprehend the subject and the lesson is very boring for me.” journal of science learning article doi: 10.17509/jsl.v5i3.44559 527 j.sci.learn.2022.5(3).520-530 discovering. however, she is bored with some verbal subjects in science. the fact that verbal topics contain long explanations may have reduced her attention in the lesson. this situation can cause the student to have difficulty understanding verbal subjects and result in a negative attitude towards understanding verbal subjects. in the theme of the most difficult part, she answered the question “which unit is the hardest for you to understand in science class? why?” for the “unit” code and stated that she had difficulties in the “cell and divisions” unit. she attributes the reason for this situation to the abstractness of the unit. the science includes abstract concepts and explanations, especially in biology and chemistry (okumuş & doymuş, 2018a). this situation negatively affects her interest and understanding of science. in this context, it can be expected for dyslexic students not to understand the subject of cell divisions, while abstract subjects affect science achievement, even for students who do not have learning difficulties. she answers 'the teacher's style' to the question “do you ever have trouble understanding science? if your answer is yes, why?” for the “teacher” code in the theme of the most difficult part. accordingly, she states that the teacher's monotonous and verbal teaching of the lesson made it difficult for her to understand it. furthermore, the teacher's teacher-centered instruction during the lesson can negatively affect her understanding of the lesson. the fact that even students who do not have learning difficulties cannot show sufficient success in a teacher-centered education system, especially since the beginning of the 2000s, updates have been made in the programs within the framework of the constructivist learning approach in turkey (ministry of education (moe), 2013; 2018). in this context, she cannot learn science at the desired level with practices such as regular or basic lectures or boring lectures (güven & aydoğdu, 2009). on the other hand, students in need of special education learn in a more complicated manner than their peers on a regular period. in this respect, using studentcentered teaching models, methods, and techniques supported by technology and giving more space to individualized teaching will effectively engage students in need of special education in the teaching process. furthermore, within the framework of the inclusive education program, students in need of special education in many schools at the secondary level receive lessons in the same classes as other students (sarı & biçer, 2020). however, it is stated in the literature that teachers do not have enough knowledge and skills to acquire special students and make them active in the lesson (gök & erbaş, 2011; sucuoğlu & akalın, 2010). furthermore, odom (2000) emphasizes that teachers play a main role in successful and qualified inclusive practices. therefore, it will be effective for teachers of all subjects, including science, to receive in-service theoretical and practical training on practices that will enable students with special education needs to participate actively in the process. in response to the question “do you have a harder time understanding verbal or numerical content in science class? why?” asks for the "content" code in the mdp theme, and she states that she has a more challenging time understanding verbal subjects. she also states that she can only fully understand the subject when she reads it herself. the student has more difficulty understanding verbal subjects in science. this may be due to her weakness in reading skills because she states that she can only fully understand the subject when she reads it herself. the science includes many scientific concepts, and this makes the course challenging. the fact that she has reading difficulties also increases the lesson's difficulty. in this context, applications such as the use of exciting texts and the case study method will effectively improve the student's reading skills and increase his/her interest in verbal subjects in science. in understanding the theme of the topic, she answers that she does not have any difficulties with the question, “do you have difficulty attending class in science class? if your answer is yes, why?” asks for the “focusing” code. in response to the question “how many minutes can you listen to the lecture without interruption in science class? do you have problems concentrating in the lesson? if your answer is yes, why?” another question asks for the table 4 findings from the ssif (continued) theme code student's statements ut focusing “no, i'm not having trouble. it's fun to answer questions.” “i can focus on the lesson without being distracted from the beginning to the end of the lesson, but until 1 year ago, i could get distracted very quickly. i could stay fixed in one place for a long time and just dive in until someone warned me. later, with the support of my family, teachers, and most importantly, my belief in myself, i was able to overcome this very big problem.” activity “no, i'm not struggling. on the contrary, i have a lot of fun doing it. understanding by doing makes me more permanent. for example, whenever i see the subject that we did the experiment or the subjects that i learned in the form of a game, the experiment we did that day comes to my mind and i can remember it immediately.” content “i have difficulty in understanding when there is a lot of verbal content and the experiment is not done and passed. but when i experiment and understand the subject, reading the activities becomes enjoyable and then i don't have any difficulties doing it anymore.” journal of science learning article doi: 10.17509/jsl.v5i3.44559 528 j.sci.learn.2022.5(3).520-530 “focusing” code in the ut theme, she states that she can focus throughout the course. when the student looks at it from her perspective, it has been determined that she sees herself well in the lessons. however, it was stated above that she had problems understanding the subjects, writing and reading, participating in the lesson, and expressing herself. she also states that she has improved herself on distraction and receives support in focusing this year. she mentions that with the support of her family and teachers, her self-confidence increases, and her focusing problem decreases. in response to the question “do you have difficulty in understanding activities such as experiments, games, and problem-solving in science class? if your answer is yes, which parts are the most difficult?" for the “activity” code in the ut theme, she states that she does not have difficulty in understanding activities such as experiments, games, and problem-solving in the science lesson. on the contrary, these applications make it easier to understand the lesson, and she attains more permanent learning when she learns by doing. therefore, student-centered applications and activities supported by technology can make it easier for students to understand the lesson and enable them to learn science in a more fun way. the results obtained in studies conducted with secondary school students show parallelism with these results (çalışkan & kapucu, 2021; demirel & özcan, 2021; eyecioğlu & yeşilyurt, 2021; i̇din & aydoğdu, 2021). within the framework of these results, applying different methods and techniques will successfully enable dyslexic students to learn science. in response to the question “do you have difficulty reading and understanding the activities in the science book? if your answer is yes, which parts are the most difficult?” asks for the “content” code in the ut theme, she states that she has difficulty in understanding when there is much verbal content, and the experiment is not done and passed. on the other hand, she says that she understands the activities supported by experiments more effortlessly and better. since experiments constitute a large part of the science content, it can be understood from the student’s explanation how important it is to include experiments in the course process. in studies conducted with secondary school students, generally, experiments are more effective in understanding and remembering the subject and improving scientific process skills (aydoğdu & ergin, 2008; er nas et al., 2019; gilbert & reiner, 2000). similarly, experiments can be important in the science learning of dyslexic students with learning difficulties. 4. conclusion it is crucial to identify dyslexic students, recognize dyslexia correctly, and analyze these symptoms well. it is even more critical for the teacher to be aware of a dyslexic student and to manage the process considering this situation, especially in lessons that include abstract concepts such as science lessons. thus, the teacher can help the student to overcome the problems encountered in the learning process. in this study, the problems faced by a dyslexic student in learning science are examined, the results are generally compatible with the literature, and it is concluded that this situation has a moderate effect on science learning. it is deduced that her reading skills, writing skills, motor skills, attention, memory and comprehension, language skills, use of sensory organs, and math skills are effective in learning science and that the development of these skills is necessary for understanding and comprehending science. technology-based studentcentered activities and individualized teaching may benefit from developing these skills, which aligns with the constructivist philosophy. in addition, it may be helpful to give reading extracts containing scientific texts, tasks involving social interaction, and writing tasks to improve students' reading, writing, and language skills. also, educational and attention-grabbing activities may improve students' math skills. in addition, it will be appropriate to use experiment activities that facilitate remembering and learning and that appeal to many senses so that the dyslexic student can be provided with permanent and practical learning in science learning. considering that there are inclusive students in almost every school in turkey, it will be easier for teachers from all branches of education to receive theoretical and practical in-service training on special education at regular intervals and to initiate updates in training in the education process of children with special education needs. it is crucial to act according to experts' recommendations and provide psychological support to adapt and include dyslexic students in the science learning process and facilitate learning. in addition, the importance of helping and guiding all members of the family and school organization in this process cannot be denied. acknowledgment this study was produced from the project named "difficulties encountered by a dyslexic secondary school student in learning science and suggestions for solutions," carried out within the framework of tübi̇tak 2209/a university students research projects support program. the first author acted as the director, and the second author acted as the consultant for the project. we want to thank tübi̇tak for the project support. references american association for the advancement of science [aaas]. (1993). benchmarks for science literacy. new york, ny: oxford university press. aydoğdu, b., & ergin, ö. (2008). the effects of open-ended and inquiry-based laboratory techniques on students’ science process skills. ege journal of education, 9(2), 15-36. journal of science learning article doi: 10.17509/jsl.v5i3.44559 529 j.sci.learn.2022.5(3).520-530 balcı, e. (2019). percentage and some characteristics of dyslexia in primary and secondary schools students. millî eğitim, 48(special issue, 1), 717-736. balcı, e., & çayır, a. (2017). the impact of multiple sensory learning in increasing the phonological awareness of a 4th grade student with dyslexia. journal of mother tongue education, 5(2), 201-216. başar, m., & göncü, a. (2018). clearing misconceptions of primary school teachers' about learning disabilities and evaluation of teacher opinions. h. u. journal of education, 33(1), 185-206. https://doi.org/10.16986/huje.2017027934 benedek-wood, e., mason, l. h., wood, p. h., hoffman, k. e., & mcguire, a. (2014). an experimental examination of quick writing in the middle school science classroom. learning disabilities: a contemporary journal, 12(1), 69-92. bishop, d. v. m., mcdonald, d., bird, s., & hayiou-thomas, m. (2009). children who read words accurately despite language impairment: who are they and how do they do it? child development, 80, 593– 605. https://doi.org/10.1111/j.1467-8624.2009.01281.x bonifacci, p., montuschi, m., lami, l., & snowling, m. j. (2014). parents of children with dyslexia: cognitive, emotional and behavioural profile. dyslexia, 20(2), 175-190. https://doi.org/10.1002/dys.1469 conti-ramsden, g., & botting, n. (2004). social difficulties and victimization in children with sli at 11 years of age. journal of speech, language, and hearing research. https://doi.org/10.1044/10924388(2004/013) çalışkan, t., & kapucu, s. (2021). the effect of argumentation-based learning approach on students’ conceptions of learning science and approaches to learning science in the subject of astronomy. yyu journal of education faculty, 18(1), 316-353. https://doi.org/10.33711/yyuefd.863217 çeliktürk sezgin, z., & akyol, h. (2015). improving reading skills of fourth grade elementary student who has reading disability. turkish journal of education, 4(2), 4-16. delimehmet dada, ş., & ergül, c. (2020). vocabulary and reading comprehension skills of students with learning disabilities and with typical development. ankara university faculty of educational sciences journal of special education, 21(1), 1-22. https://doi.org/10.21565/ozelegitimdergisi.544840 dilber, y. (2017). examination of the instructional process carried out by the science teachers with mainstreaming students diagnosed with learning disabilities. retrieved from yok tez merkezi (471999). demirel, r., & özcan, h. (2021). the effect of argumentation based science and engineering practices on 7th grade students’ achievement about light topic. aksaray üniversitesi sosyal bilimler enstitüsü dergisi, 5(1), 100-111. https://doi.org/10.38122/ased.912508 dündar, h., & akyol, h. (2014). a case study regarding definition and solution of reading and comprehensive problems. education and science, 39(171), 361-377. ekşi sınır, g. (2020). comparison of problem solving, attention skills, and executive functions of students with and without special learning difficulties. retrieved from yok tez merkezi (610722). er nas, s., gülay, a., pehlevan, m., & delimehmet dada, ş. (2018). investigation of the science and nature activities instructional process carried out by special education teachers. atatürk university kazım karabekir eğitim faculty of education, 36, 60-77. er nas, s., şenel çoruhlu, t., çalık, m., ergül, c., & gülay, a. (2019). investigating the effectiveness of the science experiments guidebook for students with learning disabilities. ankara university faculty of educational sciences journal of special education, 20(3), 501534. https://doi.org/10.21565/ozelegitimdergisi.484937 eyecioğlu, e., & yeşilyurt, m. (2021). the effect of using digital experiments in science course on student success. nişantaşı university journal of social sciences, 9(1), 29-35. https://doi.org/10.52122/nisantasisbd.927422 fırat, t., & koçak, d. (2018). investigating the opinions of class teachers’ on the concept of learning diffuculty. abant i̇zzet baysal university journal of faculty of education, 18(2), 915-931. gebbels, s., evans, s. m., & murphy, l. a. (2010). making science special for pupils with learning difficulties. british journal of special education, 37(3), 139147. https://doi.org/10.1111/j.14678578.2010.00463.x gilbert, j. k., & reiner, m. (2000). thought experiments in science education: potential and current realization. international journal of science education, 22(3), 265-283. griffin, c. c., simmons, d. c., & kameenui, e. j. (2006). investigating the effectiveness of graphic organizer instruction on the comprehension and recall of science content by students with learning disabilities. journal of reading, writing & learning disabilities, 7, 355-376. gök, g., & erbaş, d. (2011). okul öncesi eğitimi öğretmenlerinin kaynaştırma eğitimine ilişkin görüşleri ve önerileri [preschool teachers' opinions and suggestions about inclusive education]. international journal of early childhood special education, 3(1), 66-87. güven, e., & aydoğdu, m. (2009). the effectiveness of the portfolio on achievement and permanence in the “systems of body structures” unit in the sixth grade science and technology lesson. journal of turkish science education, 6(2), 115-128. helland, w. a., lundervold, a. j., heimann, m., & posserud, m. b. (2014). stable associations between behavioral problems and language impairments across childhood-the importance of pragmatic language problems. research in developmental disabilities, 35(5), 943–951. https:// doi.org/10.1016/j.ridd.2014.02.016 horton, s. v., lovitt, t. c., & bergerud, d. (1990). the effectiveness of graphic organizers for three classifications of secondary students in content area classes. journal of learning disabilities, 23(1), 1222. https:// doi.org/10.1177/002221949002300107 i̇din, ş., & aydoğdu, c. (2021). the effect of enriched educational practices on science course achievements of 7th grade students, attitudes towards science and retention of knowledge. gazi university journal of gazi educational faculty (gujgef), 41(1), 525549. i̇lik, ş. ş. (2009). an evaluation of the effectiveness of direct teaching methods on learning concepts of children with mild learning difficulties related to science and technology course. retrieved from yok tez merkezi (235060). johnson, m. m. (2013). the relationship between spelling ability and reading fluency and comprehension in elementary students. (unpublished masters thesis), northern michigan university, michigan. kaldenberg, e. r., watt, s. j., & therrien, w. j. (2014). reading instruction in science for students with learning disabilities: a meta-analysis. learning disability quarterly, 38(3), 160-173. https:// doi.org/10.1177/0731948714550204 karaer, g. & melekoğlu, m. a. (2020). review of studies on teaching science to students with specific learning disabilities. ankara university faculty of educational sciences journal of special education, 4, 789-818. https:// doi.org/10.21565/ozelegitimdergisi.532903 kesikçi, h., & amado, s. (2005). okuma güçlüğü olan çocukların fonolojik bellek, kısa süreli bellek ve wisc-r testi puanlarına ait bir inceleme [an investigation of phonological memory, short-term memory and wisc-r test scores of children with reading difficulties]. türk psikoloji dergisi, 20(55), 99-113. kirman bilgin, a., & yiğit, n. (2017). investigation of student' responses on revelation of the relation between "particulate nature of matter" topic and contexts. mersin university journal of the faculty of education, 13(1), 303-322. http://dx.doi.org/10.17860/mersinefd.306003 kolburan, ş. g., & erbay, e. ö. (2015). a case of dyslexia in example of an educational leadership. aydın i̇nsan ve toplum dergisi, 1(1), 131. korkmazlar, ü. (2003). özel öğrenme bozukluğu [specific learning disorder]. in a. kulaksızoğlu (ed.), farklı gelişen çocuklar [children who develop differently]. i̇stanbul: remzi kitabevi. journal of science learning article doi: 10.17509/jsl.v5i3.44559 530 j.sci.learn.2022.5(3).520-530 korkmazlar, ü. o. (2009). özel öğrenme bozukluğu/güçlüğü ve özel eğitim, ii [special learning disorder/difficulty and special education, ii]. eğitim psikolojisi sempozyumu. laine, t. h., nygren, e., dirin, a., & suk, h. j. (2016). science spots ar: a platform for science learning games with augmented reality. educational technology research and development, 64(3), 507-531. melekoğlu, m. a., & çakıroğlu, o. (2017). öğrenme güçlüğü olma riski taşıyan öğrencilere yönelik değerlendirme süreçleri [assessment processes for students at risk of learning disabilities]. in m. a. melekoğlu & o. çakıroğlu (eds.), özel öğrenme güçlüğü olan çocuklar [children with special learning difficulties] (pp. 101-122). ankara: vize yayıncılık. ministry of education [moe]. (2013). i̇lköğretim kurumları (ilkokullar ve ortaokullar) fen bilimleri dersi (3, 4, 5, 6, 7 ve 8. sınıflar) öğretim programı [primary education institutions (primary and secondary schools) science lesson (3, 4, 5, 6, 7 and 8th grades) curriculum]. ankara. ministry of education [moe]. (2018). fen bilimleri dersi öğretim programı: (ilkokul ve ortaokul 3, 4, 5, 6, 7 ve 8. sınıflar) [science lesson curriculum: (primary and secondary school 3rd, 4th, 5th, 6th, 7th and 8th grades)]. ankara. national research council [nrc]. (2000). inquiry and the national science education standards: a guide for teaching and learning. washington, dc: national academy press. njcld. (2014). the state of learning disabilities: facts, trends and emerging issues. new york: national center for learning inc. odom, s. l. (2000). preschool inclusion: what we know and where we go from here. topics in early childhood special education, 20(1), 2027. okumuş, s., & doymuş, k. (2018a). the effect of implementing the seven principles for good practice with cooperative learning and models on 6th graders’ academic achievement in science. journal of bayburt education faculty, 13(25), 203-238. okumuş, s., & doymuş, k. (2018b). the effect of using models with seven principles and cooperative learning on students’ conceptual understandings. abant i̇zzet baysal university journal of faculty of education, 18(3), 1603-1638. olson, j. l., & platt, j. c. (2004). teaching children and adolescents with special needs. saddle river, nj: merrill. ormsbee, c. k., & finson, k. d. (2000). modifying science activities and materials to enhance instruction for students with learning and behavioral problems. intervention in school and clinic, 36(1), 10-21. https:// doi.org/10.1177/105345120003600102 rodriguez-hernandez, b. a., & silva-maceda, g. (2021). impact of instructional sequence to teach argumentative writing to disadvantaged students using the opinion article. international journal of instruction, 14(4), 103-118. sarı, o. t., & biçer, e. (2020). öğrenme güçlüğü olan çocuklarda bireysel destek eğitim programı uygulaması [application of individual support education program for children with learning disabilities]. dicle university jounal ziya gökalp faculty of education, 38(1), 1-17. http://dx.doi.org/10.14582/duzgef.2020.149 scruggs, t. e., mastropieri, m. a., & boon, r. (1998). science education for students with disabilities: a review of recent research. studies in science education, 32(1), 21-44. http://dx.doi.org/10.1080/03057269808560126 shaywitz, s. e., fletcher, j. m., & shaywitz, b. a. (1994). issues in the definition and classification of attention deficit disorder. topics in language disorders, 14, 125. seçkin yılmaz, ş., & sarı, k. r. (2020). examination of oral language skills of students with dyslexia. kastamonu education journal, 18(3), 1102-1114. https://doi.org/10.24106/kefdergi.3896 seçkin yılmaz, ş., & şemşedinovksa, b. (2020). examination of pragmatic language skills of students with and without learning disabilities. journal of language, speech and swallowing research, 3(3), 335-355. seçkin yılmaz, ş., & yaşaroğlu, h. (2020). investigating of reading, vocabulary knowledge and verbal memory performances of students with learning disabilities. ankara university journal of faculty of educational sciences, 53(2), 751-780. https://doi.org/10.30964/auebfd.588849 snowling, m. j., hayiou‐thomas, m. e., nash, h. m., & hulme, c. (2019). dyslexia and developmental language disorder: comorbid disorders with distinct effects on reading comprehension. journal of child psychology and psychiatry, 61(6), 672-680. https://doi.org./10.1111/jcpp.13140 stake, r. (1995). the art of case study research. thousand oaks, ca: sage. sucuoğlu, b., & akalın, s. (2010). eco-behavioral assessment: an alternative method for assessing mainstreamed classrooms in turkey. ankara university journal of faculty of educational sciences, 11(1) 1942. tangkakarn, b., & gampper, c. (2020). the effects of reading-whilelistening and listening-before-reading-while-listening on listening and vocabulary. international journal of instruction, 13(3), 789-804. https://doi.org/10.29333/iji.2020.13353a taymaz sarı, o., & biçer, e. (2020). öğrenme güçlüğü olan çocuklarda bireysel destek eğitim programı uygulaması [application of individual support education program for children with learning disabilities]. dicle university jounal ziya gökalp faculty of education, 38, 1-17. http://dx.doi.org/10.14582/duzgef.2020.149 therrien, w. j., taylor, j. c., hosp, j. l., kaldenberg, e. r., & gorsh, j. (2011). science instruction for students with learning disabilities: a meta-analysis. learning disabilities research & practice, 26(4), 188– 203. http://dx.doi.org/ 10.1111/j.1540-5826.2011.00340.x. turan, z., & atila, g. (2021) augmented reality technology in science education for students with specific learning difficulties: its effect on students’ learning and views. research in science & technological education, 39(4), 506-524, http://dx.doi.org/10.1080/02635143.2021.1901682 vatansever bayraktar, h. (2015). student motivation in classroom management and factors that affect motivation. turkish studies, 10(3), 1079-1100. http://dx.doi.org/10.7827/turkishstudies ward, l. (2010). teaching science to dyslexic children. primary science, 112, 29-32. world health organization [who]. (2011). world report on disability. retrieved from http://www.who.int/disabilities/world_report/2011/report.pdf? ua=1 yin, r. (2003). case study research: design and methods (3rd ed.). thousand oaks, ca: sage. http://www.who.int/disabilities/world_report/2011/report.pdf?ua=1 http://www.who.int/disabilities/world_report/2011/report.pdf?ua=1 a © 2021 indonesian society for science educator 210 j.sci.learn.2021.4(3).210-218 received: 01 january 2021 revised: 19 march 2021 published: 01 july 2021 the effect of implementation of inquiry-based learning with socioscientific issues on students' higher-order thinking skills sitti nurul qamariyah1, sri rahayu1*, fauziatul fajaroh1, naif mastoor alsulami2 1department of chemistry education, faculty of mathematics and science, universitas negeri malang, indonesia 2graduate studies, university of jeddah, jeddah, saudi arabia *corresponding author sri.rahayu.fmipa@um.ac.id abstract this study investigates the effect of inquiry-based learning with socio-scientific issues on students' higher-order thinking skills in the first year based on chemistry topics. this study used a quasi-experiment design as a method. a sample of 96 students in three classes was selected and was divided into two groups. an experimental group used two classes (68 students) that received the instruction by inquiry-based learning with socio-scientific issues, while the control group (28 students) received instruction using verification learning. the data were collected using pretest and post-test. the results were analyzed using spss 16.0 for windows software by employing anova and effect size. this study showed that the experimental groups have a higher score in higher order thinking skills than the control group students, and there was a significant difference between the experimental groups and the control group with a large effect size. thus, this study concluded that inquiry-based learning with socio-scientific issues helps conduct the classroom's learning strategies to improve students' higher-order thinking skills. keywords inquiry, socio-scientific issues, higher order thinking skills 1. introduction the development of science and technology has both positive and negative impacts. the positive impact of this development is everything becomes fast and easy to improve life quality. meanwhile, the negative impact is individuals' exposure to various problems related to ethics, morals, and global issues that can threaten human dignity and survival (rahayu, 2016). it is necessary to build a society with scientific literacy abilities through the educational process. scientific literacy skills are essential to be mastered by students because of their relation to concern for the surrounding environment and social issues. scientific literacy skills will create a desire to solve problems or social issues around them (graber, neumann, erdmann, & schlieker, 2006). solving problems related to the environment or social issues will expose a person to several different scientific, social, or moral viewpoints (zeidler & nichols, 2009). therefore, meaning-making needs to be done to connect what they learn with environmental problems around them (sadler, 2009). the creation of meaning will occur when individuals can transfer what they know to other conditions. the transfer process is unlimited to remembering the knowledge that is owned. still, it is also related to using knowledge or things that have been learned to other experiences or conditions (brookhart, 2010). thus, the learning process is not enough to transfer information to remember and store it in the brain, but it needs education that helps students have thinking skills (rahayu, 2016; subiantoro, 2017). thinking skills refer to mental activities that allow individuals to make meanings for various information obtained to form relevant knowledge to solve problems (subiantoro, 2017). thinking skills encourage individuals to think critically and creatively in decision-making and problem-solving. such thinking skills well-known as higher-order thinking skills (hots). hots will cause individuals became an adapt to the development of science and technology. hots include critical, logical, reflective, metacognitive, and creative thinking skills (nurohman, wibowo, & widhi h, 2013; shidiq, masykuri, & susanti, 2015). hots based on bloom's taxonomy includes analysis, evaluation, and synthesis skills. higher-order thinking skills analysis, evaluation, and synthesis or creation are hots categories as transfer (brookhart, mailto:sri.rahayu.fmipa@um.ac.id journal of science learning article doi: 10.17509/jsl.v4i3.30863 211 j.sci.learn.2021.4(3).210-218 2010). not all individuals can spontaneously think at high levels, but they need encouragement, guidance, or habituation in these skills. chemistry is closely related to social science problems; for example, global warming and ecosystem damage can direct individuals to relate conceptual material to issues around them (mahanani, rahayu, & fajaroh, 2020). it makes chemistry becomes a subject that can contribute to training student's hots. however, the presence of a lot of chemistry content can cause students unable to know how to form what relationships are obtained from the learning process with how to give meaning to what is learned (tsaparlis, 2020). it also causes the assumption that chemistry is boring, difficult to understand, and irrelevant to everyday life (cardellini, 2012), so students will only tend to memorize. therefore, it is necessary to have learning chemistry, which allows students to experience learning by constructing meaning, which means students are actively involved in building concepts independently. one of the methods that accommodate is learning by inquiry. the inquiry-based learning process allows students to find, decide, and use various sources of information and ideas, which are used to improve understanding of problems or phenomena (kaltakci & oktay, 2011). inquiry-based learning will ensure that students get the knowledge and become actively involved in the process (malik, ertikanto, & suyatna, 2015). previous research has shown that inquiry-based learning can improve students' hots, including inquiry-based learning to enhance hots on reaction rate material (hendryarto & amaria, 2013). another research compares conventional learning and inquiry-based learning in improving hots (madhuri, kantamreddi, & goteti, 2012). based on these results, it can be concluded that inquiry-based learning in other chemistry topics, one of which is acid-base, may improve hots. however, the research of hendryarto & amaria (2013) and madhuri, kantamreddi, & goteti (2012) that has been carried out only focuses on the topics of learning. if students focus on the topics, it can cause students not to relate the temporal context to the events around them (subiantoro, 2017; sadler, 2009). there is a gap between abstract concepts and the reality of events around them (mahanani, rahayu, & fajaroh, 2020). therefore, learning was needed to relate to everyday life, namely by providing relevant context. the teacher should shape the context to lead students to engage in the learning process to understand, reflect, and create meaningful knowledge (sadler & zeidler, 2004; subiantoro, 2017). one context that can be used is socio-scientific issues (ssi). ssi are problems related to science and social content that are unstructured, have uncertain solutions, complex, and related to morals and ethics (eastwood, et al., 2012; sadler, 2004; sadler & zeidler, 2004; zeidler, sadler, applebaum, & callahan, 2009). ssi describes social dilemmas related to conceptual, procedural, or technological relationships in social science problems (sadler & zeidler, 2004). ssi is controversial and requires a level of moral reasoning and associated evaluation/ethics in the decision-making process for solving problems (zeidler & nichols, 2009). ssi will encourage individuals to reflect and relate relevantly between science and several scientific points of view, resulting in conflicts with held beliefs or others' beliefs (zeidler, sadler, applebaum, & callahan, 2009). the resulting conflict will lead individuals to analyze, evaluate, and synthesize sources, knowledge, or evidence to produce justification (kitchener & king, 1981; zeidler, sadler, applebaum, & callahan, 2009). ssi has a role in providing global issues and making individuals prepare themselves to deal with this (sadler & murakami, 2014). the involvement of the ssi context will create learning where individuals face complex decision-making problems that are factually, conceptually, and ethically related to ethics and morals (paraskeva-hadjichambi, hadjichambis, & korfiatis, 2015). it means that ssi learning will train students to analyze problems, evaluate the sources to be used, and create solutions. in learning by using ssi, there is an interaction between three components; educators (lecturers), students, and the context of the problems that must be solved (imaduddin & khafidin, 2018; kristiana, afandi, & wahyuni, 2019). that description is proven by similar research but on different variables and topics: inquiry-based learning on the contextbased ssi can improve critical thinking skills (mahanani, rahayu, & fajaroh, 2020). based on the research described, research studies on inquiry-based learning in the context of ssi to advancing hots need to be done. it caused to provide alternative learning information that the lecturer can use to be advancing students' hots. therefore, this study aims to examine the effectiveness of inquiry-based learning with ssi as context by making the reflective ability available to improve students' higher-order thinking skills. 2. method this research used a between-group design with quasiexperiments pre-and post-test methods (creswell j. w., 2012). quasi-experiments pre-and post-test method was used because it is impossible to randomize the existing group of subjects. the experiment design of this study is shown in table 1. table 1 the research design of quasi pre-and posttest group pretest treatment posttest eg cg o1 o1 x y o2 o2 explanation: eg: experimental group cg : control group o1 : pretest hots used 20 multiple choice questions x : treatment using process inquiry-based learning with socio-scientific issues as context y : treatment by utilizing verification learning o2 : posttest hots used 20 multiple choice questions journal of science learning article doi: 10.17509/jsl.v4i3.30863 212 j.sci.learn.2021.4(3).210-218 this research was conducted on first-year students who took introductory chemistry. this research was held in the even semester of 2019/2020. this research used two classes as the experimental groups (eg) (68 students) and one class as a control group (cg) (28 students). two classes as eg were carried out to expand the research findings. two classes as eg were given the same instruction, inquiry-based learning with ssi, while a cg was assigned verification learning. the researcher conducted the treatment in the eg, and the lecturer gave the treatment in cg. this research used chemistry topics by utilizing online media. the details of treatment are four asynchronous assignments and two synchronous assignments via zoom application. the treatment was carried out because it adjusted to the pandemic situation. in this situation, learning was carried out asynchronous and synchronous by online media. the researcher developed the instruments used in this research-based competency in subject achievement and adjusted them to hots levels. research instruments that were used included treatment (worksheets & ssi article) and measurement instruments. more concisely, the treatment scheme in this research is described in table 2. the measurement instruments form a hots test that consists of 20 multiple-choice questions. the test consists of eight questions of analyzing type (c4), seven questions of evaluating the type (c5), and five questions of creating type (c6). a higher-order thinking skill test was developed based on the indicator "action verbs" of bloom's taxonomy revised (anderson & krathwohl, 2001). hots tests were developed based on acid-base topics, which multiple-choice items. the score is given a numeric value of one to correct answer and zero for incorrect. that instrument has been tested for reliability on 98 students. a validity test was conducted to determine the hots instrument's suitability with the ability to be measured. two expert lecturers carried out the validity test. based on the validity test, 10 questions out of 30 are suggested not to be used. this is because it does not match with the hots level. furthermore, there are 2 questions from 20 need to revise. validity and reliability test were conducted using spss 16.0 for windows. reliability score obtained is equal to 0.812 > r table (0.199). the reliability score belongs to table 2 comparison of treatments in the experimental and control groups meeting to discussion topics time allocation experimental groups (inquiry-based learning with ssi) control group (verification learning) 1 introduction to acids and bases 3 x 50 minutes students introduced to surrounding subjects related to acid-base, then perform analysis of other topics related to acid-base. this introduction will provide an overview of the relationship between the concept of acid-base and daily life events so that it will attract students' interest to participate actively in learning. students were given books on acid-base materials to study independently. this learning will make students learn according to the desired learning style but focused on the lecturer's resources. acid-base theory students are given the worksheet that presents examples and non-examples of each acid-base theory, then formulated concepts based on leading questions. such learning will train students to illustrate examples that can build ideas. based on this description, students were taught to analyze ideas to create conclusions about the acid-base theory studied. students were asked to read various acid-base theories, then given the assignment to understand the understanding of the acid-base theory proposed by experts 2 damage to coral reefs 3 x 50 minutes students were given coral reef damage, then analyzed the relationship between the acid-base concept and the problems presented and made possible solutions. such learning will help students practice their ability to analyze information in reading and evaluate existing sources and data to determine the relationship between coral reef damage and acid-base material. students understood and learned acid-base calculations based on material provided by the lecturer. this learning will make students fixated on solving the problems given. acid-base calculation students in an inquiry using worksheet learned acid-base calculations to then applied to a problem. such learning will train students to apply the knowledge/concepts obtained to similar issues or concerns that have been modified. students applied the formulas that had been learned through doing assignments given by the lecturer. learning can cause students to train to use the formula obtained in the same case. journal of science learning article doi: 10.17509/jsl.v4i3.30863 213 j.sci.learn.2021.4(3).210-218 the high category based on arikunto (2014). the difficulty index of 20 question: four questions with difficult (n=0.259 – 0.283) category, and 16 questions with moderate (n=0.322 – 0.566) category. based on these results, the instrument of a hots test is declared decent to use. examples of the hots test are presented in table 3. data analysis that was carried out in this study includes a preconditioning test and a different test. a precondition test was used to determine the average hots before the treatment is carried out. the difference test was carried out to determine how much affect the treatment for advancing students' hots. the details of data analysis in this research are described as follows. precondition test. it includes normality and homogeneity tests followed by oneway anova. the normality test used the shapiro-wilk method, while the homogeneity test used levene statistic. each test was utilizing spss 16.0 for windows as assistance. the difference test used anova followed by an effect size test. both tests were utilizing spss 16.0 for windows as assistance by using the post-test score. 3. result and discussion the pretest data showed that the experimental and control groups' hots level was normal (p=0.100 for eg; p=0.147 for cg) and homogeny (p=0.976). the one-way anova test results showed no significant difference in students' pre-higher order thinking skills achievement scores for the cg and students in the eg (p=0.405). the resulting anova is described in table 4. after treatment, post-test data showed that there was advancing on the average score of the hots test. the resulting score of hots tests before and after treatment is described in table 5 and figure 1. furthermore, the normality and homogeneity tests were carried out based on post-test to determine data distribution. the normality and homogeneity test showed table 2 comparison of treatments in the experimental and control groups (continued) meeting to discussion topics time allocation experimental groups (inquiry-based learning with ssi) control group (verification learning) 3 acid rain 3 x 50 minutes students associated the concept of acid-base reactions with acid rain events by practicing, analyzing, reading, and evaluating various sources to create solutions. the acid rain event will make students relate the material of acidbase reaction equations and acid-base theory. students were given videos related to the acid-base titration process and the application of calculations to be studied independently. students can know the process of change that the sense of sight can directly observe. acid-base titration students in an inquiry analyzed the processes that occur in the titration implementation with the worksheet's help. then they were given a similar problem to solve based on the inquiry steps that had been carried out. in this titration learning, students were introduced to titration and trained to understand the process during titration. it will make students analyze the reactions that occur to evaluate what substances are present during the response and formulate acid/base calculations, formulate buffer solutions, and salt hydrolysis. it will make students apply formulas, analyze differences, and evaluate problems related to events during the reaction process. students were given assignments in the form of questions related to acid-base titration as a strengthening of understanding. students could practice knowledge by applying problem-solving steps, but thinking for other problems was not trained. 4 verification 3 x 50 minutes students conducted face-to-face discussions and lectures via zoom to discuss concepts that are not yet understood and addressed the relationship between the problem and the acid-base idea. students were given questions to work on as an exercise before the measurement process was carried out figure 1 the percentage achieving each level of hots c4 c5 c6 c4 c5 c6 pretest posttest cg 48% 37% 33% 38% 43% 36% eg 42% 36% 32% 60% 63% 60% 0% 10% 20% 30% 40% 50% 60% 70% p e rc e n ta g e journal of science learning article doi: 10.17509/jsl.v4i3.30863 214 j.sci.learn.2021.4(3).210-218 that data of eg (p=0.200) and cg (p=0.535) was normal and homogeny (p=0.60 9). caused by the results, anova was conducted to compare the students' hots achievement in the cg and eg by utilizing spss 16.0 for windows as assistance. table 6 showed that score of significance < 0.05. it means there was a significant difference in students' hots level for the cg and eg. details of the differences in students' hots level eg and cg are shown in figure 1. figure 1 showed that in each level of higher-order thinking skills in the students who treatment by inquirybased learning with ssi as context were higher than the cg table 3 examples questions of hots questions level questions indicator questions c6 creating presented with statements and examples of lewis acidbase reaction, students can determine the right hypothesis about the fundamental development of lewis acid-base theory. the limitation of the bronsted-lowry acid-base concept is explaining reactions involving compounds without h+, such as the reaction between copper (ii) and ammonia ions that produce complex tetraamincopper (ii) ions. this limitation then became the basis for the development of lewis's acid-base theory. based on this statement, determine the correct hypothesis based on lewis acid-base theory development! a. lewis's acid-base theory is based on proton donors and acceptors, where ammonia acts as a proton donor, while the copper (ii) ion acts as a proton acceptor. b. lewis's acid-base theory is based on electron-pair donors and acceptors to form covalent bonds between the bonding species, in which ammonia acts as a base and the copper ion acts as an acid. c. lewis's acid-base theory is based on the formation of coordinate covalent bonds. the species donating an electron pair acts as a base, and the electronpair acceptor acts as an acid. d. lewis acid-base theory is based on the donor and acceptor of the central atom's lone pair. the ammonia species as an electron-pair donor acts as a base, and the copper ion as an acceptor is acidic. e. lewis acid-base theory is based on the formation of stable compounds, in which species that have unstable lone pairs will bind to species without more stable lone pairs. c5 evaluating given a submicroscopic representation of the species present in a solution, students can predict pictures that are unlikely to occur in a weak acid solution. (make guesses based on particular criteria) submicroscopic figures can represent species present in acidic and alkaline solutions. in the bronsted lowry acid-base context, the most unlikely submicroscopic representation of a diprotic acid solution, h2a, is in an aqueous solution? (water molecule not drawn) table 5 pre-and posttest data of higher-order thinking skills class average score pretest posttest eg 40 68 cg 38 40 table 4 the result of one-way anova pre-higher order thinking skills category sum of squares df mean square f sig. between groups 127.103 1 127.103 .701 .405 within groups 17047.637 94 181.358 total 17174.740 95 journal of science learning article doi: 10.17509/jsl.v4i3.30863 215 j.sci.learn.2021.4(3).210-218 who treatment by verification learning. it means that inquiry-based learning with ssi as the context affected students' hots. that meaningfulness is also followed by the effect size score, which shows a score of 1.909. it led to a high contribution to the learning method based on becker (2000) to advancing hots among classes that treat inquiry-based learning with ssi as context and verification learning. the result test effect size showed in table 7. higher-order thinking skills are related to meaningful learning (aksela, 2005), increasing reading comprehension (indriyana & kuswandono, 2019), and scientific literacy (rahayu, 2016; subiantoro, 2017). the results showed that treatment acid-base inquiry with ssi as the context has hots that better verifies learning with a high contribution. this result is possible because students in the eg are accustomed to being actively involved in understanding concepts by analyzing problems and evaluating various sources and evidence to synthesize them into understanding. teaching strategy that involves students to active would play an essential role in long-term memory. it will enhance meaningful learning that leads students to apply it in other conditions (bahrick & hall, 1991; slavin, 2019). although the data pretest showed that both groups have equal level hots, students already understand the concept. however, students in the cg have not solved problems that require higher thinking skills than just memorizing, understanding, and applying. ssi in the eg will train students to analyze the issues and evaluate evidence and sources to solve them. the implementation table 3 examples questions of hots (continued) questions level questions indicator questions c4 analyzing students can analyze errors in the practical steps to make a particular ph solution, based on the facts and reasonable measures. (analyze the elements by separating the problem into its components) nahda wants to make a sodium hydroxide solution with a ph of 10.3 using solid naoh. the steps used by nahda are as follows! 1. weighing solid naoh using glass as much as 8.00 g. 2. dissolve solid naoh in a beaker with distilled water. the volume of distilled water is ± 10 ml. 3. after all solid naoh dissolves, please put it in a 1000 ml measuring flask. then add distilled water to mark the limit. shaken until homogeneous 4. 10 ml of the solution were taken using a volume pipette. then put in another 1000 ml measuring flask. 5. diluted by adding distilled water to mark boundaries. shaken until homogeneous. 6. the solution formed is then taken 10 ml to measure the ph using a ph meter. based on the experiments that nahda has done, it turns out that the ph of the solution formed is ± 11.3. determine which step of the nahda experiment was incorrect! a. step 2. in this step, nahda should immediately put the solid naoh into the measuring flask so that no naoh is left in the beaker geals. b. step 1. in this step, nahda should not have weighed naoh immediately but did the calculations first so that the amount of solid naoh was considered right. c. steps 3 and 4. in these steps, nahda should use a 100 ml measuring flask to dilute the solution formed. d. step 5. in this step, shaking should not be needed because it allows the naoh solution to spill so that the water content is reduced and the solution becomes more concentrated than before. e. step 4. in this step, you should use a measuring cup to take the solution so that the accuracy level is higher than before. table 6 the result of one-way anova post-higher order thinking skills category sum of squares df mean square f sig. between groups 15764.741 1 15764.741 60.636 .000 within groups 24439.092 94 259.990 total 40203.833 95 table 7 the results of effect size aspect effect size mean difference mean std. error difference std. deviation partial eta squared cg eg cg eg corrected model intercept group hots 1,90976 28.193 39.57 67.76 2.984 10.905 17.800 0,392 0,903 0,392 journal of science learning article doi: 10.17509/jsl.v4i3.30863 216 j.sci.learn.2021.4(3).210-218 of ssi can encourage students to actively engage in dialogue, discussion, and debate to provide a challenge to evaluate knowledge and provide an opportunity to rebuild mastery of concepts related to the concepts they have (cahyarini, rahayu, & yahmin, 2016). the issues in treatment related to acid rain, damage to coral reefs, and the use of steam electric power generating (sepg) are the contexts of ssi conflicting. for example, using sepg as a power plant will produce substances that can cause acid rain or damage coral reefs (middlecamp, et al., 2012). on the other side, sepg in indonesia is the primary electricity source (yatim, 2007). such problems are complex and related to morals and ethics that can lead to conflict. because of conflict in the context of ssi, it can lead students to analyze, evaluate, and create solutions (kitchener & king, 1981; zeidler, sadler, applebaum, & callahan, 2009). the existence of problems with the ssi context will provide many opportunities for students to exchange ideas with one another. because ssi problems involve science and social problems, arguments need to be built from various perspectives (sadler, 2004). such a process will make students think carefully by paying attention to various sources before making a final decision (pratiwi, rahayu, & fauziatul, 2016). in the process of consideration, students will train three essential aspects, namely (1) students need to analyze so that they can understand and describe ssi problems; (2) students formulate several possible solutions to problems; and (3) students need to evaluate decisions that have been made before the decisions are submitted (ratcliffe & grace, 2003). based on that study, it can be concluded that the context of ssi in the treatment affects students' higherorder thinking skills (analyzing, evaluating, and creating). students in the treatment class have also been accustomed to getting a concept with the inquiry process presented in each topic. the inquiry process will actively involve students through activities, namely understanding problems, identifying problems, then making conclusions to make concepts or solutions (aksela, 2005; hendryarto & amaria, 2013; kaltakci & oktay, 2011). these activities will encourage students to practice hots. the lecturer in inquiry learning plays an essential role as a motivator, facilitator, and director in learning. the lecturer as a motivator has a function to encourage students to provide opinions on the ssi. as a facilitator, the lecturer functions to provide context ssi that can make students think actively and have a function to show solutions if there are obstacles in the learning process. the lecturer as a director has a function to lead students in the thinking process to achieve the expected goals. previous research has shown that inquiry-based learning needs to be done to improve higher-order thinking skills (malik, ertikanto, & suyatna, 2015). inquiry showed success in improving higher-order thinking skills (aksela, 2005; kaltakci & oktay, 2011; madhuri, kantamreddi, & goteti, 2012). inquiry-based learning with ssi can enhance critical thinking skills (mahanani, rahayu, & fajaroh, 2020). based on the study and the results of these studies, it can be concluded that inquiry-based learning applied to treatment also affects students' higher-order thinking skills. inquiry learning provides benefits for students, including (a) students learn to be responsible for gaining knowledge and assignment that given; (b) students are free to use various media, sources, and technic of constructing knowledge; and (c) students learn to develop his ability and solve the problem on his way. in addition to the context and methods used in the treatment class, the learning process that involves discussion for verifying students' understanding is also a factor in training students' higher-order thinking skills to understand acid-base. confirming students' understanding used in the treatment class was not intended to teach or provide training. the comprehension verification process is used to determine whether students understand what they have learned (slavin, 2019). the treatment class's discussion process was carried out by being briefed with questions to get the correct concept. fisher & frey (2007) state that educators can use questions as an examination of understanding. questions asked to encourage students to think about ideas will help them understand concepts (sadker & sadker, 2013; slavin, 2019). meanwhile, in the classroom with verification learning, students only build their understanding, but there is less guidance in obtaining the correct concept. an independent learning process in the verification class also allows students to practice hots. the separate learning process will cause students to adjust to learning styles, but it will also have different impacts (slavin, 2019). however, experiments have also shown that learning with verification is no better than inquiry-based learning, either with a guided inquiry or open inquiry (aksela, 2005; hendryarto & amaria, 2013; mahanani, rahayu, & fajaroh, 2020; malik, ertikanto, & suyatna, 2015); as well as other contextual learning, such as problem-based learning (magsino, 2014; sismawarni, usman, hamid, & kusumaningtyas, 2020). besides, in verification learning, students are not trained intensively to lead to wrong understanding or students only to memorize and apply what the educators give. students in the verification class are also not prepared to relate various problems around them, causing them to focus on concepts without applying them to situations around them that tend to be the same. 4. conclusion this study concluded that process inquiry-based learning with ssi as context could be advancing students' higher-order thinking skills. it showed an enhancement of eg's average post-test score and the result anova that followed the effect size. higher-order thinking skills are successful because in inquiry-based learning with ssi environment, real-life problem scenarios are used. it also journal of science learning article doi: 10.17509/jsl.v4i3.30863 217 j.sci.learn.2021.4(3).210-218 encourages students to engage themselves in the learning process. inquiry-based learning with ssi as context gives a high contribution to advance students' higher-order thinking skills. references aksela, m. (2005). supporting meaningful chemistry learning and higherorder thinking through computer-assisted inquiry : a design research approach. helsinki: university of halsinky. anderson, l. w., & krathwohl, d. r. (2001). a taxonomy for learning, teaching, and assessing. boston: ma: allyn and bacon. arikunto, s. (2014). prosedur penelitian suatu pendekatan praktik (research procedure a practical approach). jakarta: rineka cipta. bahrick, h. p., & hall, l. k. (1991). lifetime maintenance of high school mathematics content. journal of experimental psychology: general, 20-33. becker, l. a. (2000). effect size ( es ). universitat de valència’. https://www.uv.es/~friasnav/effectsizebecker.pdf brookhart, s. m. (2010). how to asses higher order thinking skills in your classroom. usa: acsd. cahyarini, a., rahayu, s., & yahmin. (2016). the effect of 5e learning cycle instructional model using socioscientific issues (ssi) learning context on students' critical thinking. jurnal pendidikan ipa indonesia, 222-229. cardellini, l. (2012). chemistry: why the subject is difficult? / química: ¿por qué la disciplina es difícil? educación química, 305-310. creswell, j. w. (2012). educational research: planning, conducting, and evaluating quantitative and qualitative research. boston: pearson education, inc. eastwood, j. l., sadler, t. d., zeidler, d. l., lewis, a., amiri, l., & applebaum, s. (2012). international journal of science contextualizing nature of science instruction in socioscientific issues. international journal of science education vol. 34, 2289–2315. fisher, d., & frey, n. (2007). checking for understanding: formative assessment technique for your classroom. alexandria: acsd. graber, w., neumann, a., erdmann, t., & schlieker, v. (2006). paris: a partnership between industry and school. in p. nentwig, & d. waddington, making it relevant: context based learning of science (pp. 249-272). new york: waxman munster. hendryarto, j., & amaria. (2013). penerapan model pembelajaran inkuiri untuk melatih kemampuan berpikir tingkat tinggi siswa pada materi pokok laju reaksi [application of inquiry learning model to train students' high-order thinking skills in reaction rate topics]. journal of chemical education, 151-158. imaduddin, m., & khafidin, z. (2018). ayo belajar ipa dari ulama : pembelajaran berbasis socio-scientific issues di abad ke-21 (let's learn science from ulama: learning based on socio-scientific issues in the 21st century). thabiea: journal of nature science teaching, 102-120. indriyana, b. s., & kuswandono, p. (2019). developing students' higher order thinking skills (hots) in reading: english teachers' strategies in selected junior high schools. jet (journal of english teaching), 2014-216. kaltakci, d., & oktay, o. (2011). a guided-inquiry laboratory experiment to reveal students' comprehension of friction concept: a qualitative study. balkan physics letters. ankara: bogazici university press., 180-190. kitchener, k. s., & king, p. m. (1981). reflective judgment: concepts of justification and their relationship to age and education. journal of applied developmental psychology, 89-116. kristiana, t., afandi, & wahyuni, e. s. (2019). potensi socioscientific issues dalam memberdayakan kemampuan berpikir kritis siswa (potential of socioscientific issues in empowering students' critical thinking skills). prosiding seminar nasional fkip 2019 "optimalisasi kualitas pembelajaran abad 21 di era revolusi industri 4.0 dalam menghasilkan pendidikan yang profesional" (pp. 260-266). pontianak: universitas tanjungpura. madhuri, g. v., kantamreddi, v. s., & goteti, l. n. (2012). promoting higher order thinking skills using inquiry-based learning. european journal of engineering education, 117-123. magsino, r. m. (2014). enhancing higher order thinking skills in a marine biology class through problem-based learning. asia pasific journal multidisciplinary research, 1-6. mahanani, i., rahayu, s., & fajaroh, f. (2020). the effect of inquiry based learning with socioscientific issues context on critical thinking skills and scientific explanation. jurnal kependidikan: penelitian inovasi pembelajaran, 53-68. malik, a., ertikanto, c., & suyatna, a. (2015). deskripsi kebutuhan hots assessment pada pembelajaran fisika dengan metode inkuiri terbimbing (description of hots assessment needs in physics learning with guided inquiry method). prosiding seminar nasional fisika snf 2015, (pp. 1-4). middlecamp, c. h., keller, s. w., anderson, k. l., bentley, a. k., cann, m. c., & ellis, j. p. (2012). chemistry in context: applying chemistry to society. new york: mcgraw-hill. nurohman, s., wibowo, w. s., & widhi h, p. (2013). model integrated science berbasis socio scientific issues untuk mengembangkan thinking skills dalam mewujudkan 21st century skills (integrated science model based on socio scientific issues to develop thinking skills in realizing 21st century skills). jurnal pendidikan matematika dan sains tahun i, no. 2, 158-164. paraskeva-hadjichambi, d., hadjichambis, a. c., & korfiatis, k. (2015). how students' values are intertwined with decisions in a socioscientific issue. international journal of environmental & science education, 493-513. pratiwi, y. n., rahayu, s., & fauziatul, f. (2016). socioscientific issues (ssi) in reaction rates topic and its effect on the critical thinking skills of high school students. jurnal pendidikan ipa indonesia, 164-170. rahayu, s. (2016). mengembangkan keterampilan tingkat tinggi siswa melalui pembelajaran kimia berkonteks socioscientific issues (ssi) dan nature of science (nos) [developing students' highorder thinking skills through chemistry learning in the context of ssi and nos]. seminar nasional kimia dan pembelajarannya, (pp. 11-19). ratcliffe, m., & grace, m. (2003). science education for citizenship: teaching socio-scientific issues. united kingdom: mcgraw-hill education (uk). sadker, m., & sadker, d. (2013). questioning skills. in j. m. cooper, classroom teaching skills (pp. 121-168). usa: wadsworth. sadler, t. d. (2004). informal reasoning regarding socio-scientific issues : a critical review of research. journal of research in science teaching, 513-536. sadler, t. d. (2009). situated learning in science education: socioscientific issues as contexts for practice. studies in science education, 1-42. sadler, t. d., & murakami, c. d. (2014). socio ‐ scientific issues based teaching and learning : hydrofracturing as an illustrative context of a framework for implementation and research. revista brasileira de pesquisa em educação em ciências, 331-342. sadler, t. d., & zeidler, d. l. (2004). the morality of socio-scientific issues: construal and resolution of genetic engineer dilemmas. science education, 88(1), 4-27. shidiq, a. s., masykuri, m., & susanti, e. (2015). analisis higher order thinking skills (hots) menggunakan instrumen two-tier multiple choice pada materi kelarutan dan hasil kali kelarutan untuk siswa kelas xi sman 1 surakarta (analysis of higher order thinking skills (hots) using two-tier multiple choice instruments on solubility topics for 2nd grade senior high school 1 surakarta students). seminar nasional pendidikan sains “pengembangan model dan perangkat pembelajaran", (pp. 159-166). surakarta. journal of science learning article doi: 10.17509/jsl.v4i3.30863 218 j.sci.learn.2021.4(3).210-218 sismawarni, w. u., usman, hamid, n., & kusumaningtyas, p. (2020). pengaruh penggunaan isu sosiosaintifik dalam model pembelajaran berbasis masalah terhadap keterampilan berpikir tingkat tinggi siswa (the effect of using socioscientific issues in problem based learning models on students' high order thinking skills). jambura journal of educational chemistry, 10-17. slavin, r. e. (2019). psikologi pendidikan teori dan praktik (educational psychology theory and practice). jakarta: pt indeks. subiantoro, a. w. (2017). pembelajaran biologi berbasis socio-scientific issues ( ssi ) untuk mengasah keterampilan berpikir tingkat tinggi (biology learning based on socio-scientific issues (ssi) to train higher-order thinking skills). seminar nasional pendidikan biologi. tsaparlis, g. (2020). higher and lower-order thinking skills: the case of chemistry revisited. journal of baltic science education, 467-483. yatim, e. m. (2007). dampak dan pengendalian hujan asam (impact and control of acid rain). jurnal kesehatan masyarakat, 146-151. zeidler, d. l., & nichols, b. (2009). socio-scientific issues : theory and practice. journal of elementary science education, 49-58. zeidler, d. l., sadler, t. d., applebaum, s., & callahan, b. e. (2009). advancing reflective judgment through socio-scientific issues. journal of research in science teaching, 74-101 a © 2022 indonesian society for science educator 342 j.sci.learn.2022.5(2).342-352 received: 5 april 2022 revised: 13 may 2022 published: 27 july 2022 determining the cognitive structures of secondary school students on covid-19 çiğdem çıngıl barış1* 1department of mathematics and science education, hasan ali yücel education faculty, i̇stanbul university-cerrahpaşa, i̇stanbul, turkey *corresponding author: ccingil@iuc.edu.tr abstract this research aims to determine the cognitive structures of secondary school students about covid-19. in this research, a case study, one of the qualitative research methods, was used, and demographic information and a word association test were the data sources. the study group of the research consists of 226 students who continue their education in the 6th and 7th grades of a public secondary school. the data were collected via google form, and the obtained data were analyzed by content analysis. the cognitive structures of secondary school students on the concept of covid-19 are grouped under 11 categories. these categories are covid-19 prevention methods, covid-19 reflections in press, covid-19 results, covid-19 perceptions, causes of covid-19 transmission, covid-19 treatment, covid-19 symptoms, countries where covid-19 are common, characteristics of covid-19, covid19 carriers and covid-19 diagnostic ways. when these categories were examined, it was seen that the students' cognitive structures about the concept of covid-19 focused on covid-19 prevention methods, covid-19 reflections in press, covid-19 results, and covid-19 perceptions. in addition, it has been determined that some students have misconceptions about the covid-19 virus. keywords covid-19, cognitive structure, secondary school students, word association test 1. introduction a virus is a small infectious agent that remains within the host cell. a virus is a tiny particle with similar properties to living things but is not alive. therefore, it cannot reproduce by itself (hendaus, jomha, & alhammadi, 2015). viruses are a bridge between the living (inside the host) and the dead (that show no signs of life when taken out). viruses generally range between 20 and 400 nm and are round in shape. viruses are biological entities of protein and nucleic acid (dna or rna). viruses are dependent on other living cells for reproduction. viruses cause many diseases in human beings, such as aids, hepatitis, chicken pox, herpes, measles, poliomyelitis, etc. (bosch, biesbroek, trzcinski, sanders, & bogaert, 2013; hendaus et al., 2015). in 1892, tobacco mosaic virus, 1898; foot and mouth disease, 1901; yellow fever virus was discovered (berkhout, 2015). coronavirus is a new virus that has not been recognized in humans beforehand. coronavirus is an rna virus that can cause various symptoms, including pneumonia, fever, difficulty breathing, and cough. these viruses are common in animals worldwide, but very few cases are known to affect humans. the world health organization (who) announced that the official name of the new coronavirus in 2019 is covid-19 (who, 2020). the covid-19 epidemic, which emerged in wuhan city, the capital of the hubei region of china, on december 1, 2019, was declared a pandemic by the world health organization (who) worldwide on march 11, 2020 (who, 2020). experts know the coronavirus family as a virus that previously caused loss of life due to sars-cov and mers-cov epidemics. the covid-19 virus is more dangerous than other virus types with its contagious feature. the most crucial point is that the patient with covid-19 disease is contagious and spreads the disease to the environment (karcıoğlu, 2020). the fact that the virus was seen in a very short time, first in china, in iran, then in european countries, and then journal of science learning article doi: 10.17509/jsl.v5i2.45230 343 j.sci.learn.2022.5(2).342-352 in the americas, is proof of how high the rate of spread of the new type of coronavirus is (who, 2020). on the other hand, considering its lethal effect, it was determined that in march 2020, covid-19 caused more damage and death than the sum of sars and mers diseases (karcıoğlu, 2020). despite the significant measures taken in 2020, covid-19 continued to spread rapidly worldwide by undergoing mutations in 2021 and affected many countries' people, economies, and administrations. these measures have been implemented within the scope of these measures, such as taking a break from school, travel restrictions, stopping sports competitions, and switching to flexible working hours. in addition, people are encouraged not to go out as much as possible, to stay at home, and to implement individual hygiene measures (budak & korkmaz, 2020). covid-19 affects individuals in many different ways. the highly contagious disease, especially inhalation, has caused thousands of people to become ill and continues to do so. while it used to affect older people, especially those with chronic diseases, it now affects middle-aged and young people, even children. with the number of infected cases every day, the number of deaths is also increasing. the education sector is one of the most affected sectors after the health sector due to covid-19 (gonzalez et al., 2020; telli yamamoto & altun, 2020). schools have been closed in 188 countries due to the covid-19 pandemic (unesco, 2020a). this affected approximately 92% of the worldwide student population (1.576.021.818 students). the pandemic has caused distance education to be used in primary and secondary education institutions worldwide, apart from universities (unesco, 2020b). thus, students had to stay at home for a long time. students had difficulties adapting to the new home order, which was changed by distance education, starting with a break from face-to-face education. therefore, it can be said that students, who are the main subjects of the education sector, are more affected by this situation than teachers and parents (ercan, rodopman-arman, i̇nal emiroğlu, öztop, & yalçın, 2020). these effects affect students because they are separated from their teachers and friends, and their school life is suddenly interrupted. the factors that enable secondary school students, who are accepted as adolescents, to gain spiritual, mental, motor, and social skills are peer groups and school. changes in students' daily lives and disruption of their routine habits due to covid19 may disrupt children's mental, social, academic, motor, and mental development (ercan et al., 2020). covid 19, which has turned into a widespread pandemic worldwide, has affected every aspect of our lives individually and socially. undoubtedly, this process has also affected science education, and new ways, methods, and strategies will come to the fore in terms of science education in the future. when the studies on science education in the covid-19 process are examined, it has been observed that studies on the pandemic are mainly carried out in the health field. from this point of view, it can be said that there are few studies in the field of education. in a study conducted by pınar & dönel akgül (2020), students' opinions regarding teaching science courses by distance education during the epidemic were examined. as a result of the research, the students stated that they found distance education helpful during the epidemic process but that it was a significant shortcoming that they could not experiment with distance education. dillon & avraamidou (2020), who focused on how much science education programs prepare the society for the covid-19 pandemic period in their research, argued that although the importance of science literacy in science education has been emphasized for years, the public is not functionally scientifically literate. in a study (i̇mer çetin, timur, & pehlivan, 2021) conducted to determine the metaphorical perceptions of science teachers about the virus, it was concluded that science teachers generally perceived the concept of “virus” as a negative concept during the covid-19 pandemic process. in their study, bakioğlu & çevik (2020) determined science teachers' views on distance education during the covid-19 process. teachers also stated that they face problems such as internet connection, communication with students, a low participation rate of students in the courses during the distance education process, and concerns such as not being able to complete the curriculum and laboratory activities with distance education. in their study, karatepe, küçükgençay, & peker (2020) aimed to determine the perceptions of pre-service science teachers about synchronous education and determined that pre-service science teachers were reluctant to provide online distance education in the future and did not consider themselves sufficient in this regard. cognition is the knowledge or awareness of living things about any concept or situation (yılmaz & arun, 2020). cognitive activities are perceiving stimuli from the environment through the senses, comparing with previous information, obtaining new information and storing it in memory, bringing it back when necessary, and evaluating it (özen, 2011). the more meaningful the acquired information is in memory or the better it is associated with old information, the easier it is to remember (akınoğlu & bakır, 2003). in this context, the relationships established with old information must be correct for meaningful learning to be realized while creating new information. in learning the concepts, determining the relationship between the new knowledge concepts and whether the relationship with the previous knowledge is meaningful provides a significant convenience (ayas, 2014). for this reason, it is crucial to determine how students establish relationships between concepts and their cognitive structures. the cognitive structure is fundamental for individuals to make sense of information, correctly journal of science learning article doi: 10.17509/jsl.v5i2.45230 344 j.sci.learn.2022.5(2).342-352 associate it with each other, and even remember this information when necessary (ceylan, 2015). in this context, some techniques have gained importance to reveal the students' cognitive structure, the connections between the concepts in this structure, and also to determine whether the relations between the concepts are sufficient (bahar, nartgün, durmuş, & bıçak, 2006). for this purpose, some strategies that provide and measure conceptual understanding and conceptual change have been developed. the main purpose of these techniques is to reveal the harmony or differences between the students' mental structure and the events in the outside world by creating meaningful learning in students. these techniques include concept analysis tables, concept networks, concept maps, structured grids, analogy, diagnostic branched tree, word association test, etc. (bahar, 2003; bahar et al., 2006; ercan, taşdere, & ercan, 2010). word association tests are an alternative measurement and evaluation tool that is widely used to determine the relationships between the information existing in the minds of individuals and whether these relationships are established in a meaningful way (hovardas & korfiatis, 2006; bahar, johnstone, & sutcliffe, 1999). when the literature is examined, it is seen that word association tests are used for different purposes, such as revealing cognitive structure, misconceptions, and conceptual changes (bahar et al., 1999; çelikkaya & kürümlüoğlu, 2019; shavelson, 1974; varoğlu, şen, & yılmaz, 2020). when the covid-19 literature is examined, it is seen that the covid-19 epidemic is the subject of study for many researchers. many studies have been found, mainly in the health field, aiming to address different issues related to the covid-19 pandemic (bellini et al., 2021; nalbandian et al., 2021). the covid-19 disease has taken the whole world under its influence and has had enormous effects on individuals. secondary school students, who switched to distance education and had to stay at home with the interruption of face-to-face education, may have created different cognitive structures in their minds regarding covid-19. the cognitive structures in students' minds regarding covid-19 have the potential to give important ideas in shaping the education to be carried out after the epidemic. questioning students' thoughts regarding this process will enable them to look at the educational activities given in the continuation or after the epidemic process from a different perspective. therefore, it is thought that teachers can help students more easily through the concepts obtained from the research. in this context, this research aims to determine the cognitive structures of secondary school students regarding the concept of covid-19. for this purpose, the following sub-problems are included: 1. have the students had covid-19 disease? 2. have any students had covid-19 disease in their close circle? 3. did the students have a relative who died from covid-19 disease? 4. are there any healthcare personnel in the family of the students? 5. what are the students' cognitive structures regarding the concept of covid-19? 6. what are the students’ misconceptions about covid-19? 2. method 2.1 research design the case study design, one of the qualitative research designs, was used in this research. a case study is a research method that studies a phenomenon within its real-life framework and examines situations in a versatile, systematic, and in-depth manner (cohen & manion, 1997; patton, 1990; yıldırım & şimşek, 2006). a case study includes the stages of limiting the situation, identifying the research case, searching the data set, creating the findings, making comments, and writing the results (denzin & lincoln, 1996; bassey, 1999). in this study, the cognitive structures of secondary school students regarding covid19 and, in this context, their alternative concepts about covid-19 were determined through a case study. 2.2 research group this research was conducted in the spring term of the 2020-2021 academic year. 226 students in the 6th and 7th grades of a secondary school in the fatih district of i̇stanbul participated in türkiye. the purposeful working group was chosen in this research due to the need to collect detailed and high-quality data on the subject and the fact that covid-19 is on the world agenda. some criteria have been taken into account to minimize the problems in selecting purposeful samples (coyne, 1997; given, 2008; knight et al., 2013; patton, 1990). in this context, criteria such as having taken a science course in previous years, willingness to participate in the study voluntarily, and easy accessibility of the researcher were considered in the students' selection. although there is no virus subject in the science curriculum, the students learned the classification of living things and microscopic living things in the 5th grade. in the study, it is important for the study to choose a purposeful sample group, especially to obtain detailed information about the students' cognitive structures on the concept of covid-19. 113 (50%) of the students were female, and 113 (50%) were male. in addition, 94 (41.59%) 6th grade students and 132 (58.4%) 7th grade students. before the study, the students were informed about the measurement tools, and they were assisted while filling the measurement tools. 2.3 data collection tools the data collection tool in the research consists of two parts. in the first part, in addition to the demographic characteristics of the students, such as gender and class, "have you ever had a covid-19 disease?", "has anyone journal of science learning article doi: 10.17509/jsl.v5i2.45230 345 j.sci.learn.2022.5(2).342-352 in your close circle suffered from covid-19?", "did you have a relative who died due to covid-19 disease?" and "are there any healthcare personnel in your family?" were asked. in the second part, a word association test (wat) was employed to determine the students' cognitive structures regarding the concept of covid-19. the word association test is one of the alternative measurementevaluation techniques that allow us to determine the cognitive structure of the student and the connections between the concepts in this structure, and whether the relationships between concepts in long-term memory are sufficient or meaningful (bahar et al., 1999). while using wat, basic concepts related to the subject are selected, and students are asked to list the words associated with the concept. thus, it is possible to have an idea about the subject by looking at the number of words the student associates with the basic concept (başol, 2016). in this research, students were asked about the concept of covid-19 to complete the independent wat. the concept of covid-19 was presented as a stimulant word, and spaces were left for students to write the words they thought related to the concepts in the wat. an online form was used to get the opinions of the students. two hundred twenty-six students filled out the online form on 31.05.2021. all data were collected via google form. stimulant concept: covid-19 covid-19: ……………………… covid-19: ……………………… covid-19: ……………………… covid-19: ……………………… covid-19: ……………………… related sentence: …………………………… students should first write the five words that come to mind about the concept and then make a sentence about the words they wrote. the constructed sentences provide information about their cognitive structures and an evaluation process regarding students' misconceptions. therefore, it is crucial for the consistency of the study. 2.4 data analysis in the first part, frequency and percentage were used to determine the demographic characteristics of the students and their views on the covid-19 outbreak. in the second part, content analysis was used to analyze the data obtained by wat. the main purpose of content analysis is to reach concepts and relationships that can explain the collected data (yıldırım & şimşek, 2006). the process of analyzing and interpreting the content was carried out in four stages: (1) coding and extraction stage: the data obtained with the google form was transferred to excel and arranged in frequency tables. as a result, 1130 frequencies were obtained. (2) category development stage: the concepts examined were arranged by considering the categories expressed by ekici & kurt (2014). (3) validity and reliability phase, (4) reporting phase (yıldırım, & şimşek, 2006). on the other hand, the mindmeister program was used to model the students' cognitive structures about covid-19. 2.5 validity and reliability in order to ensure the validity of the research, (1) the data coding and how to reach the conceptual category within the data analysis process are explained in detail (hruschka et al., 2004), (2) the results obtained in the research for each of the categories, examples from student opinions were selected and presented in the findings section (roberts & priest, 2006) and (3) an attempt was made to ensure consistency between related studies in the comments. two independent experts analyzed data to ensure the reliability of the study. expert opinions were calculated using the formula [agreement / (agreement + disagreement) x 100] (miles & huberman, 1994). the average reliability among coders was found to be 94%. the categories were similarly created by dönmez & gürbüz (2020) in their studies to determine the cognitive structures of university students regarding the covid-19 virus, except for the "covid-19 diagnostic ways" category. the fact that almost all of these categories are created similarly reveals the validity of the research. 3. result 3.1 findings from “have the students had covid-19 disease?” sub-problem in the first sub-problem of the research, the students were asked whether they had covid-19, and the results are presented in figure 1. figure 1 shows the answers given by the students about whether they have had covid-19 or not. 18.6% (f = 42) of the students stated that they had covid-19, 81.43% (f = 184) stated that they did not. 3.2 findings from “have any of the students had covid-19 disease in their close circle?” sub-problem in the second sub-problem of the research, it was asked whether the students had covid-19 in their close circle, and the results are presented in figure 2. figure 2 shows the answers given by the students about whether they have had covid-19 in their close circle. figure 1 the status of students having covid-19 disease 19% 81% have the students had covid-19 disease? yes no journal of science learning article doi: 10.17509/jsl.v5i2.45230 346 j.sci.learn.2022.5(2).342-352 66.8% (f = 151) of the students stated that they had covid-19 in their close circle, while 33.2% (f = 75) stated that they did not. 3.3 findings from “did the students have a relative who died due to covid-19 disease?” sub-problem in the third sub-problem of the study, it was asked whether the students had a relative who died due to covid-19, and the results are presented in figure 3. figure 3 shows the answers given by the students about whether they have a relative who died due to covid-19. 16.8% (f = 38) of the students stated that they had a relative who died due to covid-19, while 83.2% (f = 188) stated that they did not. 3.4 findings from “are there any healthcare personnel in the family of the students?” sub-problem in the fourth sub-problem of the study, the students were asked whether there was healthcare personnel in their families, and the results are presented in figure 4. figure 4 shows the answers given by the students about whether there is healthcare personnel in their family. 16.8% (f = 38) of the students stated that healthcare personnel was in their family, and 83.2% (f = 188) stated that they did not. 3.5 findings from “what are the cognitive structures of the students about the concept of covid-19?” subproblem in figure 5, students’ cognitive structures about covid-19 are divided into 11 categories, including covid-19 prevention methods, covid-19 reflections in press, covid-19 results, covid-19 perceptions, causes of covid-19 transmission, covid-19 treatment, covid-19 symptoms, countries where covid-19 are common, characteristics of covid-19, covid-19 carriers and covid-19 diagnostic ways. table 1 shows the categories and codes related to the concept of covid-19. covid-19 prevention methods are listed as mask, social distance, restrictions, cleaning, disinfectant, and glove in the first category. four hundred twenty-five concepts are listed in this category. in the second category, 185 concepts, including quarantine, stay at home, pandemic, epidemic, healthcare personnel, case numbers, and minister of health, were determined as covid-19 reflections in press. in the third category, covid-19 results were examined. in this category, 151 concepts are listed as disease and death. in the fourth figure 2 the status of the students’ close circles to have covid-19 disease 67% 33% have any of the students had covid-19 disease in their close circle? yes no figure 3 the status of the death of one of the relatives of the students due to covid-19 17% 83% did the students have a relative who died due to covid-19 disease? yes no figure 4 the status of students having healthcare personnel in their families 17% 83% are there any healthcare personnel in the family of the students? yes no figure 5 students’ covid-19 cognitive structures journal of science learning article doi: 10.17509/jsl.v5i2.45230 347 j.sci.learn.2022.5(2).342-352 category, covid-19 perceptions are listed. one hundred twelve concepts were determined, including distance learning, fear, health, longing, stress, anxiety, danger, school, zoom, and the year 2020. in the fifth category, causes of covid-19 transmission, 94 concepts, including virus, corona, microbe, and covid, were determined. covid-19 treatments are listed in the sixth category. eighty concepts, including vaccine, hospital, intensive care, and intubated, have been determined in this category. in covid-19 symptoms, the seventh category, 23 concepts, including shortness of breath, cough, fever, loss of smell, loss of taste, and headache, have been determined. it is seen that 23 concepts are listed in the eighth category of countries where covid-19 is common, including china and türkiye. in the ninth category, characteristics of covid-19, ten concepts, including infectious, mutation and fatal, were determined. in the tenth category, covid-19 carriers, 11 concepts were determined, including family and bat. in the category of covid-19 diagnostic ways is seen, which is the eleventh category, seven concepts, including test and filiation, have been determined. under this category headings, the sentences formed by the students for the concepts were analyzed. finally, to reveal the students' cognitive structures about the covid19 concept in detail, the sentences they expressed about this concept were examined according to their relationship with the concept and categorized according to their meanings. at this stage, it was determined that some students did not write sentences. category 1. covid-19 prevention methods s32: we can protect ourselves by wearing masks. s41: let's wear a mask and keep social distance. s48: restrictions increased due to covid-19. s16: mask, distance, and cleaning. let's follow these three rules, take measures, and protect our loved ones. s4: we need masks and disinfectants very much during the pandemic period. s51: it is important to wear gloves while shopping. under this category, it is seen that the students emphasize concepts such as masks, social distance, restrictions, cleaning, disinfectant, and gloves. these concepts indicate the students' thinking about the prevention methods for covid-19. category 2. covid-19 reflections in press s139: if we get covid-19, we will have to go into quarantine. s148: we have to stay at home during the pandemic. s54: we are tired of the pandemic and social distance. s99: the epidemic has taught us to live differently. s181: since the virus entered our lives, healthcare personnel have made an effort for us. s19: case numbers are constantly decreasing and increasing. we are not as comfortable as in our life before covid-19. s170: the minister of health announced that masks and cleaning should be necessary. under this category, it is seen that the students emphasize concepts such as quarantine, stay at home, pandemic, epidemic, healthcare personnel, case numbers, table 1 the distribution of the cognitive structures of students obtained by the wat on the concept of covid-19 by categories categories code total covid-19 prevention methods mask-148 social distance-89 restrictions-72 cleaning-58 disinfectant-55 glove-3 425 covid-19 reflections in press quarantine-48 stay at home-39 pandemic-33 epidemic-30 healthcare personnel-19 case numbers-10 minister of health-6 185 covid-19 results disease-87 death-64 151 covid-19 perceptions distance learning-32 fear-18 health-15 longing-11 stress-10 anxiety-9 danger-2 school-9 zoom-4 year 2020-2 112 causes of covid19 transmission virus-74 corona-11 microbe-6 covid-3 94 covid-19 treatment vaccine-49 hospital-24 intensive care-5 intubated-2 80 covid-19 symptoms shortness of breath-7 cough-4 fever-3 loss of smell-3 loss of taste-3 headache-3 23 countries where covid-19 is common china-18 türkiye-5 23 characteristics of covid-19 infectious-12 mutation-4 fatal-3 19 covid-19 carriers family-8 bat-3 11 covid-19 diagnostic ways test-5 filiation-2 7 total 48 words 1130 journal of science learning article doi: 10.17509/jsl.v5i2.45230 348 j.sci.learn.2022.5(2).342-352 and minister of health. it can be thought that these concepts may have taken place in students' minds with the introduction of new definitions of covid-19 into our daily lives thanks to the press. category 3. covid-19 results s40: a deadly disease affecting the whole world. s174: i don’t want to die. under this category, it is seen that the students emphasize concepts such as disease and death. it was determined that the students emphasized the negative consequences of covid-19 on health. category 4. covid-19 perceptions s78: due to the virus, distance education has been started. s26: i am terrified of this epidemic. s220: let's wear our masks and stay healthy. s197: i miss my relatives; i miss the days before covid-19. s222: covid-19 is stressing me out. s215: the thought of getting sick worries me. s98: this virus is dangerous for the earth. s136: i want to go to school. s206: we started distance education due to covid-19. zoom has become indispensable. under this category, it is seen that the students emphasize concepts such as distance learning, fear, health, longing, stress, anxiety, danger, school, zoom, and year 2020. therefore, it can be thought that these concepts are the concepts that have the most place in the minds of the students during the long-term restrictions. category 5. causes of covid-19 transmission s169: let's focus on the social distance to protect from the virus. s25: i was separated from my loved ones due to corona. s193: if we get covid-19, we must be treated by doctors and use medicine to get rid of microbes. s45: i want to get rid of covid. under this category, it is seen that the students emphasize concepts such as virus, corona, microbe, and covid. these concepts express the causes of the transmission of covid-19. category 6. covid-19 treatment s146: let the vaccine end corona. s9: we went to the hospital because of the virus. s162: if we don't wear masks, we can get corona and stay in intensive care in the hospital for days. under this category, it is seen that the students emphasize concepts such as vaccine, hospital, intensive care, and intubated. these concepts are listed for the treatment forms of covid-19. category 7. covid-19 symptoms s87: shortness of breath is a horrible thing. s66: cough and fever are symptoms of covid 19. under this category, it is seen that the students emphasize concepts such as shortness of breath, cough, and fever. these concepts are listed by considering the symptoms of people with covid-19. category 8. countries where covid-19 is common s143: the chinese ate bats; the whole world became corona. s117: covid-19 is a deadly pandemic spreading worldwide and in turkey. under this category, it is seen that the students emphasize concepts such as china and türkiye. these concepts result from the students' seeing china as the source of the spread of the covid-19 epidemic. the epidemic affected turkey as well as the whole world. however, even though the epidemic was seen in other countries, the students did not specify the concept of other countries. category 9. characteristics of covid-19 s34: a fatal and infectious disease. s173: the health minister announced that the virus had undergone a new mutation. under this category, it is seen that the students emphasize concepts such as infectious, mutation, and fatal. these concepts result from students’ mental perceptions of the characteristics of the disease process of covid-19. category 10. covid-19 carriers s177: a man in china started the spread of the virus by eating bat soup. under this category, it is seen that the students emphasize concepts such as family and bat. these concepts created by students are based on living things that can be carriers of covid-19. category 11. covid-19 diagnostic ways s172: let's follow the mask, distance, and cleaning rules. let's have a covid test if there are symptoms. under this category, it is seen that the students emphasize concepts such as test and filiation. therefore, these concepts can be evaluated as findings pointing to diagnosing covid-19. 3.6 findings from “what are the students’ misconceptions about covid-19?” sub-problem the 226 sentences formed as wat results were examined, and the sentences of the students who were thought to have misconceptions are as follows; s10: i can't go out and meet my friends because it is quarantined. i am alone. s24: many people died because covid-19 is such a great plague. s39: doctors working in the intensive care unit in wuhan, china, are getting the virus daily. s42: if we are not clean, we can become corona. s83: because humanity's ridiculous curiosity about everything, it is causing the death of people in the world right now. s143: the chinese ate bats; the whole world became corona. in sentence s10, the concept of quarantine is used instead of restrictions. however, quarantine is the separation of people who have had the disease from other people. s24 has confused covid-19 with the plague. s39 journal of science learning article doi: 10.17509/jsl.v5i2.45230 349 j.sci.learn.2022.5(2).342-352 said that the doctors working in the intensive care unit in china were infected with the virus. however, the virus infects people in other professions and all humanity worldwide. s42 stated that you could catch the covid-19 disease if you are not clean. however, not only cleaning but also other rules must be followed. according to s83, an unscientific explanation was made by stating that people died in the world because of people’s curiosity. s143 has a detection that the virus spread because the chinese ate bats. however, there are other ideas about the cause of covid-19. 4. discussion this research aims to determine secondary school students' cognitive structures and misconceptions about covid-19 by word association test during the covid19 pandemic seen worldwide. the research was conducted with 6th and 7th-grade students in a secondary school in türkiye. the reason for the planning of this research is that no other study in the literature examined the cognitive structures of secondary school students with the word association test for the concept of covid-19. learning occurs as a result of the interaction between what is taught to the student and the existing concepts in the student's mind. for this reason, identifying and revealing existing concepts in students’ cognitive structures is important in learning. it is not very easy to explain the cognitive structures of individuals as a result of learning, but by revealing their thoughts about key concepts, vital data can be provided, and individuals’ cognitive structures can be revealed (kurt & ekici, 2013). according to the findings, it was determined which concepts were related to the subject in the students' cognitive structure and how the relations/connections between these concepts were established. in addition, some misconceptions were determined by looking at the quality of the associated concepts and words. the research results show that the word association test is a diagnostic and highly effective measurement and evaluation technique. when the situation of secondary school students who have had the covid-19 disease is examined, the rate of the students participating in the research who have had the disease themselves is low, but the rate of those who have had this disease in their close circles is relatively high. it was determined that the death rate in the close circles of the students participating in the study was also low. in addition, the ratio of healthcare personnel in the families of the students participating in the research was low. the cognitive structures of secondary school students on the concept of covid-19 are grouped under 11 categories. these categories are covid-19 prevention methods, covid-19 reflections in press, covid-19 results, covid-19 perceptions, causes of covid-19 transmission, covid-19 treatment, covid-19 symptoms, countries where covid-19 are common, characteristics of covid-19, covid-19 carriers and covid-19 diagnostic ways. it is seen that the most repeated area among these categories is for covid-19 prevention methods. under this category, it is seen that it is mainly repeated as a mask, social distance, restrictions, and cleaning. these findings can be interpreted as an indication that the students are worried about catching the disease caused by the covid-19 virus because there are people who often have the disease in their close circle and focus on remedies to protect themselves. in particular, the mask and social distance concepts among subjective measures have been mentioned by many students, and this issue's importance has been emphasized in literature (til, 2020). while the precautions mentioned by the students contain valuable information of vital value in the covid19 epidemic, it is an undeniable fact that it is important to spread this information to the whole society. another result obtained under this category is the increasing importance given to cleanliness, which many students mention because the benefit of personal cleaning and hygiene in this disease is undeniably great (i̇şsever, i̇şsever, & öztan, 2020). similarly, in a study conducted by dönmez & gürbüz (2020) to determine the cognitive structures of university students regarding the covid-19 virus, they stated that the category of ways of protection from covid-19 is the second most substantial cognitive structure and that the students often stated concepts such as mask, cleaning, social distance, and disinfectant. when the second category, covid-19 reflections in the press, is examined, it is seen that the students frequently repeat the new concepts such as quarantine, pandemic, and epidemic. this is an indication that students are serious about covid-19 and that they have knowledge about the subject. it is thought that the reason for the emergence of such data may be the media and the information channels they follow (görgülü-arı & hayır-kanat, 2020). in the third category, disease and death were shown by students as a result of covid-19. therefore, it would be correct to say that this epidemic had a negative impact on students. this can be explained by the fact that students, who constantly hear about deaths due to covid-19 disease and the increasing number of positive cases on the media agenda, associate this epidemic with negative thoughts. it is seen that the concepts of distance learning, fear, health, and longing are mostly repeated under the category of covid-19 perceptions. with the secondary school students having to take distance learning during the epidemic process, the students' perceptions against distance education stand out in this category. virus, corona, and covid are shown to cause the covid-19 transmission. the fact that people with covid-19 have a highly contagious effect plays a critical role in this process (jin et al., 2020; shen et al., 2020). journal of science learning article doi: 10.17509/jsl.v5i2.45230 350 j.sci.learn.2022.5(2).342-352 therefore, it can be said that the concepts stated by the students regarding the epidemic support this situation. the most mentioned concept in the covid-19 treatment category was the vaccine. as soon as covid-19 was identified, studies on developing a vaccine against the disease began. various types of vaccines have been developed by various centers and are still being developed. although there are many different discourses about when and how the epidemic will end, it is known that the vaccines developed are essential. therefore, it can be interpreted that students’ belief that the virus will disappear through vaccination permanently may be a reflection of their belief in the current health system and the vaccines applied (görgülü-arı & hayır-kanat, 2020). in addition, it can be thought that this subject is on the students' agenda as it is frequently emphasized in the information sources. in the category of covid-19 symptoms, the concepts of shortness of breath, cough, and fever are often mentioned. however, common symptoms of covid-19 have been described as fever, fatigue, and cough. in addition, shortness of breath, sore throat, and diarrhea in very few individuals have also been reported (who, 2020). this information suggests that students know the symptoms of the covid-19 disease by hearing or seeing them in their close circle. china comes to the fore in the category of countries where covid-19 is common. since china is shown as the starting point of the virus on the agenda of the whole world (who, 2020), it can be thought that this concept has an important place in students. in the covid-19 characteristics category, it was emphasized by the students that the disease was primarily infectious. therefore, it can be said that the results obtained from the study conducted by dönmez & gürbüz (2020) are parallel to the results of this research. in the category of covid-19 carriers, the concepts of family and bat are specified. it is thought that the students say the concept of family because the ways of transmission of the virus are from person to person, and attention to social distance is emphasized. in addition, bats are accepted as the covid-19 source and have an important place in the transmission of the disease. as a result, bats are thought to be the covid-19 cause (malik et al., 2020), and it is estimated that the students also started the bat concept because this situation was emphasized in the press. in the category of covid-19 diagnostic ways, the students focused on the concept of testing. a definitive diagnosis of covid-19 is provided by detecting covid19 ribonucleic acid (rna) taken from suspected individuals. among the available methods, the most preferred is the pcr test. this test's accuracy and reliability level is 70% (corman et al., 2020; choe et al., 2019). it can be thought that the students said this concept because this test is widely used, and the way it is done is also included in the media. it has been observed that very few of the students have misconceptions about the covid-19 virus. however, misconceptions may be caused by following unreliable sources. therefore, ensuring appropriate access to the mass media and disseminating accurate information can reduce misunderstandings about the covid-19 pandemic (bakebillah, billah, wubishet, & khan, 2021). conclusion covid-19 is not the first virus to threaten humans; it probably will not be the last. although pandemics are rare, they have a potentially devastating impact on people’s lives. it is stated that the epidemic has turned into a global trauma with its social, political, economic, and psychological reflections and that the situation experienced with the epidemic is not only a health problem but also a mental health problem with effects such as anxiety and social isolation (aşkın, bozkurt, & zeybek, 2020). for these reasons, the need to know people's thoughts on covid19 has arisen. according to the results obtained from the subproblems of the research, the rate of the students participating in the research who have had the disease themselves is low, but the rate of those who have had this disease in their close circles is relatively high. it was determined that the death rate in the close circles of the students participating in the study was also low. in addition, the ratio of healthcare personnel in the families of the students participating in the research was low. the cognitive structures of secondary school students on the concept of covid-19 are grouped under 11 categories. it has also been determined that students have some misconceptions about covid-19. determining the cognitive structures of secondary school students, who will be adults of the future, against covid-19; it will be beneficial in terms of developing strategies on issues such as crisis management, coping with stress, and ways of protection. in addition, it has been well understood during the epidemic process how viruses, one of the essential subjects of science education, can affect daily life. for this reason, the virus can be re-evaluated in the science curriculum, and it can be possible to organize the science education content by considering the categories obtained from this research. since the conceptual structures of secondary school students regarding the concept of covid-19 are important in structuring many concepts, especially the ways of protection from the disease, it is thought that the results of this research will contribute to the development of science education programs and the literature. references akınoğlu, o., & bakır, s. a. (2003). i̇lköğretim öğrencilerinin sosyal bigiler dersinde coğrafya konularını öğreneleriyle ilgili durum analiz [situation analysis of the geography subjects in the social studies journal of science learning article doi: 10.17509/jsl.v5i2.45230 351 j.sci.learn.2022.5(2).342-352 course of primary school students.]. marmara coğrafya dergisi, 8, 83106. aşkın, r., bozkurt, y., & zeybek, z. (2020). covid-19 pandemisi: psikolojik etkileri ve terapötik müdahaleler [covid-19 pandemic: psychological implications and therapeutic interventions]. i̇stanbul ticaret üniversitesi sosyal bilimler dergisi, 37, 304-318. ayas, a. (2014). kavram öğrenimi [concept learning]. in s. çepni (ed.), kuramdan uygulamaya fen ve teknoloji öğretimi [teaching science and technology from theory to practice]. pegem akademi. bahar, m., johnstone, a. h., & sutcliffe, r. (1999). investigation of students’ cognitive structure in elementary genetics through word association tests. journal of biological education, 33(134), 134-141. bahar, m. (2003). misconceptions in biology education and conceptual change strategies. educational sciences: theory & practice, 3(1), 55-64. bahar, m., nartgün, z., durmuş. s., & bıçak, b. (2006). geleneksel ve alternatif ölçme ve değerlendirme öğretmen el kitabı [traditional and alternative assessment and evaluation teacher's handbook]. ankara: pegema. bakebillah, m., billah, m. a., wubishet, b. l., & khan, m. n. (2021). community’s misconception about covid-19 and its associated factors in satkhira, bangladesh: a cross-sectional study. plos one, 16(9), e0257410. bakioğlu, b., & çevik, m. (2020). covid-19 pandemisi sürecinde fen bilimleri öğretmenlerinin uzaktan eğitime ilişkin görüşleri [views of science teachers on distance education during the covid-19 pandemic]. turkish studies, 15(4), 1-16. bassey, m. (1999). case study research in educational settings. usa: open university. başol, g. (2016). eğitimde ölçme ve değerlendirme. (genişletilmiş 4. baskı) [measurement and evaluation in education. (extended 4th edition)]. ankara: pegem akademi. bellini, m. i., pengel, l., potena, l., segantini, l., & esot covid‐19 working group. (2021). covid‐19 and education: restructuring after the pandemic. transplant international, 34(2), 220-223. berkhout, b. (2015). with a little help from my enteric microbial friends. front. med., 2, 30. bosch, a. a., biesbroek, g., trzcinski, k., sanders, e. a., & bogaert, d. (2013). viral and bacterial interactions in the upper respiratory tract. plos pathogens, 9(1), e1003057. budak, f., & korkmaz, ş. (2020). covid-19 pandemi sürecine yönelik genel bir değerlendirme: türkiye örneği [a general assessment of the covid-19 pandemic process: the case of turkey]. sosyal araştırmalar ve yönetim dergisi, 1, 62-79. ceylan, ö. (2015). fen öğretiminde kavram karikatürü kullanmanın 7. sınıf öğrencilerinin akademik başarılarına ve bilişsel yapılarına etkisinin incelenmesi [investigation of the effect of using concept cartoons in science teaching on 7th grade students' academic achievement and cognitive structures]. (unpublished master's thesis). sakarya üniversitesi eğitim bilimleri enstitüsü, sakarya. choe, j. y., kim, j. w., kwon, h. h., hong, h. l., jung, c. y., jeon, c. h., ... & kim, s. k. (2020). diagnostic performance of immunochromatography assay for rapid detection of igm and igg in coronavirus disease 2019. journal of medical virology, 92(11), 25672572. cohen, l., & manion, l. (1997). research methods in education. london: routledge. corman, v. m., landt, o., kaiser, m., molenkamp, r., meijer, a., chu, d. k., ... & drosten, c. (2020). detection of 2019 novel coronavirus (2019-ncov) by real-time rt-pcr. eurosurveillance, 25(3), 2000045. coyne, i. (1997). sampling in qualitative research. purposeful and theoretical sampling: merging or clear boundaries? journal of advanced nursing, 26(3), 623-630. çelikkaya, t., & kürümlüoğlu, m. (2019). sosyal bilgiler dersi “demokrasinin serüveni” ünitesi ile öğrencilerin bilişsel yapılarının ve kavramsal gelişim süreçlerinin incelenmesi [investigation of cognitive structures and conceptual development processes of students with the "adventure of democracy" unit in social studies course]. ajesi-anadolu journal of educational sciences international, 9(1), 56-86. denzin, n. k., & lincoln, y. s. (1996). strategies of qualitative inquiry. london: sage publications. dillon, j., & avraamıdou, l. (2020). towards a viable response to covid-19 from the science education community. journal for activist science & technology education, 11(2), 1-6. dönmez, i̇., & gürbüz, s. (2020). üniversite öğrencilerinin covid-19 virüsü hakkında bilişsel yapılarının belirlenmesi [determining the cognitive structures of university students about the covid-19 virus]. manas sosyal araştırmalar dergisi, 9(4), 2159-2172. ekici, g., & kurt, g. (2014). öğretmen adaylarının “aids” kavramı konusundaki bilişsel yapıları: bağımsız kelime ilişkilendirme testi örneği [pre-service teachers' cognitive structures on the concept of "aids": an example of independent word association test]. türkiye sosyal araştırmalar dergisi, 183(183), 267-306. ercan, f., taşdere, a., & ercan, n. (2010). kelime ilişkilendirme testi aracılığıyla bilişsel yapının ve kavramsal değişimin gözlenmesi [observation of cognitive structure and conceptual change through word association test]. türk fen eğitimi dergisi, 7(2), 136-154. ercan, e. s., rodopman-arman, a., i̇nal emiroğlu, n., öztop, d. b., & yalçın, ö (2020). turkish child and adolescent psychiatry association, psychosocial and spiritual support guide for family, children and adolescents during the covid-19 (corona) virus epidemic. https://www.ankara.edu.tr/wpcontent/uploads/sites/6/2020/03/cogepdercovid19rehberi30mart2020.pdf.pdf given, l. m. (ed.). (2008). the sage encyclopedia of qualitative research methods. sage publications. gallego, t. g., de la rubia, m., hincz, k., lopez, m. c., subirats, l., fort, s., & moñivas, s. g. (2020). influence of covid-19 confinement in students’ performance in higher education. görgülü-arı, a., & hayır-kanat. m. (2020). prospective teacher’ views on covid-19 (coronavirus). van yüzüncü yıl university the journal of social sciences institute, outbreak diseases special issue, 459-492. hendaus, m. a., jomha, f. a., & alhammadi, a. h. (2015). virusinduced secondary bacterial infection: a concise review. therapeutics and clinical risk management, 11, 1265. hovardas, t., & korfiatis, k. j. (2006). word associations as a tool for assessing conceptual change ın science education. learning and instruction, 16, 416-432. hruschka, d. j., schwartz, d., st. john, d. c., picone-decaro, e., jenkins, r. a., & carey, j. w. (2004). reliability in coding openended data: lessons learned from hiv behavioral research. field methods, 16(3), 307-331. i̇mer çetin, n., timur, s. & pehlivan, h. (2021). fen bilimleri öğretmenlerinin covid-19 pandemi sürecinde “virüs” kavramına yönelik metaforik algılarının incelenmesi [examining the metaphorical perceptions of science teachers towards the concept of "virus" during the covid-19 pandemic process.]. international journal of eurasia social sciences (ijoess), 12(43), 47-59. i̇şsever, h., i̇şsever, t., & öztan, g. (2020). covid-19 epidemiyolojisi [covid-19 epidemiology]. sağlık bilimlerinde i̇leri araştırmalar dergisi, 3(s1), 1-13. jin, y. h., cai, l., cheng, z. s., cheng, h., deng, t., fan, y. p., ... & wang, x. h. (2020). a rapid advice guideline for the diagnosis and treatment of 2019 novel coronavirus (2019-ncov) infected pneumonia (standard version). military medical research, 7(1), 1-23. karatepe, f., küçükgençay, n., & peker, b. (2020). öğretmen adayları senkron uzaktan eğitime nasıl bakıyor? bir anket çalışması [how do prospective teachers view synchronous distance education? a survey study]. journal of social and humanities sciences research, 7(53), 1262-1274. karcıoğlu, ö. (2020) what is coronaviruses, and how can we protect ourselves? anka tıp dergisi, 2(1), 66-71. knight, s. l., nolan, j., lloyd, g., arbaugh, f., edmondson, j., & whitney, a. (2013). quality teacher education research: how do journal of science learning article doi: 10.17509/jsl.v5i2.45230 352 j.sci.learn.2022.5(2).342-352 we know it when we see it? journal of teacher education, 64(2), 114116. kurt, h., & ekici, g. (2013). biyoloji öğretmen adaylarının bağımsız kelime ilişkilendirme testi ve çizme-yazma tekniğiyle “osmoz” kavramı konusundaki bilişsel yapılarının belirlenmesi [determining the cognitive structures of biology teacher candidates on the concept of "osmosis" with independent word association test and drawing-writing technique]. turkish studies, 8(12), 809-829. malik, y. s., sircar, s., bhat, s., sharun, k., dhama, k., dadar, m., & chaicumpa, w. (2020). emerging novel coronavirus (2019-ncov)current scenario, evolutionary perspective based on genome analysis and recent developments. veterinary quarterly, 40(1), 68-76. miles, m. b., & huberman, a. m. (1994). qualitative data analysis. thousand oaks, ca: sage. nalbandian, a., sehgal, k., gupta, a., madhavan, m. v., mcgroder, c., stevens, j. s., ... & wan, e. y. (2021). post-acute covid-19 syndrome. nature medicine, 27(4), 601-615. özen, k. (2011). özel öğrenme güçlüğü tanısı almış 7-9 yaş çocukların geliştirdikleri zihin kuramı yetenekleri ile sosyal gelişimleri arasındaki ilişkinin incelenmesi ve sağlıklı gelişim gösteren grup ile karşılaştırılması [examination of the relationship between the theory of mind skills developed by children aged 7-9 with a diagnosis of special learning disability and their social development and comparison with the healthy development group]. (unpublished master's thesis). maltepe üniversitesi sosyal bilimler enstitüsü, i̇stanbul. patton, m. q. (1990). qualitative evaluation and research methods. usa: sage. pınar, m. a., & dönel akgül, g. (2020). the opinions of secondary school students about giving science courses with distance education during the covid-19 pandemic. journal of current researches on social sciences, 10(2), 461-486. roberts, p., & priest, h. (2006). reliability and validity in research. nursing standard, 20, 41-45. shavelson, r. j. (1974). methods for examining representations of a subject-matter structure ın a student’s memory. journal of research in science teaching, 11(3), 231-249. shen, k., yang, y., wang, t., zhao, d., jiang, y., jin, r., ... & gao, l. (2020). diagnosis, treatment, and prevention of 2019 novel coronavirus infection in children: experts’ consensus statement. world journal of pediatrics, 16(3), 223-231. telli yamamoto, s. g., & altun, d. (2020) coronavirüs ve çevrimiçi (online) eğitimin önlenemeyen yükselişi [coronavirus and the unstoppable rise of online education]. üniversite araştırmaları dergisi, 3(1), 25-34. til, a. (2020). yeni koronavirüs hastalığı hakkında bilinmesi gerekenler [what to know about the new coronavirus disease]. ayrıntı dergisi, 8(85), 53-57. unesco. (2020a). school closures caused by coronavirus (covid-19). unesco. retrieved from https://en.unesco.org/covid19/educationresponse unesco. (2020b). startling digital divides in distance learning emerge. unesco. retrieved from https://en.unesco.org/news/startlingdigital-divides-distance-learning-emerge varoğlu, l., şen, ş., & yılmaz, a. (2020). üniversite öğrencilerinin periyodik tablo ile ilişkili bilişsel yapılarının incelenmesi [examining the cognitive structures of university students related to the periodic table]. yyü eğitim fakültesi dergisi, 17(1), 511-533. who. (2020). coronavirus disease (covid-19) pandemic. world health organization. retrieved from https://www.who.int/emergencies/diseases/novel-coronavirus2019 yıldırım, a., & şimşek, h. (2006). sosyal bilimlerde nitel araştırma yöntemleri (5. baskı) [qualitative research methods in the social sciences (5th edition)]. seçkin yayıncılık. yılmaz, a. c., & arun, k. (2020). bilişsel yapı, bilişsel stil ve öğrenilmiş güçlülük arasındaki ilişki [the relationship between cognitive structure, cognitive style and learned resourcefulness]. uluslararası yönetim ve sosyal araştırmalar dergisi, 7(13), 78-87. https://www.who.int/emergencies/diseases/novel-coronavirus-2019 https://www.who.int/emergencies/diseases/novel-coronavirus-2019 a © 2023 indonesian society for science educator 222 j.sci.learn.2023.6(2).222-235 received: 14 january 2023 revised: 25 may 2023 published: 5 july 2023 effects of soil textures, soil settlements, and soil water-holding capacity on landslides: an experimental study for science teachers donna hembra gabor1* 1division of physical sciences and mathematics, university of the philippines visayas, philippines *corresponding author: dhgabor@up.edu.ph abstract experimentation is a contributing factor to the interest and meaningful learning of science. in geology and earth science, the effects of soil textures, settlements, and water-holding capacity are parameters for landslides in barotac viejo and other flooded areas. landslides are triggered during heavy rainstorms, causing severe property damage and casualties. this experimental study aims to determine how these parameters are factors for landslides and give accurate information to science teachers. the study uses two methods to provide ease and continuity of measurements and settings using the fourier transform infrared(ftir) spectroscopy in analyzing the soil textures. the imhoff cone instrument is for the settling and water-holding capacity of the soil. ftir soil analysis reveals that contents of clay and organic matter directly affect soil water-holding ability due to the larger surface area. a landslide-prone zone has a lesser settling time except for the sand that settles fastest due to larger masses. this study is crucial for science teachers teaching geology and earth sciences besides forecasting and preventing geohydrological processes and developing better landslide warning strategies to mitigate risks and reduce socioeconomic damage. keywords landslide, ftir soil analysis, spectroscopy, imhoff cone, teacher’s learning 1. introduction teaching geology and earth science courses are difficult for secondary teachers who are not in an experimental study. information from this research is only for teaching resources and materials in earth science and geology that can help teachers and students. landslides are one of the world’s geohazards that threaten both exposed areas in urban and rural areas and cause severe consequences on human lives and economic losses. due to climate change, there is an increase in the frequency and intensity of heavy rainfall along with a shift of locations and recurrence of heavy rain that increases landslide risk in landslide-prone areas. expansion of urban areas due to population growth, redevelopments of mountains, and shortening of the coastal area caused by roads and railways constructions and deforestation increase exposure to the landslide hazard (pajalic et al., 2021). there are so many factors that cause landslides to happen. one factor is soil quality which includes its physical and chemical properties. second, is the soil water holding capacity and settling ability. lastly, the angle of elevation can be evaluated statistically and interpreted. a landslide refers to a soil mass's slow or rapid downward movement due to gravity. it is triggered when the shear stresses developed inside the soil exceed those which can resist. landslides are caused by the liquefaction of small grain silt sand layers or due to a general failure in combination with increased loads due to an earthquake, increased pore pressure, and reduction in the available shear strength of the soil. in particular, panay island is prone to typhoons, storm surges, and flash floods which constantly suffer from the effects of the outbreak of such destructive phenomena. for this reason, it is necessary to monitor the sources of landslides, and the mechanisms they present, to proceed with the analysis of stability and the calculation of safety factors (dariagan, atando & asis, 2021). on the other hand, the soil is a heterogeneous system, an upper layer of earth in which plants grow, a black or dark brown material typically consisting of a mixture of organic remains, clay, and rock particles. soil mechanisms and processes are complex and back-breaking for you to be understood and require analytical techniques. some traditional methods describe the relationship between soil properties such as physical, chemical, and its main soil journal of science learning article doi: 10.17509/jsl.v6i2.54618 223 j.sci.learn.2023.6(2).222-235 components. simple and accurate soil testing procedures in the field and laboratory are necessary for advanced research in landslide monitoring. (mohamed, saleh, belal & gad, 2018). the fourier transform infrared (ftir) spectroscopy is a unique tool for mineral and organic components of soil samples. the ft infrared spectroscopy offers a sensitive characterization of minerals and soil organic matter (som) and mechanistic and kinetic aspects of mineral–som interactions that underlie biogeochemical processes (margenot, calderón, goyne, mukome & parikh, 2017). ftir spectroscopy has been used in advanced research in soil composition properties for characterizing soil mineral components, including mineral identification, structural assessment, soil quality, and in situ monitoring of pedogenic processes (e.g., mineral formation) for landslide monitoring (margenot, calderón, goyne, mukome & parikh, 2017). also, the imhoff cone tests for soil-settling and waterholding ability estimate erosion ranging from loam to loamy sand in texture as a predictor of soil erosion and sediment concentration affecting land fields and human livelihood (sojka, carter & brown, 1992). the soil water-holding ability depends on precipitation patterns and holding capability. changes in weather or region precipitation patterns and the amount of water a soil can hold cause a landslide. the holding capacity of water within the pores of soils depends on capillary action and the size of the pores between soil particles. sandy soils have large particles and large pores that do not have a sizeable ability to hold water making sandy soils drain excessively. clay soil has small particles and pores that can hold water that tend to have a high water-holding capacity (margenot, calderón, goyne, mukome & parikh, 2017). this research study aims to determine how parameters like soil textures, soil water holding capacity, and settlement ability are factors for landslides and uses the results to give information to geology and earth science teachers. in this study, the researcher used two processes; where first is the development of methods to identify landslide parameters through ftir spectroscopy and imhoff cone to test soil textures, soil settlements, and soil water-holding capacity. the second process is to use these results to teach geology and earth science, particularly in landslide monitoring. the first process divides into two methods; the use of ftir spectroscopy for soil textures and the second is the imhoff cone for soil settling and holding capacity. in the first method, the fourier transforms infrared (ftir) spectroscopy analyzes soil components to provide accurate and valid identifications of soil components investigated. it aimed to identify soil components to ensure that the soil tested in the laboratory is also a factor for soil erosion leading to landslides. the use of the imhoff cone for the settling and water-holding capacity of soil to test how much the soil samples can hold water. using 1000 milliliter (ml) of water to the five soil samples for settling sediment and 250 ml for water-holding volume capacity with the imhoff cones. these samples with different textures came from places in the flooded area in iloilo province as soil sediment material. the researcher’s additional aim is to provide correct information to science teachers through this experimental approach and the ability to develop a module for landslide parameters. 1.1 experimental approach in teaching sciences incorporating technology into science courses is becoming a practice in universities, particularly in the experimental approach to science teaching, where technology is part of teaching work in any educational institution. however, it is hard in geosciences to find technology that can carry a role between the experimental method and the traditional classroom setting. ftir spectroscopy and imhoff cones are ideally suited to bridge this gap. here, i fully integrate the ftir spectroscopy and imhoff cones as an educational tool for graduate-level k– 12 in-service teachers who teaches earth science and geosciences with the developed experimental approach in a classroom setting for the student to understand landslide parameters. the researcher has established classroom course objectives and integrates technology through procedures that entail hands-on activities that engage and motivate students to learn in the geoscience classroom. i assess the impact of this experimental approach on teachers teaching in these courses through the surveys that developed the ability of students to analyze and solve problems and improve innovation for the teachers. the experiment provides a simple and engaging framework for familiarizing teachers with landslide parameters through a developed procedure with ftir spectroscopy and imhoff cone. also, the researcher shows how experiments give accurate information to the teachers in the classroom environment. according to soares, de campos, thomaz, da cruz pereira & roehrs (2016), the existence a set of factors that hinder the learning of science teaching; not only teachers being responsible but also a lack of interest, a lack of laboratories, overcrowded classrooms, school infrastructure, among many other factors. to lessen this problem in any institution, teachers should learn an experimental approach that motivates and seeks students' interests that relate to their daily life. science teachers used experimentation to arouse interest among students in various levels of learning. students attribute the experimental approach as a motivating character that increases learning ability and permits them to involve in the topics addressed during theoretical classes based on the experimental results. with the occurrence of non-significant learning, the covid-19 pandemic and other related issues help curriculum planners and supervisors analyze the importance of giving accurate information to teachers journal of science learning article doi: 10.17509/jsl.v6i2.54618 224 j.sci.learn.2023.6(2).222-235 through an experimental approach. therefore, with experimentation dedicated to the fundamental level, students will be provided with a better and more concrete base in science education as an attempt to introduce students to advanced laboratory approaches in learning natural sciences, geology, and earth sciences. 1.2 transform infrared (ftir)spectroscopic analysis of soil textures in recent decades, most studies have focused on soil components like organic matter, texture, and mineralogy (hassani, bahrami, noroozi & oustan, 2014). nandiyanto, oktiani & ragadhita (2019) stated that fourier transforms infrared (ftir) is one of the advanced analytical techniques for researchers to characterize samples in liquids, solutions, pastes, powders, films, fibers, and gases. ftir analysis is also for analyzing the material on the surfaces of the substrate that is rapid, accurate, and relatively sensitive (jaggi & vij, 2006). in the ftir procedural analysis, samples with infrared (ir) radiation affect atomic vibrations of molecules that result in the specific absorption and transmission of energy, making the ftir determines specific molecular vibrations contained in the sample. the infrared (ir) spectrum has three wavenumber regions: far-ir spectrum (<400 cm-1), midir spectrum (400-4000 cm-1), and near-ir spectrum (4000-13000 cm-1). mid-ir is the most widely used in the sample analysis, but the farand near-ir range also provides information about the samples analyzed. in this study, the researcher used the mid-ir wavelength that is into four regions; the single bond region (2500-4000 cm1), the triple bond region (2000-2500 cm-1), the double bond region (1500-2000 cm-1), and the fingerprint region (600-1500 cm-1). in imaging spectroscopy, the spectroscopic modes emitted electromagnetic energy from a light source collides with the given phenomenon, light rays were absorbed, reflected, and the other part passed through it. spectroscopy is a quantitative calibration of reflection, absorption, or passing. one of the advantages of fourier transforms infrared spectroscopy is that this is a nondestructive technique with no hazard or destruction to the environment (guerrero, viscarra & mouazen, 2010). fourier transforms infrared spectroscopy is for farming and environmental studies on soils within two visible and infrared ranges. in addition, infrared spectroscopy may also provide for specifying various soil components. most studies have shown that soil spectral reflectance is affected by soil properties like humidity, texture, structure, and quantity of organic matter (soriano-disla, janik, viscarra & macdonald, 2014). the soil spectral reflectance is within visible and near-infrared (nir) ranges at wavelengths (3502500 nm) (iurian & cosma, 2014). of other important soil properties, which affect the quantity of the given spectral reflection, one can refer to the type and frequency of clay minerals, carbonates, hydroxyl groups in water and soil, organic compounds, and iron and aluminum oxides. the reflective spectra may act as a tool in analyzing many soil properties. organic carbon is one of the foremost soil properties estimated by satellite images and spectroscopic technologies with high precision because of the accumulated organic carbon in the surface layer of soil. this property has various spectral behaviors because of the existing complexity of organic matter (viscarra, walvoort, mcbratney, janik & skjemstad, 2006). the most absorbent characteristics from organic carbon often occur at the wavelengths about 1730 nm and 2330 nm, while the little absorbent is at wavelengths about 1150, 1670, 1765, 2070, 2110, 2140, 2190, 2280, 2310, and 2390 nm. the absorbent bands adjacent to 1400 and 1900 nm may be due to the existing water in organic compounds (babaeian, homaee, montzka, vereecken & norouzi, 2015). the carbonate minerals often possess strong absorbent characteristics near 2345 nm and are relatively weaker adjacent to 1860, 1990, and 2140 nm (viscaraa et al., 2006). particle size noticeably affects soil spectral behavior as the size of particles becomes sizable with an increase in the light path through soil particles is more absorbed, and reduced reflection leads to spectral curves. in reduction, the size of soil particles increases soil reflectance with faded color minerals like silicates and carbonates. oxide and hydroxide minerals have small spectral reflectance, and the level of soil spectral reflectance decreases as the size of soil particles decreases (summers, lewis, ostendorf & chittleborough, 2011). the spectral behavior of soil is a function of its constituent elements, and its chemical components like oxygen, silica, and aluminum lack strong absorbent characteristics within visible and near-infrared ranges. however, the soil's pieces, such as iron oxides, clay, and organic substance, may highly affect spectral curves and absorbent characteristics. the iron oxides influence the reflectance in the visible zones of organic carbon and clay in the infrared zones (summers lewis, ostendorf & chittleborough, 2011). in recent research studies, soil chemical properties used spectral reflectance to achieve favorable results related to the soil structures and other components like clay, silt, and organic matter that can estimate by soil spectral data with very high precision (summers, lewis, ostendorf & chittleborough, 2011). this study used spectral data for the information on soil components present in the area of study for iloilo province to monitor the physical and chemical properties of soil in the flooded area. in general, the findings of other research studies indicate that using soil spectral data may be employed as an indirect technique for the estimation of the physical and chemical properties of soil. the present study only focuses on the area in the province of iloilo, and properties in other places in the philippines with different effects on soil's spectral behavior are not part of the study. it also incorporates technology through an experimental journal of science learning article doi: 10.17509/jsl.v6i2.54618 225 j.sci.learn.2023.6(2).222-235 approach to promoting student understanding of concepts associated with landslide parameters by geosciences and earth teachers. 1.3 soil settling and water holding capacity during landslides, debris flows are a transport that results in large amounts of material delivered rapidly downstream. it has destructive capabilities because of its high impact force due to the velocities reached and the mass in transit that lifts and carries large objects such as boulders and trees, resulting in fluidization of the surface soil layers (luino et al., 2022). nowadays, the knowledge of the physical and environmental factors influencing landslide activation within the panay, especially in iloilo province, is still lacking and incomplete. recent studies have investigated soil settling and holding capacities and their relations with landslide occurrence, making this study a source of information and additional information for other researchers. according to luino et al. (2022), several empirical and physically based approaches define thresholds for hydrological conditions—including rainfall, soil water holding, and settling capacity that results in landslides. several authors have proposed different methods to identify rainfall thresholds for the possible initiation of landslides where soil can hold. dariagan, atando & asis (2021), the predominant soil types in the region are clay loam of sta. rita and alimodian series, sandy loam of sara, umingan, louisiana series and clay for panay island; guimbalaon clay, silay fine sandy loams of volcanic origin for negros occidental while for guimaras, it is faraon clay, gravely loam, and sara sandy loam. this type of soil is moderately deep and has a high degree of permeability that has connected pore spaces that allow water to flow from one to another. low permeability soil has isolated pore spaces that trap waters within them, while in a lump of clay, most pore spaces block the water where it cannot flow easily. recent studies have only considered rainfall variables as the most vital and easy-to-quantify landslide-triggering factor for the landslide (dariagan, atando & asis, 2021). most works find difficulties in correlations between soil water settling and holding capacity and landslide occurrences, especially when the investigation is on warning systems aimed at mitigating the possibly severe consequences and damage to property and population. these works have considered soil water settling and holding capacity as one of several variables for landslide occurrence. on the contrary, several studies have investigated landslide-conditioning factors, including the activation of mass movements that affect soil water settling and holding capacity. based on recent research, the researchers presented different simulations and mathematical models to estimate the rainfall conditions that affect settling and water-holding capacity leading to the activation of landslides. infinite slope stability analysis or modeling of water infiltration dynamics and groundwater pressure in soils causes a landslide (luino et al., 2022). places in iloilo province (barotac viejo, leganes, oton, and miagao) where the researcher conducted three different methods to investigate the occurrence of shallow landslides to mitigate its effect and give accurate information to these communities are the focus of this study. based on the works of alamanis, papageorgiou, xafoulis & chouliaras (2020), the activation of mass movements is due to soil moisture (settling and holding capacity), soil type, and slope acclivity. in this paper, the researcher will present a study based on three methods that use soils from four places in iloilo province as data for the possible occurrence of shallow landslides that have occurred in the past, with the aim of (i) identifying the soil properties that most significantly influence the activation of flood and landslides in the study area, (ii) differentiating soil water settling and holding capacity within the study area, (iii) identifying slopes or elevation angle for the possible occurrence of shallow landslides and mud-debris flows in the five different study area, and finally (iv) comparing the soil settling and holding capacity based on imhoff cone test. 2. method 2.1 purpose of the study the current study wanted to determine how parameters like soil textures, soil water holding capacity, and settlement ability are factors for landslides and uses the results to give information to geology and earth science teachers. through these, the study wants to develop a module from this experimental approach for landslide parameters that motivate students learning in earth science and geology courses. 2.2 context of the study barotac viejo, oton, leganes, and miagao are places in the province of iloilo prone to landslide and flooded areas. the researcher tried to improve the landslide monitoring from these areas and give accurate information on how soil textures, soil settling, and water-holding capacity can be parameters of a landslide which was one of the priorities identified by the local government of iloilo and their respective municipalities. developing and implementing advanced education for earth science and geoscience teachers' plans incorporating technology through experimental methods was done as the first step of the study. in the last year of 2022, the province of iloilo opened its doors to improve their landslide monitoring and share this information with teachers for learners to know about landslides and floods. the researcher used a hybrid approach, combining experimental methods, modular modality (developed module), technology, and face-to-face classes. students learned theoretical concepts through the journal of science learning article doi: 10.17509/jsl.v6i2.54618 226 j.sci.learn.2023.6(2).222-235 modules provided by teachers, followed up with lectures and skills during face-to-face meetings. this approach allowed students to better understand the lessons and ideas explored to make their learning more relevant. it allowed learners to bridge the gap between the concepts learned through the module and the help offered by the teachers during the short face-face classes. the pilot test has five (5) earth science teachers and additional ten (10) earth science teachers and their respective students from senior high school, with only fifty (50) students participating in this study and taking 1.5 hours of face-to-face classes. the same students went through the modules. these ten teachers are assigned to facilitate face-to-face learning in all phases of the course. 2.3 developed module the researcher developed a module that allowed learners to use this as a self-learning module (slm) to help the earth science teachers apply the shared information to the learner. three earth science teachers from the university of the philippines and two graduate students from the same school who teaches the same course were participants in the pilot testing and as content validators. teachers went through the developed module and asked for their comments on the content of the module. the module was revised after the collaboration of ideas from the content validators. the revised module was pilot tested on the five teachers and the researcher's students. the printed module examined students' performance and only resource material in learning landslide parameters of earth science/ geosciences courses. lessons were delivered via module and assisted and supplemented the teachers' instruction. a series of topics in a module was part of the subject teaching, and students answered the assigned activities in the module. in addition, the module has text and graphics that consists of learning objectives. the content and context of the modules were results from the experimental approach done on the first steps in the research study that promotes critical thinking skills and enhance the core content of the earth science subject with the concepts of landslide parameters used by the teacher to design the content subject. at the end of each module, students answered a short multiple-choice test to assess their understanding of the key concepts in the module. this assessment verified the student's completion of the unit of study and part of the research-based learning outcome. students received grades for each module. 2.4 instrument after the module lesson went through, students answered survey questions that were administered to science students anonymously from november to december 2022 and after a class discussion of the module as teaching resource materials for the stem second quarter of the year. the survey included learners' perspectives on the developed module, the experimental method applied, and earth science teaching that incorporates technology instruction that improves learners' performance. it also indicates how helpful teachers facilitate learning of landslide parameters and provide honest feedback on learners' experiences using the experimental method. a total of 50 senior high school students of up visayas participated in the trial and completed the survey based on improvements in actual results. the study took place in the first week of october 2022, the second quarter of face-to-face interaction after nearly two years of distance learning. pretests, posttests, and survey questions verified how technology instruction and experimental methods improve learners' performances in earth science and geology. the descriptive statistics and t-test analyzed the statistical significance of the various measures in this study. 2.5 participants for pilot testing, the participants are the three (3) teachers teaching earth science from the university of the philippines visayas, and the two graduate students from tacurong sur national high school and janiuay national high school. for final testing, the sample class was determined using a random sampling technique from the senior stem classes that attended the earth science and geology course. it was because each student (class) has relatively the same character, academic ability (preliminary test results), access to information (learning modules), and advanced technology (ftir spectroscopy and imhoff cone). the subjects of this study were 50 stem participants. this study of an experimental learning approach used printed modules, advanced technology, and face-to-face meetings as part of learning materials and data sources that answered the researchers’ questions. the characteristics of the participants are in table 1. table 1 participant’s characteristics gender frequency percent valid percent cumulative percent valid male female 31 19 62.0% 38 .0% 62.0% 38 .0% 62.0 100 .0 total 50 100 .0 % grade level frequency percent valid percent cumulative percent valid grade 11 grade 12 17 33 34.0% 66.0% 54.0% 66.0% 34.0 100 .0 total 50 100 .0% journal of science learning article doi: 10.17509/jsl.v6i2.54618 227 j.sci.learn.2023.6(2).222-235 2.6 data gathering procedure there were five stages in this research (fig 1): 1) experimental approach, 2) developed experimental procedure, 3) developed and validate module, 4) revision of the learning module as teaching materials, carrying out a pretest to determine students' prior knowledge; 5) measuring student-learning outcomes through post-test and analyzing teachers’ perceptions from answered survey questions. first is the experimental approach, which divides into two. the first method is the collection of samples which are fragmented by hand and dried for 14 days, and the second is the soil testing through mid-ftir spectroscopy. the chosen areas in the province of iloilo are prone to landslide areas according to the national mapping and resource information authority (namria), particularly in barangays of barotac viejo and flooded areas such as oton, leganes, and miagao. the sand was from up visayas miagao instead of the soil because the miagao is partly coastal, and most soil has a sand content in that area. the sand and five different soil samples were collected, gently fragmented by hand, and air-dried for ten days without sieving. figure 2 contains five different soils and sand from flooded and prone landslide areas of oton, leganes, barotac viejo, and miagao. textures and properties of soil were analyzed using fourier transform infrared spectroscopy. soil organic matter was determined using the infrared wave ranges. an electronic balance identifies the weight of soil grains from 0.0000 g to 0.0200g. second, under the experimental method, samples of each soil were placed in an imhoff cone with 1000ml running water to test the settling and water holding capacity. the ph paper determines the acidity or alkalinity of a liquid used. for the settling ability in figure 3, each soil suspension was hand stirred for five minutes then a timer took time for the soil to settle. each sample and the sand underwent a settling test as one of the parameters for floods and landslides occurrence. with a ph number of 7 and using the same imhoff cone apparatus and a timer, 100 g of each five different soil figure 1 procedural flowchart of the research study figure 2. five different soil samples and sand from iloilo province (a) (b) figure 3. (a). soil water settling set-up.(b) four different samples underwent a settling test. journal of science learning article doi: 10.17509/jsl.v6i2.54618 228 j.sci.learn.2023.6(2).222-235 samples and sand tested its soil water holding capacity with 250 ml of water added. the imhoff cone measured the sediment volume that settled after 4 hours. the reading volume of settled sediment required gentle leveling of the surface of the settled sediment in the imhoff cones with one-finger taps at the bottom of the cone. the contents were carefully decanted and reweighed using weightbalanced (hardened). air dry the soil within 24 hours to remove water. filter papers were removed, dried at 37 °c overnight, and reweighed to determine sediment weight. this process was done three times for each soil. the 100 g soil contained in 1 l of water was the independent variable, where the maximum settling volume is four hours for landslide soil settling ability. this approach uses the imhoff cone for a given mass of soil in volume suspension that could vary, ensuring no residue was lost in the filtering process; the suspended soil mass was an absolute parameter for each sample. in determining the soil water holding capacity for every 100 grams of each soil sample, 250 ml water was added to the imhoff cone and set for four hours for each soil sample to absorb the water added (fig 4). the third process is the researcher's development and validation of a module. the module contains the results of soil textures, settling, and water-holding capacity information. a pilot test ensures the validity of the developed module. the module was try-out by five earth science and geosciences teachers for content validity. a survey questions about the content and how a module can help motivate students in learning about landslide parameters. the fourth process was revising the learning module as teaching materials, and carrying out a pretest to determine students' prior knowledge. the module was revised based on the content validators' suggestions and recommendations. a pretest test was conducted on fifty (50) participants of the university of the philippines visayas, janiuay national high school, and tacuyong sur national high school. the last remaining process is measuring the effectiveness of the developed module based on the collaboration of experimental research and technology. measuring student-learning outcomes through post-test and analyzing teachers’ perceptions from answered survey questions. final testing was on another ten teachers and their respective students. students from each school participated in their own specific discussion time to discuss the same topics (definitions, context, and content of modules). the module in landslide parameters was a teacher follow-up through face-to-face meetings. students performed module activities, and if students needed clarifications on the lessons, an online platform was available where students exchanged ideas with definitions, context, and content and followed up with teacher assistance during a short face-toface class. for the third week, the teacher prepared and conducted an actual study inside the classroom. the collected data of both groups were analyzed, interpreted, and concluded using frequencies, mean-standard deviation, t-test, and anova. the results of the research were in favor of a student’s learning and teaching approach. the researchers suggested that collaboration between advanced technology and experimental method were an alternative to teaching to help students adjust to the new classroom environment due to the covid-19 pandemic. 2.7 data analysis quantitative and qualitative data were collected from respondents and analyzed separately using the excel program and the pspp application, open-source software or program for statistical analysis of sampled data (pspp) intended as a free alternative for ibm spss statistics. the frequencies of nominal variables and descriptive statistics like percentages, standard deviations, and means of categorical variables were analyzed using pspp also. t-test was used to test the relationships of variables. the 50 respondents of grade 11 and grade 12 stem tracks students answered all parts of the research questionnaires and no missing value from the participants' answers. 3. result and discussion 3.1 fourier transform infrared (ftir) spectroscopic analysis of soil properties. in figure 5, sample one had peaks containing a single bond area (2500-4000 cm-1). no hydrogen bond in the material because there is no broad absorption band presence. a sharp at 3423 cm-1 is an alcohol and hydroxy compound, especially a hydroxyl group, and h-bonded oh stretch is present, and 3621 cm-1 replies to the existence of free alcohol hydroxyl compound(oh). no aromatic structure because there are no peaks between 3000 and 3200 cm-1. a narrow sharp of less than 3000 cm figure 4. soil water holding set-up https://en.wikipedia.org/wiki/spss journal of science learning article doi: 10.17509/jsl.v6i2.54618 229 j.sci.learn.2023.6(2).222-235 1, especially at 2360 cm-1, indicates the presence of carbon dioxide. no specific peak for aldehyde has between 2700 and 2800 cm-1. no triple bond region (2000-2500 cm-1) was detected, informing no c≡c bond in the material. regarding the double bond region (1500-2000 cm-1), the sample has some organic nitrates from the soil or ammonia at about 1652 cm-1. there is no specific peak for aldehyde between 2700 and 2800 cm-1. this material has thiols and a thio-substituted compound. since the peaks were only about eight, the material should be a small organic compound. the result showed that several peaks were detected, informing the complex structure material. in addition to the single bond area (2500-4000 cm-1), there are several peaks. in the double bond region (1500-2000 cm-1), several peaks were also detected: in the fingerprint region (600-1500 cm-1), a sharp at 1033 cm-1 informed the alkylrelated compound present. in figure 6, sample two had peaks containing a single bond area (2500-4000 cm-1). no hydrogen bond in the material because there is no broad absorption band presence. a sharp at 3442 cm-1 is a secondary amino, especially heterocyclic amine, is present from ammonia decomposition, and 3621 replies to the existence of an alcohol hydroxyl compound or free o-h. no aromatic structure because there are no peaks between 3000 and 3200 cm-1. a narrow sharp of less than 3000 cm-1, especially at 2360 cm-1, indicates the presence of carbon dioxide. no specific peak for aldehyde has between 2700 and 2800 cm-1. no triple bond region (2000-2500 cm-1) was detected, informing no c≡c bond in the material. regarding the double bond region (1500-2000 cm-1), the sample has some organic nitrates from the soil or ammonia at about 1637 cm-1. samples from the residential area so urea or carbamide compound formed as the end product of the metabolism of protein and excreted in the urine of mammals. it is synthesized in large quantities from ammonia and carbon dioxide for use in fertilizers, animal feed, and manufacturing polymers known as ureaformaldehyde resins, used in making plastics. there is no specific peak for aldehyde between 2700 and 2800 cm-1. since the peaks were only about eight peaks, the material should be a small organic compound. the result showed figure 5. buray, oton ftir analysis figure 6. botong, oton ftir analysis https://www.merriam-webster.com/dictionary/compound https://www.britannica.com/science/carbon-dioxide https://www.britannica.com/topic/feed-agriculture https://www.britannica.com/topic/feed-agriculture journal of science learning article doi: 10.17509/jsl.v6i2.54618 230 j.sci.learn.2023.6(2).222-235 that several peaks were detected, informing the complex structure material. in addition to the single bond area (2500-4000 cm-1), there are several peaks. in the double bond region (1500-2000 cm-1), several peaks were also detected: in the fingerprint region (600-1500 cm-1), a sharp at 1033 cm-1 informed the alkyl-related compound present. in figure 7, sample three had peaks containing a single bond area (2500-4000 cm-1). no hydrogen bond in the material because there is no broad absorption band presence. a sharp at 3419 cm-1 is an alcohol and hydroxy compound, especially heterocyclic amine is presently produced by the reaction of water with nitriles, and 3621 replies to the existence of an alcohol hydroxy compound. no aromatic structure because there are no peaks between 3000 and 3200 cm-1. a narrow sharp of less than 3000 cm1, especially at 2360 cm-1, indicates the presence of carbon dioxide. no specific peak for aldehyde has between 2700 and 2800 cm-1. no triple bond region (2000-2500 cm-1) was detected, informing no c≡c bond in the material. regarding the double bond region (1500-2000 cm-1), the sample has some organic nitrates from the soil or ammonia at about 1652 cm-1 (olefinic (alkene)). there is no specific peak for aldehyde between 2700 and 2800 cm-1. this material has thiols and a thio-substituted compound. since the peaks were only about eight, the material should be a small organic compound. the result showed that several peaks were detected, informing the complex structure material. in addition to the single bond area (2500-4000 cm1), there are several peaks. in the double bond region (15002000 cm-1), several peaks were also detected: in the fingerprint region (600-1500 cm-1), a sharp at 1033 cm-1 informed the alkyl-related compound present. at 464 cm1, the thiols, and thio-substituted compounds, especially aryl disulfides (s-s stretch are present in the sample. in figure 8, sample 4 had peaks that contained a single bond area (2500-4000 cm-1). no hydrogen bond in the material because there is no broad absorption band presence. a sharp at 3419 cm-1 is an alcohol and hydroxy compound, especially heterocyclic amine is present, and 3691 replies to the existence of an alcohol hydroxy compound. no aromatic structure because there are no peaks between 3000 and 3200 cm-1. a narrow sharp of less figure 7. leganes ftir analysis figure 8. barotac viejo residential area ftir analysis journal of science learning article doi: 10.17509/jsl.v6i2.54618 231 j.sci.learn.2023.6(2).222-235 than 3000 cm-1, especially at 2360 cm-1, indicates the presence of carbon dioxide. no specific peak for aldehyde has between 2700 and 2800 cm-1. no triple bond region (2000-2500 cm-1) was detected, informing no c≡c bond in the material. regarding the double bond region (1500-2000 cm-1), the sample has some organic nitrates from the soil or ammonia at about 1683 cm-1 (aromatic ring (aryl). there is no specific peak for aldehyde between 2700 and 2800 cm1. since the peaks were only about eight, the material should be a small organic compound. the result showed that several peaks were detected, informing the complex structure material. in addition to the single bond area (2500-4000 cm-1), there are several peaks. in the double bond region (1500-2000 cm-1), several peaks were also detected: in the fingerprint region (600-1500 cm-1), a sharp at 1029 cm-1 informed the alkyl-related compound present. at 466 cm-1, the thiols, and thio-substituted compounds, especially aryl disulfides (s-s stretch are present in the sample. figure 9. shows the analysis of sample five, the soil from the landslide-prone area of barotac viejo. the results conclude as follows: (1) regarding the number of peaks, there are more than five peaks, informing that the analyzed chemical is not a simple chemical. (2) the peaks contained a single bond area (2500-4000 cm-1). no broad absorption band was found, indicating no hydrogen bond in the material. there is a sharp bond peak at about 3623 cm-1 and 3697 cm-1, replying to the existence of secondary alcohol and oh stretch. no peaks between 3000 and 3200 cm-1 indicate no aromatic structure. no specific peak for an aldehyde is between 2700 and 2800 cm-1. (3) one triple bond region (2000-2500 cm-1) was detected, indicating a carbon dioxide bond in the material. (4) regarding the double bond region (1500-2000 cm-1), a sharp peak at about 1633 cm-1 informs some simple hetero-oxy compounds, especially the organic nitrate, which can be from artificial fertilizer and ammonia. this peak at about 1633 cm-1 informs a c=c bonding in the material. and at the fingerprint region (600-1500 cm-1), there is an aliphatic organohalogen compound from an industrial product that comes from pesticides. based on the above interpretation, conclusions on ample five has to do with farming due to a figure 9. barotac viejo landslide prone area ftir analysis figure 10. sand ftir analysis (sio2) from the department of geology at the university of tartu (https://spectra.chem.ut.ee/paint/fillers/sand/). journal of science learning article doi: 10.17509/jsl.v6i2.54618 232 j.sci.learn.2023.6(2).222-235 compound found in pesticides and fertilizers used by farmers for their plants and animals. the material should be a small organic compound since the peaks were only about ten spikes. in figure 10, the infrared spectrum of the clean sand sample depicts an adsorption peak at 830 cm−1 depicting the symmetric and asymmetric stretch vibration for si―o, respectively. absorption peaks around 620 cm−1 characterize the bending of the si―o functional group in asymmetric and symmetric vibration regions with the presence of pure silica as the chief component in the sand sample. the clean sand particles show peaks at 2,851.85 and 2,924.64 cm−1, which indicates the symmetric and asymmetric ‐ch2 stretch. the absorption band around 2,920, 2292, and 2236 cm−1 in the three spectrograms shown in figure 9 depict the c―h symmetric vibration of the saturated hydrocarbons and o―h band due to stretching vibration at 3,435 cm−1. c¼o stretching peaks appeared between 1,627 and 1,870 cm−1, confirming carbonyl components like acids and aliphatic esters in the sand particles (saxena, kumar & mandal, 2018). 3.2 soil settling and water holding capacity imhoff cones with dry soil to determine the settling time for five soils and sand. this settling time was affected by the agitation method (handshaking and stirring). the information on the soil settling capacity of the samples and sand conducted in the laboratory is in table 2. compared to other soil samples, the results revealed that soil from a landslide-prone area has a lesser settling time except for the sand that settles fastest due to larger masses. granulated soil samples have a longer settling time required. a soil with higher clay contents had more suspended clay-sized particles after hand stirring for 5 minutes. with different particle size distributions, the slope of the settling relationship is difficult to determine because of the inability to account for the mean aggregate size of the soil sample or the sample's aggregate stability due to the limited functions of the instrument used. these properties may vary from soil to soil and change with time and exposure to various environmental influences. a 100g of soil and sand were placed inside the imhoff cone to determine the soil-water-holding capacity. using the imhoff cone, the accuracy of these methods is good, but they are very time-demanding. results revealed that the soil water holding capacity depends on soil texture (particle sizes) and organic matter. soil texture, smaller particle sizes, such as in clay, have a larger surface area. the larger the surface area, the easier the soil to hold onto water leading to a higher water-holding capacity. compared to sand, which has large particle sizes but a smaller surface area. the sand has a smaller surface area leading to low water-holding ability. also, sand is gritty and therefore does not hold more water because pores are so large that water can rapidly move through it. soil organic matter is another factor that can help increase water-holding capacity (reichert et al., 2009). soil organic matter has a natural magnetism to water. botong and buray oton is farm area causing an increase in the percentage of soil organic matter leading to an increase in soil water holding capacity due to a decayed material from a living organism (plant or animal material) (table 3). 3.3 student’s perception of developed module as technology changes, the living styles and tendencies of people toward social communicational, economic, and educational aspects, people need to reach information or any resources fast and easily. in addition, they need to exchange and share resources on an informational base. with the help of technology, today it is easy to reach table 2 soil settling time soil settling capacity time( hrs.) samples type location trial 1 trial 2 trial 3 1 soil buray, oton 4 .333 5.167 5 .75 2 soil botong, oton 3.333 2.16 2.75 3 soil leganes 2.16 2.43 4.33 4 soil residential, barotac viejo 3.11 3.167 4.33 5 soil landslide prone area, barotac viejo 2.19 2.42 2.15 6 sand up visayas, miagao 0.45 0.35 1.15 table 3 soil holding capacity time soil water holding capacity time( hrs.) samples type location trial 1 trial 2 trial 3 1 soil buray, oton 1.54 1.5 2.46 2 soil botong, oton 2.07 2.3 2.2 3 soil leganes 0.5 1.2 2.46 4 soil residential, barotac viejo 0.56 1.41 2.3 5 soil landslide prone area, barotac viejo 1.41 2.16 2.3 6 sand up visayas, miagao 0.035 0.0235 0.038 journal of science learning article doi: 10.17509/jsl.v6i2.54618 233 j.sci.learn.2023.6(2).222-235 information and share it with others with technology instruction like an online module that motivates learners. in this study, conventional learners are responsible for building connections between the knowledge they acquire and the situation in which they apply this knowledge; however, some learners do not use much of the knowledge they have gained from learning experiences but through experimental methods that help them remember. in table 4, according to gender group statistics, females have a very satisfactory rating in reading the module that discusses landslide parameters, with an average mean of 4.47. the male participants, with a mean average of 4.23, liked to learn in the development module. female participants with a 4.17 mean average learn about landslide parameters using the development module. and same female participants are satisfied with the learning modules. in table 4, in the grade level group statistics, the developed module significantly affects to participant's learning method that primarily influences learning, allowing students to explore and find relevant information to solve problems embedded in a complex social context. this application of experimental approaches with technology instruction allowed students to associate their knowledge of solution procedures with real-world-like problem situations. 3.4 student performances on the developed module in table 5, more than 50 percent of the participants got the answer wrong during the pretest, and after they were users of the development module (table 6), students table 4 gender group statistics gender n mean std. deviation s.e. mean how do you feel overall about the module? male 31 1.19 .40 .07 female 19 4.47 .70 .16 do you like learning landslide parameters from the develop module? male 31 4.23 .67 .12 female 19 1.42 .61 .14 how much do you learn about landslide parameters using the module? male 31 1.26 .51 .09 female 19 4.16 .37 .09 how helpful have your teachers been while you've been studying landslide parameters through module? male 31 4.19 .40 .07 female 19 3.74 .99 .23 are you satisfied with the modules you're using for discussing landslide parameters? male 31 3.68 .83 .15 female 19 1.21 .42 .10 table 5 grade level group statistics sum of squares df mean square f sig. how do you feel overall about the module? between groups 6.35 1 6.35 18.43 .000 within groups 16.53 48 .34 total 22.88 49 do you like learning landslide parameters from the develop module? between groups .02 1 .02 .06 .814 within groups 14.86 48 .31 total 14.88 49 how much do you learn about landslide parameters using the module? between groups 1.38 1 1.38 11.08 .002 within groups 6.00 48 .12 total 7.38 49 how helpful have your teachers been while you've been studying landslide parameters through module? between groups 12.62 1 12.62 23.41 .000 within groups 25.88 48 .54 total 38.50 49 are you satisfied with the modules you're using for discussing landslide parameters? between groups .01 1 .01 .09 .771 within groups 7.99 48 .17 total 8.00 49 journal of science learning article doi: 10.17509/jsl.v6i2.54618 234 j.sci.learn.2023.6(2).222-235 increased their scores by more than 50 percent. technology and experimental methods can help learners construct knowledge and deduce meanings by providing rich learning environments. as one of the most widely used and easily accessible technologies today, ftir spectroscopy has many intrinsic capabilities that can complement the methods of instruction. the interactivity of the module is to engage mental processes and enhance performance and productivity. the learner using the module actively participated in the teaching and learning process. with the abilities of advanced technology, the module can enable a case-based approach wherein learners encounter real-life problem cases and solve them using appropriate knowledge or concepts. research and development of technology and experimental method have evolved recently to provide learners with environments that enhance the application of knowledge. 4. conclusion the present study generates the following conclusions; 1. the ft infrared spectroscopy analysis of the contents of soil (clay) and organic matter directly affect soil water-holding capacity due to the larger surface area of the clay soil. 2. the smaller particle sizes, such as in the case of clay, have a larger surface area. the larger the surface area, table 6 pretest frequencies of student value label value frequency percent valid percent cum percent 1 the movement of earthy materials from a higher region to a lower region due to the gravitational pull wrong 0 34 68.00 68.00 68.00 correct 1 16 32.00 32.00 100.00 total 50 100.0 100.0 2 downhill movement of the earth is mainly caused by wrong 0 34 68.00 68.00 68.00 correct 1 16 32.00 32.00 100.00 total 50 100.0 100.0 3 movement of heavy materials on the unstable sloppy region creates wrong 0 41 82.00 82.00 82.00 correct 1 9 18.00 18.00 100.00 total 50 100.0 100.0 4 device that detects landslide wrong 0 25 50.00 50.00 50.00 correct 1 25 50.00 50.00 100.00 total 50 100.0 100.0 5 to prevent landslide, improving soil cultivation prevent the effect of wrong 0 43 86.00 86.00 86.00 correct 1 7 14.00 14.00 100.00 total 50 100.0 100.0 table 7 posttest frequencies of student value label value frequency percent valid percent cum percent 1 downhill movement of the earth is mainly caused by wrong 0 25 50.00 50.00 50.00 correct 1 25 50.00 50.00 100.00 total 50 100.0 100.0 2 movement of heavy materials on the unstable sloppy region creates wrong 0 20 40.00 40.00 40.00 correct 1 30 60.00 60.00 100.00 total 50 100.0 100.0 3 device that detects landslide wrong 0 11 22.00 22.00 22.00 correct 1 39 78.00 78.00 100.00 total 50 100.0 100.0 4 to prevent landslide, improving soil cultivation prevent the effect of wrong 0 18 36.00 36.00 36.00 correct 1 32 64.00 64.00 100.00 total 50 100.0 100.0 5 the movement of earthy materials from a higher region to a lower region due to the gravitational pull wrong 0 17 34.00 34.00 34.00 correct 1 33 66.00 66.00 100.00 total 50 100.0 100.0 journal of science learning article doi: 10.17509/jsl.v6i2.54618 235 j.sci.learn.2023.6(2).222-235 the easier the soil to hold onto water leading to a higher water-holding capacity. 3. soil from a landslide-prone area has a lesser settling time except for the sand that settles fastest due to larger masses. imhoff cone is not enough instrument to determine the settling ability of soil samples that has different particle size distributions. settling relationship with other factors in landslide monitoring is difficult to identify because of the inability to account for the mean aggregate size of the soil sample or the sample's aggregate stability due to the limited functions of the instrument used. these properties may vary from soil to soil and change with time and exposure to various environmental influences. accurate data through any experimental approach leads to teachers' mastery of sciences. experimentation in an investigative and communicative way provides the teacher with a different way of teaching but stimulates students to study and construct their knowledge. the experimental approach had the potential to the formation of students ability who understand and value science and its contributions to daily routines. data results will be shared with the community of teachers handling earth science and geology classes. acknowledgements the author thanks the university of the philippines visayas and the following; (1) the division of physical sciences and mathematics for the physics instrument and laboratory room, (2) staff and chairman of the department of chemistry for the help with the execution of the ftir soil analysis and the instrument used in the study. references alamanis, n., papageorgiou, g., xafoulis, n., & chouliaras, i. (2020). effects of landslides and soil settlements on the built environment: a metaanalysis. 7. 6-15. babaeian, e., m. homaee, c. montzka, h. vereecken, and a.a. norouzi. (2015). towards retrieving soil hydraulic properties by hyperspectral remote sensing. vadoze zone j. 14(3),doi: 10.2136/ vzj2014.07.0080. dariagan, j.d., atando, r.b. & asis, j.l.b. (2021) disaster preparedness of local governments in panay island, philippines. nat hazards 105, 1923–1944. https://doi.org/10.1007/s11069-02004383-0. guerrero, c., viscarra r.r.a, & mouazen, a. m. (2010). diffuse reflectance spectroscopy in soil science and land resource assessment. geoderma. 158: 1-2. hassani, a., bahrami, h., noroozi, a., & oustan, s. (2014). visible-near infrared reflectance spectroscopy for assessment of soil properties in gypseous and calcareous soils. journal of watershed engineering and management, 6(2), 125138. iurian, a. r, cosma, c. (2014). a practical experimental approach for the determination of gamma-emitting radionuclides in environmental samples. nucl. is unknown and the path forwards. plant and soil, 375, 1e19. jaggi, n., & vij, d. (2006). fourier transform infrared spectroscopy. in handbook of applied solid state spectroscopy. boston: springer, 411450. luino, f., de graff, j., biddoccu, m., faccini, f., freppaz, m., roccati, a., ungaro, f., amico, m., turconi, l. (2022). the role of soil type in triggering shallow landslides in the alps (lombardy, northern italy). land 2022, 11, 1125. https://doi.org/ 10.3390/land11081125. margenot a.j., calderón f.j., goyne k.w., mukome f.n.d & parikh s.j. (2017). ir spectroscopy, soil analysis applications. in: lindon, j.c., tranter, g.e., and koppenaal, d.w. (eds.) the encyclopedia of spectroscopy and spectrometry, 3(2), 448-454. mohamed, e., saleh, a., belal, a., & gad, a. (2018). application of near-infrared reflectance for quantitative assessment of soil properties. the egyptian journal of remote sensing and space science, 21(1), 1-14. https://doi.org/10.1016/j.ejrs.2017.02.001. nandiyanto, a. b.d., oktiani,r., & ragadhita, r. ( 2019). how to read and interpret ftir spectroscope of organic material. journal of science & technology, 4(1), 97-118. doi: http://dx.doi.org/10.17509/ijost.v4i1.15806. pajalic, s., peranic, j., maksimovic, s., ceh, n., jagodnik, v., & arbanas, ž. (2021). monitoring and data analysis in small-scale landslide physical model. applied sciences, 11, 5040. https://doi.org/10.3390/ app11115040. reichert, j. m., albuquerque, j. a., kaiser, d. r., reinert, d. j., urach, f. l., & carlesso, r. (2009). estimation of water retention and availability in soils of rio grande do sul. revista brasileira de ciência do solo, 33, 1547-1560. rossel, r. v., walvoort, d. j. j., mcbratney, a. b., janik, l. j., & skjemstad, j. o. (2006). visible, near infrared, mid infrared or combined diffuse reflectance spectroscopy for simultaneous assessment of various soil properties. geoderma, 131(1-2), 59-75. saxena, n., kumar,s., & mandal, a.( 2018). adsorption characteristics and kinetics of synthesized anionic surfactant and polymeric surfactant on sand surface for application in enhanced oil recovery. asia-pacific journal of chemical engineering. 13(4), 1-14. https://doi.org/10.1002/apj.2211 soares, b. c., de campos, m. e. c., thomaz, j. r., da cruz pereira, g., & roehrs, r. (2016). the importance of experimentation in the teaching of sciences to elementary school. revista monografias ambientais, 1-17. doi:10.5902/2236130827003. sojka,r. e., carter, d. l., & brown, m. j. ( 1992). imhoff cone determination of sediment in irrigation runoff. soil science society of american journal, 56:884-890. soriano-disla, j. m., janik, l. j., viscarra rossel, r. a., macdonald, l. m., & mclaughlin, m. j. (2014). the performance of visible, near, and mid-infrared reflectance spectroscopy for prediction of soil physical, chemical, and biological properties. applied spectroscopy reviews, 49(2), 139-186. http://dx.doi.org/10.1080/ 05704928.2013.811081. summers, d., lewis, m., ostendorf, b., & chittleborough, d. (2011). visible near-infrared reflectance spectroscopy as a predictive indicator of soil properties. ecological indicators, 11(1), 123-131 https://doi.org/10.1007/s11069-020-04383-0 https://doi.org/10.1007/s11069-020-04383-0 https://doi.org/10.1016/j.ejrs.2017.02.001 http://dx.doi.org/10.17509/ijost.v4i1.15806 https://doi.org/10.1002/apj.2211 a © 2021 indonesian society for science educator 309 j.sci.learn.2021.4(4).309-315 received : 17 march 2021 revised : 22 april 2021 published : 1 august 2021 adapted primary literature in authentic science: students’ perception toni hidayat1*, nuryani rustaman1, parsaoran siahaan1 1sekolah pascasarjana, universitas pendidikan indonesia, bandung, indonesia *corresponding author. tonihidayatbinhasan@gmail.com abstract science textbooks only present scientific facts that cause the textbooks to seem not to promote scientific reasoning. science learning also becomes not in line with the nature of science. then, science learning requires applying authentic science as an approach. adapted primary literature (apl) is assumed to be able to achieve this objective. therefore, this study aims to examine how apl is a source of science learning in actualizing authentic science. this research using a quasi-experimental method, 81 students from two 7th grade groups were involved as participants. one group analyzed apl using a jigsaw, another using a numbered heads together (nht). a perception questionnaire of apl was given. quantitative analysis was only performed on data from 44 students due to other students’ participation that is lacked by a pandemic. the result showed both classes possess a high perception of apl. a mean difference test also showed that there were no differences in perception between the two classes. these indicated that analyzing apl was perceived as an authentic science. science teachers can utilize apl as an alternative way to actualize authentic science with a relatively more straightforward model, strategy, and method than commonly applied. keywords adapted primary literature, authentic science, students’ perception 1. introduction science textbooks only present various scientific facts without inviting students to think scientifically (ariely, livnat & yarden, 2019a; rusilowati, 2014; vesterinen, aksela & lavonen, 2013; wahyu & markos, 2016). when learning science, students do not read texts that reflect authentic scientific reasoning (ariely, livnat & yarden, 2019a, 2019b). reading science texts does not mean doing science if the reading activities only obtain facts (cervetti & pearson, 2012). this causes the difference between science in school and real science. meanwhile, science in school must be in line with the nature of science (mccoach, gable & madura, 2013; tala & vesterinen, 2015; upahi, ramnarain, & ishola, 2018). a means to actualize science learning that reflects natural science is an authentic science. authentic science is a science learning approach that provides opportunities for students to think (crawford, 2013; labouta et al., 2018), work (anker-hansen & andrée, 2019; chapman & feldman, 2017; crawford, 2013; koomen, rodriguez, hoffman, petersen, & oberhauser, 2018; machluf, gelbart, ben-dor & yarden, 2017) and feel like scientists. authentic science is also an approach to teach science based on the inquiry process (anker-hansen & andrée, 2019; broder et al., 2019; hellgren & lindberg, 2017; koomen, rodriguez, hoffman, petersen, & oberhauser, 2018; labouta et al., 2018; olitsky, becker, jayo, vinogradov, & montcalmo, 2020; tsybulsky, 2019). therefore, science teaching and learning will be studentcentered (anker-hansen & andrée, 2019; buxton, 2006; labouta et al., 2018; rivera maulucci, brown, grey, & sullivan, 2014; ward et al., 2016) and also require students’ independence (koomen, rodriguez, hoffman, petersen, & oberhauser, 2018; labouta et al., 2018; machluf, gelbart, ben-dor, & yarden, 2017; olitsky, becker, jayo, vinogradov & montcalmo, 2020; rivera maulucci, brown, grey, & sullivan, 2014; ward et al., 2016). authentic science can be actualized with researchbased learning activities (chapman, 2013; chapman & feldman, 2017; rivera maulucci, brown, grey, & sullivan, 2014), in the laboratory (chapman & feldman, 2017; labouta et al., 2018; munn, knuth, van horne, shouse & levias, 2017; olitsky, becker, jayo, vinogradov, & montcalmo, 2020; rivera maulucci, brown, grey, & sullivan, 2014) or out of school (hellgren & lindberg, 2017; koomen, rodriguez, hoffman, petersen, & journal of science learning article doi: 10.17509/jsl.v4i4.32731 310 j.sci.learn.2021.4(4).309-315 oberhauser, 2018; ward et al., 2016). authentic science can also be actualized by inviting scientists into science classes (tsybulsky, 2019), inviting students to have tours in universities and the laboratories (chapman & feldman, 2017; olitsky, becker, jayo, vinogradov, & montcalmo, 2020; tsybulsky, 2019), conducting science exhibitions (koomen, rodriguez, hoffman, petersen, & oberhauser, 2018; rivera maulucci et al., 2014), and visiting museums (achiam, simony & lindow, 2016). nevertheless, the learning models, strategies, and methods mentioned above seem to require plenty of teachers' and students’ effort to actualize authentic science. science teachers need a more straightforward model, strategies, or methods. authentic science imitates what scientists do. the scientists employ about 23% of their work time to read (phillips & norris, 2009). of course, the intended reading is primary scientific literature (psl) to support their work as scientists. however, scientific language in psl is complex for students to understand (ariely, livnat & yarden, 2019a). then, several obstacles allegedly arise when science teachers assign students to read psl. the teachers possibly need to use an adapted version of psl, namely adapted primary literature (apl). the genre, structure, writing content, and science presentation in apl are maintained equal to psl (ariely, livnat & yarden, 2019a; yarden, 2009; yarden, norris, & phillips, 2015a). the differences are in the objectives, authors, and readers target (ariely, livnat & yarden, 2019a; yarden, 2009; yarden, norris, & phillips, 2015a). even though apl is an adaptation, utilizing apl in science learning can actualize authentic science. moreover, science teachers in middle school possess a high positive perception toward apl (hidayat, rustaman & shidiq, 2020). therefore, this study has the first research question: can apl actualize authentic science? if the result shows what is assumed, authentic science can be actualized by applying a learning model, strategy, and method that is relatively more straightforward. on the other hand, middle school students experience a transition in reading skills, from learning-to-read to reading-to-learn. several students encounter barriers in this stage (fang, 2006). in line with that, indonesian middle school students' reading skills and science achievement are still below the average of the oecd country (oecd, 2019). science learning that requires them to read allegedly reduces their involvement in the learning activity. furthermore, students are assigned to read scientific texts. even though students read the adapted version of the psl, researchers suspected that these obstacles may still arise. cooperative learning models should be applied to allow students to assist each other (slavin, 2012). this is expected to boost their involvement in the learning activity because cooperative learning may be better than traditional techniques when the learning outcome is at a high cognitive level (slavin, 1980), such as analyzing apl. jigsaw and numbered heads together (nht) were determined as the learning models in this study. these two learning models are believed to be widely known among teachers. consequently, science teachers can be relatively easy to implement the result of this study into their teaching and learning activity. moreover, the jigsaw can be primarily used in learning that reading text is an essential part of the learning activity (li & lam, 2013). the jigsaw can also improve students reading comprehension (nurbianta & dahlia, 2018), enhance student motivation to learn science (tarhan, ayyıldız, ogunc & sesen, 2013), enhance student involvement in a learning activity (goolsarran, hamo, & lu, 2020), and improve student learning outcomes (doymus, 2008; (goolsarran, hamo & lu, 2020); tarhan, ayyıldız, ogunc & sesen, 2013; yoruk, 2016). the nht can improve students' reading skills, enhance student participation in learning, particularly in the question and answer session (purnomo, 2012), and improve student learning outcomes (leasa & corebima, 2017; mulyana, hanifah, & jayadinata, 2016). the researchers consider that learning science using apl will be more effective if performed in these two types of cooperative learning. a question arises between the two, which is more effective. therefore, this study has the second research question: which model is considered more effective in actualizing authentic science, jigsaw, or nht? finding the answer to this second research question can aid the science teachers to determine which learning models they can apply in their learning to be more effective in actualizing authentic science. in addition, the context of this study has to determine carefully. environmental pollution is considered an appropriate topic in this study. they consider that environmental pollution has occurred on soil, water, and the air around students’ live areas. the pollution seems to be getting worse. rice fields are polluted by heavy metals from untreated textile factory waste (komarawidjaja, 2017). water in the dug well was contaminated by coliform bacteria. the bacteria are thought to have come from animal cages, septic tanks, plantations, and fish ponds (prilia & kamil, 2011). the level of carbon monoxide in the blood of people who work in places with air pollution sources correlates with the levels of carbon monoxide in the air of that place (fitriana & oginawati, 2012). by raising the topic of problems that occur in the students' lives, it is expected to increase the possibility of actualizing authentic science. 2. method 2.1. research method this study used a quantitative approach and a quasiexperimental method (fraenkel, wallen, & hyun, 2012). students learn about environmental pollution by reading, analyzing, and discussing apl. after that, students were given a posttest to ask their perceptions about the learning conducted in actualizing authentic science. since apl is journal of science learning article doi: 10.17509/jsl.v4i4.32731 311 j.sci.learn.2021.4(4).309-315 allegedly not widely utilized by science teachers in indonesia, particularly in bandung. to capture student perceptions of apl, students require to be acquainted with apl by using it in their learning activity. 2.2 sample participants were determined according to the availability of school to be a location of this research. a total of 81 seventh-grade students of a public middle school in bandung were invited to this study as participants. there was no randomization of students in this study. student groups are the same as existing classes. 2.3 treatment two groups of students were involved in this study. one group was designated as experimental class 1, and another group was designated as experimental class 2. each group learned about environmental pollution in three meetings. one apl was used as a learning source at each meeting. students in each group were assigned to analyze the apl, beginning from the background, methods, results, discussion to conclusions. the apl analyzed by students were three apl with soil, water, and air pollution that occur in bandung. researchers adapted the apl from psl published by a department of environmental engineering journal from a state university in bandung. the three apl can be viewed in associated content sections 1 to 3. a worksheet was used in this learning as a guide for students in analyzing the apl. the experimental class 1 and 2 applied a jigsaw and an nht as the learning models, respectively. when this research was conducted, a pandemic occurred in the form of the coronavirus outbreak that caused covid-19. this condition caused all learning activities in schools to shift to the home and conducted online. therefore, providing treatment to groups of students in this research was also conducted online. the media used in this learning was whatsapp group (wag). this media was determined for three considerations. first, the school science teacher assessed that the students’ gadgets are not yet eligible to use video-based streaming media. second, wag was believed by the teacher and researchers as the most widely used communication media by students. third, wag can send messages in various forms of media, namely texts, sounds, images, videos, and files. thus, wag was considered the most suitable media to be used in this learning. 2.4 data collection after the treatment was conducted, the two groups of students were requested to fill out a questionnaire via google form. we told students to fill out the questionnaire honestly and according to what they experienced. we also told students that filling out the questionnaire had no impact on their science scores. the questionnaire contained 14 statements about students' perceptions of apl in actualizing authentic science. the statements in the questionnaire were in the form of positive and negative statements. students can choose a scale that matches their perception in a positive statement, scale 1 for strongly disagree and scale 4 for strongly agree. in a negative statement, the opposite applies. table 1 shows the questionnaire rubrics. this questionnaire statement's validity and reliability test item were performed at the significance level of 0.05 two-sided and rtable of 0.254. all statements have a p-value greater than rtable. then all statements are declared valid and reliable. in associated content sections 4 and 5, a complete questionnaire, validity, and reliability test are provided, respectively. data was collected from the end of april to the beginning of may 2020. from a total of 81 students, only 44 students were declared to be active in the research process from the first treatment to completing the questionnaire. therefore, at the data analysis stage, only data from 44 students were processed. participant data is shown in table 2. 2.5 data analysis to obtain how the level of students' perceptions of apl in actualizing authentic science, the total scores of each student were converted into percentages. the mean percentage of each group of students is calculated and interpreted using the criteria in table 3 (sugiyono, 2010). to examine the difference in perception between the two groups of students, a comparison test of the mean table 1 questionnaire rubric of students’ perceptions of apl in actualizing authentic science rubric of statements example statement number of items students think like a scientist when reading apl when reading apl, several questions arose in my mind about the topic in the apl 5 students work like a scientist when reading apl i made a writing based on data/information in apl that i read 4 students feel like a scientist when reading apl after reading apl, i was interested in conducting research 5 table 2 participants data class and gender all participants activeparticipants jigsaw class male female nht class male female 20 21 22 18 7 10 14 13 total 81 44 journal of science learning article doi: 10.17509/jsl.v4i4.32731 312 j.sci.learn.2021.4(4).309-315 percentage of each group was conducted. because the data came from two different groups and had non-normal distributed data, the mann-whitney test has been conducted (minium, king, & bear, 1993). this test can provide answers to which learning model is more effective in actualizing authentic science by using apl. furthermore, the effect size is also calculated to contrast the discrepancy (minium, king, & bear, 1993; thalheimer & cook, 2002). 3. result and discussion learning using internet media is growing and becoming a trend nowadays (palvia et al., 2018), particularly amid a pandemic. face-to-face teaching and learning is considered a threat to students' and teachers' health (murphy, 2020). but in its implementation, there are still obstacles in the form of limited student access to the internet (assareh & bidokht, 2011). for example, students have a poor internet connection (pitaloka, anggraini, kurniawan, erlina & jaya, 2020). therefore, as many as 40% of students experienced obstacles in accessing technology used in online learning (shahmoradi, changizi, mehraeen, bashiri, jannat & hosseini, 2018). this study also encountered similar obstacles. as a result, about 45% of students in this study were declared less active, so the number of active participants was only 44. even though student involvement in learning can be improved by jigsaw (goolsarran, hamo & lu, 2020) or nht (purnomo, 2012), the learning models applied in this study were not the two cooperative learnings; the number of active students could be possibly lower. the first research question is how students' perceptions of apl in actualizing exact science. figure 1 shows the average percentage of student's perceptions of the two indicators, and it appears that students in both classes have high perceptions of apl. all indicators also show a high perception. this can be interpreted that the use of apl in both classes has succeeded in actualizing authentic science. based on data from 44 students, the use of apl as a source of science learning has been able to actualize authentic science. assigning students to analyze apl means asking students to read the writings of scientists. one strategy for achieving an authentic science is to posit students in the perspective of scientists, namely by reading their writings (larison, 2018). although in the form of an adaptation from scientists' writings, apl can still allow students to conduct authentic scientific activities such as those of scientists (ford, 2009). reading apl is also considered an authentic scientific practice (yarden, norris, & phillips, 2015a) to improve the learning process of science based on inquiry (ford, 2009; yarden, norris, & phillips, 2015b). the second research question is whether there is a difference in perception of apl between students who analyze apl with jigsaw and students who analyze apl with nht. table 4 shows the descriptive statistics of the two classes regarding their perceptions of apl. the mean percentage of perception in the jigsaw class is higher than in the nht class. inferential analysis needs to be performed to prove the significance of the difference. all test is conducted at a significance level of 0.05. the homogeneity test for both classes produces a p-value of 0.112. this means that data from both classes are homogeneous. the normality test for data from the jigsaw class and nht class results in p-values of 0,567 and 0,038, respectively. this means that data from the jigsaw class has a normal distribution, while data from the nht class does not have a normal distribution. a non-parametric test (mann-whitney) was used to compare perceptions between the two classes. this is because the data from the nht class are generally not distributed (minium, king, & bear, 1993). the mann-whitney test produces a p-value of 0.054. this means that there is no significant difference between the perception of apl of jigsaw and nht class students in actualizing authentic science. table 3 interpretation guideline of perception percentage percentage of perception category 0 – 19 20 – 39 40 – 59 60 – 79 80 – 100 very low low moderate high very high table 2 descriptive statistics on the percentage of students' perceptions of apl descriptive statistics class jigsaw class nht number of participant 17 27 mean 72.79 67.26 standard deviation 10.82 7.18 minimum score 51.79 55.38 maximum score 92.86 82.14 figure 1 percentage of students' perceptions on each indicator 72.94 68.38 76.18 66.48 70.60 65.37 60 65 70 75 80 students think like sceintists students work like sceintists students feel like sceintists p e rc e n ta g e o f s tu d e n ts ' p e rc e p ti o n o f a p l jigsaw class nht class journal of science learning article doi: 10.17509/jsl.v4i4.32731 313 j.sci.learn.2021.4(4).309-315 scientists can conduct research individually or in groups. when the research is conducted in groups, there is a distribution of tasks. the distribution can be different (fraenkel, wallen, & hyun, 2012). for instance, in conducting a literature review, this stage can be performed by one or several researchers (s). when several researchers perform a literature review, the distribution of tasks may be similar to the distribution of tasks when students learn with jigsaw or nht. jigsaw is a cooperative learning model (goolsarran, hamo & lu, 2020), and reciprocally nht (baker, 2013). thus, both with jigsaw and nht, the learning activities are similar to what scientists do when analyzing psl. therefore, students from the two experimental classes perceived equal learning activities. both of them perceived that what they did was the work of scientists. on the other hand, the value of effect size is 0.647. this value can be interpreted as a medium effect. commonly, the effect size is calculated from an average score of the experiment and control class. nevertheless, both classes in this study are considered practical classes. since the average perception percentage in the jigsaw class is higher than nht class, it can be said that jigsaw is more effective in actualizing authentic science even though the inferential statistic shows that both classes are the same. applying jigsaw is possibly a more appropriate option for science teachers to actualizing authentic science by utilizing apl. 4. conclusion this study has confirmed that apl as a source of science learning can actualize authentic science. a significant difference did not appear between students' perceptions of apl in the jigsaw and nht classes. the students' perception of apl in both classes was also in the high category of all measured indicators of perception. thus, apl can be a means for science teachers to actualize authentic science with a more straightforward strategy, mainly if other learning strategies are almost impossible to implement because of limited resources. authentic science is conducting experiments, observations, and interacting with scientists, but this study shows that reading, analyzing, and discussing apl is also considered authentic science. these results can also be a background for strengthening the use of apl as a source of science learning. we suggested the promotion of apl to science teachers in bandung notably. thus, science learning in bandung is expected to be in line with the nature of science. this research is recognized to have several limitations. one of them is the number of participants. this was caused by this learning used online media, while online learning facilities owned by students were still not sufficient. therefore, students encountered obstacles to be present and active in learning. further research is suggested to conduct face-to-face learning and apply other cooperative learning models. the demographic factors of students can also be traced to examine their effect on their perception. acknowledgment i would like to thank the indonesia endowment fund for education (lpdp) for funding my master's study. i am also grateful to mr. sukardi and his students who have supported and were willing to become participants in this research references achiam, m., simony, l., & lindow, b. e. k. (2016). objects prompt authentic scientific activities among learners in a museum programme. international journal of science education, 38(6), 1012– 1035. https://doi.org/10.1080/09500693.2016.1178869 anker-hansen, j., & andrée, m. (2019). in pursuit of authenticity in science education. nordina, 15(1), 498–510. ariely, m., livnat, z., & yarden, a. (2019a). analyzing the language of an adapted primary literature article. science & education, 28(1), 63–85. https://doi.org/10.1007/s11191-019-00033-5 ariely, m., livnat, z., & yarden, a. (2019b). analyzing the language of an adapted primary literature article. science & education, 28(1–2), 63–85. https://doi.org/10.1007/s11191-019-00033-5 assareh, a., & bidokht, m. h. (2011). barriers to e-teaching and elearning. procedia computer science, 3, 791795.https://doi.org/https://doi.org/10.1016/j.procs.2010.12.129 baker, d. p. (2013). the effects of implementing the cooperative learning structure, numbered heads together, in chemistry classes at a rural, low performing high school. louisiana state university. broder, e. d., guilbert, k. e., tinghitella, r. m., murphy, s. m., ghalambor, c. k., & angeloni, l. m. (2019). authentic science with dissemination increases self-efficacy of middle school students. integrative and comparative biology. https://doi.org/10.1093/icb/icz140 buxton, c. a. (2006). creating contextually authentic science in a “lowperforming” urban elementary school. journal of research in science teaching, 43(7), 695–721. https://doi.org/10.1002/tea.20105 cervetti, g., & pearson, p. (2012). reading, writing, and thinking like a scientist. journal of adolescent & adult literacy, 55(7), 580–586. https://doi.org/10.1002/jaal.00069 chapman, a. (2013). an investigation of the effects of authentic science experiences among urban high school students [university of south florida]. http://scholarcommons.usf.edu/etd/4653 chapman, a., & feldman, a. (2017). cultivation of science identity through authentic science in an urban high school classroom. cultural studies of science education, 12(2), 469–491. https://doi.org/10.1007/s11422-015-9723-3 crawford, b. (2013). authentic science. in r. gunstone (ed.), encyclopedia of science education (pp. 1–3). springer netherlands. https://doi.org/10.1007/978-94-007-6165-0_144-2 doymus, k. (2008). teaching chemical bonding through jigsaw cooperative learning. research in science & technological education, 26(1), 47–57. https://doi.org/10.1080/02635140701847470 fang, z. (2006). the language demands of science reading in middle school. international journal of science education, 28(5), 491–520. fitriana, d., & oginawati, k. (2012). studi paparan gas karbon monoksida dan dampaknya terhadap pekerja di terminal cicaheum bandung. jurnal teknik lingkungan, 18(1), 21–29. https://doi.org/10.5614%2fjtl.2012.18.1.3 ford, d. j. (2009). promises and challenges for the use of adapted primary literature in science curricula: commentary. research in science education, 39(3), 385–390. https://doi.org/10.1007/s11165008-9115-8 fraenkel, j. r., wallen, n. e., & hyun, h. h. (2012). how to design and evaluate research in education. mcgraw-hill. journal of science learning article doi: 10.17509/jsl.v4i4.32731 314 j.sci.learn.2021.4(4).309-315 goolsarran, n., hamo, c. e., & lu, w.-h. (2020). using the jigsaw technique to teach patient safety. medical education online, 25(1), 1710325. https://doi.org/10.1080/10872981.2019.1710325 hellgren, j. m., & lindberg, s. (2017). motivating students with authentic science experiences: changes in motivation for school science. research in science and technological education, 35(4), 409–426. https://doi.org/10.1080/02635143.2017.1322572 hidayat, t., rustaman, n., & shidiq, a. s. (2020). science teachers’ perception of apl: what and how. proceedings of the 2020 8th international conference on information and education technology, 169– 173. https://doi.org/10.1145/3395245.3396423 komarawidjaja, w. (2017). paparan limbah cair industri mengandung logam berat pada lahan sawah di desa jelegong, kecamatan rancaekek, kabupaten bandung. jurnal teknik lingkungan, 18(2), 173–181. koomen, m. h., rodriguez, e., hoffman, a., petersen, c., & oberhauser, k. (2018). authentic science with citizen science and student-driven science fair projects. science education, 102(3), 593– 644. https://doi.org/10.1002/sce.21335 labouta, h. i., kenny, n. a., li, r., anikovskiy, m., reid, l., & cramb, d. t. (2018). learning science by doing science: an authentic science process-learning model in postsecondary education. international journal of science education, 40(12), 1476–1492. https://doi.org/10.1080/09500693.2018.1484966 larison, k. d. (2018). taking the scientist’s perspective. science & education, 27(1), 133–157. https://doi.org/10.1007/s11191-0189957-z leasa, m., & corebima, a. d. (2017). the effect of numbered heads together (nht) cooperative learning model on the cognitive achievement of students with different academic ability. journal of physics: conference series, 795(012071), 1–9. https://doi.org/doi:10.1088/1742-6596/795/1/012071 li, m. p., & lam, b. h. (2013). cooperative learning. the hong kong institute of education, 1–33. machluf, y., gelbart, h., ben-dor, s., & yarden, a. (2017). making authentic science accessible-the benefits and challenges of integrating bioinformatics into a high-school science curriculum. briefings in bioinformatics, 18(1), 145–159. https://doi.org/10.1093/bib/bbv113 mccoach, d. b., gable, r. k., & madura, j. p. (2013). affective characteristics in school and corporate environments: their conceptual definitions bt instrument development in the affective domain: school and corporate applications (d. b. mccoach, r. k. gable, & j. p. madura (eds.); pp. 1–32). springer new york. https://doi.org/10.1007/978-1-4614-7135-6_1 minium, e. d., king, b. m., & bear, g. (1993). statistical reasoning in psychology and education (third edit). john wiley & sons. mulyana, m. a., hanifah, n., & jayadinata, a. k. (2016). penerapan model kooperatif tipe numbered heads together (nht) untuk meningkatkan hasil belajar siswa pada materi kenampakan alam dan sosial budaya. jurnal pena ilmiah, 1(1), 331–340. munn, m., knuth, r., van horne, k., shouse, a. w., & levias, s. (2017). how do you like your science, wet or dry? how two lab experiences influence student understanding of science concepts and perceptions of authentic scientific practice. cbe life sciences education, 16(2), ar39. https://doi.org/10.1187/cbe.16-04-0158 murphy, m. p. a. (2020). covid-19 and emergency elearning: consequences of the securitization of higher education for postpandemic pedagogy. contemporary security policy, 1–14. https://doi.org/10.1080/13523260.2020.1761749 nurbianta, n., & dahlia, h. (2018). the effectiveness of jigsaw method in improving students reading comprehension. eternal (english teaching journal), 9(1). oecd. (2019). pisa 2018 results (volume i). oecd. https://doi.org/10.1787/5f07c754-en olitsky, s., becker, e. a., jayo, i., vinogradov, p., & montcalmo, j. (2018). constructing “authentic” science: results from a university/high school collaboration integrating digital storytelling and social networking. research in science education, 1– 24. https://doi.org/10.1007/s11165-018-9699-6 palvia, s., aeron, p., gupta, p., mahapatra, d., parida, r., rosner, r., & sindhi, s. (2018). online education: worldwide status, challenges, trends, and implications. journal of global information technology management, 21(4), 233–241. https://doi.org/10.1080/1097198x.2018.1542262 phillips, l. m., & norris, s. p. (2009). bridging the gap between the language of science and the language of school science through the use of adapted primary literature. research in science education, 39(3), 313–319. https://doi.org/10.1007/s11165-008-9111-z pitaloka, n. l., anggraini, h. w., kurniawan, d., erlina, e., & jaya, h. p. (2020). blended learning in a reading course: undergraduate efl students’ perceptions and experiences. irje |indonesian research journal in education|, 4(1), 43–57. https://onlinejournal.unja.ac.id/irje/article/view/8790 prilia, d., & kamil, i. m. (2011). penentuan kualitas air tanah dangkal berdasarkan parameter mikrobiologi (studi kasus: kecamatan ujungberung, kota bandung). jurnal teknik lingkungan, 17(2), 11– 21. purnomo, s. (2012). improving the students’ reading comprehension through numbered heads together technique. journal on english as a foreign language, 2(2), 37–44. rivera maulucci, m. s., brown, b. a., grey, s. t., & sullivan, s. (2014). urban middle school students’ reflections on authentic science inquiry. journal of research in science teaching, 51(9), 1119–1149. https://doi.org/10.1002/tea.21167 rusilowati, a. (2014). analisis buku ajar ipa yang digunakan di semarang berdasarkan muatan literasi sains. shahmoradi, l., changizi, v., mehraeen, e., bashiri, a., jannat, b., & hosseini, m. (2018). the challenges of e-learning system: higher educational institutions perspective. journal of education and health promotion, 7, 116. https://doi.org/10.4103/jehp.jehp_39_18 slavin, r. e. (1980). cooperative learning. review of educational research, 50(2), 315–342. https://doi.org/10.3102/00346543050002315 slavin, r. e. (2012). classroom applications of cooperative learning. in apa educational psychology handbook, vol 3: application to learning and teaching. (pp. 359–378). american psychological association. https://doi.org/10.1037/13275-014 sugiyono. (2010). metode penelitian pendidikan: pendekatan kuantitatif, kualitatif, dan r&d. penerbit alfabeta. tala, s., & vesterinen, v.-m. (2015). nature of science contextualized: studying nature of science with scientists. science & education, 24(4), 435–457. https://doi.org/10.1007/s11191-014-9738-2 tarhan, l., ayyıldız, y., ogunc, a., & sesen, b. a. (2013). a jigsaw cooperative learning application in elementary science and technology lessons: physical and chemical changes. research in science & technological education, 31(2), 184–203. https://doi.org/10.1080/02635143.2013.811404 thalheimer, w., & cook, s. (2002). how to calculate effect sizes from published research: a simplified methodology. work-learning research, 1, 1–9. tsybulsky, d. (2019). students meet authentic science: the valence and foci of experiences reported by high-school biology students regarding their participation in a science outreach programme. international journal of science education, 41(5), 567–585. https://doi.org/10.1080/09500693.2019.1570380 upahi, j. e., ramnarain, u., & ishola, i. s. (2018). the nature of science as represented in chemistry textbooks used in nigeria. research in science education. https://doi.org/10.1007/s11165-018-9734-7 vesterinen, v.-m., aksela, m., & lavonen, j. (2013). quantitative analysis of representations of nature of science in nordic upper secondary school textbooks using framework of analysis based on philosophy of chemistry. science & education, 22(7), 1839–1855. wahyu, e., & markos, s. (2016). analisis buku siswa mata pelajaran ipa kelas viii smp/mts berdasarkan kategori literasi sains. jurnal inovasi dan pembelajaran fisika, 3(2). journal of science learning article doi: 10.17509/jsl.v4i4.32731 315 j.sci.learn.2021.4(4).309-315 ward, t. j., delaloye, n., adams, e. r., ware, d., vanek, d., knuth, r., hester, c. l., marra, n. n., & holian, a. (2016). air toxics under the big sky: examining the effectiveness of authentic scientific research on high school students’ science skills and interest. international journal of science education, 38(6), 905–921. https://doi.org/10.1080/09500693.2016.1167984 yarden, a. (2009). reading scientific texts: adapting primary literature for promoting scientific literacy. research in science education, 39(3), 307–311. https://doi.org/10.1007/s11165-009-9124-2 yarden, a., norris, s. p., & phillips, l. m. (2015a). adapting primary literature for promoting scientific literacy. in a. yarden, s. p. norris, & l. m. phillips (eds.), adapted primary literature: the use of authentic scientific texts in secondary schools (pp. 15–31). springer netherlands. https://doi.org/10.1007/978-94-017-9759-7_2 yarden, a., norris, s. p., & phillips, l. m. (2015b). apl and reading in science classrooms. in a. yarden, s. p. norris, & l. m. phillips (eds.), adapted primary literature: the use of authentic scientific texts in secondary schools (pp. 59–80). springer netherlands. https://doi.org/10.1007/978-94-017-9759-7_4 yoruk, a. (2016). effect of jigsaw method on students’ chemistry laboratory achievement. international journal of educational sciences, 15(3), 377–381. https://doi.org/10.1080/09751122.2016.11890547. a © 2021 indonesian society for science educator 365 j.sci.learn.2021.4(4).365-374 received: 9 february 2021 revised: 28 april 2021 published: 18 september 2021 augmented reality application-based teaching material's effect on viscera learning through algorithmic thinking ebru turan güntepe1, necla dönmez usta1* 1faculty of education, giresun university, turkey *corresponding author. necla.donmezusta@giresun.edu.tr abstract the study aimed to examine ar-based teaching material's effect on viscera learning through algorithmic thinking by the primary school teacher candidates who are sophomores in the classroom teaching department in the spring term of the 2018-2019 academic year at a state university in the eastern black sea and selected by convenience method. viscera information form (vif) and application process and ar survey form (apsf) were used as data collection tools in the study. vif included subjects viscera in a human model and placed them in the skeletal structure. the other form, apsf, is about the application process and the material prepared with augmented reality. while the data obtained from vif were analyzed under the researcherdefined categories regarding the participants' showing each viscera in a human torso model and placing them in the skeletal structure, the data obtained from apsf was processed with content analysis. the study results revealed that ar-based teaching material makes a positive contribution to the learning of viscera through algorithmic thinking. in addition, this is determin ed as ar-based teaching material contributes to understanding the related basic concepts through algorithmic thinking. keywords algorithmic thinking, augmented reality, teaching material 1. introduction with technology taking a more incredible place in our lives day by day, the need for programs that provide solutions for the problems encountered in daily life has become increasingly essential. in search of solving an existing situation, programming can generate an answer to that problem thanks to a language that the computer can understand (van-roy & haridi, 2004). therefore, many programs need to be developed to meet the needs, and good programming education and teaching are provided. at present, where programs are so significant, programming education and teaching are also of equal importance, and therefore, programming education is tried to be given widely (perry, 2009). algorithms, which play a crucial role in programming, determine how to achieve a solution by showing the processes step by step in a problem to be solved or in a plan to be implemented. its definition as performing a task step by step in computer science and other disciplines is considered an indicator of its extensive use as a concept of algorithmic thinking (selby & woollard, 2013). that is to say, all finite and sequential operations used by individuals in their daily lives are carried out with algorithmic thinking (akçay & coklar, 2016). these sequential operations provide the individual with skills such as planning, offering different solutions to problems, dividing a task into subtasks, and algorithmic thinking (ziatdinov & musa, 2013). in this sense, algorithmic thinking is considered the primary programming step (nunes et al., 2017). algorithmic thinking, which is frequently sought in programming education, is addressed at various levels in the literature. while brown (2015) considers these levels to be understanding the problem, clearly presenting the problem, assessing the remedy, and creating an algorithm, vasconcelos (2007) describes the sub-levels of algorithmic thinking as understanding the problem, determining the applicable theoretical concepts, presenting the problem qualitatively, creating the solution strategy, and stating the tested solution. in another study, futschek (2006) classified the algorithmic thinking levels as presenting the problem clearly, analyzing, determining the basic actions required in the solution, creating the right algorithm, increasing the interpretation and efficiency by fully addressing the problem. zsakó & szlávi (2012) examined algorithmic thinking in 7 stages: understanding the problem, creating journal of science learning article doi: 10.17509/jsl.v4i4.32054 366 j.sci.learn.2021.4(4).365-374 and analyzing algorithm steps, writing-coding-modifying the algorithm, and dividing more complex algorithms into sub-problems. developing algorithms is essential before learning any programming language (nayak & vijayalakshmi, 2013; nunes et al., 2017). however, it is known that current teaching methods are insufficient for the programming and algorithm development process (cutts, connor, donaldson & michaelson, 2014; esteves, forseca, morgado & martins, 2011). accordingly, the learning process will be positively affected by taking advantage of augmented reality technologies, where learners can intervene in the real environments around them and which provide human-computer interaction experience (cai, wang & chiang, 2014), ensure access to the real world (wu, lee, chang & liang, 2013), increase the willingness and interest to learn (cai, wang & chiang, 2014; delello, 2014; tomi & rambli, 2013), ensure permanent learning, reduce misconceptions (yoon, anderson, lin & elinich, 2017), and offer student-centered approach (delello, 2014). additionally, the literature has many studies showing that ar technology positively affects the learning process of students, especially in science lessons (cai, wang & chiang, 2014; chiang, yang & hwang, 2014; hsiao, chang, lin & wang, 2016; ibáñez, di serio, villaran, & kloos, 2016; wang, duh, li, lin, & tsai, 2014). in this context, based on the positive effect of ar-based teaching material in the literature, it was thought that it would be helpful to use it in the learning of internal organs. in addition to this, since it is essential to array the process steps that need to be performed in the arrangement of internal organs (ziatdinov & musa, 2013), algorithmic thinking was taken as a basis in the scope of the study. thus, candidates are expected to analyze the problem and implement solutions within a particular order. in addition, this study offers a new approach to the literature. teaching ar-based teaching material with algorithmic thinking towards any subject or concept in the literature has not been encountered. in this respect, the study is considered valuable. against this background, the study examines arbased teaching material's effect on viscera learning through algorithmic thinking. the study seeks answers to the following questions: • what is the ar-based teaching material's effect on viscera learning through algorithmic thinking? • what are the opinions of the teacher candidates about the ar-based material and the application process? 2. method 2.1. research model this study examines the ar-based teaching material's effect on viscera learning through algorithmic thinking and uses the case study method. this method is a qualitative research design that helps researchers in obtaining in-depth information in a short time. the results obtained via this method are limited to the cases examined, and there is no concern for generalization. 2.2. study sample the study sample consists of 32 primary school teacher candidates who are sophomores in the classroom teaching department in the spring term of the 2018-2019 academic year at a state university in the eastern black sea and selected by convenience method. teachers candidates are between the ages of 18-22. also, the teachers' candidates’ are 20 females, 12 males. teacher candidates were coded as t1, t2, t3… t32 in the study. 2.3. data collection tools viscera information form (vif) and application process and ar survey form (apsf) were used as data collection tools in the study. vif is an information form applied as a pre-test and post-test. participants are asked to show the viscera (lungs, heart, the stomach, liver, kidneys, gall bladder, small and large intestine) in a human model to place them in the skeletal structure. expert opinion was obtained from three science education experts. the vif was updated in line with the experts' feedback and was turned into a pre-information form. the other form, apsf, is a survey form consisting of two basic open-ended questions about the application process and the material prepared with augmented reality, which is applied only as a post-test. expert opinion was obtained from one science education and one computer educational and instructional technology expert. this science educator has 15 years of experience, is skilled in technology integration in science education, and his experience to put her knowledge into practice. the other expert who computer educational and instructional technologyhas 10 years of experience and is skilled in educational technology and these technologies application on the learning environment. 2.4. data analysis in the study, the data obtained from vif were analyzed under the researcher-defined categories regarding the participants' showing each viscera in a human torso model and placing them in the skeletal structure. these categories and the criteria for them are presented in table 1. based on the categories in table 1, the researchers analyzed each teacher candidates' answers independently. first, the data obtained were categorized individually and placed in matrices, and then a consensus was achieved by comparing them. the reliability percentage in this test was calculated with miles and huberman (1994) formula (reliability = [consensus/disagreement + consensus]*100), and the percentage was reached as 0.94. the data obtained from apsf was processed with content analysis. journal of science learning article doi: 10.17509/jsl.v4i4.32054 367 j.sci.learn.2021.4(4).365-374 2.5.teaching material developed based on augmented reality application in this study, the teaching material developed based on an augmented reality (ar) application was used. this material was created based on the futschek (2006) classification, namely presenting the problem, analyzing, determining the basic actions required in the solution, making the correct algorithm, increasing the interpretation and efficiency by comprehensively addressing the issue. ar material is related to viscera. the image of each organ has been identified as a trigger, and the ar application is embedded in these triggers. spurs have been put in envelopes. there are also magnetic puzzle pieces in the envelope of each stimulus. a scientist takes part in the ar application embedded in triggers, asks riddles to the teacher candidates, expects them to find viscera in the correct way & order, and takes a detective role to solve the puzzles. this stage aims to enable the teacher candidates to present the problem through algorithmic thinking. teacher candidates participating in the ar application must solve the puzzles like a detective and find out where the triggers are in the classroom. the aim here is that teacher candidates should identify the basic actions required for analyzing and solving. puzzles are about viscera and cover instructions such as five steps to the left, three steps to the right, etc. the researchers placed these instructions in the classroom following the order in the puzzles before starting the application. the triggers representing viscera are arranged in the school in the same way as the indication and placement of human organs. for example, the heart in the scientist's puzzle was placed closer to the left lung. so, the aim is to create the correct algorithm. the teacher candidates who locate all the triggers in the correct order will also collect the puzzle pieces in the envelopes (figure 1). figure 1. placement of triggers in classroom teacher candidates who locate all puzzle pieces are expected to complete the puzzle on the classroom board. the mystery is again about the correct viscera indication and placement. the purpose of completing the puzzle is to discuss and interpret the problem with all aspects, increasing efficiency. all these stages' relationship with algorithmic thinking is summarized in table 2. table 2 relationship between algorithmic thinking and teaching material based on the ar application algorithmic thinking futschek (2006) features associated with algorithmic thinking in teaching material based on the ar application 1. presenting the problem clearly • a scientist takes part in the ar application and asks puzzles to the teacher candidates and expects them to find viscera in the correct way & table 1 categories and criteria categories criteria correct indication-correct placement (ci-cp) indicating the viscus in the correct place in the skeletal structure and correct placement in the human model correct indication-incorrect placement (ci-ip) indicating the viscus in the correct place in the skeletal structure and incorrect placement in the human model correct indication-no placement (ci-np) indicating the viscus in the correct place in the skeletal structure and no placement in the human model incorrect indication-correct placement (ii-cp) indicating the viscus in the incorrect place in the skeletal structure and correct placement in the human model incorrect indication-incorrect placement (ii-ip) indicating the viscus in the incorrect place in the skeletal structure and incorrect placement in the human model incorrect indication – no placement (ii-np) indicating the viscus in the incorrect place in the skeletal structure and no placement in the human model no placement-correct placement (np-cp) indicating no placement of the viscus in the skeletal structure and correct placement in the human model no placement-incorrect placement np-ip indicating no placement of the viscus in the skeletal structure and incorrect placement in the human model np-np indicating no placement of the viscus both in the skeletal structure and the human model eka inserted text journal of science learning article doi: 10.17509/jsl.v4i4.32054 368 j.sci.learn.2021.4(4).365-374 order and take the role of a detective to solve the puzzles. 2. analyzing • teacher candidates participating in the ar application are required to solve the puzzles like a detective and find out where the triggers are in the classroom 3. identifying the basic actions required for the solution 4. creating the correct algorithm • puzzles cover instructions such as five steps to the left, three steps to the right, etc. 5. discussing and interpreting the problem with all aspects and increasing the efficiency • completing the puzzle correctly on the classroom board 2.6. application process before the application, vif was applied as a pre-test for the teacher candidates, where the viscera were asked to be indicated in a human model and placed in the skeletal structure. teacher candidates were then asked to experience the teaching material developed based on the augmented reality (ar) application. some photos of the application are shown in figure 2. during the experience, teacher candidates are expected to take on the role of a detective (figure 2a-b), solve puzzles (figure 2d-e, find the triggers in the classroom (figure 2c), create the correct algorithm, and complete the puzzle correctly on the board (figure 2d-f). figure 3. research application process after the application, vif was applied to the teacher candidates as a post-test. furthermore, apsf was also involved as a post-test on teaching material and application process based on the ar application. the research application process is schematized in figure 3. (d.) (e.) (f.) figure 2. sample images of the application process 2.7. research ethics the participant teacher candidates' consent to data sharing was obtained, and they were ensured that they should not suffer any harm due to the research. some private dialogues between the researcher and teacher candidates during the data collection and application process were not reflected in the study as per privacy and confidentiality principles. moreover, the identities of the teacher candidates who participated in the data collection process were kept confidential, following the principles of research ethics. 3. result and discussion 3.1. the result and discussion for the first research question the results obtained from the vip as a pre-and posttest are presented in table 3. (a.) (b.) (c.) journal of science learning article doi: 10.17509/jsl.v4i4.32054 369 j.sci.learn.2021.4(4).365-374 table 3 findings obtained from viscera information form categories viscera lungs heart liver stomach gall bladder kidneys small intestine large intestine pre (f) post (f) pre (f) post (f) pre (f) post (f) pre (f) post (f) pre (f) post (f) pre (f) post (f) pre (f) post (f) pre (f) post (f) ci-cp 9 25 11 26 5 17 9 16 2 13 4 15 1 12 1 12 ci-ip 2 1 1 2 3 5 2 2 6 3 2 5 1 5 ci-np 4 4 4 4 1 7 4 8 2 9 3 8 2 10 3 10 ii-cp 2 7 3 1 2 2 2 ii-ip 8 1 5 9 2 9 2 7 8 2 12 1 12 1 ii-np 2 1 6 3 7 1 6 1 7 1 7 1 np-cp 2 1 2 1 1 np-ip 2 1 1 1 1 4 3 np-np 3 1 7 3 2 1 11 5 3 3 2 3 3 3 as shown in table 3, while 9 of the teacher candidates' answers in the pre-test for lungs were in the ci-cp and eight were in the ii-ip category, there were no answers in the np-cp category. in the post-test, 25 of the solutions of the teacher candidates are in the ci-cp category, while one is in the ii-ip. this result indicates that 25 teacher candidates showed the lung's location in the human model and skeletal structure correctly following the application. in the pre-test for the heart, 11 of the answers of the teacher candidates were in the ci-cp; 7 of them were in the ii-ip category. in the post-test, 26 of the solutions of the teacher candidates were in the ci-cp category, while there were no answers in the ii-ip class. this result also indicates that 26 teacher candidates showed the lung's location in the human model and skeletal structure, placing them correctly after the application. findings on lungs and heart (table 3) have revealed that 25 out of 32 teacher candidates gave answers for lungs in the dg-tf category in the post-test, while 25 out of them for heart in the same category. that shows that most teacher candidates indicated and placed the lungs and heart correctly in the human model and the skeletal structure. prokop & fancovicova (2006) research specified that 69.2% of the students referred to the lungs as the second viscus after the heart. similarly, in the study conducted by lee (2015), lungs are frequently indicated by the participants as the most critical organ in terms of respiratory tract learning. in addition, academic studies on body systems reveal that students know organs like lungs and heart of all age groups (pelaez, boyd, rojas & hoover, 2005; prokop & fancovicova, 2006; zvi-assaraf, dodick & tripto, 2013). the possible reason for this is that the primary body organs and systems are taught in preschool education (ahi & balci, 2017). in parallel with the literature, the fact that nine teacher candidates indicate the lungs and 11 of them indicate the heart in the correct place in the human model and place them correctly in the skeletal system model during the pre-test maybe because they have learned the basic organs in education systems at different levels starting from the preschool period. however, 25 out of 32 teacher candidates indicate lungs, and 26 indicate the heart in the correct place in the human model and place them correctly in the skeletal system model in the post-test shows the effect of ar-based teaching material. it is a fact that ar-based teaching materials and/or ar technologies contribute to attract the interest and attention of the students to the lesson (delello 2014; dönmez-usta, durukan & turan-güntepe, 2020; tomi & rambli, 2013) and increase their motivation for the lesson (kerawalla, luckin, seljeflot & woolard 2006; perez-lopez & contero 2013; tomi & rambli, 2013). furthermore, ar-based teaching materials enable learning abstract concepts by materializing them through visualization facilities (abdüsselam & karal 2012; özarslan, 2013; shelton & stevens, 2004) and facilitate understanding the complex topics (kaufmann 2003; shelton & hedley 2002). accordingly, the learner’s success increases (shelton & hedley, 2002; sirakaya, 2015). teacher candidates' indicating the heart closer to the left lung reveals creating the correct algorithm. that can be interpreted as ar-based teaching material that contributes to the creation of the right algorithm. that also indicates that as futschek (2006) stated, the presence of instructions in the ar-based teaching material to create the correct algorithm contributes to the teacher candidates in doing so. in the pre-test for liver, 5 of the answers of the teacher candidates were in the ci-cp and 9 in the ii-ip category, while 17 of them were in the ci-cp and 2 in the ii-ip category in the post-test. in the pre-test for the stomach, 9 of the answers of the teacher candidates were similarly in the ci-cp and 9 in the ii-ip category, while 16 of them were in the ci-cp; 2 in the ii-ip category in the post-test. these results indicate that the number of teacher candidates who correctly place the liver and stomach in the human model and skeletal structure has dramatically increased. the number of those who did incorrectly has decreased. the results about liver and stomach (table 3) show that the answers in the ii-ip category in the pre-test have fallen in the post-test. in addition, it was determined that the solutions in the ci-cp category increased significantly in the post-test. this increase may be due to journal of science learning article doi: 10.17509/jsl.v4i4.32054 370 j.sci.learn.2021.4(4).365-374 teacher candidates solving the puzzles like a detective and finding the triggers in the classroom. in the teaching material based on the ar application, the triggers with puzzles were placed in different parts of the class to be placed in the human model. accordingly, the teacher candidates presented and analyzed the problem clearly and identified the basic actions required for a solution. that can be claimed to be related to studying the issue through the algorithmic thinking steps and determining the basic activities necessary for the answer. similarly, pruden, levine & huttenlocher (2011) stated in their study that mental rotations could be developed through visual maps and visualization of games. in algorithmic thinking, visualizing a problem that is not easy to solve but can readily be understood can help figure out the basic concepts associated with algorithms (futschek, 2006). therefore, it can be said that the fact that the teaching material based on the ar application includes visuality helps to understand the related basic concepts through algorithmic thinking in the pre-test for gall bladder, 2 of the answers of the teacher candidates were in the ci-cp and 7 in the iiip category, while 13 of them were in the ci-cp in the post-test. also, no teacher candidate answer was in the ii-ip category. in the pre-test for kidneys, 4 of the answers of the teacher candidates were in the ci-cp and 8 in the ii-ip category, while 15 of them were in the cicp and 2 in the ii-ip category in the post-test. in the pre-test for the small intestine, 1 of the answers of the teacher candidates was in the ci-cp and 12 in the ii-ip category, while 12 of them were in the ci-cp; 1 in the ii-ip category in the post-test. in the pre-test for the large intestine, as is the case for the small intestine, 1 of the answers of the teacher candidates was in the ci-cp and 12 in the ii-ip category, while 12 of them were in the ci-cp and 1 in the ii-ip category in the post-test. pre and post-test categories of teacher candidates for the small and large intestines are similar. however, the number of those who indicate and place correctly in the post-test has increased significantly. the results about the gall bladder, kidneys, and small and large intestines (table 3) show that the answers in the ii-ip category in the pretest significantly were reduced in the post-test. besides, it was determined that the solutions in the ci-cp category increased considerably in the post-test. that may be related to the teacher candidates presenting and analyzing the problem clearly, defining the basic actions required to solve it, and creating the correct algorithm. the teacher candidates' successful implementation of almost all steps, that is, their ability to think algorithmically and give correct answers, can be reasoned by the contribution of the teaching material based on the ar application. similarly, iste (2015) suggests using algorithmic thinking skills with technologysupported activities. in addition, it is claimed that the creativity and productivity of learners who create the correct algorithm improve (yecan, ozcinar & tanyeri, 2017). in such a case, teaching material based on the ar application may have contributed to developing metacognitive skills. the results obtained from the first question of the study reveal that most teacher candidates reached the correct information by discussing and interpreting the problem with all aspects. the teacher candidates who found all the triggers in the right order in the teaching material based on the ar application will have collected the puzzle pieces. since the puzzle is about the accurate viscera indication and placement, they have to assess all organs in the right place and order, solving the problem. they must discuss and interpret the problem with all aspects, thereby increasing efficiency. the teacher candidates' correct indications in the human model and correct placements in the skeletal structure may result from this. also, since algorithmic thinking is defined as performing a step-by-step task in other disciplines and computer science (selby & woollard, 2014), it is thought that algorithmic thinking can be used frequently in the process. unlike computer education, this study used algorithmic thinking in science education. in this respect, the results of the study are valuable. 3.2. the result and discussion for the second research question the results obtained from the apsf "as post-test are presented in table 4. journal of science learning article doi: 10.17509/jsl.v4i4.32054 371 j.sci.learn.2021.4(4).365-374 the opinions of the teacher candidates are collected under six basic categories. these categories are as follows: the learning process, topic/concept learning, increasing the interest/motivation in the learning environment, learners' activeness in the process and their interaction with the material, and the use of such materials in the learning environment. in addition, the opinions in these categories are listed and exemplified as positive, neutral, and negative. as shown in table 4, most teacher candidates expressed positive opinions in all categories. however, in the variety of the ar application-based teaching material's effect on the learning process, it is noteworthy that such materials are sometimes involved too much, making the people worn out. the cognitive load theory mentions that it is essential to properly use many components such as sound, text, pictures, graphics, and animations when designing multimedia to present information effectively and efficiently (anglin, vaez, & cunningham, 2013; barron, 2004). in this context, it is expected that the teaching process should be structured effectively to reduce the cognitive load, and mental structures are formed easily by allocating the working memory space to practical bag (kılıç-çakmak, 2007). thus, by creating welldesigned multimedia contents, candidates can be prevented from seeing the relevant materials as a burden in the learning process. for example, in the topic/concept learning category, teacher candidates familiar with the topic expressed positive opinions and stated that they did not learn the concepts thanks to the material based on the ar application. table 4 findings obtained from apsf categories answers of teacher candidates (f) sample answers for the categories effect on the learning process positive 30 t13: i found the learning process entertaining and compelling as i was active during the activity. neutral 1 t19: it seems as if it varies between those on good terms with technology and those who are not. negative 1 t21: technological materials are sometimes so much involved in the process. it wears me out. the topics could be explained in a regular and fundamental way. effect on topic/concept learning positive 29 t28: the students play the detective role, and locating some things with clues increases the memorability of the learned organ. the concept explanation was illustrative, instructive, and entertaining. neutral negative 3 t25: i already knew about the organs. i graduated from a healthy high school, so the concepts did not affect my learning. increasing learner's motivation/interest in the learning environment positive 30 t22: it makes us wonder about the next step as we proceed step by step. t26: solving puzzles and finding envelopes increased the motivation during learning, as they were interesting. neutral negative 2 t14: it is difficult to achieve a suitable environment; increasing the interest depends on the environment, so it seems complicated. t29: it was a little boring. it seems to impose a lot of burden on students. the learners' activeness in the process positive 27 t5: since we are directly involved in the activity, we are active in the process, and this is good. neutral 3 t31: i think it varies a little depending on whether the student wants to be active or not. negative 2 t15: it's boring to be so active. the learner is very active during the process, and this makes it dull. learner's interaction with the material positive 26 t2: taking on the detective role increased our interaction with the material. neutral 5 t30: interaction with the material? i don't know. maybe yes, maybe no. negative 1 t7: i could not establish any interaction. use of such materials in the learning environment positive 28 t11: such materials must be used in the learning environment as they give the student an active role in the process, make it fun, and arouse curiosity. our age is the age of technology. neutral 2 t13: if the school has facilities, then we use them. otherwise, it is difficult to use. negative 2 t6: facilities of the school or learning skills may not be sufficient. it is difficult to use such materials. journal of science learning article doi: 10.17509/jsl.v4i4.32054 372 j.sci.learn.2021.4(4).365-374 one of the two teacher candidates, who stated negative opinions in the category of increasing the motivation/interest in the learning environment, expressed that s/he found it boring as it imposes a lot of burden on the student. in the category of the learners' activeness in the process, three teacher candidates gave neutral opinions. t15, one of the two teacher candidates who expressed negative views in this category, stated that being too active creates boredom. the variety of the learner's interaction with the material has the most significant number (5) of neutral answers among all categories. in using such materials in the learning environment, the technological facilities and competence of the school or teacher play a significant role in opposing and neutral responses. the findings on the second question of the study (table 4), which aimed to reveal the teacher candidates' opinion about the material based on ar and the application process, show that most of the teachers generally expressed positive (f = 30, 29, 30, 27, 26, 28) opinions. in their study on augmented reality applications in a learning environment, ramazanoğlu & aker (2019) stated that students have a favorable view. specific studies in the literature specify that the use of teaching materials based on the ar applications or ar technologies in the learning environment makes lessons entertaining (yıldırım, 2016), increases motivation and interest (chiang, yang & hwang, 2014; ramazanoğlu & solak, 2020), ensures effective learning (ibáñez, di serio, villarána & kloosa, 2014) and that these materials are desired to be used in various learning environments (küçük, yilmaz & goktas, 2014). this is consistent with the study results. on the other hand, some teacher candidates gave a neutral or negative response, namely not desiring to be very active in the process, knowing the concepts, not being able to establish the expected interaction with the material, and the school's facilities and teachers' skills preventing the use of such materials. these opinions can be accounted for because they are familiar with topic/concept learning through current teaching methods. 4. conclusion according to the results of this study ar-based, teaching materials and ar technologies attract the students' interest and attention to the lesson and increase their motivation for the task. besides, ar-based teaching materials enable learning abstract concepts by materializing them through visualization facilities and facilitate understanding the complex topics. in this context, it can be a positive effect on the success of teachers candidates'. however, unlike the results, some teacher candidates think neutral or negative, namely not desiring to be very active in the process, not establishing the expected interaction with the material, and the school's facilities and teachers' skills preventing the use of such materials. these opinions can be related to being familiar with topic/concept learning through current teaching methods. one of the results in this study, teacher candidates' indicating the heart closer to the left lung reveals that they can create the correct algorithm. in this context, ar-based teaching material can contribute to the creation of an accurate algorithm. in this direction, it is recommended to develop materials related to ar-based and algorithmic thinking on different subjects and concepts such as excretory system, digestive system, respiratory system. acknowledgment the authors would like to thank all the individuals at the university of giresun who willingly participated in this study. compliance with ethical standards conflict of interest: the authors declare that they have no conflict of interest. ethical approval: the work described has not been previously published. all authors approved this manuscript. consent statement: study participation was voluntary, and participants were required to accept (or decline) the terms of the consent forms before completing the study. references abdüsselam, m. s., & karal, h. (2012). the effect of mixed reality environments on the students’ academic achievement in physics education: 11th grade magnetism topic example. journal of research in education and teaching, 1(4), 170–181. ahi, b., & balcı, s. (2017). “where does the breath i take from my nose go? children’s knowledge about respiratory system. elementary education online, 16(1):326-341. akçay, a. & çoklar, a. (2016). a suggestion for the development of cognitive skills: programming education. in a., i̇şman, h. f. odabaşı & b. akkoyunlu (eds.). educational technology readings 2016 (s. 121-140). adapazarı: sakarya univesity. anglin, g. j., vaez, h., & cunningham, k. l. (2013). visual representations and learning: the role of static and animated graphics. in handbook of research on educational communications and technology (pp. 854-905). routledge. barron, a. e. (2004). audiotory instruction. d. h. jonassen (edt.), handbook of research on educational communication and technology (sf. 949978). mahwah, new jersey: lawrence erlbaum associates. brown, w. (2015). introduction to algorithmic thinking. retrieved march, 21, 2019. cai, s., wang, x. & chiang, f. k. (2014). a case study of augmented reality simulation system application in a chemistry course. computers in human behavior, 37, 31– 40 chiang, t. h., yang, s. j., & hwang, g. j. (2014). an augmented realitybased mobile learning system to improve students' learning achievements and motivations in natural science inquiry activities. journal of educational technology & society, 17(4), 352.365. cutts, q., connor, r., michaelson, g., & donaldson, p. (2014, november). code or (not code) separating formal and natural language in cs education. in proceedings of the 9th workshop in primary and secondary computing education (pp. 20-28). delello, j. a. (2014). insights from pre-service teachers using sciencebased augmented reality. journal of computers in education, 1(4), 295– 311. journal of science learning article doi: 10.17509/jsl.v4i4.32054 373 j.sci.learn.2021.4(4).365-374 dönmez-usta, n., durukan, ü.g. & turan-guntepe, e. (2020, november). augmented reality-based instructional material on "solar system". paper presented at the international science, mathematics, entrepreneurship and technology education congress, online. esteves, m., fonseca, b., morgado, l. & martins, p. (2011). improving teaching and learning of computer programming through the use of the second life virtual world. british journal of educational technology, 42(4), 624-637. futschek, g. (2006). algorithmic thinking: the key for understanding computer science. in international conference on informatics in secondary schools-evolution and perspectives (pp. 159-168). springer, berlin, heidelberg. hsiao, h. s., chang, c. s., lin, c. y., & wang, y. z. (2016). weather observers: a manipulative augmented reality system for weather simulations at home, in the classroom, and at a museum. interactive learning environments, 1(19), 205-223. ibáñez, m. b., di serio, a., villaran, d., & kloos, c. d. (2016). support for augmented reality simulation systems: the effects of scaffolding on learning outcomes and behavior patterns. ieee transactions on learning technologies, 9(1), 46 – 56. ibáñez, m. b., di serio, á., villarána, d., & kloosa, c. (2014). experimenting with electromagnetism using augmented reality: impact on flow student experience and educational effectiveness. computers & education, 71(1–13). iste. (2015). ct leadership toolkit. https://cdn.iste.org/wwwroot/ct-documents/ct-leadershipt-toolkit.pdf?sfvrsn=4 kaufmann, h. (2003). collaborative augmented reality in education. institute of software technology and interactive systems, vienna university of technology. kerawalla, l., luckin, r., seljeflot, s., & woolard, a. (2006). making it real: exploring the potential of augmented reality for teaching primary school science. virtual reality, 10(3–4), 163–174. kılıç-çakmak, e. (2007). the bottle neck in multimedia: cognitive overload. gazi university journal of gazi educational faculty (gujgef), 2, 1-24. küçük, s., yılmaz, r., & göktaş, y. (2014). augmented reality for learning english: achievement, attitude and cognitive load levels of students. education and science, 39(176), 393-404. lee, y. c. (2015). self-generated analogical models of respiratory patways. journal of biological education, 49(4), 370-384. nayak, a. s., & vijayalakshmi, m. (2013). teaching computer system design and architecture course—an experience. in 2013 ieee international conference in mooc, innovation and technology in education (mite) (pp. 21-25). ieee. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=67 56298 nunes, f., herpich, f., amaral, é., voss, g., zunguze, m., medina, r. & tarouco, l. (2017). a dynamic approach for teaching algorithms: integrating immersive environments and virtual learning environments. computer applications in engineering education, 25(5), 732-751. özarslan, y. (2013). the effect of augmented reality enhanced learning materials on learners’ achievement and learners’ satisfaction. (unpublished doctoral dissertation). university of anadolu, eskişehir. pelaez, n. j., boyd, d. d., rojas, j. b. & hoover, m. a. (2005). prevalence of blood circulation misconception among prospective elementary theachers. adv. physiol. educ., 29, 172-181. perez-lopez, d., & contero, m. (2013). delivering educational multimedia contents through an augmented reality application: a case study on its impact on knowledge acquisition and retention. turkish online journal of educational technology tojet, 12(4), 19– 28. perry, g. (2009). absolute beginner’s guide to programming (trans. t. aksoy). i̇stanbul: system publishing. prokop, p. & fancovicova, j. (2006). students’ ideas about the human body: do they really draw what they know? journal of baltic science education, 2(10), 86-95. pruden, s., levine, s. & huttenlocher, j. (2011). children’s spatial thinking: does talk about the spatial world matter?. developmental science, 14(6), 1417-1430. ramazanoğlu, m., & aker, a. (2019). opınıons of the teacher candıdates related to use of augmented reality technology for educatıonal purposes. turkish studies-information technologies and applied sciences, 14 (1), 91-106. ramazanoğlu, m., & solak, m. ş. (2020). attitudes of secondary school students towards the use of augmented reality applications in education: sample of siirt province. kastamonu education journal, 28(4), 1646-1656. selby, c. & woollard, j. (2014). refining an understanding of computational thinking. https://eprints.soton.ac.uk/372410/. selby, c. & woollard, j., (2013). computational thinking: the developing definition. university of southampton (eprints) 6pp. https://eprints.soton.ac.uk/356481/1/selby_woollard_bg_soton _eprints.pdf. accessed 24 march 2020 shelton, b. e., & hedley, n. r. (2002). using augmented reality for teaching earth-sun relationships to undergraduate geography students. in augmented reality toolkit, the first ieee international workshop (p. 8--pp). shelton, b. e., & stevens, r. (2004). using coordination classes to interpret conceptual change in astronomical thinking. in proceedings of the 6th international conference for the learning sciences. lawrence erlbaum & associates, mahweh, nj. sırakaya, m. (2015). effects of augmented reality applications on students' achievement, misconceptions and course engagement. (unpublished doctoral dissertation). university of gazi, ankara. tomi, a. b., & rambli, d. r. a. (2013). an interactive mobile augmented reality magical playbook: learning number with the thirsty crow. procedia computer science, 25, 123130. haridi, s., & van-roy, p. (2004). concepts, techniques, and models of computer programming. cambridge: mit press. vasconcelos, j. (2007). basic strategy for algorithmic problem solving. extraído el, 2. https://www.cs.jhu.edu/~jorgev/cs106/problemsolving.html. accessed 14 january 2019. wang, h. y., duh, h. b. l., li, n., lin, t. j., & tsai, c. c. (2014). an investigation of university students’ collaborative inquiry learning behaviors in an augmented reality simulation and a traditional simulation. journal of science education and technology, 23(5), 682-691. wu, h.k., lee, s. w. y., chang, h. y.,& liang, j. c. (2013). current status, opportunities and challenges of augmented reality in education. computers & education, 62, 41-49. yecan, e., özçınar, h., & tanyeri, t. (2017). ict teachers’ visual programming teaching experiences. elementary education online, 16(1). yildirim, s. (2016). "fen bilimleri dersinde artırılmış gerçeklik uygulamalarının öğrencilerin başarısına, motivasyonuna, problem çözme becerilerine yönelik algısına ve tutumlarına etkisi [the impact of augmented reality to student's success, motivation, and their perception and behavior related to problem solving abilities in science classes]." unpublished master thesis), ankara university, institute of educational sciences, department of computer education and instructional technologies, educational technologies, ankara. https://cdn.iste.org/www-root/ct-documents/ct-leadershipt-toolkit.pdf?sfvrsn=4 https://cdn.iste.org/www-root/ct-documents/ct-leadershipt-toolkit.pdf?sfvrsn=4 https://eprints.soton.ac.uk/372410/ https://eprints.soton.ac.uk/356481/1/selby_woollard_bg_soton_eprints.pdf https://eprints.soton.ac.uk/356481/1/selby_woollard_bg_soton_eprints.pdf journal of science learning article doi: 10.17509/jsl.v4i4.32054 374 j.sci.learn.2021.4(4).365-374 yoon, s., anderson, e., lin, j., & elinich, k. (2017). how augmented reality enables conceptual understanding of challenging science content. journal of educational technology & society, 20(1), 156-168. ziatdinov, r. & musa, s. (2013). rapid mental сomputation system as a tool for algorithmic thinking of elementary school students development. european researcher, 25(7), 1105-1110 zsakó, l. & szlávi, p. (2012). ict competences: algorithmic thinking. acta didactica napocensia, 5(2), 49-58. zvi-assaraf, o., dodick, j & tripto, j. (2013). high school students’ understanding of the human body system. research in science education, 43, 33-56. a © 2022 indonesian society for science educator 69 j.sci.learn.2022.5(1).69-78 received: 28 march 2021 revised: 30 april 2021 published: 1 march 2022 a content analysis study towards researches regarding context-based learning approach in science education by between years 2010 and 2020 in turkey büşra arik güngör1*, mustafa metin2, sibel saraçoğlu2 1talas çömlekçi middle school, turkey 2erciyes university, department of mathematics and science, turkey *corresponding author. busrarik38@gmail.com abstract the purpose of this study is to study research regarding the context-based learning approach in science education between 2010 and 2020 in turkey. in this study, a descriptive content analysis method was used to analyzed 86 articles on contextbased learning in science education published in 67 different journals. according to some variables such as the year of publication, purpose, area, method, sample, number of samples, data collection tool and data analysis techniques. the research classification form developed by the researchers was used in the analysis of the data. the data were analyzed using descriptive content analysis. most studies in the field of context-based learning approaches in science education were published in 2019. most of the studies aimed to determine the effects of the method on achievement and attitude and were carried out within science. the experimental method was preferred among the quantitative methods in the studies. most of the studies were conducted with secondary school students, and the number of participants was in the range of 0-100. achievement tests and attitude scales were used extensively as data collection tools, and content analysis and t-test were used extensively in data analysis techniques. according to the results obtained in the research, it is suggested that the researches on the context-based learning approach should be conducted in areas other than science education. keywords science education, life-based learning, context-based learning, content analysis 1. introduction today, the scientific literacy level of individuals has become an indicator of the development of countries together with rapidly developing technology and changing scientific knowledge (can & çelik, 2020). this situation has revealed the need to raise individuals who use science and technology and apply the recipe with what is given to them and develop science and technology and create new recipes. therefore, the needs of individuals have also changed during recent years, and using the knowledge efficiently has become more valuable. therefore, it is needed to prevent disconnection between learned knowledge and daily living in schools, and students should be able to solve problems met in their daily lives by using the knowledge they have learned in classes (dewi, poedjiastoeti & prahani, 2017; oduor, 2013; sugiono & purwastuti, 2017). especially science education has great importance for transferring knowledge into daily living situations among educational programs (aikenhead 2006; rosa, mundilarto, wilujeng & sulistyani, 2019; tal & dierking, 2014). however, it is stated that students couldn’t find an answer to what the science concepts learned in the course would do in their daily lives and that they couldn’t associate the concepts learned with daily life events. this situation causes a reduction of the interest of students in the lesson, having difficulties in learning, and decrease in performance (aniashi, okaba, anake & akomaye, 2019; balkan kıyıcı &aydoğdu, 2011; gilbert, 2006; stolk, bulte, de jong & pilot, 2009; yıldırım, küçük & ayas, 2013). to overcome these problems, the use of education programs allowing students to build more association with the world they are living in and to be able to use learned in different situations carries importance (de jong, 2008; genç, ulugöl & ünsal, 2017, meb, 2020). thus, it is necessary to use learning approaches that include practices attracting children's attention and journal of science learning article doi: 10.17509/jsl.v5i1.33074 70 j.sci.learn.2022.5(1).69-78 intriguing, contribute to the integration of the learned subject with daily life and improve not only knowledge but also skills, abilities, and capacities. (çepni, özmen & ayvacı, 2015; flynn, 2019; how kids learn science best, 2021; rosa, mundilarto, wilujeng & sulistyani, 2019; silander 2015). moving from these requirements, the thought of using a context-based learning approach, foundations of which are based on social structuring in educational programs, has appeared (çepni, özmen & ayvacı, 2015). in this context, in the early 1980s, a group of chemistry educators from york university started to use real-life fiction called salters stories as a way to achieve their learning goals, and when successful results were achieved, the use of this approach was widespread in many countries (bennett & lubben, 2006; ramsden, 1997; university of york, 2015). the purpose of the context-based learning approach is to provide students' learning by associating their previous knowledge and experiences in daily life. the context-based learning approach defends the thought that learning is beneficial only in case based on real-life and performed with strong interactions (cobos, castilla & lópez, 2017; çepni, özmen & ayvacı, 2015). in context-based learning approach, learning desires and efforts of students are tried to be increased by providing awareness between situations in their daily lives and science in a learning process occurring in a social environment, which supports teaching and learning (demir, 2019; gilbert, 2006; king, winner & ginns, 2011; sevian, dori & parchmann, 2018). contextbased learning is a contemporary learning approach that excites students, facilitates understanding of concepts and principles related to a subject by students, increases students' motivations during the active learning process, and allows students to take responsibility through individual learning. besides, this approach supported that students make them gain self-management skills, increases superior thinking, communication skills of students, provides an increase of interests of students, teachers towards lessons and at the same time, considers scientific developments (gilbert, 2006; kutu & sözbilir, 2011; overton & potter, 2011; özay-köse, & çam tosun, 2011). it is seen in the literature searching that many national and international studies have been conducted related to context-based learning approach, which showed many positive influences in learning and teaching (bellocchi, king & ritchie, 2016; gül & konu, 2018; john, molepo & chirwa, 2018; karslı-baydere & kurtoğlu, 2020; podschuweit & bernholt, 2018; tulum, 2019; yıldırım & dağıstanlı, 2020; wiyarsi, pratomo & priyambodo, 2020). when conducted research is examined, it is seen that subject areas, purposes, methods, data collection tools, and study groups of studies differ from each other, and each one should be solved separately. concurrent access to researches on this subject, reading, interpretation, and solving difficulties suggest the requirement for researches based on integrated analysis on this area. although review studies carried out regarding context-based learning approach, it takes attention that their number is quite limited (kabuklu & kurnaz 2019; ültay & ültay, 2014). investigation results, including integrated analyses related to the context-based learning approach, will summarize contemporary studies in the related area and facilitate researchers, teachers, and other shareholders to research data related to the subject area. critical data to be presented about the content of studies being conducted on this subject would present a possibility to researchers for seeing new and different studies in an integrated way on this subject. it would contribute to developing different viewpoints about the subject. moreover, it is expressed in the meb 2023 vision document that the science course curriculum should be designed to raise individuals who do not see what they learn as independent from the life they live and who can offer solutions to the problems encountered in daily life (meb, 2020). therefore, research results to be made in this context may also contribute to drawing the attention of program preparers on this issue. considering this fact, it is essential to conduct a holistic analysis of studies in which context-based learning approach is investigated from different perspectives. this study aims to conduct a content analysis of research regarding the context-based learning approach in science education between 2010 and 2020 in turkey. within this context, answers to the following questions were sought in the research: of researches with the subject of context-based learning approach in science education; i) how are their distribution according to i) years, ii) purpose, iii) study area, iv) approach/methods of research, v) sample group; vi) sample size, vii) data collecting tools and viii) data analysis methods? 2. method this study aims to conduct content analysis articles published in turkey between the years 2010-2020 related to the context-based learning approach in science education. content analysis is considered in three categories as meta-analysis, meta-synthesis, and descriptive analysis in the literature (çalık & sözbilir, 2014). within this study, descriptive content analysis method has been used to determine the general trend in research where both quantitative and qualitative research methods were used (cohen, manion & morrison, 2007). similar data are combined within the frame of specific concepts and themes with the descriptive content analysis method. finally, they are arranged and interpreted in a way that the reader can understand (fraenkel, wallen & hyun, 2012). 2.1. population and sample the sample of this research is formed by articles published in turkey between 2010 and 2020 years related journal of science learning article doi: 10.17509/jsl.v5i1.33074 71 j.sci.learn.2022.5(1).69-78 to the context-based learning approach in science education. while the sample of the research was selected; it was taken into consideration criteria such as executed study is in turkey, it is present in ulakbim and tr index, it can be accessed as a whole text in google academy database, and it would contain keywords of “bağlam temelli öğrenme”, “yaşam temelli öğrenme”, ‘’context-based learning” and “life-based learning”. by taking these criteria into consideration, a sample of our research consists of 86 articles published in 67 different journals comprising studies until 30.11.2020. 2.2. data collection tool a “research examination form” has been developed to evaluate articles related to the context-based learning approach in science education within the study context, and this form has been used as a data collection tool. while developing research examination form, studies previously executed by descriptive content analysis in the literature have been investigated (akkuş, sarı & uner, 2012; chang, chang & tseng, 2010; keklik, 2011; kızılaslan, sözbilir & yaşar, 2012; selçuk, palancı, kandemir & dündar, 2014) and features that should be present in a scientific article have been examined (büyüköztürk et al., 2014; cohen, manion & morrison, 2007; çepni, 2012; fraenkel, wallen & hyun, 2012). in line with examinations made, there are nine chapters as the masthead, year, purpose, study area, approach/method, sample group, sample size, data collection tools, and data analysis techniques. categories included in this examination form and options to be given under this category have been presented to the opinions of two experts studying in the area of content analysis. arrangements have been made in the form in line with expert opinions. 2.3. data analysis 86 articles complying with criteria determined within the study context have been analyzed according to the research examination form developed by researchers. two different researchers have separately evaluated the studies by considering categories determined in the form in the analysis of articles. researchers have recorded findings obtained from articles in a microsoft excel file and categorized them within the research questions frame. after the analysis of all the articles was completed, the two researchers' analyses were compared, and the analysis data were arranged in line with a common opinion. findings obtained are then presented in the form of tables and graphs. after the analysis, the data were organized, and inferences were made. 3. result and discussion 86 articles published between the years 2010-2020 in the context-based learning approach in science education, of which whole text could be accessed, have been analyzed, and findings have been presented in the form of separate graphics within the context of determining categories. the distribution of studies examined according to years has been presented in figure 1. when the distribution of articles in the area of contextbased learning according to years given in figure 1 is examined, it is seen that 16 articles were published in 2019, 15 articles in 2017, two or three studies in 2010 and 2014. thus, it is seen that articles published in the area of contextbased learning showed an increase during recent years. figure 1 distribution of articles examined in the area of context-based learning in science education according to years 0 5 10 15 20 25 2 0 1 0 2 0 1 1 2 0 1 2 2 0 1 3 2 0 1 4 2 0 1 5 2 0 1 6 2 0 1 7 2 0 1 8 2 0 1 9 2 0 2 0 2 3 4 4 5 8 10 15 8 16 11 number of articles according to years figure 2 distribution of articles examined in the area of context-based learning in science education according to purpose 18 16 13 13 9 9 6 5 4 3 3 2 2 1 0 2 4 6 8 10 12 14 16 18 20 success thought mativation permanent learnig problems i̇nterest react strategy propose of the investigetion journal of science learning article doi: 10.17509/jsl.v5i1.33074 72 j.sci.learn.2022.5(1).69-78 the distribution of examined articles according to purpose has been given in figure 2. when figure 2 is examined, it is seen that 18 studies conducted in the field of context-based learning are aimed at determining the effect of this method on success, and 16 of them are aimed at determining the effect on attitude. in addition, the opinion of the context-based learning sample was determined in 13 studies, and 13 studies were conducted to determine the effect of context-based learning on the sample's conceptual understanding. the distribution of examined articles according to subject areas they are included has been presented in figure 3. when figure 3 is examined, a more significant majority (49 articles) of subject areas of studies conducted related to subject of context-based learning have been included in the subject area of science. for example, it has been seen that 14 scientific studies have been made in physics education, 12 in biology education, and 11 in chemistry education. distribution of examined articles according to approach and method of research used has been given in figure 4. when figure 4 is examined, quantitative research method has been used in 53 articles, qualitative research method in 26 articles and mixed research method in 7 articles published between years 2010-2020. it has been determined that the most preferred method in studies where a quantitative research approach was used is the experimental research method with 42 studies. besides, it is understood from figure 4 that the most used qualitative research approach is the status study method with 11 figure 3 distribution of articles examined in the area of contextbased learning in science education according to subject areas science 49 physics 14 biology 12 chemistry 11 area of investigation figure 4 distribution of studies examined in the area of context-based learning in science education according to approach and method of investigation 0 10 20 30 40 50 e x p e ri m e n ta l s u rv e y p h e n o m e n o lo g y c as e s tu d y a c ti o n r e se ac h d o c u m e n t re v ie w u n sp e c if ie d m ix e d m e th o d quantitative (53) qualitative (26) mixed (7) 42 11 3 11 6 2 4 7 approach / methods of investigation figure 5 distribution of examined studies according to sample group 0 10 20 30 40 50 3 33 24 18 24 2 1 sample group journal of science learning article doi: 10.17509/jsl.v5i1.33074 73 j.sci.learn.2022.5(1).69-78 studies. however, in mixed studies, it is seen that methods are generally not specified, and they are expressed only as "mixed method was used". distributions of examined articles according to sample groups included in the study have been given in figure 5. when we look at figure 5, it is seen that 33 percent of researches have been conducted on middle school students. also, it has been determined that 24 types of research have been performed with high school students and teachers. however, it is seen in the research that preservice teacher were chosen as a sample in 18 studies and primary school students in 3 studies, and academicians in 2 studies. the distribution of examined articles according to sample size included in the sample group where studies were conducted has been presented in figure 6. when figure 6 is examined, 22 studies have 21-40 participants, 19 studies with 41-60 participants, 16 studies with 0-20 participants, and ten studies with 121 and more participants. therefore, it was determined that most researches (63 types of research) had been conducted with a sample size between the range of 0-100. the distribution of data collection tools used in examined articles has been given in figure 7. when the distribution of data collecting tools given in figure 7 is examined, the number of data collecting tools is more than a number of examined articles because more than one data collecting tool has been used in some studies. when figure 7 is examined, it is understood that the questionnaire was chosen in 10 studies, scale in 34 studies, and 39 studies and form in 27 studies. it is observed that primarily open-ended questionnaires (8) were used in studies where the questionnaire was used and that mostly attitude scales (17) were used in studies where scales were used. also, while mostly success tests (27) were preferred in research where the test was used, mostly interview forms (16) were used among forms. the distribution of analysis methods of data used in examined articles has been given in figure 8. when figure 8 is examined, it is seen that content analysis / descriptive analysis was used in the greater majority of studies with 32 studies and that t-test was used in 30 studies. also, it is determined that anova was used in 9 studies, mannwhitney u test in 9 studies, and ancova analysis in 4 studies this research aims to conduct a content analysis on articles published in turkey between 2010-2020 on the figure 6 distribution of studies examined in the area of context-based learning in science education according to participant number in their samples 0 5 10 15 20 25 16 22 19 1 5 9 10 4 simple size figure 7 distribution of studies examined in the area of context-based learning in science education according to data collecting tools 0 10 20 30 40 50 l ik e rt o p e n e n d e d a tt it u d e m o ti v a ti o n o th e r b e h av io u r a w ar e n es s a c h ie v em e n t te st c o n c e p t t es t c o n te x t t e st s c ie n ti fi c p ro c e ss t e st in te rv ie w o b se rv at io n d o c u m e n t re v ie w o th e r fo rm s questionnaire (10) scale (34) test (39) form (27) 2 8 17 11 7 2 1 27 7 3 1 16 4 3 4 data collection tools scale (39) test (38) journal of science learning article doi: 10.17509/jsl.v5i1.33074 74 j.sci.learn.2022.5(1).69-78 subject of context-based learning in science education. within this context, 86 articles published in 67 different journals have been analyzed. furthermore, research findings have been evaluated under determined categories considering analysis data. the first of these categories is the distribution of particles according to years. although studies related to context-based learning have started in the 1980s in the international area (bennett & lubben, 2006), it takes attention that two studies taking this approach as a subject in science education has been ma de in 2010 in turkey. the number of studies that were limited until 2014 on this subject has increased until 2017, and most researches were performed in 2019. though a reduction was seen in 2020, it is thought that this result has originated from that studies published until november 2020 have been included in the research. when studies of which printing has been completed after november 2020 were considered, possibly the number of related studies would be more than indicated. one of the reasons for the increase of studies related to context-based learning may be that this subject has been given more importance in programs. thus, it is indicated that education programs of science that was renewed in line with meb 2023 vision have been structured such as individuals can present solutions for problems they met in their daily lives, that they don’t see learned things independent of the life, they lived and that they would have gained for putting forward new technological products in line with their needs (meb, 2020). one of the most suitable approaches that facilitate accessing to indicating gains is context-based learning. it aims to update undergraduate for science teachership to develop candidates' abilities for context-based science teaching (yök, 2018). for this reason, the use of this approach has gained importance in science education, and the need for evaluating problems and solutions came by use of this approach could have directed researchers to this subject. also, this approach found more place in renewed programs causes more activities to be developed in science education. this situation requires performing more researches on this subject. it is expected that the use of context-based science education, many advantages of which have been expressed in the literature (demircioğlu, vural & demircioğlu, 2012; gilbert, 2006; sözbilir, sadi, kutlu & yıldırım., 2007), would become widespread and increase in several types of research on this subject. another category formed as a result of research findings is the distribution of researches according to purpose. on examinations made, it is seen that researches related to context-based learning approach have focused on the subject of the influence of context-based learning approach on the success of students and their attitudes towards science lessons. subjects following them are participants' thoughts related to this approach and the effect of conceptual understanding. the context-based learning approach aims to provide awareness of students on the association of real-life issues and science by presenting scientific concepts to students together with events selected from daily life and thus, to increase desire and motivations of students for learning (çepni, 2012; kutu & sözbilir, 2011; özay-köse, & çam tosun, 2011). increased desire and motivation of students contribute to increasing their interest in science and their academic performances (çepni, 2012; gilbert 2006, sözbilir et al., 2007). within this context, it is an expected result that researchers have given more places to the status of accessing targets of this approach in their researches. but, it is seen in the literature that several studies related to perception and awareness related to context-based learning are pretty limited. awareness of teachers/teacher candidates/students related to the approach used influences the quality of teaching and learning (yakar & duman, 2017). this situation brings the requirement for making more studies on this subject into the schedule. no study hasn’t been met in the literature examination related to context-based science education on students' abilities such as communication, social, and entrepreneurship. however, this approach is based on cooperation built between concepts and daily living and covers a process occurring in a social environment that supports learning (gilbert, 2006). obtained results and literature data show a need for studies examining the effects of context-based science education on high-level skills such as social skills, creativity, and entrepreneurship. therefore, the fact that there is no study on this subject in the literature is considered a deficiency. when researches examined within the context of the research are analyzed, it is seen that the more significant majority of researches have been in the area of science. figure 8 distribution of studies examined in the area of context-based learning in science education according to data analysis methods 0 10 20 30 40 50 32 30 9 9 4 2 2 2 3 analysis methods of data journal of science learning article doi: 10.17509/jsl.v5i1.33074 75 j.sci.learn.2022.5(1).69-78 when the objectives of the science course curriculum and the characteristics of the context-based learning approach are examined, it is seen that one of the most appropriate approaches that can be used to achieve these goals is context-based learning (coştu, 2009; gilbert, 2006, meb, 2020). this situation may be one of the causes why the subject area of science is more involved in researches. the fact that the target audience examined in the studies is mostly at the secondary school level may have caused such a result in the subject area. when examined in terms of research approach and methods used in studies on context-based learning approach in science education in the literature, it is seen that primarily quantitative research approach and empirical method are used. still, qualitative and mixed methods are also included. while the experimental method is used in 49% of the research examined, the mixed method ratio remains at 8%. the number of researches using case science, action research, and document examination methods is also limited. the fact that experimental researches completely put forward the cause-result association may lead to this result. because experimental researches are interventional researches and also giving education to individuals would be considered an intervention, researchers could have preferred this method in their researches (özmen, 2014). but, it can be indicated that increasing qualitative and mixed research is needed to perform in-depth research. when researches are evaluated for sample groups, it is seen that researches have been primarily conducted with secondary school students. this situation shows parallelity with the result that most studied area has been science education. secondary school level is a period of physical, mental-emotional changes in individuals. these changes may reveal some academic and psychological problems. schools have an essential role in using appropriate approaches that touch young people's lives, reduce their anxiety, meet their desire to join a group, interest, curiosity, and motivation for healthy academic, social, and emotional development (şirin, 2020). during this period, youth having required environmental support can solve their problems more efficiently. when characteristics of this age group and aims of context-based learning and science lesson program are taken together, it wouldn’t be surprising that the number of studies towards this age group is higher. a context-based learning approach is also recommended in the updated science program (meb, 2020). indicated situations may be a justification of the abundance of researches numbers at the secondary school level. several studies conducted on primary school students is quite limited in studies. due to the effects of approaches used in schools on the behavioral changes of individuals, evaluating the influence of this approach in younger age groups could make essential contributions for researchers and executers for the aspect of revealing the advantages and limitations of the approach. when studies are examined according to the sample size, most studies have been conducted with participants between a range of 0-100. most studies have been performed with participants ranging from 20-40 people. it is thought that a lower number of samples in studies is influenced by the fact that the experimental method of quantitative approach has been used chiefly, and a high number of researchers have preferred qualitative research methods. due to some ethical issues and a small number of students in a class where the application would be made, a study has been conducted with a sample of a limited number (büyüköztürk et al., 2014; fraenkel, wallen & hyun, 2012; özmen, 2014). similarly, in studies where qualitative research methods have been used, a sample with a limited number is preferred due to lack of anxiety of generalization of research result and to investigate the subject more in-depth (cohen, manion & morrison, 2007; fraenkel et al., 2012; yıldırım & şimşek, 2006). when these two situations are considered, it is seen that being sample in a limited number is unavoidable. in the investigation performed for the aspect of data collecting tools used, the number of data collecting tools is higher than the number of researches. this situation originated from the use of more than one data-collecting tool to increase validity in research. mainly used data collecting tools in research are achievement test, attitude and motivation scale as scale, and interview method as form and open-ended questionnaires. this situation overlaps with the result that the methods used have been primarily quantitative. the finding that the interview method is the most used data collecting tool in qualitative studies (yıldırım & şimşek, 2006) supports the result of research. interview forms have been chiefly used for supporting quantitative data. it is seen that the aims of examined researches and findings related to data collecting tools used have supported each other. when obtained results are evaluated for data analysis techniques, it takes attention that mostly content analysis and t-test have been used. several types of research where other analysis techniques have been used are pretty limited. when examined researches are considered, it is observed that data collection tools, by which qualitative findings are obtained, such as observation and interview form, using in both qualitative and mixed research methods. in this case, solving data obtained in the study by using content analysis is a natural situation. it is expressed in qualitative research methods that data have been solved descriptively by creating a specific theme and codes and more detail by making content analysis (çepni, 2012; yıldırım & şimşek, 2006). it is seen that another mostly chosen data analysis method is the independent t-test within the context of the study. non-parametric analyses such as dependent and non-dependent t-test, anova, ancova, manavo, mancova, chi-square test, mann-whitney test, wilcoxon signed-rank sum, and sign test are used in the analysis of data in studies in which experimental research methods are used (metin, journal of science learning article doi: 10.17509/jsl.v5i1.33074 76 j.sci.learn.2022.5(1).69-78 2014). in research examined within the context of the study, t-tests have been used to show the normal distribution in experimental research. they have more strong validity for making generalizations compared to results obtained from other non-parametric tests and descriptive analyses (frequency, percent, arithmetic average, standard deviation) (büyüköztürk et al., 2014; fraenkel, wallen & hyun, 2012; metin, 2014). it is thought that obtaining this result is natural. however, nonparametric tests are likely chosen less because they have no normal distribution and weaker validity than parametric tests (çepni, 2012; erdem, 2011; metin, 2014). investigations performed within the context of the study show parallelity with this situation. besides, it is observed in the study that advanced statistical analyses such as anova, ancova, manavo, mancova, and factor analysis haven’t been preferred too much. it is thought that this situation has originated from that research subject isn’t suitable for these types of analyses or that researchers aren’t competent enough on advanced statistical methods. conclusion in this study, a descriptive content analysis method was used to analyzed 86 articles on context-based learning in science education published in 67 different journals according to some variables such as the year of publication, purpose, area, method, sample, number of samples, data collection tool and data analysis techniques. most studies in the field of context-based learning approaches in science education were published in 2019. most of the studies aimed to determine the effects of the method on achievement and attitude and were carried out within science. the experimental method was preferred among the quantitative methods in the studies. most of the studies were conducted with secondary school students, and the number of participants was in the range of 0-100. achievement tests and attitude scales were used extensively as data collection tools, and content analysis and t-test were used extensively in data analysis techniques. the following recommendations can be presented based on the results achieved: within the scope of this study, researches on the context-based learning approach in science education were examined. it may be suggested to other researchers researching this field to examine the studies on contextbased learning approaches in different disciplines. in addition, since the number of studies at the primary school level is limited, it is believed that conducting studies with primary school students will contribute to the field. besides, it is seen that studies are conducted with a limited number of participants. therefore, quantitative studies with a large number of participants are recommended. furthermore, more emphasis can be placed on quantitative or mixed research to make more detailed and in-depth research. also, it can be suggested that comprehensive studies using different analysis techniques such as bibliometric analysis can be conducted. references aikenhead, g. s. (2006). science education for everyday life, canada, teachers college press. akkus, h., sari, s. n., & uner, s. (2012). the content analysis of graduate theses written between 2000 and 2010 in the field of chemistry education. procedia-social and behavioral sciences, 47, 729733. aniashi, s. o., okaba, l. a., anake, e. u., & akomaye, m. u. (2019). classroom science with everyday life: a means for improving performance in sciences and national development in nigeria. international journal of science and technology education research, 10(3), 25-29. balkan-kıyıcı, f. & aydoğdu, m. (2011). fen bilgisi öğretmen adaylarının günlük yaşamları ile bilimsel bilgilerini ilişkilendirebilme düzeylerinin belirlenmesi [determining the level of associating science teacher candidates' daily life with their scientific knowledge.]. necatibey eğitim fakültesi dergisi, 5(1), 43-61. bellocchi, a., king, d. t., & ritchie, s. m. (2016). context-based assessment: creating opportunities for resonance between classroom fields and societal fields. international journal of science education, 38(8), 1304-1342. bennett, j. & lubben, f. (2006). context‐based chemistry: the salters approach. international journal of science education, 28(9), 999-1015. büyüköztürk, ş., kılıç çakmak, e., akgün, ö.e., karadeniz, ş. & demirel, f. (2014). bilimsel araştırma yöntemleri [scientific research methods ] (17. baskı). ankara: pegem yayınları can, ş. & çelik, c. (2020). pre-service science teachers' universal science literacy levels across the statistical regional units. pamukkale üniversitesi eğitim fakültesi dergisi, 49, 112-133. chang, y. h., chang, c. y., & tseng, y. h. (2010). trends of science education research: an automatic content analysis. journal of science education and technology, 19(4), 315-331. cobos, t. l., castilla, r. l., & lópez, á. b. (2017). procesos de oxidación: un acercamiento a su estudio en la eso [oxidation processes: an approach to its study at eso]. boletín encic: revista del grupo de investigación hum-974, 1(1), 7-8. cohen, l., manion, l., & morrison, k. (2007). observation. research methods in education, 6, 396-412. coştu, s. (2009). matematik öğretiminde bağlamsal öğrenme ve öğretme yaklaşımına göre tasarlanan öğrenme ortamlarında öğretmen deneyimleri. [teacher experiences in learning environments designed according to the contextual learning and teaching approach in mathematics teaching]. (unpublished master's thesis). ktü fen bilimleri enstitüsü, trabzon. çalık, m. & sözbilir, m. (2014). i̇çerik analizinin parametreleri [parameters of content analysis]. eğitim ve bilim, 39(174). çepni, s. (2012). araştırma ve proje çalışmalarına giriş [introduction to research and project work]. ankara: pegema yayıncılık çepni, s., özmen, h., & ayvacı, h. ş. (2015). yaşam (bağlam) temelli, beyin temelli öğrenme kuramları, 21. yüzyıl becerileri ve fetemm yaklaşımı ve fen bilimleri öğretimindeki uygulamaları [life (context)-based, brain-based learning theories, 21st century skills and stem approach and its applications in science teaching] . s. çepni (ed.), kuramdan uygulamaya fen ve teknoloji öğretimi içinde, ankara: pegem akademi de jong, o. (2008). context-based chemical education: how to improve it?. chemical education international, 8(1), 1-7. demı̇rcı̇oğlu, h , vural, s , & demı̇rcı̇oğlu, g . (2012). “react” stratejisine uygun hazırlanan materyalin üstün yetenekli öğrencilerin başarısı üzerinde etkisi [the effect of the material prepared in accordance with the “react” strategy on the success of gifted students]. ondokuz mayıs üniversitesi eğitim fakültesi dergisi ,31 (2), 101-144 journal of science learning article doi: 10.17509/jsl.v5i1.33074 77 j.sci.learn.2022.5(1).69-78 demir, i̇. (2019). yaşam temelli öğretimin ortaokul 7.sınıf öğrencilerinin ağız ve diş hijyeni konusunda kavram öğrenmelerine, fen bilimlerine karşı tutumlarına ve motivasyonlarına etkisi [the effect of life-based teaching on secondary school 7th grade students' learning concepts about oral and dental hygiene, their attitudes and motivations towards science.] [yayınlanmamış yüksek lisans tezi]. marmara üniversitesi, eğitim bilimleri enstitüsü, i̇stanbul. dewi, i. n., poedjiastoeti, s., & prahani, b. k. (2017). elsii learning model based local wisdom to improve students’ problem solving skills and scientific communication. international journal of education and research, 5(1), 107-118. erdem, d. (2011). türkiye'de 2005–2006 yayımlanan eğitim bilimleri dergilerindeki makalelerin bazı özellikleri incelemesi: betimsel bir analiz [analysis of some features of articles in educational sciences journals published in 2005-2006 in turkey: a descriptive analysis]. eğitimde ve psikolojide ölçme ve değerlendirme dergisi , 2 (1), 140-147. flynn, s. (2019). science literacy – science education for everyday life, journal of the chartered college of teaching. https://impact.chartered.college/article/science-literacy-scienceeducation-everyday-life/ fraenkel, w., & wallen, & n. e. hyun,(2012). how to design and evaluate research in education, (eight edition). new york: mcgraw-hill. genç, m., ulugöl, s., & ünsal, s. (2017). ortaokul öğrencilerinin yaşam temelli öğrenme hakkındaki görüşleri [secondary school students' views on life-based learning]. researcher: social science studies, 5(9), 244-255. gilbert, j.k. (2006). on the nature of ‘context’ in chemical education. international journal of science education, 28(9), 957-976. gül ş. & konu m. (2018). yaşam temelli probleme dayalı öğretim uygulamalarının öğrenci başarısına etkisi [the effect of life-based problem-based teaching practices on student achievement]. yaşadıkça eğitim, 32(1), 45-68. how kids learn science best (2021). https://www.homesciencetools.com/learning-center/how-tolearn-science john, m., molepo, j. m., & chirwa, m. (2018). secondary school learners’ contextual'zed knowledge about reflection and refraction: a case study from south africa. research in science & technological education, 36(2), 131-146. kabuklu, ü. n. & kurnaz, m. a. (2019). fen eğitimi alanında türkiye’de yapılmış bağlam temelli öğretim konulu çalışmaların tematik incelemesi [thematic analysis of studies on context-based teaching in the field of science education in turkey]. asya öğretim dergisi, 7(1), 32-53. karslı-baydere, f. & kurtoğlu, s. (2020). 5. sınıf öğrencilerinin biyolojik çeşitlilik konusundaki kavramsal anlamalarına react stratejisinin etkisi [the effect of react strategy on 5th grade students' conceptual understanding of biodiversity]. yüzüncü yıl üniversitesi eğitim fakültesi dergisi, 17(1), 1015-1041. keklik, i̇. (2011). mate selection preferences of turkish university students. eurasian journal of educational research (ejer), (44), 129148 king, d. t., winner, e. & ginns, i. (2011). outcomes and ımplications of one teacher’s approach to context-based science in the middle years. teaching science, 57 (2), 26–30. kizilaslan, a., sozbilir, m., & yasar, m. d. (2012). inquiry based teaching in turkey: a content analysis of research reports. international journal of environmental and science education, 7(4), 599617. kutu, h. & sözbilir, m. (2011). yaşam temelli arcs öğretim modeliyle 9. sınıf kimya dersi" hayatımızda kimya" ünitesinin öğretimi. ondokuz mayis university journal of education, 30(1), 29-62 meb. (2020). i̇lköğretim kurumları (i̇lkokullar ve ortaokullar) fen bilimleri dersi öğretim programı [primary education institutions (primary and secondary schools) science course curriculum] (3., 4., 5., 6., 7. ve 8. sınıflar). ankara. metin, m. (2014). nicel veri toplama araçları [quantitative data collection tools]. m metin (ed) kuramdan uygulamaya eğitimde bilimsel araştırma yöntemleri (pp.161-214), ankara: pegem a yayıncılık. oduor r (2013). can science solve all the problems in the world? (for world science day for peace and development). http://writeparagraphs.blogspot.com.ng/2013/11/can-sciencesolve-all-problems-inworld.html overton, t. l. & potter, n. m. (2011). invesigating students' success in solving and attitudes towards context-rich open-ended problems in chemistry. chemistry education research and practice, 12, 294–302. özay-köse, e. & çam tosun, f. (2011). yaşam temelli öğrenmenin sinir sistemi konusunda öğrenci başarılarına etkileri [the effects of lifebased learning on student achievement in the nervous system]. türk fen eğitimi dergisi, 8(2), 91-106. özmen, h. (2014). deneysel araştırma yöntemi [experimental research method]. m metin (ed) kuramdan uygulamaya eğitimde bilimsel araştırma yöntemleri, (pp.47-76), ankara: pegem a yayıncılık. podschuweit, s. & bernholt, s. (2018). composition-effects of contextbased learning opportunities on students’ understanding of energy. research in science education, 48(4), 717-752. ramsden, j.m. (1997). how does a context-based approach infuence understanding of key chemical ideas at 16+?. internotional journal of science education, 19, 657-710. rosa, f. o., mundilarto, wilujeng, i. & sulistyani, a. m. (2019). science in everyday life to build science literacy. internatıonal journal of scıentıfıc & technology research, 8, 12, 1148-1151. sözbilir, m., sadi, s., kutu, h., & yıldırım, a. (2007). kimya eğitiminde içeriğe/bağlama dayalı (context-based) öğretim yaklaşımı ve dünyadaki uygulamaları [context-based teaching approach in chemistry education and its applications in the world], i. ulusal kimya eğitimi kongresi, 20-22 haziran, i̇stanbul. selçuk, z., palancı, m., kandemir, m., & dündar, h. (2014). eğitim ve bilim dergisinde yayınlanan araştırmaların eğilimleri: i̇çerik analizi [trends of research published in the education and science journal: content analysis]. eğitim ve bilim, 39(173), 430-453. sevian, h., dori y. j. & parchmann, i. (2018). how does stem contextbased learning work: what we know and what we still do not know. international journal of science education, 40, 10,1095-1107. silander, p. (2015). rubric for phenomenon based learning. http://www.phenomenaleducation.info/phenomenon-basedlearning.html stolk, m. j., bulte, a. m. w., de jong, o. & pilot, a. (2009). towards a framework for a professional development programme: empowering teachers for context-based chemistry education. chemistry education research and practice, 10, 164–175. sugiono, l. r. & purwastuti, a. (2017). local wisdom-based character education model in elementary scgool in bantul yogyakarta indonesia. sino-us english teaching vol.14, no. 5, 299-308. doi:10.17265/15398072/2017.05.003 şirin, s. (2020). yetişin gençler [pemuda dewasa]. i̇stanbul: doğan kitap tal, t. & dierking l. d. (2014). learning science in everyday life. journal of research in science teaching, 51,3, 251-259. tulum g. (2019). fen bilimleri dersi ışık konusuna yönelik geliştirilen bağlam temelli materyalin akademik başarı üzerine etkisi [the effect of the contextbased material developed for the subject of light in science course on academic achievement]. [yayınlanmamış yüksek lisans tezi]. ondokuz mayıs üniversitesi, eğitim bilimleri enstitüsü, samsun university of york. (2015). ocr a level salters’ advanced chemistry student book (4th ed.). oxford: open university press. [google scholar] ültay, e. & ültay, n. (2014). bağlam temelli fizik çalışmaları: literatürdeki çalışmaların içerik analizi [context-based physics studies: content analysis of studies in the literature]. hacettepe üniversitesi eğitim fakültesi dergisi (h. u. journal of education) 29(3), 197-219. wiyarsi, a., pratomo, h., & priyambodo, e. (2020). vocational high school students’ chemical literacy on context-based learning: a case of petroleum topic. journal of turkish science education, 17(1), 147161. yakar, a. & duman b. (2017). duyuşsal farkındalığa dayalı öğretimin akademik başarı ve öğretmenlik mesleğine yönelik tutumlar üzerine etkisi [the effect of teaching based on affective awareness on https://impact.chartered.college/article/science-literacy-science-education-everyday-life/ https://impact.chartered.college/article/science-literacy-science-education-everyday-life/ https://www.homesciencetools.com/learning-center/how-to-learn-science https://www.homesciencetools.com/learning-center/how-to-learn-science http://writeparagraphs.blogspot.com.ng/2013/11/can-science-solve-all-problems-inworld.html http://writeparagraphs.blogspot.com.ng/2013/11/can-science-solve-all-problems-inworld.html http://www.phenomenaleducation.info/phenomenon-based-learning.html http://www.phenomenaleducation.info/phenomenon-based-learning.html journal of science learning article doi: 10.17509/jsl.v5i1.33074 78 j.sci.learn.2022.5(1).69-78 academic achievement and attitudes towards the teaching profession.]. eğitim kuram ve uygulama araştırmaları dergisi 3 (3), 3047. yıldırım, h. i̇. & dağıstanlı, f. (2020). yaşam temelli öğrenme yaklaşımı ile destekli çevre eğitiminin ortaokul 7. sınıf öğrencilerinin çevreye yönelik tutum, davranış ve başarı düzeylerine etkisi [the effect of environmental education supported by life-based learning approach on the attitudes, behaviors and achievement levels of secondary school 7th grade students.]. mehmet akif ersoy üniversitesi eğitim fakültesi dergisi, (54), 106-132. yıldırım, n., küçük, m. & ayas, a. (2013). a comparison of effectiveness of analogy-based and laboratory-based instructions on students' achievement in chemical equilibrium. scholarly journal of education, 2(6), 63-76. yıldırım, a., & şimşek, h. (2006). sosyal bilimlerde nitel araştırma yöntemleri [qualitative research methods in the social sciences]. ankara: seçkin yayıncılık yök. (2018). fen bilgisi öğretmenliği lisans programı https: //www.yok.gov.tr /documents/kurumsal/egitim_ogretim_dairesi/yeniogretmen-yetistirme-lisans programları / fen_bilgisi_ogretmenligi_lisans_programi.pdf. a © 2021 indonesian society for science educator 337 j.sci.learn.2021.4(4).337-346 received: 22 march 2021 revised: 01 june 2021 published: 01 august 2021 the effect of the augmented reality applications in science class on students’ 21st century skills and basic skills* irem dilmen1, nurhan atalay2 1primary teacher, ministry of national education, hakkari, turkey 2department of primary education, faculty of education, niğde ömer halisdemir university, turkey corresponding author: nurratalay@gmail.com abstract this study investigates augmented reality (ar) applications on 21st-century skills and basic skills of elementary school students. the research method is determined as a mixed method. the sample group of the research consisted of 62 randomly selected elementary school 4th-grade students. in the experimental group, courses were taught using ar, and in the control group, the lessons were taught with activities included in the science class. in the research, the 21st-century learning and innovation scale and basic skills scale was used as a quantitative data collection tool, and semi-structured interview and researcher's journal were used as a qualitative data collection tool. according to the results, the study of science using augmented reality applications has positively affected the skills of elementary school 4th-grade students. the qualitative results of the research concluded that students' creativity and innovation, critical thinking, making inferences, problem-solving, collaboration, and communication skills developed. during the interviews with students and the classroom teacher, they stated that the ar application makes information more permanent by embodying the information in their minds. the lessons are fun and exciting; their interest in technology is increased and should be used in other lessons. keywords science, augmented reality, 21st century learning, and innovation skills, basic skills 1. introduction people are asked to produce in the 21st-century information age and question and criticize information through developing technology. leading countries worldwide have developed different training programs on what to teach in science (akgündüz et al., 2015). the training programs focus on stem education and 21stcentury skills. stem education applies at all levels, from pre-school to higher education, including technology and engineering practices based on science and mathematics (özsoy, 2017). in this regard, the place and types of technologies used in educational environments increase day by day and become critical. the recent use of augmented reality (ar) is also an essential aspect of technology. augmented reality is a technology that enables real and virtual objects to interact in a simultaneous artificial environment and enables users to examine virtual * this article was produced from the master thesis of the same name accepted in 2020. objects in the real world (azuma, 1997; zhou, duh, & billing-hurst, 2008). ar bridges the gap between real and virtual smoothly (chang, morreale, & medicherla, 2010). the concepts of augmented reality and virtual reality are expressed together but do not have the same meaning. vr is combining real-world and 3d objects in interactive virtual environments. augmented reality is blending the real world and real-time in a computer environment and enriching it. in other words, while virtual reality transports objects to the virtual world, augmented reality focuses on enriching reality (somyürek, 2014). ar appears to be an exciting technology for education (luckin and fraser, 2011). frequent use of advanced technologies in life increases the importance of skills. developing and advancing technology brings innovation to societies. 21st-century skills are the high-level skills that individuals need to acquire advanced technology, mailto:nurratalay@gmail.com journal of science learning article doi: 10.17509/jsl.v4i4.32900 338 j.sci.learn.2021.4(4).337-346 produce, select and analyze information, and use it in their daily lives (anagün, atalay, kılıç, & yaşar, 2016). 21stcentury learning and innovation skills are defined as skills that involve individuals' life and career skills in their learning processes; and use media, knowledge, and technology skills they use in their daily lives (atalay, 2015). scientific process skills are the skills that individuals use to recognize problems they face, produce solutions, and make decisions (taşdemir, 2016). the research aims to reveal the effect of augmented reality applications in science classes of elementary school students on 21st-century skills and basic skills. for this purpose, answers to the following questions are sought. 1. do augmented reality applications in science lessons of elementary school 4th-grade students affect their 21stcentury learning and innovation skills? 2. do augmented reality applications in science lessons of elementary school 4th-grade students affect their basic skills? 3. what are the opinions of elementary school 4th-grade students and classroom teachers on augmented reality applications? 2. method 2.1 research model in this study, embedded experimental design, one of the mixed methods, was used to examine the effect of augmented reality applications on 21st-century skills and basic skills in science class. the embedded experimental design is a mixed method in which qualitative and quantitative data are obtained sequentially or simultaneously (figure 1). data is collected to support the other group of data (creswell & plano clark, 2007). 2.2 participants this study was conducted with 62 students in an elementary school in antakya, hatay, in the 2018-2019 academic year. table 1 shows 16 female students and 15 male students in the experimental group and 12 female students and 19 male students in the control group. 2.3 data collection tools in the study, the "21st century learning and innovation scale", which consists of 39 items and three subdimensions developed by atalay (2015), was used to measure the effect of ar practices on students' 21st-century skills. meanwhile, the "basic skills scale" developed by padilla, cronin, and twiest (1985) and adapted into turkish by aydoğdu and karakuş (2015) was used to measure the basic skills of the students. the cronbach’s alpha coefficient of the 21st century learning and innovation scale was calculated as 0.70. the cronbach's alpha coefficient of the basic skills scale was calculated as 0.81 semi-structured interview form, and the researcher's journal was also used in the study. 2.4 implementation process at the first stage of the research process, the "earth's crust and the movement of our world" unit was determined from the science curriculum. in the second stage, lesson plans and worksheets were prepared according to the 5e model. in the third stage, the preparation process of materials for the ar application has been completed. in the fourth stage, the necessary programs for the augmented reality applications, internet access, and technical support were provided to tablets and made ready for implementation. the 15-course hour lecture process in the experimental group was continued with ar applications and with the current curriculum in the control group. hp reveal and space 4d + cards were used to choose the ar application materials. the researcher prepared the experimental materials such as qr codes, fossil photographs, and mine cards. the study groups were randomly selected, and the class environment was set up to a cluster layout for students, and the implementation process continued in this way. in the first week of the implementation process, "the ground shell of the earth's crust consists of the rock layer. it is not covered in the classification of rocks." learning outcome was processed. the qr codes prepared for the students were distributed and requested to be examined. a mind map of the word "rock" has been created. with the table 1 distribution of the students by gender experimental group control group gender f % f % female 16 48 12 36 male 15 52 19 64 figure 1 embedded experimental design (creswell & plano clark, 2007) journal of science learning article doi: 10.17509/jsl.v4i4.32900 339 j.sci.learn.2021.4(4).337-346 help of qr codes and augmented reality, the information showed that the rocks are made of minerals. during the assessment, the students completed the structured grid forms using the augmented reality application. in the second week, "relates rocks with minerals and discusses the importance of rocks as raw materials. it is referred to many rocks and minerals in turkey; gold, boron, marble, lignite, copper, hard coal, silver, etc., examples are given." learning outcome was processed. the students were given a video about mines, and their opinions were taken on the subject of the video. after the necessary information was provided, the material titled "mining cards" prepared by the researcher was presented, and the cards were introduced. both sides of the cards are defined in the hp reveal program. the raw states of the mines on one side of the cards and the areas of use on the other side are shown using the augmented reality application. in the third week, "explains the formation of fossils. fossil types are not covered," the learning outcome was processed. a fossil experiment was performed first in this learning outcome. after the experiment was completed, the students were given photos of the identified fossils and asked to examine them. they first predicted the species whose fossil photographs were examined, and they have also observed them through augmented reality applications. in the fourth week, "explains the difference between the earth's rotation and revolution movements," the learning outcome was processed. first, the cartoon about the rotation and revolution movement of the earth has been shown to the students in this learning outcome. then, they were asked to give examples from daily life. the students have first introduced to space 4d+ cards. then, they were asked to examine the earth and solar system card. the fifth week was started by repeating the difference between rotation and revolution movements of the earth and "explains the events that occurred as a result of the movements of the earth." a. the rotation of the earth is mentioned. b. the revolution of the earth is mentioned. c. it is mentioned that the position of the earth changes depending on the revolution around the sun. d. the formation of day and night is mentioned. d. the concepts of day, year, and time are given." learning outcome was processed. the students were given space 4d+ cards. solar system card inspection requested. at the end of the implementation, one-on-one interviews were conducted with four low, four medium, and four high-level students and teachers, taking into account the students' scores on the "21st century learning and innovation scale” (figures 2-3). figure 2 implementation process review of the curriculum and preparation of teaching plans according to 5e model preparing augmented reality application materials and providing tablet support 1st week “the ground shell of the earth’s crust consists of rock layer. it is not covered in the classification of rocks.” learning outcome was processed. 2nd week “relates rocks with minerals and discusses the importance of rocks as raw materials. it is referred to many rocks and minerals in turkey; gold, boron, marble, lignite, copper, hard coal, silver, etc. examples are given." learning outcome was processed. 3rd week "explains the formation of fossils. fossil types are not covered." learning outcome was processed. 4th week "explains the difference between the movements of the earth’s rotation and revolution." learning outcome was processed. 5th week “explains the events that occurred as a result of the movements of the earth.” learning outcome was processed. journal of science learning article doi: 10.17509/jsl.v4i4.32900 340 j.sci.learn.2021.4(4).337-346 2.5 data analysis as a quantitative data collection tool, 21st century learning and innovation skills scale and basic skills scale, as qualitative data collection tool; semi-structured interview form and researcher's journal were used in the research. in addition, analyses were conducted for the research problems. spss software was used to analyze quantitative data, and a descriptive analysis method was used to interpret the analysis of qualitative data. 3. result and discussion in this study, shapiro wilks normality analysis was performed to test whether the experimental and control group data showed normal distribution. the normality test is essential in determining which statistical formula can be used to test the research hypothesis. if the group size is less than 50, the p valu es found due to the shapiro wilks normality analysis test, which results from normality analysis, are determined as more significant than .05. ttests from parametric tests were used in the pre-post test scores of the experimental and control groups. the pre-test scores of the experimental and control groups are shown in table 2. when table 2 is examined, there is a difference of .99 between the students' average scores in the experimental and control groups from 21st century learning and innovation skills pre-test scores. it has been observed that the p-value based on the t-test result performed to determine whether this difference is statistically significant is .30. when the table is examined for the basic skills scale, there is a difference of 2.46 points between the pre-test scores obtained by students in the experimental and control group. it has been observed that the p-value based on the t-test result performed to determine if it is statistically significant is .22. the result shows that the difference between the averages of both groups is not significant. therefore, there is no statistically significant difference before the study regarding the 21st century learning and innovation skills scale and basic skills scale of the students in the experimental and control groups. in the context of this skill, it can be said that the groups have similar characteristics. according to the study, the dependent sample t-test results of the 21st century learning and innovation skills scale and the basic skills scale pre-post test scores of the experimental and control groups students are given in table 3. when table 3 is examined, there is a 5.55 point difference between the students' average scores in the experimental and control groups from the 21st century learning and innovation skills scale pre-post test scores. it has been observed that the p-value based on the t-test result was .00, which is used to determine whether this difference is statistically significant. there is a 2.10 point difference between the average scores obtained by the students in the experimental group from the basic skills scale pre-post test scores. it has been observed that the pvalue based on the t-test result was .00, which is used to determine whether this difference is statistically significant. the result shows that the difference between pre-post test scores of the experimental group is significant. therefore, there is a statistically significant difference between the prepost test scores of the students in the experiment group for the 21st century learning and innovation skills and basic skills. figure 3 application examples table 2 independent sample t-test results of experimental group and control group students’ 21st-century learning and innovation skills scale and basic skills scale pre-test scores groups n 𝐗 ss sd t p 21st century learning and innovation skills scale exp. cont. 31 31 90.61 91.60 11.10 8.74 62 -.399 .30 basic skills scale exp. cont. 31 31 20.60 18.14 3.78 4.69 59 2.26 .22 table 3 dependent sample t-test results of experimental group and control group students’ 21st-century learning and innovation skills scale and basic skills scale pre-post test scores tests n 𝐗 ss sd t p 21st century learning and innovation skills scale pre-test post-test 31 31 89.51 95.06 11.39 10.81 30 3.85 .00 basic skills scale pre-test post-test 31 31 20.70 22.80 11.39 10.81 59 2.26 .00 journal of science learning article doi: 10.17509/jsl.v4i4.32900 341 j.sci.learn.2021.4(4).337-346 according to the study, independent sample t-test results of the 21st century learning and innovation skills scale and basic skills scale post-test scores of the experimental and control group students are given in table 4. when table 4 is examined, there is a 6.32-point difference between the students' average scores in the experimental and control groups from the 21st century learning and innovation skills scale post-test scores. it was observed that the p-value based on the t-test result was .61, which is used to determine whether this difference is statistically significant. there is a 3.84 point difference between the students' average scores in the experimental and control groups from the core basic skills scale posttest scores. it was observed that the p-value based on the t-test result was .29, which is used to determine whether this difference is statistically significant. the result shows that the difference between the averages of both groups is not significant. therefore, there is no statistically significant difference between the 21st century learning and innovation skills and basic skills post-test scores of the experimental and control groups. sub-dimensions of the 21st century learning and innovation skills scale were examined in the study. dependent sample t-test results of pre-post test scores regarding the 21st century learning and innovation skills scale of experimental group students’ creativity and innovation, critical thinking and problem solving, collaboration and communication skills sub-dimension and basic skills scale prediction and communication skills sub-dimension are given in table 5. the results of the data show that there is a significant difference between the 21st-century learning and innovation skills scale of the experimental group students' creativity and innovation, critical thinking and problem solving, collaboration and communication skills subdimension pre-post test scores, a nd basic skills scale prediction and communication skill sub-dimension posttest scores (figure 4). qualitative data has resulted in students' interest in creativity, curiosity, exposure, and technology using augmented reality in the science class. students expressed the effect of the application on their creativity and table 4 independent sample t-test results of experimental group and control group students’ 21st-century learning and innovation skills scale and basic skills scale post-test scores groups n 𝐗 ss sd t p 21st century learning and innovation skills scale exp. cont. 31 31 95.06 88.74 10.81 10.46 60 2.33 .61 basic skills scale exp. cont. 31 31 22.80 18.96 4.08 4.70 59 3.40 .29 table 5 dependent sample t-test results of pre-post test scores regarding the 21st-century learning and innovation skills scale of experimental group students' creativity and innovation, critical thinking and problem solving, collaboration and communication skills sub-dimension, and basic skills scale prediction and communication skills sub-dimension test n 𝐗 ss sd t p creativity and innovation pre-test post-test 31 31 46.51 48.58 5.74 5.88 30 -2.54 0.01 critical thinking and problem solving pre-test post-test 31 31 25.70 28.45 4.76 3.72 30 -3.83 0.00 collaboration and communication pre-test post-test 31 31 17.29 18.03 3.77 3.14 30 -1.97 0.05 prediction skill pre-test post-test 31 31 4.35 4.87 0.91 0.56 30 -3.73 0.00 communication skill pre-test post-test 31 31 2.06 2.45 0.96 0.80 30 -2.44 0.02 figure 4 creativity and innovation skill creativity and innovation skill creativity exposure interest in technology curiosity journal of science learning article doi: 10.17509/jsl.v4i4.32900 342 j.sci.learn.2021.4(4).337-346 innovation skills in semi-structured interviews as follows; s1, “yeah, i imagined a watch. because there were many technological stuffs and i wanted to make a watch. you press a button like this, hot air was coming, your cold was passed, you were pushing another button, it was like an air conditioner, and the clock was coming when you opened it in a normal clockwise direction, i had imagined something like that, and i had never tried it. because it is not like the normal picture, it is very vivid, i mean augmented one is like fully in it." and s2, “i wanted to design spacecraft. for instance, you will be able to see the ground while in the air. for example, you want something. it will take it directly to the moon. there are animated films like this about space. i used to think that such technology like tablets or phones was not working, but now i have learned. i also did it at home. i have done two things like yours. i have written on it, put pictures in it and tried it.” expressed the positive effect of applying on their creativity with their views—s3, "if i wanted to design something, for example, normal cars, small models. for instance, when you jump up, the gravity decreases, or you fly up, or you can have something to make them heavier, something like thorny, and it does not fly—no, not flying. that is the one who goes to mars, something like that, the spacecraft." and s4 “there was always more (curiosity in me). for instance, you guys did something, and i went and started things like that on youtube. yes. i opened it again, and i did it. how do they get these things out? how do they do that?... so there was a channel called space adventure, and there was a nasa channel or something, and you are doing something, and you are learning from nasa. sometimes you can come up with some crazy stuff, like an alien. however, it is beautiful." expressed the positive effect of the application on their curiosity and interest level regarding technology in semi-structured interviews (figure 5). it shows that using augmented reality applications in science class of the students has gained these skills in learning new information by doing research and noticing and eliminating misconceptions. students expressed the positive effects of the application on critical thinking and problem-solving in semi-structured interviews as follows; s5 “yeah, well, at first, i did not know the earth was moving. and then i was looking at the tablet, and well, it was circling the sun. i also learned that the sun is constant. so, for example, i have become more interested in science…" and s6 “my friend, for instance, said that the world is from east to west, but i said it is from west to east. i knew the false information then, but then i learned the correct information.” views expressed. in the researcher’s journal, “they loved augmented reality application cards. observing the movements and consequences of the earth has helped students embody abstract terms and gain permanent information. they want to learn new things about space and the universe, and they have a discussion among themselves why turkey does not have a spacecraft like the other countries. it was lovely and impressive.” views expressed. it was concluded that the students had improved their collaboration and communication skills in augmented reality application in science class. students expressed the positive effects of the application on collaboration and communication skills in semi-structured interviews as follows; s7 “i was the president of that group, and i was the only one holding it, and we did it in binary order. he was very, very enthusiastic. everybody wanted to hold it like we are taking turns, and i will hold it twice, and i will hold it three times. yeah. i remember when my friends were looking and giving it to a friend in front of me, and he was looking, and everybody was seeing, and everybody was understanding." and s8 “teamwork. for example, we once said who will do it, who will do it, and then we took turns doing it. yes, it did. my communication with my friends, like, has gotten better, and i have started talking more, and i have been spending more time." views expressed. it was concluded that the students improved their prediction and communication skills in augmented reality application in science class with the qualitative data. students expressed the positive effects of the application on prediction and communication skills in semi-structured interviews as follows: s9 “firstly, we thought it was a baby bear, and then we looked at it as a crab. now we see on the tablet... many different things came out of that, which we wrote. we understood which animal it was, and we can predict better. however, it was hard because it was a regular paper. then we looked at it from the tablet, it was very, very different.” and s10 “no teacher, i had a picture of a dinosaur that looked like a flying reptile, so i wrote dinosaurs about it, i mean about my guess. i got it better. i saw something similar to marble, crab, and i wondered what it was, and i thought it was marble. so i wrote it, and i learned that it was a crab in augmented reality application. so it got better, it happened here too, when i finally looked with the augmented reality application, i found out that it was fish." expressed their views about the contributions of the implementation process. following semi-structured interviews with the students involved in the applications and implementation process, the data were discussed under seven topics (figure 6). students stated that the implementation was realistic, provided more permanent learning to them, should be used interdisciplinary, aroused curiosity and interest, embodied figure 5 critical thinking and problem solving critical thinking and problem solving skill noticing misconceptions learning new information by doing research journal of science learning article doi: 10.17509/jsl.v4i4.32900 343 j.sci.learn.2021.4(4).337-346 abstract concepts, developed observation and prediction skills, and improved career awareness. s4 "we use fossils in the application. for instance, it was best to see them like that. i did not think i would do that in a science lesson. it was very beautiful. it has been great that the technology has gone this far." and s8 "i would like it to be fun with the tablet. when you talk like this, it happens with this application. i would like it with augmented reality. they are so real. that is it, on the name." expressed their views on the creativity and innovation aspects of the application. the researcher expressed their views as “it was the first time that qr codes and augmented reality applications were introduced. they wanted to learn about what to do and what kind of a lesson it would be. the students' motivation towards the lesson increased after the application was introduced.” in the researcher’s journal (researcher’s journal, 20.09.2018). the teacher stated that the augmented reality application provides prediction and curiosity, creativity, selectivity in internet usage, and increased teamwork and communication in oneon-one meetings. göçmen (2019), in her study titled “an augmented reality-driven design for effective learning of the solar system and beyond the subject," stated that the implementation makes learning more accessible and makes lessons enjoyable. kurtoğlu (2019), in his study titled "the effect of augmented reality applications on learning processes in information technologies and software courses," stated that the implementation increases the motivation of the students, the active participation in the course, and the ambition to use the application in other courses. omurtak (2019), in her study titled "investigation of the effectiveness of augmented reality applications in a biology course and student opinions," stated that the implementation embodies the concepts. demirel (2017), in her study titled "the effect of augmented reality activities supported by augmentation approach on academic achievement, critical thinking skills, motivation towards science and technology course and argumentation skills," found that academic achievement and motivation skills of students have increased. their ability to make inferences, critical thinking skills, and argumentation skills have improved. gün (2014), in her study titled “effects of augmented reality applications on students' spatial abilities," stated that the implementation embodies the abstract concepts and makes the lessons interesting regarding students' views. these research results are similar to our research (figure 7). it has been stated by classroom teachers that augmented reality applications attract students' attention. they will now use technology more consciously. they can visualize abstract concepts more efficiently in their minds, and they significantly have improved creativity. the teachers also stated that teachers should be given in-service training, and the infrastructure of the schools should be improved. the teacher stated that the application had improved selectivity in internet usage, creativity, and prediction skills of the students with his/her views as “the fantasy worlds are so wide. children dream of everything. they live with dreams; some figure 6 student views on the implementation student vi̇ews realistic provides permanent learning usable in different lessons mbodying abstract terms arousing curiosity and interest career awareness observation and prediction skill journal of science learning article doi: 10.17509/jsl.v4i4.32900 344 j.sci.learn.2021.4(4).337-346 children tell us things that we have not. they are curious, and they are exploring online. at least in this way, they have also seen that instead of watching or following things that are useless or cause them to gain bad behaviors on the internet, this type of research can both add better things to themselves and expand their horizons. i think they will be selective now. i mean, they do not follow whatever comes to them. i think they would be more selective to follow things that might open up different ways of thinking that might add something to them. i mean, they are more worried. they asked themselves like "what will come out? what is this? how will this work?". i think it also helped their creativity. regarding curiosity and prediction, the children were inquisitive about what would come out. they understand one thing. did i do it right or not?" the teacher stated that the augmented reality application contributes to the collaboration and communication skills of the students with his/ her views as “students do the science or social studies class as group work. they are preparing their subjects. they bring their experiments. however, this app is something very different from theirs, and they liked it. they love tablets. "i looked," "you look too," "let us look together." they had that kind of sharing. of course, some students want to use it by themselves. this is what we encounter a lot in our daily lives. they had to share the tablets they used in the house alone but did not make an issue out of it, in turn, they somehow found a solution, and then they were again excited while comparing their predictions with the notes they took. they surprised and said, "oh, i guessed it right." of course, there are some problems with the children in sharing. nevertheless, it is good that it was supported by such activity.” matcha and rambli (2013), in their study titled "exploratory study on collaborative interaction through the use of augmented reality in science learning," concluded that augmented reality technology has improved cooperation and communication skills in science education. these research results are similar to our research results. the teacher stated that he/she teach fun lessons with the students through the application and that technology should be used more in education with his/her views in general as "we usually use notebooks and books when we teach the lessons. we open the internet from the smart board where we use the most technology. smartboard does not open everything in schools; it is restricted and does not open private channels. we do not have the opportunity to do such activities since it only opens channels allowed by the ministry of national education. we are not equipped in this regard either. i liked that application very much. the children saw many different things and waited with excitement. we are pleased with your work, and if we had any training on that subject, i would like to implement it in the same way. of course, it helped. they were much happier to see them in 3d rather than seeing them from a notebook, book, or photo in one-dimensional view. it aroused their curiosity and waited with great curiosity. parents were very happy about it. as the children were happy, they liked it too. i think it is useful. i hope they have not forgotten; they will be curious about it and continue their research. some students will be curious and continue. it was very suitable for them." büyükuygur (2018), in his study, stated that augmented reality application improves the vocabulary of the students by using it in foreign language teaching and makes the lessons more interesting. sırakaya and alsancak sırakaya (2018) examined augmented reality use in science education on attitude and motivation and found that the students were interested in astronomy subjects. these research figure 7 teacher views on the implementation teacher vi̇ews curiosity prediction communication interdisciplinary relationship creativity teamwork selectivity in internet usage journal of science learning article doi: 10.17509/jsl.v4i4.32900 345 j.sci.learn.2021.4(4).337-346 results are similar to our research results. yıldırım's (2016) study showed a positive increase in students' perception and attitudes towards problem-solving skills. squire and jan (2007) found in their research that augmented reality technology has improved 21st-century skills. these research results are similar to our research results. it was observed that some of the students wanted to use the technological tools and equipment used in the application individually, and they could not be problem-solving groups much. however, it was determined that students who wanted to work individually in other weeks were included in the group, and groups have solved problems they encountered by collaborating. another result obtained under the title of cooperation and communication is that because students are in randomly created groups, their communication with their classmates increases, and a positive classroom climate is achieved. ibanez and kloss (2018), in their research titled "augmented reality for stem learning: a systematic review," compiled research on the use of stem and augmented reality and suggested the use of augmented reality application in the stem discipline. linder, rineow, and jürgens (2019) were highlighted the relationship between augmented reality and stem activities in their research. these research results are similar to our research results. delello (2014), in her study titled "insights from pre-service teachers using science-based augmented reality," concluded that the implementation positively affected the classroom environment and increased motivation and participation. demirer and erbaş (2015), in their research titled “investigation of mobile augmented reality applications and evaluation of educational perspective," recommended applications that should be used in an educational environment. kara (2018), in his research titled "investigation of research on the use of augmented reality practices in education," concluded that the effect of augmented reality applications is a matter of curiosity in education, and the studies have increased since 2012. the studies have been focused on science and engineering. soylu (2019), in his research titled “the effect of the education program based on augmented reality practices on pre-school teacher candidates' attitudes and views," concluded that pre-school teacher candidates' interest in technology increased. these research results are similar to the results of the study. based on the findings and results obtained in the study, the following recommendations can be made. 4. conclusion the research investigates the effect of ar on students’ 21st-century learning and innovation skills and basic skills. after implementation, the experimental group students' 21st-century learning and innovation skills differ significantly between pre-test and post-test scores. it reveals that the effective use of augmented reality in lessons has positively affected 21st-century skills. there was a significant difference in favor of the post-test score was observed when pre-test and post-test scores of the experimental group were compared regarding the results of quantitative data on the dimensions of "creativity and innovation", "critical thinking and problem solving", and "collaboration and communication", which are identified as the sub-dimensions of the 21st-century learning and innovation skills scale it was noticed in the activities that the students realized their misconceptions at the end of the implementation process, began to question the accuracy of the information they encountered and gained critical thinking skills. it was also observed that the students learned when they were curious about it throughout the process from various sources through questioning and research. it was concluded that while students had difficulty solving the problems they encountered in the activities and during the implementation process, they tried to solve and reached a solution by finding different solutions towards the end of the process. when the basic skill scores of the experimental and control group students were analyzed within the groups, there was no significant difference in the basic skills scale scores of the students in the control group. in contrast, a significant difference was found in the scores of the experimental group students. this result shows that augmented reality application has a positive effect on students' basic skills. acknowledgement the present study was not supported by any foundations, institutions, or others. therefore, there is no funding to report for this submission. we confirm that all co-authors contributed equally to the study. to the best of our knowledge, there is no conflict of interest.. references akgündüz, d., aydeniz, m., çakmakçı, g., çavaş, b., çorlu, m. s., öner, t., & özdemir, s. (2015). stem eğitimi türkiye raporu [stem education turkey report]. i̇stanbul: scala basım. anagün, s. ş., atalay, n., kılıç, z., ve yaşar, s. (2016). the development of a 21st century skills and competences scale directed at teaching candidates: validity and reliability study. pamukkale eğitim fakültesi dergisi, (40), 160-175. atalay, n. (2015). slowmation application to student's learning and innovation skills improvement in science course (unpublished doctoral thesis). instıtue of educational sciences, anadolu university eskişehir. aydoğdu, b., & karakuş, f. (2015). the adaptation study to turkish of basic process skills scale towards primary students. mehmet akif ersoy üniversitesi eğitim fakültesi dergisi, 1(34), 105-13. azuma, r. t. (1997). a survey of augmented reality. teloperators and virtual environments, 6(4), 355 -385. büyükuygur, m. (2018). developing of educational game application for improving primary school students' english learning via using augmented reality technology (unpublished master's thesis). i̇stanbul aydın university, i̇stanbul. chang, g., morreale, p., & medicherla, p. (2010). applications of augmented reality systems in education. in d. gibson & b. dodge journal of science learning article doi: 10.17509/jsl.v4i4.32900 346 j.sci.learn.2021.4(4).337-346 (eds.), proceedings of society for information technology & teacher education international conference 2010, 1380-1385. chesapeake, va: aace. creswell, w. j. & clark, p. v. (2007). karma yöntem araştırmaları tasarım ve yürütülmesi (dede y., & demir b. s., trans). anı yayıncılık, ankara. delello, j. a. (2014). insightsfrompre-service teachersusingsciencebasedaugmentedreality. journal of computers in education, 1(4) , 295–311. http://doi.org/10.1007/s40692-014-0021-y demirel, t.(2017). argümantasyon yöntemi destekli artırılmış gerçeklik uygulamalarının akademik başarı, eleştirel düşünme becerisi, fen ve teknoloji dersine yönelik güdülenme ve argümantasyon becerisi üzerindeki etkisinin incelenmesi [investigation of the effect of augmented reality applications supported by argumentation method on academic achievement, critical thinking skills, motivation for science and technology course and argumentation skills] (unpublished doctoral thesis). çukurova üniversitesi sosyal bilimler enstitüsü, adana. demirer, v., & erbaş, ç. (2015). mobil artırılmış gerçeklik uygulamalarının incelenmesi ve eğitimsel açıdan değerlendirilmesi [examination and educational evaluation of mobile augmented reality applications]. mersin üniversitesi eğitim fakültesi dergisi, 11(3), 802-813. göçmen, h. (2019). güneş sistemi ve ötesi konusunun etkili öğrenimi için artırılmış gerçeklik odaklı bir tasarım [an augmented realitydriven design for effective learning of the solar system and beyond the subject]. afyon kocatepe üniversitesi fen bilimleri enstitüsü, afyonkarahisar. gün, e. (2014). artırılmış gerçeklik uygulamalarının öğrencilerin uzamsal yeteneklerine etkisi [effects of augmented reality applications on students' spatial abilities] (unpublished master’s thesis). gazi üniversitesi, eğitim bilimleri enstitüsü, ankara. ibanez, m. b., & kloos, d.c. (2018). augmented reality for stem learning: a systematic review. computers ve education, 123, 109-123. https://doi.org/10.1016/j.compedu.2018.05.002 kara, a. (2018). artırılmış gerçeklik uygulamalarının eğitimde kullanılmasına yönelik araştırmaların incelenmesi [examination of research on the use of augmented reality applications in education] (master’s thesis). atatürk üniversitesi eğitim bilimleri enstitüsü, erzurum. kurtoğlu, y. b. (2019). artırılmış gerçeklik uygulamalarının bilişim teknolojileri ve yazılım derslerinde öğrenme süreçlerine etkisi [the effect of augmented reality applications on learning processes in information technologies and software courses] (doctoral dissertation), lisansüstü eğitim enstitüsü (bilgisayar ve öğretim teknolojileri eğitimi). lindner, c., rienow, a., & jürgens, c. (2019). augmented reality applications as digital experiments for education – an example in the earth-moon system. acto astronoutica, 16, 66-74 https://doi.org/10.1016/j.actaastro.2019.05.025 luckin, r., & fraser, d. s. (2011). limitless or pointless? an evaluation of augmented reality technology in the school and home. international journal of technology enhanced learning, 3(5), 15, doi:10.1504/ijtel.2011.042102 matcha, w., & rambli, d. r. a. (2013). exploratory study on collaborative interaction through the use of augmented reality in science learning. procedia computer science, 25, 144-153. http://doi.org/10.1016/j.procs.2013.11.018 . omurtak, e. (2019). biyoloji dersinde artırılmış gerçeklik uygulamalarının etkililiğinin incelenmesi ve uygulamalara ilişkin öğrenci görüşleri [investigation of the effectiveness of augmented reality applications in a biology course and student opinions] (master's thesis), karamanoğlu mehmetbey üniversitesi. özsoy, n. (2017). stem ve yaratıcı drama [stem and creative drama]. ahi evran üniversitesi kırşehir eğitim fakültesi dergisi, 3(18), 633-644. sırakaya , m., & alsancak sırakaya, d. (2018). artırılmış gerçekliğin fen eğitiminde kullanımının tutum ve motivasyona etkisi [the effect of the use of augmented reality in science education on attitude and motivation]. kastamonu education journal, 26(3), 887-896. padilla, m., cronin, l., & twiest, m. (1985). the development and validation of the test of basic process skills. paper presented at the annual meeting of the national association for research in science teaching, french lick, in. somyürek, s. (2014). öğrenme sürecinde z kuşağının dikkatini çekme: artırılmış gerçeklik. eğitim teknolojisi kuram ve uygulama dergisi, 4(1), 63-80. soylu, s. m. (2019). artırılmış gerçeklik uygulamalarına yönelik eğitim programının okul öncesi öğretmen adaylarının tutum ve görüşlerine etkisi [the effect of the training program on augmented reality applications on he attitudes and opinions of pre-school teacher candidates] (master’s thesis). pamukkale üniversitesi eğitim bilimleri enstitüsü, denizli. squire, k. d., & jan, m. (2007). mad city my story: developing scientific argumentation skills with a place-based augmented reality game on handheld computers. journal of science education and technology, 16(1), 5–29. http://doi.org/10.1007/s10956-006-9037-z taşdemir, a. (2016). bilimin doğası ve bilimsel süreç becerileri [the nature of science and scientific process skills]. in m. demirbaş, fen bilimleri öğretiminde bilimin doğası [the nature of science in science teaching] (p. 190-221). ankara: pegem akademi. yıldırım, s. (2016). fen bilimleri dersinde artırılmış gerçeklik uygulamalarının öğrencilerin başarısına, motivasyonuna, problem çözme becerilerine yönelik algısına ve tutumlarına etkisi [the effect of augmented reality applications in science lesson on students' success, motivation, perception and attitudes towards problem solving skills] (master’s thesis). ankara üniversitesi fen bilimleri enstitüsü, ankara. zhou, f., duh, h. b. l., & billinghurst, m. (2008). trends in augmented reality tracking, interaction and display: a review of ten years of ismar. in 2008 7th ieee/acm international symposium on mixed and augmented reality (pp. 193-202). ieee. http://doi.org/10.1007/s40692-014-0021-y https://doi.org/10.1016/j.compedu.2018.05.002 https://doi.org/10.1016/j.actaastro.2019.05.025 http://doi.org/10.1016/j.procs.2013.11.018 http://doi.org/10.1007/s10956-006-9037-z a © 2022 indonesian society for science educator 1 j.sci.learn.2022.5(1).1-13 received: 12 february 2021 revised: 2 june 2021 published: 1 march 2022 #a part of this article was presented as an oral presentation at the 28th international congress of educational sciences (ices/uebk) held in ankara/turkey on 25-28 april 2019 the effect of using digital concept cartoons in science lesson on students’ achievement# arda danıştay şanlıtürk1, gülçin zeybek2* 1mareşal mustafa kemal secondary school, ministry of education, karaman, turkey 2department of educational sciences, faculty of education, karamanoğlu mehmetbey university, merkez/karaman, turkey *corresponding author: gulcinzeybek@kmu.edu.tr abstract in the digital age we live in, electronic teaching materials can effectively increase students' participation in the learning and teaching process. in addition, it is known that using visual materials in science education has a positive effect on learning in this direction. the study aims to determine the effect of digital concept cartoons in the sixth-grade science course on the academic achievement of students and the retention of what is learned in the lesson and to determine the students' views on the concept cartoons applied in the lesson. this research was carried out with sequential mixed design, one of the mixed designs in which quantitative and qualitative research methods are used. the quantitative dimension of the study was conducted with a random experimental design with the pretest-posttest control group, and the qualitative dimension was carried out with a case study design to support the quantitative data and make an in-depth analysis of the data. a significant difference in favor of the experimental group was found between the post-test and retention test mean scores of the experimental and control group students participating in the study. based on these findings, it can be said that the use of digital concept cartoons in the sixth-grade science course "reproduction, growth, and development in plants and animals" unit is effective in ensuring academic achievement and the retention of what is learned in the lesson. this finding of the study is parallel with qualitative findings. keywords instructional technologies, electronic learning materials, concept cartoons 1. introduction approaches and methods based on the active participation of students in the process of science teaching have taken their place in the curricula of countries. based on students' learning by doing, experiencing, and establishing relationships between their prior knowledge and new knowledge, these education programs aim to raise individuals who research and produce knowledge. therefore, constructivism has been an essential orientation for research and applications in science teaching (geelan, 1997). in the constructivist approach, the student should actively participate in the learning process to be an active learner and structure knowledge (kroasbergen & van luit, 2005). for this reason, it is important to use visual tools that are thought to enable students to actively participate in the process in science lessons arranged according to the constructivist approach. in the constructivist approach, learning is considered a subjective process. from this perspective, learningteaching strategies and materials to be used in the teaching process are very important in increasing the quality of learning (çepni, 2010). for example, concept cartoons are shown among highly effective teaching materials that can increase the quality of learning products (chin & teou, 2009). 1.1 concept cartoons concept cartoons are recommended as tools that can be used in science teaching in terms of both preparing a teaching environment suitable for the constructivist approach and minimizing the problems related to the classroom environment (naylor & mc murdo, 1990; keogh & naylor, 1997; keogh, naylor, & wilson, 1998). concept cartoons, which were first put forward in 1992 by brenda keogh and stuart naylor to develop an innovative teaching strategy, are drawings in the form of cartoons that include daily events containing scientific elements and in which different opinions about these daily events are mailto:gulcinzeybek@kmu.edu.tr journal of science learning article doi: 10.17509/jsl.v5i1.32097 2 j.sci.learn.2022.5(1).1-13 presented by the characters (keogh & naylor, 1999b). in other words, concept cartoons; are drawings designed as a stimulus to generate scientific ideas, spark discussion, attract attention, and ask questions (keogh, naylor, de boo, & feasey, 2001). the constructivist learning approach, which argues that the knowledge is interpreted and created by the individual, is the basis of it (morris, merritt, fairclough, birrell, & howitt, 2007). concept cartoons show different characteristics compared to regular cartoons. while cartoons are used to make individuals laugh, concept cartoons entertain students and question their knowledge (keogh & naylor, 1999b). thanks to concept cartoons, scientific thoughts are adapted to daily life in order for students to establish a connection between daily life and science (keogh et al., 1998). therefore, the focus of each concept caricature is the daily life events related to the learners' own experiences (naylor & keogh, 1999). according to dabell (2008), the way to follow while applying concept cartoons can be listed as follows: students are directed to the confusion of concepts with the help of cartoons. a suitable environment is provided for students to express their personal views on the topic in a discussion environment. students are encouraged to have small group discussions. a common result is tried to be reached. it is ensured that the common ideas reached are shared with the class. outcome ideas of small groups are discussed with all class members. how the students' ideas have changed is evaluated. reinforcement and application are made to ensure that the right ideas are permanent. 1.2 benefits of using concept cartoons in education concept cartoons; it is effective in visualizing the subject in science teaching, encouraging students to discuss and ensuring active participation in the lesson, enabling students to compare their thoughts with each other, seeking reasons on which students can base their ideas, and justifying their ideas (morris et al., 2007). in addition, presenting the concepts as drawings in cartoons attracts the students' attention and increases their interest in the concepts. these tools can be used as a starting point to reveal students' thoughts and encourage students to discuss. according to kinchin (2004), concept cartoons are helpful tools for focusing and participating in class discussions. in addition, concept cartoons used in the discussion environment help reveal characteristics such as how students show their behaviors and express their opinions in science classrooms (coll, france, & taylor, 2005). the characters in the concept cartoons reveal different views. they show that there can be different perspectives about an event and make this event a problem for learners (keogh & naylor, 1999b). in addition, concept cartoons offer learners alternative views, new and different perspectives that they did not think of before in many cases (keogh & naylor, 1999a). each student interprets the situation given in the concept cartoons by associating it with the current information (keogh & naylor, 2000). therefore, it can be said that concept cartoons enable students to use their existing knowledge, create new perspectives, and see every alternative idea in concept cartoons as acceptable. giving students both correct and incorrect statements about an event in concept cartoons enables a cognitive conflict to occur in mind (naylor & keogh, 1999). it may not be possible to represent the daily event under investigation in actual conditions accurately. many environmental conditions can cause students not to see the event correctly. for this reason, it is stated that concept cartoons will change the conditions and offer a better opportunity to observe different situations (stephenson & warwick, 2002). concepts that are mental tools enable individuals to think and communicate meaningfully; it plays a vital role in ensuring that the individual understands his / her environment and events correctly (mulhan, 2007). concept cartoons can also help students to identify their prior knowledge and to realize their misconceptions. however, since concept cartoons do not directly give the correct answer to students, students' existing misconceptions can be changed with the help of discussion and a cognitive balancing process (martinez, 2004). for this reason, concept cartoons function as a stimulus that leads to the change and development of students' thoughts and helps students to question their thoughts, increase their interest and motivation (keogh & naylor, 1996). technology-based knowledge and skills that individuals should have in the 21st century are much different than in the past. therefore, developments in technology must be reflected in the educational environments of the new generation. the use of technology and digital materials in teaching can be an effective way to meet the needs of today's students. through the preparation of concept cartoons in an electronic environment, digital concept cartoons, which are the subject of this study, emerge. 1.3 purpose and importance of the research as naylor, downing, and keogh (2001) stated in their studies; concept cartoons can be used as a stimulus in science lessons where students can conduct applied research to solve different opinions that arise from their discussions. in the literature, there are studies (chin, 2001; keogh et al., 2001; klavir & gorodetsky, 2001; stephenson & warwick, 2002; dabell, 2004; parkinson, 2004; balım, i̇nel, & evrekli, 2008; akamca & hamurcu, 2009; chin & teou, 2009; i̇nel, balım, & evrekli, 2009; naylor & keogh, 2009; sexton, gervesoni, & brandenburg, 2009; cengizhan, 2011; evrekli, i̇nel & balım, 2011; i̇nel & journal of science learning article doi: 10.17509/jsl.v5i1.32097 3 j.sci.learn.2022.5(1).1-13 balım, 2011; atasoy, tekbıyık, & gülay, 2013; tokcan & alkan, 2013; çinici et. al., 2014; demirel & aslan, 2014; topcubaşı & polat, 2014; akbaş & toros, 2016; gümüş, kavanoz, & yılmaz, 2017; yaşar, 2017; özdemir, yildizdurak, karaoğlan yılmaz, & yılmaz, 2018; şahin, 2018; cerrah özsevgeç, yurtbakan, & uludüz, 2019; demirci, 2019) that reveal that concept cartoons are essential tools in ensuring active participation of students in the lesson, in revealing and increasing their conceptual understanding, in determining and eliminating misconceptions, in revealing alternative concepts and individual thinking styles, in developing academic achievement , inquisitive learning skills and affective characteristics. according to the science curriculum, which has a constructivist structure, it is necessary to create classroom environments that will allow students to think creatively and critically, be open to communication and participatory, and investigate and question. therefore, one of the critical purposes of science lessons is to allow students to examine daily life examples to construct concepts scientifically in their minds while teaching in active environments (ministry of education, 2005). in this context, it is crucial to benefit from visual tools such as concept cartoons that enable students to participate in the process actively and support them to reach solutions by questioning the problems they encounter in daily life in science lessons arranged according to the constructivist theory. it is predicted that the use of concept cartoons in the science course will be effective in ensuring that students who are intensely confronted with concepts gain a positive attitude towards the lesson, making the lesson more enjoyable and learning more permanent. in addition to these, the use of technology in classroom activities gains importance daily and brings a new digital dimension to teaching. it is known that the education to be given to students who need to keep up with the rapid development and change of information and communication technologies by using these technologies in lessons contributes to them from different angles. therefore, it is vital to carry out studies on the use of digital concept cartoons, which are different teaching materials that attract students' attention, in science lessons. it is thought that this study will contribute to the field in terms of being an example for the researches that can be done in this field in learning environments today, where technology integration in education is supported. in addition to this, the research contribution to science is considered necessary due to quantitative and qualitative data collection methods together in the study, providing data diversity and enriching the research process. in this direction, the study aims to determine the effect of digital concept cartoons in the sixth-grade science course on the academic achievement of students and the retention of what is learned in the lesson and to determine the students' views on the concept cartoons applied in the lesson. 2. method this section includes information on the research design, study group, data collection tools, data collection process, and data analysis. 2.1 research design this research was carried out with sequential mixed design, one of the mixed designs in which quantitative and qualitative research methods are used. sequential mixed designs mean mixed-method patterns in which successive study stages occur within a specific time sequence (qualitative, quantitative, or vice versa) (teddlie & tashakkori, 2009: 172). the quantitative dimension of the study was conducted with a randomized pretest-posttest control group experimental design. firstly, in this design, two groups are formed by random assignment from the previously determined sample pool. one of the groups is determined randomly as the experimental group and the other as the control group. then, the measurements of the subjects in the two groups about the dependent variable are taken. in the application process, the experimental procedure whose effect is tested is given to the experimental group but not to the control group. finally, the measurements of the subjects in the groups for the dependent variable are tested using the same tool or peer form (büyüköztürk, kılıç çakmak, akgün, karadeniz, & demirel, 2012). a pre-test in the model enables us to know the groups' pre-experiment similarity levels and arrange the post-test results accordingly. furthermore, the opportunity to work with the experimental and control groups in this design contributes to determining the actual effect of the experimental process. these two advantages make the pattern a solid experimental design. the qualitative dimension was carried out with a case study design to support the quantitative data and analyze the data. in the case study, the factors (environment, individuals, events, processes, etc.) related to a situation are investigated in a holistic approach and focuses on how they affect the relevant situation and how they are affected by the relevant situation (yıldırım & şimşek, 2018). 2.2 study group the study group of the research consisted of 42 students studying in the 6th grade of a secondary school in karaman city in turkey in the 2017-2018 academic year. at the beginning of the 2017-2018 academic year, 6th-grade students were divided into four classes by random method. before the research, one of these four classes was randomly determined as the experimental and control groups. both groups consisted of 21 students, and the average age was 11. while 10 of the students in the experimental group were girls and 11 were boys, 9 of the students in the control group were girls, and 12 were boys. in order to determine whether the experimental and control group students participating in the study are equivalent in terms of academic achievement on "reproduction, growth, and journal of science learning article doi: 10.17509/jsl.v5i1.32097 4 j.sci.learn.2022.5(1).1-13 development in plants and animals" unit of the science lesson before the practical application, an independent samples t-test was applied on the pre-test scores. the results are given in table 1. when table 1 is examined, there is no significant difference between the experimental and control group students (p≤0.05). based on this finding, it can be said that the experimental and control group students are equivalent in terms of academic achievement on the sixth-grade science lesson unit of "reproduction, growth, and development in plants and animals". 2.3 data collection tools academic achievement test a multiple-choice achievement test consisting of 27 questions was prepared by the researchers to be applied as a pre-test and a post-test in the study. relevant course and question books were used while determining the questions to be included in the academic achievement test. after the determined questions were corrected in the context of the opinions of the field and language experts, the achievement test was applied to 136 students studying in another secondary school in the same province for a reliability study. after the pilot application, five items were removed from the test due to low discrimination (rjx) values, and the test was finalized. according to bloom's taxonomy, the questions in the final test belonged to the level of knowledge, comprehension, and application. the reliability of the final test consisting of 22 questions was calculated with the kr-20 method, and the reliability coefficient was found to be 0.87. therefore, looking at this value, it is possible to say that the achievement test is reliable. student interview form the researchers developed a semi-structured interview form to determine the experimental group students' views on digital concept cartoons applied in science lessons. this interview form was finalized by benefiting from the opinions of the experts. to give examples of the questions in the interview form can be specified as; "at which stage did you have the most difficulty while preparing/presenting digital concept cartoons? why?", "what were the phases you enjoyed most during the implementation process? why?", "did this method you use contribute to learning the subject? if you think it contributed, what kind of contribution did it make?", "how do you think the effect of the application process on the classroom environment and friendship relations?", "how did this practice affect your interest in a science lesson?" 2.4 implementation process and data collection the experimental application was carried out for six weeks in the sixth-grade science lesson "reproduction, growth, and development in plants and animals" unit. during this period, digital concept cartoons were applied in the experimental group, while the current program was continued to be implemented in the control group. at the beginning of the application process in the experimental group, students were informed about concept cartoons, and digital concept cartoons were introduced. students were directed to the confusion of concepts through concept cartoons in the first stages of teaching based on concept cartoons. a suitable environment was created for them to express their personal opinions and participate in the discussion. in addition, students were encouraged to investigate the accuracy of different thinking styles in the cartoon and reinterpret the cartoon's ideas in light of the research findings. subsequently, a typical result was tried to be reached. these common ideas were shared with the class, how the students' ideas changed were evaluated, and reinforcement and implementation were made to ensure that the right ideas reached were permanent. as the experimental process progressed, the activity of the students increased, and they created their digital concept cartoons, and they conducted research and discussions through these cartoons. therefore, digital concept cartoons were used as readymade materials for the course at the beginning of the experimental process; in the following weeks, it can be said that the students prepared it. that is, it was a student product during the lesson. while these operations were carried out in the experimental group, the current program was continued to be executed simultaneously in the control group. accordingly, the lesson in the control group was taught by the teacher with the method of the lecture as before. while the experimental process (application of digital concept cartoons), whose effect was tested in the application process in the study, was given only to the experimental group, there were no interventions in the control group. all these things are thought to contribute to the determination of the actual effect of the experimental process. the course teacher and the students used two different online software during the digital concept cartoon application process. the same academic achievement test was applied to both groups as a post-test after the experimental application and as a retention test 3 weeks after the experimental process. at the end of the experimental process, one-by-one interviews were done with the students by the researchers in order to get the opinions of the experimental group students on the digital concept cartoons. 2.5 data analysis descriptive statistical techniques and unrelated samples t-test were used in the analysis of the quantitative data of the study. descriptive analysis technique was used to table 1 t-test results of the students' pre-test scores according to the group group n x ̄ s sd t p experiment 21 8.38 3.06 40 -1.407 0.167 control 21 9.76 3.30 p≤0.05 journal of science learning article doi: 10.17509/jsl.v5i1.32097 5 j.sci.learn.2022.5(1).1-13 analyze the qualitative data collected through the interview form in the study. the data obtained from the students' learning diaries were arranged and interpreted according to the themes determined at the beginning of the research process. the following validity and reliability measures were applied in the quantitative and qualitative dimensions of the study. 1. in the study, in which quantitative and qualitative data collection methods were used together, the researchers became a natural part of the research process by spending time in the field both during the experimental process and the interview process and by doing direct interviews with the participants. therefore, in this study, the researchers were in a participatory role. in addition, their proximity to the participants increased the validity (creswell, 2013). 2. the processes followed in preparing the achievement test and the interview form used in the research are explained under data collection tools. 3. the research process has been enriched with qualitative and quantitative data collection techniques. in addition, different data collection techniques were used together to increase the credibility of the research. 4. the study's qualitative findings were clearly stated, and participant confirmation was obtained for the data collected by one by one interviews with the students. the findings obtained from the interviews were presented through direct quotations to increase the transmissibility, and the researcher's opinions were reflected in the interpretation phase after the data were collected and analyzed. 5. the experts examined the analysis results of the collected data. 3. findings 3.1 quantitative findings of the study in order to compare the post-test scores of the students according to the group, independent samples t-test was performed, and the results are given in table 2. when table 2 is examined, a significant difference is observed between the post-test scores of the students in the experimental and control groups in favor of the experimental group (p≤0.05). based on this finding, it can be said that the use of digital concept cartoons in the sixthgrade science course "reproduction, growth, and development in plants and animals" unit has a positive effect on academic achievement. in order to compare the permanence test scores of the students according to the group, independent samples ttest was performed, and the results are given in table 3. when table 3 is examined, it is seen that there is a significant difference between the retention test scores of the experimental and control group students in favor of the experimental group (p≤0.05). based on this finding, it can be said that the use of digital concept cartoons in the sixthgrade science lesson "reproduction, growth, and development in plants and animals" unit is effective in ensuring the permanence of what has been learned. 3.2 qualitative findings of the study the findings obtained from the interviews with the students participating in the study are presented in table 4. these findings were expressed under the titles of "general opinions", "pleasant stages", “difficult stages”, "positive opinions," and "suggestions" according to student responses. as stated in table 4, considering the participant student views in the research under general opinions, their enjoyable, entertaining, and instructive expressions came to the fore. to exemplify some student quotations regarding this situation: table 2 t-test results of the students' post-test scores according to the group group n x ̄ s sd t p experiment 21 19.19 1.78 40 9.863 0.000 control 21 14.05 1.60 p≤0.05 table 3 t-test results of the students' retention test scores according to the group group n x ̄ s sd t p experiment 21 13.14 1.85 40 3.488 0.001 control 21 11.05 2.04 p≤0.05 table 4 findings from interviews with students general opinions enjoyable, fun, educational pleasant stages generating ideas, discussing difficult stages adding background, character, and speech bubble, entering to program, and saving the study positive opinions quick thinking and quick responses taking notes, summarizing, doing homework topic repetition and understanding the topic willingness to attend class, motivation, concentration, energy helping and solidarity benefiting from different ideas, meeting on common ground suggestions application in other lessons development of the mobile application providing the ability to create your character and record sound journal of science learning article doi: 10.17509/jsl.v5i1.32097 6 j.sci.learn.2022.5(1).1-13 s1: "in the past, it was a bit boring to listen and write in science classes all the time, but thanks to this application, i attended the class with fun." s5: "it was both fun and instructive for us to process this unit like this." s6: "i think it is very fun and something i did myself, so i liked it." s8: "i think it was a very good event. because in this way, i learned the subject of 'reproduction in plants and animals very well." s10: "our teacher was lecturing and solving tests in other lessons. the lesson could be boring from time to time. but the lessons we taught through digital concept cartoons were much more enjoyable." s21: "it was fun because when i design something, i enjoy it very much." s22: “we always use books and notebooks in other lessons. it was fun to use technology in this lesson." considering the participant student views in the research under the title of pleasant stages, generating ideas and discussing expressions came to the fore. to exemplify some student quotations regarding this situation: s1: "i enjoyed adding text to cartoons the most. because i liked thinking, finding, and answering the question." s3: "it was an enjoyable stage to generate and discuss ideas." s8: "setting a background was enjoyable for me." considering the participant student views in the research under the title of difficult stages; adding background, character and speech bubble; entering to program and saving the study expressions came to the fore. to exemplify some student quotations regarding this situation: s1: “i had a hard time adding characters. because there were so many characters that it was difficult for me to choose." s2: “i had difficulty entering the application. because he often gave errors." s3: "continuous shifting of the background made it difficult for me to work." s11: “i had difficulty making the speech bubbles. it was very difficult to fit the text and ensure it did not slip." s13: "i had the most difficulty in the recording. this is because the program was in a foreign language." s15: “i had difficulty in speaking my ideas. because i was so excited." s22: "i had difficulty making the characters talk." nil: did you know that when living things produce individuals similar to themselves, it is called "reproduction"? omar: of course, it is also called "reproduction". all living things can reproduce, but reproduction is not necessary for the living thing to survive. figure 1 reproductive a: reproduction in living things is of two types, "sexual" and "asexual". b: asexual reproduction too; there are five types: "division", "budding", "regeneration", "vegetative", and "spore reproduction". figure 2 sexual and asexual reproduction journal of science learning article doi: 10.17509/jsl.v5i1.32097 7 j.sci.learn.2022.5(1).1-13 considering the participant student views in the research under the title of positive opinions; quick thinking and quick responses; taking notes, summarizing, doing homework; topic repetition and understanding the topic; willingness to attend class, motivation, concentration, energy; helping and solidarity; benefiting from different ideas, meeting on common ground expressions came to the fore. to exemplify some student quotations regarding this situation: s1: "now i can think faster and answer questions faster." s12: "it made it easier for me to take notes, summarize and do my homework." s14: "i made my subject repetitions through concept cartoons." s18: “i think it contributed to my understanding." s19: "i think that seeing the different studies of my friends contributed to my learning." s1: "science lessons are more fun now. my desire to participate has increased." s14: "thanks to digital concept cartoons, i entered science lessons more motivated and energetic and concentrated better." s1: "i think it taught us all that we can help each other and ask each other on matters we do not know." s10: "we talked with our friends about the cartoons we created even between classes." s11: "i think it was good because we all asked each other more things." s13: “i think we learned to meet at a common point." s14: "we also had the opportunity to listen to each other's ideas and benefit from them." s19: "in-class solidarity increased thanks to asking our friends what we could not understand." c: the new creatures that have emerged have the same features as the ancestor. d: it does not cause diversity in living things. c: it happens with the proliferation of the cell. d: it happens quickly and is seen in primitive organisms. c: in other words, it is seen in unicellular, fungi, some plants, and animals. figure 3 asexual reproduction e: how about talking about asexual reproduction types of reproduction by division? f: splitting single-celled organisms into two means "reproduction by dividing". e: a creature that has reached sufficient maturity is divided into two halves. f: living things formed by the division have the same features as each other. e: bacteria, paramecium, euglena multiply by dividing. figure 4 division by multiplication journal of science learning article doi: 10.17509/jsl.v5i1.32097 8 j.sci.learn.2022.5(1).1-13 considering the participant student views in the research under the title of suggestions; application in other lessons, development of the mobile application, providing the ability to create our character a nd record sound expressions came to the fore. to exemplify some student quotations regarding this situation: s1: "i would like to apply it in other lessons as well because it is an application that will make us like the lessons we do not like." s3: “it is a fun and easy method. i think it should be used in every lesson at least once a week." g: let us talk a little bit about budding. h: of course. first of all, a protrusion forms on the main creature's body. g: the resulting protrusion develops over time and separates from the main creature. h: i have also heard that sometimes they do not break up. they can also live in colonies. g: bu yaratıklar ne biliyor musun? h: creatures such as brewer's yeast, hydra, jellyfish, sponge, coral. figure 5 budding selin: hello nil. can i ask you a question? nil: of course, selin. i am listening to you. selin: how does the life cycle begin? nil: the life cycle begins with reproduction. the process involving the birth, growth, development, reproduction, and death of living creatures in nature is called the life cycle. selin: hmm, now i remember. reproduction is one of the common features of living things. creating structures similar to living things in order to continue their generation is called reproduction. nil: well, then let us talk a little bit about breeding patterns. selin: sexual and asexual reproduction? nil: yes, i will suggest you an easy way to distinguish between them. selin: waiting impatiently. nil: as the name suggests, reproduction that occurs with sex (reproductive) cells is called sexual reproduction. selin: in this case, creating new individuals with the same characteristics as a living thing without sex (reproductive) cells is called asexual reproduction. figure 6 reproduction and life cycle journal of science learning article doi: 10.17509/jsl.v5i1.32097 9 j.sci.learn.2022.5(1).1-13 s5: "this application can only be opened on computer and smartboard. it should be developed to be opened on tablets and phones." s10: "nice, but it can be a little more fun. in fact, i think we can create characters ourselves, and we can record and make those characters speak." j: come, let us play a game with you. k: what kind of game? j: let me tell you the characteristics of a reproductive cell, and guess which reproductive cell it is. k: well, let us try it. j: large, abundant cytoplasm, immobile, in ovaries and is produced in small numbers. k: i got it! the female reproductive cell, namely the egg. j: congratulations, correct answer. well, there comes another question... it is mobile thanks to its tail. it is produced in the testicles and in large numbers. it is small and has little cytoplasm. k: i think i can do that too. male reproductive cell, or sperm. j: bravo! again correct answer. you are great. figure 7 female and male reproductive cells tekir: hello tekir, do you know what comes to my mind? do you think the stages of people's pre-bırth development are like ours? bekir: i will share what i know with you. reproductive cells are formed as a result of meiosis of the reproductive mother. the structure formed due to the fertilization of the female reproductive cell... what was the say? fetus? embryo? i found i found. zygote. tekir: then what? bekir: then embryo occurs when the fertilized egg undergoes repeated mitosis. tekir: then this structure continues to develop, and will a baby after the second month? bekir: no, it takes the name of the fetus. however, after about forty weeks, the baby is born. figure 8 zygote-embryo-fetus-baby journal of science learning article doi: 10.17509/jsl.v5i1.32097 10 j.sci.learn.2022.5(1).1-13 3.3 digital concept cartoon example the examples of digital concept cartoons developed by students within the scope of the experimental application and their translations from turkish into english are presented. when the cartoon in figure 1 is examined, there are explanations and essential features of two concepts that are synonymous with each other. in figure 2, reproduction in living things was divided into two categories, and the types of asexual reproduction were mentioned. the characters in figure 3 talk about the characteristics of asexual reproduction, how it happens, and in which creatures it occurs. the cartoon in figure 4 shows the definition of the concept of "reproduction by division", its features, how it occurs, and in which creatures it is seen can be seen. in figure 5; the definition of the concept of "budding", one of the asexual reproduction varieties, its features, how it takes place, and in which creatures it is seen can be seen. when figure 6 is examined, it is seen that "reproduction", "life cycle", "sexual reproduction", "asexual reproduction", and the similarities and differences between these concepts are discussed, and misconceptions are tried to be prevented by this way. in figure 7, the characters talk about two separate concepts and their properties over a game. in figure 8, the characters describe the concepts of zygote-embryo-fetus-baby in order of formation. when figure 9 is examined, the characters discuss the metamorphosis process of frogs and other creatures and how this process develops. 4. result and discussion a significant difference in favor of the experimental group was found between the post-test and retention test mean scores of the experimental and control group students participating in the study. based on these findings, it can be said that the use of digital concept cartoons in the sixth-grade science course "reproduction, growth, and development in plants and animals" unit is effective in ensuring academic achievement and the retention of what is learned in the lesson. this finding of the study is parallel with qualitative findings. the students participating in the research stated that they learned by having fun with digital concept cartoons. the application made understanding easier that they remembered better what they learned in this way, that they enjoyed a lot despite having difficulties at some stages while making the application. they stated that the application provided interest, curiosity, and motivation towards the lesson. it increased helping each other. the students also stated that the lessons they continued with this method were more enjoyable, exciting, and energetic than the others, provided more accessible and practical learning. squirrel: hello, dear frog, i have a question about your growing-up process. frog: as always, you are so curious, dear squirrel. let us ask. squirrel: there was a different stage in your life cycle. what was his name? frog: are you saying that we undergo structural changes after hatching and become similar to the main creature? squirrel: yeah yeah. exactly i am asking it. frog: it is called metamorphosis. squirrel: so, are there any creatures that undergo metamorphosis other than you? frog: of course, arthropods, sponges, corals, butterflies also undergo metamorphosis. figure 9 metamorphosis journal of science learning article doi: 10.17509/jsl.v5i1.32097 11 j.sci.learn.2022.5(1).1-13 therefore they wanted to use these and similar methods in different course units and other lessons. the findings showed the diversity of students' opinions. in the research findings section, examples of digital concept cartoons prepared by students during the experimental application process are given. 1. when cartoon 1 is examined, which is about "reproductive", there are explanations and essential features of two concepts that are synonymous with each other. the aim here is to introduce these concepts and their properties. 2. in cartoon 2, which is about "sexual and asexual reproduction,"; reproduction in living things was divided into two categories, and the types of asexual reproduction were mentioned. here, the classification of the concept of asexual reproduction is emphasized. 3. the characters in cartoon 3, which is about "asexual reproduction,"; talk about the characteristics of asexual reproduction, how it happens, and in which creatures it occurs. the aim here is to provide basic information on the subject of asexual reproduction. 4. when cartoon 4 is examined, which is about "division by multiplication,"; the definition of the concept of "reproduction by division", its features, how it occurs, and in which creatures it is seen can be seen. here, general information on the subject of reproduction is given by dividing. 5. in cartoon 5, which is about "budding,"; the definition of the budding concept, one of the asexual reproduction varieties, its features, how it takes place, and in which creatures it is seen can be seen. the aim here is to give general information about budding. 6. when cartoon 6, which is about "reproduction and life cycle," is examined, it is seen that "reproduction", "life cycle", "sexual reproduction", "asexual reproduction", and the similarities and differences between these concepts are discussed. thus, misconceptions are tried to be prevented by this way. 7. in cartoon 7, which is about "female and male reproductive cells, " two concepts and features are mentioned by the characters based on a game. 8. cartoon 8, which is about "zygote-embryo-fetusbaby", describes the concepts of zygote-embryo-fetusbaby in order of formation. in this way, the development of an infant is discussed chronologically and based on related concepts. 9. when cartoon 9, which is about "metamorphosis," is examined, the characters discuss the metamorphosis process of frogs and other creatures and how this process develops. in this way, the concept of metamorphosis and the steps of the process of metamorphosis were tried to be comprehended. when all of the examples are analyzed, students use concept cartoons to define, classify, sort, and specify their characteristics. it has also been observed that synonymous concepts can be learned, and misconceptions can be eliminated in this way. concept cartoons have emerged to provide a practical and innovative approach in science teaching (keogh & naylor, 1996). concept cartoons can be used as a stimulus to initiate discussion in science lessons (keogh, naylor, & downing, 2003). it is thought that the use of concept cartoons in science lessons can create learning and information structuring environments where students can focus their attention on the lesson and discuss their views. concept cartoons are effective both in determining and eliminating misconceptions. concept cartoons are suggested as tools that can be used in science teaching in terms of both preparing the environment for the learning activities envisaged by the constructivist approach and minimizing the problems related to classroom organization (naylor & mc murdo, 1990; keogh & naylor, 1997; keogh et al., 1998 ). various factors affect the success of teaching based on concept cartoons. according to keogh et al. (1998), concept cartoons should be presented in association with daily events to provide students with the desired motivation and successful teaching. sentences should be as short, legible, and similar as possible. it is stated that concept cartoons prepared in this way will contribute to developing the student's problem solving, critical thinking, scientific thinking skills and help attract attention by making the subject more interesting (keogh & naylor, 1999b). conclusion 1. the positive effects of the use of concept cartoons in lessons on academic achievement, retention of what is learned in the lesson, attitude, and motivation have been proven by various studies. for this reason, it may be a good idea to include concept cartoons in textbooks frequently and to prepare concept cartoons as posters and hang them on classroom walls. 2. concept cartoons can be helpful, especially in lessons and subjects where many concepts and concept confusion can be observed at a high rate. 3. many tools such as concept maps, meaning analysis tables, concept puzzles, concept networks, v diagrams, descriptive branched tree, structured grid, conceptual change text are used in concept teaching. nowadays, considering the importance of technology integration in education and the interest of children and young people in technology, benefiting from the digital applications of these materials may be necessary for terms of attracting students' attention, motivating them to the lesson, enabling them to learn by researching, discussing, and having fun. 4. this research was carried out with 21 experimental and 21 control group students for six weeks. more extended studies can be carried out with larger study groups. an academic achievement test developed by the journal of science learning article doi: 10.17509/jsl.v5i1.32097 12 j.sci.learn.2022.5(1).1-13 researcher was used to obtain quantitative data. by using an attitude scale, the attitudes of the experimental and control groups towards the lesson can be examined, or the change in attitudes of the students before and after the application can be investigated. 5. it is known that the more senses students participate in the learning process, the more efficiently and effectively they learn. in this sense, it may be essential to develop and implement similar multimedia tools in lessons. examples of these materials are digital picture books, web presentations, animations, digital stories. references akamca, g. ö., & hamurcu, h. (2009). science and technology education based on analogies, concept cartoons and predictobserve-explain techniques. e-journal of new world sciences academy, 4(4), 1186-1206. https://dergipark.org.tr/en/pub/nwsaedu/issue/19826/212385 akbaş, y., & toros, s. (2016). the effects of using interactive cartoons and concept maps on academic achievement in social studies teaching. electronic turkish studies, 11(9), 53-68. http://dx.doi.org/10.7827/turkishstudies.9727 atasoy, ş., tekbıyık, a., & gülay, a. (2013). the effect of concept cartoons on understanding of sound concept of fifth grade students. journal of turkish science education, 10(1), 176-196. http://tused.org/index.php/tused/article/view/284 balım, a. g., i̇nel, d., & evrekli, e. (2008). the effects the using of concept cartoons in science education on students’ academic achievements and enquiry learning skill perceptions. elementary education online, 7(1), 188-202. http://ilkogretimonline.org.tr/index.php/io/article/view/1866 büyüköztürk, ş., kılıç çakmak, e., akgün, ö. e., karadeniz, ş., & demirel, f. (2012). bilimsel araştırma yöntemleri [scientific research methods]. pegem akademi publishing. cengizhan, s. (2011). prospective teachers’ opinions about concept cartoons integrated with modular instructional design. education and science, 36(160), 93-104. http://egitimvebilim.ted.org.tr/index.php/eb/article/view/338 cerrah özsevgeç, l., yurtbakan, e., & uludüz, ş. (2019). the effect of concept cartoon on elimination of fourth grade students’ misconceptions related with “mass and weight”. fen bilimleri öğretimi dergisi, 7(1), 51-67. https://dergi.fead.org.tr/wpcontent/uploads/31.pdf chin, c. (2001). eliciting students’ ideas and understanding in science: diagnostic assessment strategies for teachers. teaching and learning, 21(2), 72-85. http://hdl.handle.net/10497/352 chin, c., & teou, l. (2009). using concept cartoons in formative assessment: scaffolding students’ argumentation. international journal of science education, 31(10), 1307-1332. https://doi.org/10.1080/09500690801953179 coll, r. k., france, b., & taylor, i. (2005). the role of models/and analogies in science education: implications from research. international journal of science education, 27(2). 183-198. https://doi.org/10.1080/0950069042000276712 creswell, j. w. (2013). qualitative research methods; qualitative research and research design according to five approaches (m. bütün & s. b. demir, trans.). ankara: siyasal kitabevi publishing. çepni, s. (2010). fen ve teknoloji öğretimi [science and technology teaching]. ankara: pegem akademi publishing. çinici, a., özden, m., akgün, a., herdem, k., karabiber, h. l., & deniz, ş. m. (2014). investigating the effectiveness of argumentation based activities supported with concept cartoon. adıyaman üniversitesi sosyal bilimler enstitüsü dergisi, 7(18), 571-596. http://dx.doi.org/10.14520/adyusbd.839 dabell, j. (2004). the maths coordinator’s file using concept cartoons. london: pfp publishing. dabell, j. (2008). using concept cartoons. mathematics teaching incorporating micromath, 209, 34-36. https://eric.ed.gov/?id=ej815105 demirci, s. (2019). comparison of the effects of concept cartoons supported by digital and worksheets on conceptual understanding in teaching of geography topics in social studies course (unpublished master’s thesis). trabzon university, trabzon. demirel, r., & aslan, o. (2014). the effect of science and technology teaching promoted with concept cartoons on students’ academic achievement and conceptual understanding. journal of theory and practice in education, 10(2), 368-392. https://dergipark.org.tr/tr/pub/eku/issue/5460/74070 evrekli, e., i̇nel, d., & balım, a. g. (2011). a research on the effects of using concept cartoons and mind maps in science education. necatibey faculty of education electronic journal of science and mathematics education, 5(2), 58-85. https://dergipark.org.tr/en/download/article-file/39830 geelan, d. r. (1997). epistemological anarchy and the many forms of constructivism. science and education, 6, 15-28. https://doi.org/10.1023/a:1017991331853 gümüş, h., kavanoz, s., & yılmaz, m. b. (2017). the effect of delivering concept cartoons in mobile learning environments on learning english idioms in secondary education. yyu journal of education faculty, 14(1), 815-855. http://dx.doi.org/10.23891/efdyyu.2017.31 i̇nel, d., & balım, a. g. (2011). the effect of concept cartoons assisted problem based learning method on motivation about science learning of secondary students. uşak üniversitesi sosyal bilimler dergisi, 4(1), 169-188. https://dergipark.org.tr/en/pub/usaksosbil/issue/21649/232751 i̇nel, d., balım, a. g., & evrekli, e. (2009). the opinions of students about the use of concept cartoon in science and technology education. necatibey faculty of education electronic journal of science and mathematics education, 3(1), 1-16. https://dergipark.org.tr/tr/download/article-file/39768 keogh, b., & naylor, s. (1996, september). teaching and learning in science: a new perspective. paper presented at the meeting of the annual bera conference, lancaster. keogh, b., & naylor, s. (1997). starting points for science. sand bach: mill gate house. keogh, b., naylor, s., & wilson, c. (1998). concept cartoons: a new perspective on physics education. physics education, 33(4), 219-224. https://doi.org/10.1088/0031-9120/33/4/009 keogh, b., & naylor, s. (1999a). science goes underground. adults learning, 10(5), 6-8. https://eric.ed.gov/?id=ej578970 keogh, b., & naylor, s. (1999b). concept cartoons, teaching and learning in science: an evaluation. international journal of science education, 21(4), 431-446. https://doi.org/10.1080/095006999290642 keogh, b., & naylor, s. (2000). teacher and learning in science using concept cartoons: why dennis wants to stay in at playtime. australian primary and junior science journal, 16, 10-14. https://www.scirp.org/(s(vtj3fa45qm1ean45vvffcz55))/reference /referencespapers.aspx?referenceid=771111 keogh, b., naylor, s., de boo, m., & feasey, r. (2001). formative assessment using concept cartoons: initial teacher training in the uk. in b. helgard (ed.), research in science education past, present and future. usa: kluwer academic publishers. keogh, b., naylor, s., & downing, b. (2003). children’s interactions in the classroom: argumentation in primary science. paper presented at the 4th european science education research association conference, noordwijkerhout, netherlands. kinchin, i. (2004). investigating students' beliefs about their preferred role as learners. educational research, 46(3), 301-312. https://doi.org/10.1080/001318804200277359 journal of science learning article doi: 10.17509/jsl.v5i1.32097 13 j.sci.learn.2022.5(1).1-13 klavir, r., & gorodetsky, m. (2001). the processing of analogous problems in the verbal and visual-humorous (cartoons) modalities by gifted/average children. gifted child quarterly, 45(3), 205-215. https://doi.org/10.1177/001698620104500305 kroasbergen, e. h., & van luit, j. e. (2005). constructivist mathematics education for students with mental reterdation. european journal of special needs education, 20(1), 107-116. https://doi.org/10.1080/0885625042000319115 martinez, y. m. (2004). does the k-w-l reading strategy enhance student understanding in honors high school science classroom? (unpublished master’s thesis). california state university, fullerton. ministry of education (2005). i̇lköğretim 4.ve 5. sınıflar için fen ve teknoloji öğretim program [science and technology curriculum for primary education 4th and 5th grades]. ministry of education publishing, ankara. morris, m., merritt, m., fairclough, s., birrell, n., & howitt, c. (2007). trialing concept cartoons in early childhood teaching and learning of science. teaching science, 53(2), 42-45. https://researchrepository.uwa.edu.au/en/publications/trialling-conceptcartoons-in-early-childhood-teaching-and-learning mulhan, m. (2007). seventh classes civilization and human rights education lesson's effect on education and teaching practice of learning concepts with social, economical and cultural factors (unpublished master’s thesis). marmara university, i̇stanbul. naylor, s., downing, b., & keogh, b (2001, august). an empirical study of argumentation in primary science, using concept cartoons as the stimulus. paper presented at the meeting of the 3rd european science education research association conference, thessaloniki, greece. naylor, s., & keogh, b. (1999). constructivism in classroom: theory into practice. journal of science teacher education, 10(2), 93-106. https://doi.org/10.1023/a:1009419914289 naylor, s., & keogh, b. (2009). active assessment. mathematics teaching, 215, 35-37. https://eric.ed.gov/?id=ej859747 naylor, s., & mcmudro, a. (1990). supporting science in schools. timperley: breakthrough educational publications. özdemir, b. g., yildiz-durak, h., karaoğlan yılmaz, f. g., & yılmaz, r. (2018, july). use of digital concept cartoons and digital concept maps for elimination of misconceptions in secondary school mathematics course: examination of opinions of teachers. paper presented at the meeting of the edulearn conference, palma, mallorca, spain. parkinson, j. (2004). improving secondary science teaching. canada-usa: routledge falmer. sexton, m., gervasoni, a., & brandenburg, r. (2009). using a concept cartoon to gain insight into children’s calculation strategies. australian primary mathematics classroom, 14(4), 24-28. https://files.eric.ed.gov/fulltext/ej885812.pdf stephenson, p., & warwick, p. (2002). using concept cartoons to support progression in students’ understanding of light. physics education, 37, 135-141. https://doi.org/10.1088/00319120/37/2/306 şahin, y. (2018). the effect of concept cartoons usage, enriched with 5e model, on students’ concept learning and critical thinking skills in geography education (unpublished master’s thesis). karadeniz technical university, trabzon. teddlie, c., & tashakkori, a. (2009). foundations of mixed methods research: integrating quantitative and qualitative approaches in the social and behavioral sciences. london: sage publishing. tokcan, h., & alkan, g. (2013). the effect of the concept cartoons to the students of the social studies teaching. ahi evran üniversitesi kırşehir eğitim fakültesi dergisi, 14(2), 1-19. http://kefad2.ahievran.edu.tr/archieve/pdfler/cilt14sayi2/jkef _14_2_2013_1-19.pdf topçubaşı, t., & polat, s. (2014). the effect of concept cartoons to student achievement in social science teaching. international journal of new trends in arts, sports & science education, 3(2), 48-61. http://www.ijtase.net/ojs/index.php/ijtase/article/view/300 yaşar, z. (2017). the effect of concept cartoons activities on the speaking skills of primary school 3rd grade students (unpublished master's thesis). bartın university, bartın. yıldırım, a., & şimşek, h. (2018). sosyal bilimlerde nitel araştırma yöntemleri [qualitative research methods in the social sciences]. ankara: seçkin yayınevi. a © 2021 indonesian society for science educator 357 j.sci.learn.2021.4(4).357-364 received: 15 april 2021 revised: 24 may 2021 published: 01 august 2021 the effect of nature education activities on biophilia levels of science teacher candidates abdurrahman sefalı1, esra ozay kose2* 1department of mathematics and science education, bayburt university, bayburt, turkey 2department of biology education, kazım karabekir education faculty, atatürk university, erzurum, turkey *corresponding author: esraozay2002@yahoo.com abstract as a result of increasing the level of interest in living and living systems (biophilia), it will be possible to educate more protectionist individuals against nature and the environment. this study was conducted to investigate the effect of nature education activities on biophilia levels of science teacher candidates. this research was conducted at public research university in bayburt, turkey, in the department of science education during the spring semester of the 2017-2018 academic year. the activities were done in an integrated manner to the environmental science course. the study group included a total of 62 science teacher candidates. these students were distributed equally to the experiment and control group. the study was used a quasi-experimental method with a group of experiments and control. in the study, the biophilia scale was applied as a data collection tool. independent samples t-tests and dependent samples t-tests were used for analyzing the data. as a result of the analysis, the findings indicated that nature education and in-class activities positively affected teacher candidates' biophilia levels. the findings also indicated that the knowledge and experience they gained due to students' scrutiny, touching, smelling, and even tasting the species at in-class and nature activities affected their biophilia levels. keywords nature education, out-of-school activities, in-class activities, biophilia, science education 1. introduction human beings have been a part of nature since they came into existence by acting as a part of the ecosystem, just like other living things. today, human beings have become destructive to nature because they change their environment on a macro scale. the inevitable end for all living spaces on earth is imminent due to nature's selfcentered and utilitarian perspective. if our view on nature is not a priority, nature but only human-centered, the situation will inevitably worsen. although the idea of “protecting nature” is thought to come to the fore with environmental pollution, it can be said that it started with protecting areas and animals considered sacred to people and based on ancient times (kurdoğlu, 2007). according to the 1995 report of the world wide fund for nature (wwf), sanctuaries have probably been the most ancient habitat protection method on the planet (wild, mcleod, & valentine, 2008). whatever the reason is, it is a fact that protected natural areas have a great biological heritage. however, the conflict between human activities and biodiversity conservation efforts is increasing daily in all european ecosystems (young et al., 2005). since anatolia has hosted many civilizations for centuries, its natural resources have been severely damaged (aslım, yiğit, i̇zmirli & yaşar, 2012). biological diversity is threatened by elements such as overfishing, vegetation from nature (karagöz et al., 2010), invasive species, environmental problems, habitat loss, natural disasters, genetically modified organisms, and endangered creatures (çakmak, 2008). it is undoubtedly possible to protect nature with education, to be conscious and realistic. however, especially in developing countries, the use of natural areas under the name of investment cannot be prevented despite tens of binding national and international protection laws (kurdoğlu, 2007). therefore, in order for our people to become more sensitive to the environment (başlar & şahin, 1993), it is essential to implement the necessary training (karagöz, özbek, & sarı, 2016). informal education, the student spends a large class of his time. it is known that this education in schools is mailto:esraozay2002@yahoo.com journal of science learning article doi: 10.17509/jsl.v4i4.33434 358 j.sci.learn.2021.4(4).357-364 different from the realities of life (ramey-gassert, 1997). it is a situation that students are confined to classrooms in their education life and may have been kept away from real life (özür, 2010). informal education should also not be neglected in addition to formal education by making use of its opportunity for the education of these individuals (özyıldırım & durel, 2017). activities such as free lectures, programs, and camp training, which are widespread in institutions, most of which are in the establishment's status, are developments in increasing participants' sensitivity to their environment (meydan, bozyiğit, & karabulut, 2012). for this reason, the number of these institutions providing only informal education is increasing day by day (meydan et al., 2012). in this context, the environments that individuals offer to act comfortably and collaboratively and make autonomous decisions enable us to reach the desired learning outcomes (atal, & koçak usluel, 2011). furthermore, individuals in out-of-school learning activities can acquire practical skills by having experience (çebi, 2018). humberstone and stan (2010) emphasized that out-of-school learning activities provide beneficial experiences for students and teachers. for this reason, it is known that experiences gained from out-of-school activities are remembered for a long time (lakin, 2006). school outside activities about nature education have recently become popular in turkey's field (erdogan, 2011). with nature education, individuals can learn about the natural environment and thus understand the cause-andeffect relationships (erdoğan, bahar, & uşak, 2011; yerkes & haras, 1997). nature education, including nature-related issues, aims to provide participants with a broad vision of ecology (tübi̇tak, 2018). the broad vision of ecology mentioned here is understanding and recognizing natural environments (ozaner, 2004; keleş, uzun, & uzun, 2010). in general, it can be said that nature education is utilitarian education carried out in nature. however, it is possible to turn the elements of nature into educational material (keleş, 2011). wilson (2007) stated that humans have an emotional connection with nature and other living things genetically, and therefore the preference to be in natural environments is a genetic phenomenon. biophilia is a hereditary need and tendency that emerges to connect with living systems and survive (kahn, 1997). biophilia is a word of latin origin, translated as "bio-philia = living-interest", meaning a tendency to vitality. the reasons for our instinctive closeness to nature, green, and animals are hereditary and date back to ancient times. therefore, human beings depend on nature and living systems. today, we live in a rapidly urbanizing world where access to nature is limited (turner, nakamura, & dinetti, 2004). in this context, as we move away from nature, we experience physical and mental diseases. because human health and well-being are affected by the quality of living environments (lopez, 2012), it is thought that being in natural environments is good for some mental illnesses or is related to beneficial factors (bratman et al., 2019). stress and anxiety decrease even in rooms with windows to natural environments (view) (yin et al., 2020). naturedesigned residential areas, biophilic designs, culturally maintain people's connection with nature (kellert, 2012,2018; kellert, heerwagen, & mador, 2011). people often want to be close to living things and living systems to increase human welfare by choosing nature itself instead of the environments they create (lumber, richardson, & sheffiel, 2017). if biophilia in humans is determined, this level can be increased by species-based, appropriate nature education activities. with the increasing level of biophilia, it will be possible to raise individuals who are more concerned and protective of nature and the environment (cooney & dickson, 2005; katcher & wilkins 1993; simaika & samways, 2010;). the deep love for nature (biophilia) is also known as the responsibility of a person to protect nature (perkins, 2010). therefore, individuals' biophilia levels can be increased with nature education activities by helping to develop environmental awareness, sense of responsibility, and environmental knowledge (erdoğan & özsoy, 2007; yerkes & haras, 1997). 1.1 purpose of the study this research aims to determine how biophilia levels of students studying in the third grade of science education undergraduate program change with nature education activities. for this purpose, answers to the following questions were sought during the research process: regarding the biophile scale (bs) at the end of nature education activities; • is there a significant difference between the pre-test and post-test scores of the experimental group? • is there a significant difference between the pre-test and post-test scores of the control group? • is there a significant difference between the post-test scores of the students in the experimental and control groups? 2. method 2.1 research model in this study, a quasi-experimental design including experimental and control groups was used. unfortunately, education studies do not provide many opportunities for real experimental studies because it is impossible to distribute individuals to groups in school and classroom environments (özmen, 2014). in this context, no special effort was made to match the individuals in the experimental and control groups through an unbiased assignment. however, care was taken to ensure that they were as similar as possible (büyüköztürk, çakmak, akgün, karadeniz, & demirel, 2016). journal of science learning article doi: 10.17509/jsl.v4i4.33434 359 j.sci.learn.2021.4(4).357-364 2.2 study group the number of students in the study group is 62 in total. the convenience sampling method was used among the non-random sampling methods. convenience sampling is a sampling aiming to prevent loss of time, money, and overtime (büyüköztürk et al., 2016). thirty-one of these students are in the experimental group, and 31 are in the control group. by comparing the biophile scale pre-test scores of the groups, it was revealed that they were equivalent. in the study, nature education activities were carried out in the experimental group and classroom activities in the control group within the framework of the environmental science course. 2.3 data collection tools in the study, biophile scale (bs) was used as a data collection tool. bs was first developed by glock, meyer, and wertz (1999) and consisted of expressions created to understand the connection of children and young people to nature. these expressions were translated into turkish as 39 items, and it is a scale that contains 25 items. furthermore, validity and reliability were ensured with the data collected from 868 university students. bs was developed and used within the scope of the doctoral study (sefalı, 2019). thirty minutes were given for the application of bs, which consists of 25 items with the 5-point likerttype, and was applied as a pretest-posttest. the reliability coefficient of the test was determined to be 0.88. content validity of the test was provided by taking expert opinions. 2.4 data collection and implementation process in the study, the process steps shown in table 1 were applied to the group of students with nature education activities and the group of students who did in-class activities. as shown in table 1, the interaction of the students in the experimental group was ensured, and a species-based activity path was followed. this situation was carried out in parallel in the control group, and museum samples and video/slide shows of living things were used. the implementation lasted for an academic period. photos of the activities related to the experimental and control groups are given in figure 1. 2.5 examined organisms the experiment and control group participants were allowed to see and examine the living things closely during the activities in the study. in the experimental group, the species of the plants and animals were examined in living and natural environments. in contrast, the control group was examined museum materials (herbarium plant samples and animal samples preserved in formaldehyde) and visual presentations. when looking at the living things examined in activities, the species of asteraceae, fabaceae, brassicaceae, lamiaceae, and poaceae were examined as turkey's common flowering plants (day, kemp, yáñezarancibia, & crump, 2012). also, the species of arthropods, beetles (coleoptera), locusts (orthoptera), and scorpions (arachnida); some fish, frog, bird, and mammal species were studied as animal species. at the same time, habitat investigations were made. 2.6 data assessment in the study, independent samples t-test analysis was conducted to test how biophilia levels changed between the experimental group students in which nature education table 1 process steps followed in experimental and control groups experimental group activities control group activities meeting, making the purpose and promotion of nature education activities to be held meeting, making the purpose and promotion of nature education activities to be held implementation of bs (as pre-test) implementation of bs (as pre-test) fish examination activity (examination of some bulbous plants in spring plants at the end of this activity) activity of examination of fish (samples of fish species preserved in alcohol and formaldehyde were used. also, examination of herbarium samples of some bulbous plants which are among the first spring plants) bird watching activity (observation of birds in the dump, examining the quarries as an example of habitat loss and examining the plants in the countryside around the dump) bird watching activity (sample powerpoint presentations were prepared and photographs and visuals parallel to the experimental group and a documentary about birds were used ) examination of endemic plants (examination of steppe tulip). in addition, water birds were observed during this activity. examination of endemic plants (examination of herbarium samples of steppe tulip). in addition to this activity, waterfowl powerpoint presentation was used. examination of reptiles (snakes and lizards) in the study of reptiles (snakes and lizards), samples of snake and lizard species preserved in alcohol and formaldehyde were used. activity of studying mammals (anatolian yellow scorpion and grasshopper species among arthropods were also examined). a powerpoint presentation was prepared using visuals parallel to the animals examined in the experimental group for the study of mammals, and scorpion alcohol samples were analyzed. conducting frog and salamander inspection activity (plants and songbirds were also observed). in the frog and salamander study, samples of frog and salamander species preserved in alcohol and formaldehyde were used (also a documentary about nature was used). implementation of bs (as a final test). implementation of bs (as a final test). journal of science learning article doi: 10.17509/jsl.v4i4.33434 360 j.sci.learn.2021.4(4).357-364 activities were applied and the control group students. they were applied in-class activities using bs as a data collection tool. the dependent samples t-test was used to determine whether there was a difference before and after the experimental procedure by examining the biophilia levels of the control group and experimental group students. 3. results in the study, the findings related to the dependent samples t-test applied to determine whether there is a significant difference between the bs pre-test and post-test results applied to the control group are presented in table 2. according to the findings in table 2, it was determined that there is a significant difference between the pre-test and post-test of the control group (t(30) = -5.31, p<.05). in the control group where in-class activities were applied, the average pre-test biophilia levels of the students were (x̄ = 2.52), while the average post-test biophilia levels were (x̄ = 3.08). in addition, when the effect size of the difference between the pre-test and post-test scores of the control group was calculated, cohen d was found to have a significant effect size (d = 0.316) (cohen, 1988). in the study, the findings related to the dependent samples t-test applied to determine whether there is a significant difference between the bs pre-test and post-test results applied to the experimental group are presented in table 3. according to the findings in table 3, it was determined that there is a significant difference between the pre-test and post-test of the experimental group (t(30) = -9.16, p<.05). in the experimental group in which nature education activities were applied, the average pre-test biophilia levels of the students were (x̄ = 2.66), while the average post-test biophilia levels were (x̄ = 3.76). in addition, when the effect size of the difference between the pre-test and post-test scores of the experimental group is calculated, it is seen that cohen d, (d = 0.482) has a significant effect size (cohen, 1988). the post-test scores of the experimental and control groups were compared to determine the change in biophilia levels of the participants with nature education and classroom activities. as a result of the independent samples t-test, a significant difference was found between the posttest scores of both groups (p<.05). the findings obtained are presented in table 4. figure 1 photos of activities in the research. a-b-c: photos of the control group, a. investigation of “steppe tulips” of plants activity, b. fish activity, c. examination of the alcohol sample of the “kocabaş snake”, d-e-f: photographs of the experimental group, d. examination of the “nalburunlu bat” in the mammals activity, e. study of “water snakes”, f. study of the “küpeli water snake” in nature. table 2 analysis of the pre-and post-test bs in control group control group n x ̄ sd t sd p eta square bs pre-test 31 2.52 .536 -5.31 .582 .000 .316 bs post-test 31 3.08 .572 table 3 analysis of the pre-and post-test bs in experimental group experimental group n x ̄ sd t sd p eta square bs pre-test 31 2.66 .448 -9.16 .669 .000 .482 bs post-test 31 3.76 .591 journal of science learning article doi: 10.17509/jsl.v4i4.33434 361 j.sci.learn.2021.4(4).357-364 at the end of the application in table 4, it was found that there is a statistically significant difference between the groups according to the data obtained from the bs (t(60) = 4.65, p = .000). according to the data in this table, the arithmetic mean and standard deviation value of the experimental group (x̄ = 3.76, sd = .572) and the arithmetic mean, and standard deviation value of the control group (x̄ = 3.08, sd = .591) were calculated. therefore, it was determined that there is a significant difference in terms of biophilia levels of the students according to the bs posttest scores. in addition, when the effect size of the difference between the post-test scores of the experimental and control groups is calculated and the cohen d value is examined (d = 0.265), it is seen that it has a significant effect size (cohen, 1988). 4. discussion in the light of all the data obtained from bs, in-class and nature education activities applied to increase the biophilia levels of the students gave positive results in both experimental and control groups and significantly increased biophilia levels. however, it was observed that the biophilia levels of the experimental group students who participated in the nature education activities increased significantly compared to the control group students who participated in the classroom activities. studies are showing that out-of-school activities are loved and fun by students. because the students do not see the activities, they do as a lesson and spend time as a game or a hobby. many studies show the positive effects of this informal education in out-of-school learning environments on students (bamberger, & tal, 2008; bodur, 2015; bozdoğan, 2007/2008; dewitt & osborne, 2010; sandford, duncombe, & armor, 2008). nature itself is a science laboratory. it is possible to organize all kinds of studies, researches, and training on science here. it has been determined that extracurricular activities related to science education positively affect students' attitudes (bozdoğan, okur, & kasap, 2015; güler, 2011; jarvis & pell, 2002/2005). öner (2018) reported how vital the role of nature in science teaching is and that students who have little interaction with nature have difficulty relating their learning. this study was implemented in nature, which is one of the learning environments out of school. it aimed to increase the biophilia levels by exposing the students to interact with the studied creatures for a long time. in the study, when the findings obtained as a post-test with the bs applied to compare the biophilia levels of the experimental and control groups after the activities were examined, it was found that the biophilia levels of the students in the experimental group were higher than the students in the control group. the high biophilia levels of the students in the experimental group with nature education activities are thought to be since the students in this group can examine the materials used in the activities live, touch, and feel these creatures. they have examined these creatures in their natural environment for a long time. some studies show that contact with living creatures is good for people and even therapy (allen, 1997; clements, benasutti, & carmone, 2003; frumkin, 2001; jhonson & meadows, 2002; lust, ryan-haddad, coover, & snell, 2007; shore, douglas, & riley, 2005; ). the fact that there is a significant difference between the pre-test and post-test scores of the control group shows that in-class activities related to nature education also increase the biophilia levels of the students. however, the fact that the students in the control group did their activities over the dead materials and the activities performed as a presentation and documentary might have caused the increase in biophilia levels not as much as in the experimental group. it was determined that the participants' perspectives on the world were positively affected by nature education studies, and the experiences gained were shared with other people (güler, 2010). similarly, it is possible for the knowledge acquired through nature education to become permanent and faster (erten, 2004; farmer, knapp, & benton, 2007; ozaner, 2004). they also drew attention to how nature travels develop an awareness of the relationships between the individual himself, others, and the natural world (kaplan & kaplan, 1989; keleş et al., 2010). since the concept of biophilia is an unpredictable value, it is vital to protect genes, populations, species, or interactions (cooney & dickson, 2005). since our connection with nature is experiential, biophilia is best derived from learned experiences (tidball, 2012). so biophilia is not an innate love; instead, it is learned by instilling responsibility into nature (katcher & wilkins, 1993). some researchers working on the concept of biophilia have drawn attention to the aspects of biophilia that will be learned later and utilitarian in protecting nature rather than being inherited (katcher and wilkins 1993; miller 2005; samways, 2007; stokes 2006; tidball, 2012). it appears that accepting biophilia as a learned trait to help preserve biodiversity is far more helpful than trying to claim it is innate (sala, 2009; simaika & samways, 2010). it is known that emotional factors play an essential role in people's identification of species (martınez-lopez, montes, table 4 independent sample t-test analysis of the bs post-test scores of the experimental and control groups groups n x ̄ sd t sd p eta square experimental 31 3.76 .572 4.65 .148 .000 .265 control 31 3.08 .591 journal of science learning article doi: 10.17509/jsl.v4i4.33434 362 j.sci.learn.2021.4(4).357-364 & benayas, 2007), and well-informed people have ecological-scientific thoughts (tidball, 2012). therefore, it is possible for individuals to become more concerned and protective towards nature and the environment due to increasing their biophilia levels by learning (cooney & dickson, 2005; katcher & wilkins 1993; simaika & samways, 2010). 5. conclusion as a result, it was ensured that the participants got close to living beings with both in-class (control group) and nature education (experimental group) activities. mainly, with the nature education activities conducted in the experimental group, the students were made to touch, smell, and taste (some plants) living things and perceive the vitality and natural environments more clearly. therefore, the biophilia levels of students have a better view and study of living things and nature have increased their biophilia levels. therefore, it is essential to determine and increase individuals' biophilia levels to protect nature and understand the importance of living things. in addition, the importance of the concept of biophilia has been revealed in this study. 6. recommendation some suggestions have been made within the scope of this study. looking at these suggestions. it is essential to determine and increase individuals' biophilia levels to protect nature and understand the importance of living things. in nature education activities, different species should be examined as much as possible to increase the biophilia levels. suppose it is assumed that a sufficient number of species are examined in this study. in that case, the fact that endemic rare, invasive, and endangered species have also been examined can be effective in increasing the biophilia level. the general and distinctive features of the species to be studied in nature education activities should be well known. if species are very close to each other, examining that species with its closest relatives can facilitate species distinctions. one of the most striking aspects of nature education activities is studying the species that people fear or disgust. for students, their attitude to these species depends on the researcher's attitude towards these creatures. therefore, the researcher should be well planned to show the fundamental aspects of these creatures by explaining them, and the way they approach the living things will not be adversely affected. for example, the researcher's attitude towards dangerous species (poisonous) or grasping an aggressive snake like the big head snake (non-venomous) by the head and showing how to hold it. the use of some examples in nature in the classroom environment will effectively increase the biophilia level of students. for example, for nature education activities, museum samples of living things (such as herbarium samples) can be prepared by teachers and used in the classroom environment. in addition, it is possible to benefit from mineral stones, dried plant samples, seashells collection, and feather collection. gender variable can be used as advice while determining the change in the biophilia level differences of the participants in nature education activities. references allen, d. t. 1997. effects of dogs on human health. journal of the american veterinary medical association, 210, 1136–1139 aslım, g., yiğit, a., i̇zmirli, s., & yaşar, a. (2012). hayvan koruma kavramı ve biyoetik çerçevesinde yaban hayatı koruma ve yaban hayatı geliştirme sahaları [wildlife protection and wildlife development areas within the framework of animal protection concept and bioethics]. kafkas univ vet fak derg, 18(4), 657-662. atal, d., & koçak usluel, y. (2011). i̇lköğretim öğrencilerinin okul içinde ve dışında teknoloji kullanımları [technology use of primary school students in and out of school]. hacettepe üniversitesi eğitim fakültesi dergisi, 41(41), 24-35. bamberger, y., & tal, t. (2008). an experience for the lifelong journey: the long-term effect of a class visit to a science center. visitor studies, 2(11), 198-212. doi: 10.1080/10645570802355760 başlar, s., & şahin, n. (1993). ekolojik denge ve yok olan değerlerimiz [ecological balance and our disappearing values]. çevre dergisi, 9, 15-20. bodur, z. (2015). sınıf dışı etkinliklerin güneş sistemi ve ötesi ünitesinde ortaokul yedinci sınıf öğrencilerinin akademik başarıları, bilimsel süreç becerileri ve motivasyonları üzerine etkisi [the effect of extracurricular activities on the academic achievement, scientific process skills and motivation of secondary school seventh grade students in the solar system and beyond unit] (master's thesis). yükseköğretim kurulu ulusal tez merkezi’nden edinilmiştir. (tez no. 412440) bozdoğan, a. e. (2008). planning and evaluation of field trips to informal learning environments: case of the energy park. journal of theory and practise in education, 4(2), 282-290. bozdoğan, a. e., okur, a., & kasap, g. (2015). planlı bir alan gezisi için örnek uygulama [example application for a planned field trip]. karadeniz sosyal bilimler dergisi, 7(14). bozdoğan, a.e. (2007). bilim ve teknoloji müzelerinin fen öğrenimindeki yeri ve önemi [the place and importance of science and technology museums in science education] (phd thesis). yükseköğretim kurulu ulusal tez merkezi’nden edinilmiştir. (tez no. 207028) bratman, g. n., anderson, c. b., berman, m. g., cochran, b., de vries, s., flanders, j., ... & kahn, p. h. (2019). nature and mental health: an ecosystem service perspective. science advances, 5(7), eaax0903. doi: 10.1126/sciadv.aax0903 büyüköztürk, ş., çakmak-kılıç, e., akgün, ö.e., karadeniz, ş., & demirel, f. (2016). bilimsel araştırma yöntemleri (20. baskı) [scientific research methods (20th ed.)]. ankara: pegem akademi. clements, p. t., benasutti, k. m., & carmone, a. (2003). support for bereaved owners of pets. perspectives in psychiatric care, 39(2), 49-54. cohen, j. (1988). statistical power analysis for the behavioral sciences (2nd ed.). hillsdale, nj: lawrence earlbaum associates cooney, r., & b. dickson. (2005). the precautionary principle. earthscan, london. çakmak m. (2008). conservation of biological diversity by law and public interest. aühfd, 133-166. çebi, h. (2018). farklı okul dışı öğrenme ortamlarının, öğrencilerin fen bilimleri dersine karşı ilgi ve tutumlarına etkisi [the effect of different out-ofschool learning environments on students' interests and attitudes towards science course] (master's thesis). yükseköğretim kurulu ulusal tez merkezi’nden edinilmiştir. (tez no. 519021) journal of science learning article doi: 10.17509/jsl.v4i4.33434 363 j.sci.learn.2021.4(4).357-364 day, j. w., kemp, w. m., yáñez-arancibia, a., & crump, b. c. (eds.). (2012). estuarine ecology. john wiley & sons. dewitt, j., & osborne, j. (2010). recollections of exhibits: stimulatedrecall interviews with primary school children about sciencecentervisits. international journal of science education, 32(10), 1365-1388. erdogan, m. (2011). the effects of ecology-based summer nature education program on primary school students' environmental knowledge, environmental affect and responsible environmental behavior. educational sciences: theory and practice, 11(4), 2233-2237. erdoğan, m., & özsoy, a. m. (2007). graduate students’ perspectives on the human and environment relationship. türk fen eğitimi dergisi, 4(2), 21-30. erdoğan, m., bahar, m., & uşak, m. (2011). 007 yılında uygulanmaya başlanan lise 9-12. sınıf biyoloji dersi ögretim programlarında çevre eğitimi [environmental education in classroom biology curriculum]. kuram ve uygulamada egitim bilimleri, 12(3), 2217-2235. erten, s. (2004). çevre eğitimi ve çevre bilinci nedir, çevre eğitimi nasıl olmalıdır? [what is environmental education and environmental awareness, how environmental education should be?]. çevre ve i̇nsan dergisi, çevre ve orman bakanlığı yayın organı, 65-66. farmer, j., knapp, d., & benton, m. g. (2007). an elementary school environmental education field trip: long-term effects on ecological and environmental knowledge and attitude development. the journal of environmental education, 38(3), 33-42. frumkin, h. (2001). beyond toxicity: human health and the natural environment. american journal of preventive medicine, 20, 234–240. glock, j., meyer, m., & wertz, s. (1999). discovering the naturalist intelligence: science in the schoolyard. zephyr press. güler, a. (2011). planlı bir müze gezisinin ilköğretim öğrencilerinin tutumuna etkisi [the effect of a planned museum visit on the attitude of primary school students]. i̇lköğretim online, 10(1), 169179. güler, t. (2010). ekoloji temelli bir çevre eğitiminin öğretmenlerin çevre eğitimine karşı görüşlerine etkileri [the effects of an ecology-based environmental education on teachers' views on environmental education]. eğitim ve bilim, 34(151). humberstone, b., & stan, i., (2010). outdoor learning: primary pupils’ experiences and teachers’ interaction in outdoor learning. international journal of primary, elementary and early years education, 39(5), 529-540. doi: 10.1080/03004279.2010.487837 jarvis, t., & pell, a. (2002). the effect of the challenger experience on elementary children’s attitudes to science. journal of research in science teaching, 39(10), 979-1000. jarvis, t., & pell, a. (2005). factors influencing elementary school children's attitudes toward science before, during and after a visit to the uk. national space centre. journal of research in science teaching, 42(1), 53-83. doi: 10.1002/tea.20045 johnson, r. a., & meadows, r. l. (2002). older latinos, pets, and health. western journal of nursing research, 24(6), 609-620. doi: 10.1177/019394502320555377 kaplan, r., & kaplan, s. (1989). the experience of nature: a psychological perspective. cambridge university press. karagöz a., zencirci n., tan a., taşkın t., köksel h., sürek m., toker c. & özbek k. (2010). bitki genetik kaynaklarının korunması ve kullanımı [conservation and use of plant genetic resources]. türkiye ziraat mühendisliği vii teknik kongresi. bildiriler (i): 11-15 ocak, ankara (p. 155-177). karagöz, a., özbek, k., & sarı, n. (2016). türkiye’nin bitkisel biyolojik çeşitliliğinin korunması ve sürdürülebilir kullanımına ilişkin sorunlar ve çözüm önerileri [problems and solutions regarding the conservation and sustainable use of turkey's plant biodiversity]. tarla bitkileri merkez araştırma enstitüsü dergisi, 88-99. katcher, a., & wilkins, g. (1993). dialogue with animals: its nature and culture. the biophilia hypothesis, 173-197. keleş, ö. (2011). fen öğretiminde okul dışı öğrenme ortamları [out-ofschool learning environments in science teaching]. in c. laçinşimşek (ed.), doğa eğitimleri [nature education] (p. 65-84). ankara: pegem akademi. keleş, ö., uzun, n., & uzun, f.v. (2010). öğretmen adaylarının çevre bilinci, çevresel tutum, düşünce, ve davranışlarının doğa eğitimi projesine bağlı değişimi ve kalıcılığının değerlendirilmesi [evaluation of the change and permanence of teacher candidates' environmental awareness, environmental attitudes, thoughts and behaviors depending on the nature education project]. elektronik sosyal bilimler dergisi, 9(32), 384-401. kellert, s. r. (2012). birthright: people and nature in the modern world. yale university press. kellert, s. r. (2018). nature by design: the practice of biophilic design. yale university press. kellert, s. r., heerwagen, j., & mador, m. (2011). biophilic design: the theory, science and practice of bringing buildings to life. john wiley & sons. kurdoğlu, o. (2007). dünyada doğayı koruma hareketinin tarihsel gelişimi ve güncel boyutu [the historical development and current dimension of the nature conservation movement in the world]. artvin çoruh üniversitesi orman fakültesi dergisi, 8(1), 59-76. lakin, l. (2006). science beyond the classroom. journal of biological education, 40(2), 88-90. lopez, r.p. (2012). the built environment and public health (1st ed.). san francisco: jossey-bass. lumber, r., richardson, m., & sheffield, d. (2017). beyond knowing nature: contact, emotion, compassion, meaning, and beauty are pathways to nature connection. plos one, 12(5), 1-24. doi: 10.1371/journal.pone.0177186 lust, e., ryan-haddad, a., coover, k., & snell, j. (2007). measuring clinical outcomes of animal-assisted therapy: impact on resident medication usage. the consultant pharmacist, 22(7), 580-585. martınez-lopez, b., montes, c. & benayas, j. (2007). the noneconomic 'motives behind the willingness to pay for biodiversity conservation. biological conservation, 139, 67–82. meydan, a., bozyiğit, r., & karakurt, m. (2012). ekoloji temelli doğa eğitimi projelerinin katılımcı beklentilerini karşılama düzeyleri [the levels of meeting participant expectations of ecology-based nature education projects]. marmara coğrafya dergisi, 25, 238-255. miller, j. r. (2005). biodiversity conservation and the extinction of experience. trends in ecology and evolution, 20, 430–434. ozaner, f. s. (2004). türkiye’de okul dışı çevre eğitimi ne durumda ve neler yapılmalı? [what is the situation of out-of-school environmental education in turkey and what should be done?]. in the v. national ecology and environment congress (pp. 67-98). abantbolu: abant i̇zzet baysal university and i̇zmir biologists association. öner, z. (2018). çevre eğitimine yönelik hazırlanan formal ve informal uygulamaların akademik başarı, tutum ve öğrenci kazanımları açısından incelenmesi [examination of formal and informal practices prepared for environmental education in terms of academic achievement, attitude and student achievements] (phd thesis). yükseköğretim kurulu ulusal tez merkezi’nden edinilmiştir. (tez no. 528448) özmen, h. (2014). deneysel araştırma yöntemleri [experimental research methods]. in m. metin (ed.), kuramdan uygulamaya eğitimde bilimsel araştırma yöntemleri [scientific research methods in education from theory to practice] (pp. 47-76). ankara: pegem akademi yayıncılık. özür, n. (2010). sosyal bilgiler dersinde sınıf dışı etkinliklerin öğrenci başarısına etkisi [the effect of out-of-class activities on student achievement in social studies course] (phd thesis). yükseköğretim kurulu ulusal tez merkezi’nden edinilmiştir. (tez no. 279700) özyıldırım, h., & durel, e. (2017). okul dışı öğrenme ve okul dışı öğrenme ortamları. balkan eğitim araştırmaları [out-of-school learning and out-of-school learning environments. balkan educational research]. edirne: trakya üniversitesi. perkins, h. e. (2010). measuring love and care for nature. journal of environmental psychology, 30(4), 455-463. journal of science learning article doi: 10.17509/jsl.v4i4.33434 364 j.sci.learn.2021.4(4).357-364 ramey-gassert, l. (1997). learning science beyond the classroom. the elementary school journal, 97(4), 433-450. sala, o. e. (2009). biodiversity change and human health: from ecosystem services to spread of disease. island press. samways, m. j. (2007). rescuing the extinction of experience. biodiversity conservation, 16, 1995–1997. sandford, r., duncombe, r., & armour, k. (2008). the role of physical activity/sport in tackling youth disaffection and anti-social behaviour. educational review, 60(4), 419-435. doi: 10.1080/00131910802393464 sefalı, a. (2019). doğa eğitimi etkinliklerinin fen bilgisi öğretmen adaylarının biyofili seviyelerine etkisinin araştırılması [investigation of the effects of nature education activities on the biophilia levels of pre-service science teachers] (phd thesis). atatürk üniversitesi, eğitim bilimleri enstitüsü.erzurum. shore, e. r., douglas, d. k., & riley, m. l. (2005). what's in it for the companion animal? pet attachment and college students' behaviors toward pets. journal of applied animal welfare science, 8(1), 1-11. doi: 10.1207/s15327604jaws0801_1 simaika, j. p., & samways, m. j. (2010). biophilia as a universal ethic for conserving biodiversity. conservation biology, 24(3), 903-906. stokes, d. l. (2006). conservators of experience. bioscience, 56, 6–7. tidball, k. g. (2012). urgent biophilia: human-nature interactions and biological attractions in disaster resilience. ecology and society, 17(2). turner, w. r., nakamura, t., & dinetti, m. (2004). global urbanization and the separation of humans from nature. bioscience, 54(6), 585590. tübi̇tak. (2018). 4004-doğa eğitimi ve bilim okullarıdestekleme programı [supporting program for nature education and science schools]. retrieved from https://www.tubitak.gov.tr/sites/default/files/13209/4004_cagri _metni_27072018.pdf wild, r., mcleod, c., & valentine, p. (2008). sacred natural sites: guidelines for protected area managers (no. 16). iucn. wilson, e, o. (2007). biophilia and the conservation ethic. evolutionary perspectives on environmental problems. washington dc. 249-257. yerkes, r., & haras, k. (1997). outdoor education and environmental responsibility. eric digest. yin, j., yuan, j., arfaei, n., catalano, p. j., allen, j. g., & spengler, j. d. (2020). effects of biophilic indoor environment on stress and anxiety recovery: a between-subjects experiment in virtual reality. environment international, 136, 105427. young, j., watt, a., nowicki, p., alard, d., clitherow, j., henle, k., ... & niemela, j. (2005). towards sustainable land use: identifying and managing the conflicts between human activities and biodiversity conservation in europe. biodiversity & conservation, 14(7), 16411661. a © 2021 indonesian society for science educator 327 j.sci.learn.2021.4(4).327-336 received: 22 february 2021 revised: 20 may 2021 published: 01 august 2021 opinions of secondary school students on the use of mobile augmented reality technology in science teaching gonca keçeci1, pelin yıldırım2*, fikriye kırbağ zengin2 1department of mathematics and science education, faculty of education, firat university, elaziğ, turkey 2institute of education sciences, firat university, elaziğ, turkey corresponding author: yildirim.pelin92@gmail.com abstract mobile augmented reality (mar) draws attention in terms of providing a flexible learning process and environment. it is thought that learning environments can be more effective using mar technology. the research aimed to determine the secondary school students' views on the use of mar technology in science teaching. the research was conducted with 143 students studying in the sixth grade at two different secondary schools in turkey. in the research, convergent parallel patterns from mixedmethod patterns were used. for eight weeks, the research was carried out to cover the systems unit in our body, situated in t he 6thgrade science course curriculum. during the process, the anatomy 4d application was used in the experimental group during the procedure, and the science course curriculum was used in the control group. in this paper, a part of the mixed research, the qualitative data collected from the experimental group's semi-structured interview form and diaries were analyzed. descriptive analysis was used to analyze the data. as a result of the research, although the students do not have a clear idea about ar and mar technologies at the beginning of the application, it is observed that their thoughts change over time, and mar applications are effective on the success of the course. keywords mobile augmented reality, science teaching, secondary school students, student opinions. 1. introduction in recent years in which science is integrated with technology and increasingly dominates the world, it is necessary to educate individuals who can adapt to the needs of the age and developments in technology and choose what they need from the knowledge and skills. today, human capital, the most valuable part of technologically advanced, developing, or underdeveloped countries, depends on education in general. in addition, the 21stcentury skills that are expected to be from individuals are generally technology-centered. questions on how to use the training in order to improve the quality of education lead the studies in the areas of education and training technology in recent years (heinecke, milman, washington, & blasi, 2001; wang & hannafin, 2005; watson, 2001; kozma & anderson, 2002). effective use of information technologies by individuals is accepted as an indicator of the advanced level of societies. various policies and complete projects have been initiated to integrate information technologies such as turkey, italy, germany, japan, united states of america (usa) into educational environments (keles, oksüz & bahcekapili, 2013). it has become imperative for various reasons like this to develop educational environments and teaching programs to enable technological tools. at this point, the concept of mobile augmented reality (mar) comes to the forefront in order to provide a flexible learning process and an effective learning environment for individuals called z generations who are born between 1995 and 2010, growing with tablet computers, mobile phones (ay, 2018). the mar concept, acknowledged in our day to be an indispensable part of daily life, thanks to technologies developed for especially mobile devices, can be defined as applications through mobile devices providing integration of virtual objects into real-world environments in augmented reality (ar) (ifenthaler & eseryel, 2013). thus, mar applications work on conveying the captured images of an object obtained through various information technologies due to processing it to the real world through mobile devices. ar applications work with the principle of transmitting the image of the object obtained through various mailto:yildirim.pelin92@gmail.com file:///c:/users/sertan/desktop/tojet.docx%23ref1 file:///c:/users/sertan/desktop/tojet.docx%23ref1 file:///c:/users/sertan/desktop/tojet.docx%23ref2 file:///c:/users/sertan/desktop/tojet.docx%23ref3 file:///c:/users/sertan/desktop/tojet.docx%23ref4 file:///c:/users/sertan/desktop/tojet.docx%23ref5 journal of science learning article doi: 10.17509/jsl.v4i4.32310 328 j.sci.learn.2021.4(4).327-336 information technologies to the real environment. in ar applications, the reality is created due to the synchronic enrichment of the real-world image with the virtual data formatted and revealed in a computer environment. ar applications are grouped under two headings as imagebased and location-based. it is primarily made understandable through virtual objects, locator pointer codes, and physical objects in image-based ar applications. afterward, it is provided to convert the pointer image obtained by various mobile devices into 3d objects with the help of ar imaging programs (icten & güngör, 2017; yilmaz, 2014). image-based ar applications are also divided into two, in itself, pointerbased and non-pointer-based. in pointer-based ar applications, objects in the real environment can only be introduced, provided that pointers are introduced to the system. in non-pointer ar applications, existing objects in the environment are used instead of adding a pointer to the real environment. location-based ar applications are based on the determination of the location through the features of mobile devices such as wireless networks and gps, and the inclusion of virtual objects to the designated location, that is, to the real world (somyürek, 2014). with the development of technology, the mar studies that have entered our lives are increasing daily. cheng & tsai (2013), in their research, have identified two major ar technology in science education, are named as image-based ar (eursch, 2007; kerawalla, luckin, seljeflot & woolard, 2006) and location-based ar (dunleavy, dede, & mitchell, 2009; rosenbaum, klopfer & perry, 2007; squire & jan, 2007). when the use of mar applications in the field of education is examined, it is understood that location-based mobile augmented reality applications are in the majority (ardito & lanzilotti, 2008; papagiannakis et al., 2005; efstathiou, kyza & georgiou, 2018; mcmahon, cihak & wright, 2015; turan, meral & sahin, 2018). there are also many pieces of research determining the effects of mar; outdoor learning (arvanitis et al., 2009), students’ socioscientific reasoning (chang, hsu, wu, & tsai, 2018), students’ historical reasoning (efstathiou et al., 2018), science interest and collaboration skills (bressler & bodzin, 2013). the number of studies conducted to determine students' opinions about mar technology in science teaching in turkey is limited. however, these studies have been carried out in recent years. considering the conducted studies, the objective of research carried out by kücük (2015) was to determine the effects of mar applications in anatomy education on academic successes and cognitive loads of medical students, and the students' views on these applications. as a result of the analysis of the data obtained through data collection tools, results have been obtained that mar application embodies the subject, increases the interest in the lesson, provides a flexible learning environment, and will be helpful in individual studies. another research conducted by sentürk (2018) aimed to determine the effect of mar applications in the process of science courses on the academic achievements, motivations, attitudes towards science, technology, and ar applications of secondary school students. according to the findings obtained from the research, it is concluded that ar technology facilitates learning, concretizes the subject, is fun, engaging, and realistic, and students are willing to use the ar application. this research has been carried out based on the facts that the ar technology has been demonstrated to be among the promising educational technologies in the recent issues of horizon reports which have been orderly published since 2004; that it is thought to occupy an important place in education; that there are statements in the body of literature expressing that the studies directed towards the use of ar applications in education are in the infancy stage (ifenthaler & eseryel, 2013; kücük, 2015; sentürk, 2018; sirakaya, 2015; yilmaz, 2014; zhang, sung, hou, & chang, 2014) and that there is a need for studies to determine the effects for created of their use in science education. 1.1 problem of research in the 21st century, it has been emphasized that science education has a vital role in reaching the target to be raised as individuals who can design content, use technology effectively and efficiently, have imagination, learning, and critical thinking skills (ecevit & kaptan, 2021). with the development of technology, the applications used in science education have increased. ar applications are one of them. however, the number of studies investigating the effect of using ar applications in science education on these goals is limited. nevertheless, the results of the researches conducted with ar and mar show that students' views of the use of these technologies are positive (abd majid, mohammed, & sulaiman, 2015; bressler & bodzin, 2013; chang, wu, & hsu, 2013; jamali, shiratuddin, wong, & oskam, 2015). on the other hand, different devices (smartphones, tablets, etc.) are needed for this application (abd majid, mohammed & sulaiman, 2015). whether the lack of these tools because of different socio-economic statuses will influence opinions on this issue is a matter of research. therefore, this research focuses on these problem situations. 1.2 research focus this research will play an essential role in determining secondary school students' views on mar in science education and the effects of mar application. in addition, the research results will provide researchers and teachers with a perspective on the use of mar technology, which is promising and will be a new generation of technologies that will significantly affect the quality of the education offered. file:///c:/users/sertan/desktop/tojet.docx%23ref6 file:///c:/users/sertan/desktop/tojet.docx%23ref7 file:///c:/users/sertan/desktop/tojet.docx%23ref7 file:///c:/users/sertan/desktop/tojet.docx%23ref8 file:///c:/users/sertan/desktop/tojet.docx%23ref9 file:///c:/users/sertan/desktop/tojet.docx%23ref9 file:///c:/users/sertan/desktop/tojet.docx%23ref10 file:///c:/users/sertan/desktop/tojet.docx%23ref10 file:///c:/users/sertan/desktop/tojet.docx%23ref11 file:///c:/users/sertan/desktop/tojet.docx%23ref12 file:///c:/users/sertan/desktop/tojet.docx%23ref12 file:///c:/users/sertan/desktop/tojet.docx%23ref13 file:///c:/users/sertan/desktop/tojet.docx%23ref14 file:///c:/users/sertan/desktop/tojet.docx%23ref15 file:///c:/users/sertan/desktop/tojet.docx%23ref16 file:///c:/users/sertan/desktop/tojet.docx%23ref16 file:///c:/users/sertan/desktop/tojet.docx%23ref17 file:///c:/users/sertan/desktop/tojet.docx%23ref18 file:///c:/users/sertan/desktop/tojet.docx%23ref19 file:///c:/users/sertan/desktop/tojet.docx%23ref15 file:///c:/users/sertan/desktop/tojet.docx%23ref20 file:///c:/users/sertan/desktop/tojet.docx%23ref20 file:///c:/users/sertan/desktop/tojet.docx%23ref21 file:///c:/users/sertan/desktop/tojet.docx%23ref21 file:///c:/users/sertan/desktop/tojet.docx%23ref22 file:///c:/users/sertan/desktop/tojet.docx%23ref5 file:///c:/users/sertan/desktop/tojet.docx%23ref21 file:///c:/users/sertan/desktop/tojet.docx%23ref22 file:///c:/users/sertan/desktop/tojet.docx%23ref24 file:///c:/users/sertan/desktop/tojet.docx%23ref6 file:///c:/users/sertan/desktop/tojet.docx%23zhang file:///c:/users/sertan/desktop/tojet.docx%23zhang journal of science learning article doi: 10.17509/jsl.v4i4.32310 329 j.sci.learn.2021.4(4).327-336 2. method in this research, which is based on a mixed-method, parallel pattern converging from mixed-method patterns was used (creswell & plano clark, 2015). convergent parallel patterns are patterns in which the quantitative and qualitative data are collected concurrently and analyzed independently following the research context. then, combine two sets of data to produce meaningful results. in this research which carried out in line with the convergent parallel pattern, on the one hand, the diaries reflecting the positive/negative emotions and thoughts, observations, suggestions of the application, which were expressed at the end of each course by the students, were analyzed using qualitative analytical methods. on the other hand, data on students' academic achievements and attitudes were collected and analyzed by quantitative methods. for eight weeks, the research was carried out to cover the systems unit in our body, situated in the 6th-grade science course curriculum. during the process, the anatomy 4d application was used in the experimental group during the procedure, and the science course curriculum was used in the control group. since the ar application was not carried out with the students in the control group, the students' opinions were not included in qualitative data analysis. in this paper, a part of the mixed research, the qualitative data collected from the experimental group's semi-structured interview form and diaries were analyzed. 2.1 participants in this research-based mixed method, the convergent parallel design, one of the mixed-method designs, was used. in the research, two schools were selected among 30 secondary schools classified as low and high socioeconomic levels in elazig to determine the effectiveness of students' socio-economic levels on various variables. yearend averages of the branches were taken into account in determining the experimental and control groups. the experimental and control groups of the research were randomly selected among twelve classes that did not differ according to the year-end averages. in turkey, the 8-year compulsory education period has been increased to 12 years since the 2012-2013 academic year. 4+4+4 the education system, which is formulated in the form and consists of 3 levels, is organized as a first-tier 4-year primary school, a second-tier four-year secondary school, a thirdtier 4-year high school. the research was carried out with 143 students studying in the 6th grade of barbaros hayrettin paşa secondary school and bahçelievler secondary school in elazig province in the 2017-2018 academic year. barbaros hayrettin paşa secondary school was coded as primary school, and bahçelievler secondary school was the second school in the research. the research was carried out with two experimental and two control groups, one experiment, and one control group in each school. in the research, the experimental and control group in barbaros hayrettin paşa secondary school was named as experiment-1 and control-1 group, and the experimental and control group in bahçelievler secondary school was named as experiment-2 and control-2 group. the photos of the anatomy 4d application used in the teaching of the lesson in the experimental groups are shown in figure 1. 2.2 demographic characteristics of students the student information form was used to determine the demographic characteristics of the students who formed the research group. when we look at the distribution of students by schools and gender, the first group in the experiment-1 group 23 (13 girls, 10 boys), the control-1 group consisted of 23 (13 girls, 10 boys) students. in the second school, the experiment-2 group consisted of 48 (23 girls, 25 boys), and the control-2 group consisted of 49 (27 girls, 22 boys) students. 2.3 working process the research was carried out with 143 students attending the 6th grade of barbaros hayrettin pasa secondary school and bahcelievler secondary school in elazig province in the 2017-2018 academic year. in this research, a total of 2 experiments and 2 control groups, one experimental and one control group in each school, were carried out. the year-end averages of the branches for the previous year were considered in determining the experimental and control groups. in this research, a semistructured interview form was applied to the students in the experimental groups, and the students in the experimental groups kept diaries. the research included eight weeks for the systems unit in our body, one of the units in the curriculum of the 6th-grade science course. during the process, the anatomy 4d application, a mar application, was used in support and movement system, respiratory figure 1 sample photos of mar application in experimental groups journal of science learning article doi: 10.17509/jsl.v4i4.32310 330 j.sci.learn.2021.4(4).327-336 system, circulatory system in the experimental groups, and the control groups were subjected to the existing textbook the course. during the process, the researcher has made the anatomy 4d application available on the smartboard through a computer program. afterward, students who had tablet computers among the experimental groups were asked to download the anatomy 4d application to their computers and bring their tablet computers throughout the process. in the first week of the research, experiments and control groups were determined. students in experimental groups were divided into groups with at least one tablet computer in each group, were informed about mar technology and anatomy 4d application, and pre-tests were applied. in order to permit each student in the experimental groups to actively use the anatomy 4d application both in the classroom and outside the classroom environment during the research period, two worksheets were developed with ar technology were given. in the 2nd week of the application, the anatomy 4d application was activated on smartboard and tablet computers, and the subject of support and movement system was processed. throughout the process, on the one hand, the places of the bones in our body were shown actively. on the other hand, the visual was enriched with theoretical knowledge, and simultaneously, learning was performed. furthermore, from time to time, groups were given time to use the application from their tablet computers, and their active participation in the process was ensured. in the 3rd week of the application, after the anatomy 4d application was activated on both the smartboard and the tablet computers, questions were asked about the bone and joint types in order to repeat what was done in the previous course, and the students were asked to show the locations of various bones and joints through the application. subsequently, the transition to the muscular system and the muscles' location through the application are shown actively. in the 4th week, the application of anatomy 4d was activated on smartboard and tablet computers, and the respiratory system subject was processed. throughout the process, the primary organs of the respiratory system were shown in 3d, and theoretical information was given about the organs. in addition, the students were asked to learn about the respiratory system organs and give feedback from time to time. in the 5th week, the application of anatomy 4d was activated on the smartboard and tablet computers, and the respiratory system subject was processed. throughout the process, the primary organs of the respiratory system were demonstrated in 3d, and theoretical information was given about the organs. from time to time, the students were asked to give information about the respiratory system organs and give feedback. subsequently, a short introduction was made to the circulatory system's topic, and the students' readiness levels were observed. at the 6th and 7th weeks of the application, the circulatory system's last subject has been processed. in addition to the "human body" worksheet developed with the ar technology used in the processing of support and movement system and respiratory system issues, the "heart" worksheet was also used in order to make learning about the structure of the heart, to concretize the abstract concepts that are difficult to understand. in the last week of the application, the research was terminated by applying a semi-structured interview form to the experimental groups. the steps followed during the research and the schematic structure for this process are given in figure 2 3. data collection tools in this research, semi-structured interview forms and diaries kept by the students in the experimental groups were used as data collection tools. in research, a semistructured interview form consisting of 6 questions was used to determine the students' opinions in the experimental groups on the practice used and the effects of this application on various variables. the semi-structured interview form was applied to 65 students in the experimental groups, and the students were given 40 minutes to answer the questions in the interview form. in addition, within the scope of the research, the students in the experimental groups were asked to keep the diaries during which they could reflect their positive/negative figure 2 flow chart regarding the steps followed during the research journal of science learning article doi: 10.17509/jsl.v4i4.32310 331 j.sci.learn.2021.4(4).327-336 feelings and thoughts, observations, and suggestions about the application. 3.1 data analysis descriptive analysis was used to analyze the data obtained from the semi-structured interview form conducted for the students in the experimental groups. the data obtained in this process were coded using the concepts that are likely to play a crucial role in answering the research questions developed, and then these codes were categorized under broader themes. in this way, the basis for linking the themes was prepared, and the variables in the data set were compared. in addition to this, direct quotations from the data sources were included, and the students' opinions were put forward effectively and contributed to the validity of the research. interpretations to be made concerning the relevance of these themes and their predictions and predictions will make the analysis more important (cepni, 2014). during the research period, descriptive analysis was used to analyze the data obtained by the students in the experimental groups. in the descriptive analysis, a framework has been formed from the research's conceptual framework to determine which themes will be organized and presented the data obtained through the students' diaries. the data obtained following the frame created has been read and arranged. the edited data has been made understandable by supporting it with direct quotations in the necessary parts of the research. finally, some conclusions have been made by interpreting the findings that have been made clear (cepni, 2014). the students' diaries in the experimental groups were collected under the themes of 'feelings and thoughts about ar technology and anatomy 4d application, problems encountered during the research period'. 4. results with this research, secondary school students' opinions on the use of mar technology in science teaching were determined. results are presented in two subtitles, "semistructured interview form results" and "students' diary results". 4.1 results from semi-structured interview it is aimed to reach the concepts and themes that can clearly explain the data so that the readers can clearly understand the data obtained through the semi-structured interview form consisting of 6 questions. accordingly, an inductive logic based on coding was preferred (yildirim & simsek, 2006). in this process, data resembling each other were categorized under certain concepts and themes, and sensitivity was shown to reflect the answers given by the students without deviating their meanings. a semi-structured interview form was applied to 65 students (35 girls, 30 boys) in the experimental groups. in the research, student names were presented by coding. the answers to the questions in the form, frequencies related to the answers to the questions, the percentage of these frequencies, and some students' answers are given below. the first question, 'can you explain what mobile augmented reality (mar) technology is?'. the frequencies related to the answers given by the students to this question and the percentages related to these frequencies are shown in table 1. table 1 describes what mobile augmented reality (mar) technology is. it is observed that respond in the form that 24.62% of students respond to the question as 3d image, 18.46% of an application that introduces our body, anatomy 4d and 7.69% of as the use of mobile devices in applications that enable us to organize and direct our daily lives. these results show that most students are familiar with mar technology and are based on the anatomy 4d application used during the research while table 1 frequency and percentage distribution of students' answers to their thoughts about mar technology answers f % 3d image 16 24.62 an application that introduces our body, anatomy 4d 12 18.46 enrichment of our environment with simultaneous addition of digital objects such as sound, image, animation, hologram 11 16.92 using applications such as aurasma, layer, alive, 2d, and 3d visuals depending on some criteria for various purposes 6 9.23 use of mobile devices in applications that enable us to organize and direct our daily lives 5 7.69 a view of a new perception environment as direct or indirect 3d 4 6.15 i don’t know 3 4.62 various applications such as aurasma, layer, alive 2 3.08 virtual reality 1 1.54 transfer what we see in real life to a virtual environment 1 1.54 integration of ar technology into mobile devices 1 1.54 demolition of the wall between the real world and the virtual world 1 1.54 an application to help students understand the subject better 1 1.54 i know, but i can't explain 1 1.54 file:///c:/users/sertan/desktop/tojet.docx%23ref25 file:///c:/users/sertan/desktop/tojet.docx%23ref25 journal of science learning article doi: 10.17509/jsl.v4i4.32310 332 j.sci.learn.2021.4(4).327-336 defining this technology. some of the students' answers to the relevant question are given below: emir: ‘enrichment of the environment in which we are in real-time with objects such as sound, image, animation, hologram.’ side: 'like virtual reality, it's like seeing in 3d and showing detail.' mert: ‘mobile technologies can enable us to organize and direct our daily lives with mobile applications we use during the day. one of the applications used in mobile devices is augmented reality technology.’ 2nd question 'have you used augmented reality (ar) or mobile augmented reality (mar) technologies and used applications for these areas? can you explain?'. sixty-three students answered yes, and two students answered no to this question. these results show that almost all students are aware of the ar and mar technologies and have started to benefit from the work done. some of the students' answers to the relevant question are given below: ali: ‘yes, i used it. i wore 3d glasses in some games and when i went cinema.’ melis: ‘no, i didn't use it. for the first time, i met with ar and mar.’ sevgi: ‘no, i did not use it, but after i started to use it in the science course, i started to use it. it helped me a lot. because in our writings we can study anatomy 4d for working on those issues.’ 3rd question 'do you think the use of mobile augmented reality (mar) application during the course contributes to the lesson's success? can you explain?'. fifty-nine students answered yes to this question, two students at medium level, and four answered no. the frequencies related to the answers given by the students to this question and the percentages related to these frequencies are shown in table 2. when table 2 is examined, it is seen that 33.85% of the students responded to this question as facilitating understanding and learning, 30.77% providing precise and detailed learning, and 9.23% making the lesson fun. these results show that the use of the mar application plays an active role in the development of the students' academic achievement levels and contributes to the learning of the current subject by having fun. the answer given by one of the students to the relevant question is given below: furkan: ‘yes, it is remaining in my mind. i think it is helping to classes, and i think the science course more tasteful and memorable with visual.’ sertan: 'yes, i think. because with the application of anatomy 4d, we see parts of our bodies as if they were really in front of us. that's why i think it's contributing.' sila: 'according to me yes. because i used to love science less, i was less interested. now i love it more, and i'm interested.' 4th question is 'did the use of mobile augmented reality (mar) application affect the attitude of the course. can you explain?'. thirty-eight students answered yes, two students at medium level, and 25 students answered no to this question. the frequencies related to the answers given by the students to this question and the percentages related to these frequencies are shown in table 3. when table 3 is examined, it is seen that 56.92% of the students responded as changing the attitude towards science positively, 20% changing the attitude towards technology positively, and 16.92% providing an increase in knowledge. the answer given by one of the students to the relevant question is given below: omer: 'thanks to the mobile augmented reality application, i made more contributions to the science course. i liked it more than before.' tarik: 'yes, it's okay. i'm not much of a student who loves science. however, when using mar, i realized that science is more fun and important lesson.' buse: 'yes, it's okay. because beforehand, i understood this application very simple. however, in every science table 2 frequency and percentage distribution of students' response to the success of mar application use in the course answers f % facilitate learning and understanding 22 33.85 provide clear and detailed learning 20 30.77 to embody abstract concepts 20 30.77 making learning permanent 9 13.85 allowing of applied learning 7 10.77 making the lesson fun 6 9.23 making sense of reality 5 7.69 provide an increase in the success in the course 3 4.62 love the lesson, interested in the lesson 3 4.62 ensuring adaptation to the course 1 1.54 allowing technology-based training 1 1.54 table 3 frequency and percentage distribution of students' response to the contribution of mar application used to the attitude towards the course answers f % changing the attitude towards science in a positive direction 37 56.92 changing the attitude towards technology in a positive direction 13 20.00 provide the increase in knowledge accumulation 11 16.92 facilitate understanding 9 13.85 making the lesson fun 7 10.77 provide participation in the course 3 4.62 making learning permanent 2 3.08 making sense of reality 1 1.54 provide clear and detailed learning 1 1.54 journal of science learning article doi: 10.17509/jsl.v4i4.32310 333 j.sci.learn.2021.4(4).327-336 course at the moment, i want to open this application. i thought science class was boring before. however, now i like science more than other courses.' when the students' answers about to whether mar application contributes to scientific process skills are examined, it was determined that 50 students responded in the form of no, and 15 students in the form of yes. when the students' answers about the contribution of mar application to scientific process skills are examined; it was determined that 13 students responded in terms of observation, seven students as classification, and seven students as the definition. the answer given by one of the students to the relevant question is given below: omer: ‘observation: we were able to examine the systems in our body with augmented reality.’ ali: 'observation: i didn't normally know the inside of the human body, but i saw both the human body and the muscles with this practice.' ayse: ‘defining: we can better tell what we have learned.’ students all have the advantage when the students' answers about the advantages and disadvantages of using the mar application in the course processing are examined. among the 65 students, nine students have the disadvantage. one student was identified to respond in the form of an no idea. the students 'what are the advantages and disadvantages of using mobile augmented reality (mar) in the course?' the frequencies of the answers they gave to the question and the percentage ratios depending on these frequencies are shown in table 4 and table 5. when table 4 is examined, it is seen that 44.62% of the students responded to this question as facilitating learning and understanding, 24.62% providing precise and detailed learning, and 13.85% embodying abstract concepts. when table 5 is examined, it is seen that two students each responded as the application was not suitable for use on every tablet, causing eye fatigue. the answer given by one of the students to the relevant question is given below: arda: 'helping us to understand the important topics in the course better. disadvantage = this application only works on certain tablets. it doesn't work in some.' ayşe: 'the lesson began to be more fun and different. we have seen everything more in detail. so i think no disadvantage.' duygu: 'disadvantages = my eyes were tired, the internet was slow, people were playing the game from the tablet, the application does not work in ios. advantages = i began to get to know the organs well, my participation in the classes was excellent, it added a lot to me.' 4.2 results from students' diaries the descriptive analysis technique was used to analyze qualitative research data obtained through the students' diaries in the experimental groups during the research period. after reading the logs several times, codes were formed according to the concepts obtained from the data. after the codes were compiled and analyzed, common aspects between the codes were found and categorized. the students' diaries were collected under the themes of 'feelings and thoughts about ar technology and anatomy 4d application, problems encountered during the research period'. results related to the theme of the students' feelings and thoughts related to ar technology and anatomy 4d application when the diaries of the students in the experimental groups were examined, sentences that reflect the new hearings of the students' ar and mar concepts were frequently encountered. as time progressed, it was observed that the students had mastered both these concepts and the application used. in their diaries, the table 4 frequency and percentage distribution of students' responses to advantages of using mar application in the course answers f % facilitate learning and understanding 29 44.62 provide clear and detailed learning 16 24.62 to embody abstract concepts 9 13.85 contributing to making an observation 7 10.77 provide the increase in knowledge accumulation 6 9.23 making the lesson fun 6 9.23 making learning permanent 4 6.15 love the lesson, interested in the lesson 3 4.62 providing a learning environment by living by doing 1 1.54 provide the participants to the course 1 1.54 provide the adapt to the course 1 1.54 table 5 frequency and percentage distribution of students' responses to the disadvantages of using mar application in the course answers f % application isn't suitable to be used in each tablet 2 3.08 causes eye fatigue 2 3.08 internet is slow 1 1.54 use of for other purposes outside the class hours of tablets 1 1.54 the content of the mar application is in english 1 1.54 the slowing down of the functioning of the course 1 1.54 limited activities in mar applications 1 1.54 journal of science learning article doi: 10.17509/jsl.v4i4.32310 334 j.sci.learn.2021.4(4).327-336 students expressed that the teacher played a decisive role in their love for the lesson and learned the lesson by having fun. during the application, the students have been enabled to actively use the practice to play an active role in the course. thus, an educational approach that takes the students to the center has been adopted. in addition, each student was given application-specific research papers to personalize their learning process independently of time and place by using their mobile devices. below are direct quotations from some students: 'the first day was wonderful. i liked the teacher, the lesson and the subject. i think the anatomy 4d application is nice. the application, our muscle system, in short, telling all the organs in our body. if we come to the useful videos that we watch in the course, they are standing in front of us with the most ornamented form, and since all of these privileges are for our class, we have handled the lesson proudly.' 'we had so much fun in science today. hodja was very good, kind, and sometimes made us laugh. the course was great. it was very spectacular to see the human body and organs in 3 dimensions.' 'what the teacher told things us was wonderful, we had so much fun. the functioning of the lesson with the tablet was very nice. i hope it applies to other classes...' results on the theme of problems students live through the process when the students' diaries were examined, some of the students stated that the tablets used were not fixed and, therefore, the image could not be clarified fluently. however, over time, students have found a solution to eliminating this problem, which they use by fixing their working papers and tablets with the help of various objects. in addition, the students in the experimental-1 group sometimes stated that the course of the lesson slowed down, and the reason for this was the noise caused by the classmates' conversations. during the process, the utmost care has been taken to ensure classroom management to minimize the impact of this problem. below are direct quotations from some students: 'today, we learned the structure of the bones in the course. our body is a great miracle. in this course, i also learned topics i never knew. it was great to see our body in 4d, but the internet connection was distressed. it was just boring. as usual, this course is great.' 'there is noise in the same way in the class. so, it's not from my teacher, but the students were because of that noise...' 5. discussion the data obtained from the semi-structured interview form and the diaries that the students had kept during the research period showed that a large proportion of students heard ar and mar concepts for the first time, they did not use ar and mar technologies before, and they did not use applications for these areas. in addition, it is seen that a significant number of students associate mar technology with the application used during the process and associate mar technology with the application of anatomy 4d. although the students do not have a clear idea about both the application to be used during the research process and the ar and mar technologies, it is observed that their thoughts change over time, and mar practices are effective on the success of the course. in the results obtained, various factors play an essential role. the mar application used in the course gives a sense of reality, allowing practical learning, understanding, facilitating learning, and providing precise and detailed learning. when the related literature is examined, it is seen that the number of studies that determine the views of mar technology in the course of the course is quite limited. sirakaya (2015), in his research, investigated students' views on ar material. as a result of the research, the ar reached the opinion that the learning material embodied the abstract issues, made the subjects more comprehensible, made the lesson more exciting and fun, increased communication, and became more effective at the course. fleck & simon (2013) investigated the effect of ar technology on students' achievements and conceptual misconceptions. he found that the use of ar technology embodies abstract concepts by presenting a rich learning environment and thus facilitates understanding. the results of the studies carried out by sirakaya (2015) and fleck & simon (2013), which include the students 'views on mar application, are consistent with the student's views on the use of mar technology in the course. all of the students in the experimental groups stated that the use of the mar application was advantageous in the course of the course. in contrast, a small part stated some disadvantages of using the mar application in the course. the students stated that the use of mar application has several advantages: understanding facilitating learning, providing precise and detailed learning, concreting abstract concepts, increasing knowledge, making the lesson fun, and making learning permanent. in addition to this, some students stated that the mar application used in the course is not suitable for every tablet. the internet is slow, and the tablets are used for different purposes other than course hours. furthermore, the content of the mar application is in english. within the scope of the research, sentürk (2018), di serio, ibáñez & kloos (2013) have concluded that the ar technologies used in the course processing have disadvantages arising from technical reasons. the results obtained from the researchers ' studies support the results of this research on the disadvantages of using the mar application in the course. file:///c:/users/sertan/desktop/tojet.docx%23ref24 file:///c:/users/sertan/desktop/tojet.docx%23ref27 file:///c:/users/sertan/desktop/tojet.docx%23ref24 file:///c:/users/sertan/desktop/tojet.docx%23ref27 file:///c:/users/sertan/desktop/tojet.docx%23ref22 file:///c:/users/sertan/desktop/tojet.docx%23ref28 journal of science learning article doi: 10.17509/jsl.v4i4.32310 335 j.sci.learn.2021.4(4).327-336 6. conclusion this experimental research was carried out with middle school students to determine their views on mar technology used in science teaching. as a result of the research, students stated that mar has several advantages, such as facilitating learning and providing precise and detailed learning. from this point of view, researchers can provide technology-based education by integrating technological tools into educational environments. however, the research is limited to a small group of students and one mar application available. in order to expand the field of this limitation, the application needs to be integrated into activities that address different learning areas. there is a need for people in the software field in the development of such applications. at this point, richer learning environments can be created by collaborating with science educators. different socio-economic levels of students and the difference in access to technology resulting from this did not affect their views. it has been found that students find solutions to eliminate the technological problems they face during the research. this can outweigh the desire to use it, even if it is difficult to access the technology. notes this research is based on a master thesis named "the impact of science teaching made with mobile augmented reality technology on science and technology attitudes and academic achievement of secondary school students". gonca keçeci’s orcid number: 0000-00022582-3850. pelin yıldırım's orcid number: 0000-00034425-2472. references abd majid, n. a., mohammed, h., & sulaiman, r. (2015). students’ perception of mobile augmented reality applications in learning computer organization. procedia-social and behavioral sciences, 176, 111-116. https://doi.org/10.1016/j.sbspro.2015.01.450 ardito, c., & lanzilotti, r. (2008). “isn’t this archaeological site exciting!”: a mobile system enhancing school trips. in avi’08 proceedings of the working conference on advanced visual interfaces (pp. 488–489). napoli: acm. doi: 10.1145/1385569.1385670 arvanitis, t. n., petrou, a., knight, j. f., savas, s., sotiriou, s., gargalakos, m., & gialouri, e. (2009). human factors and qualitative pedagogical evaluation of a mobile augmented reality system for science education used by learners with physical disabilities. personal and ubiquitous computing, 13, 243–250. doi: 10.1007/s00779-007-0187-7 ay, b. (2018). user profile of the 21st century: y z and alpha belt. iii. international information and document management department student congress, ankara. http://bby2018kongre.ankara.edu.tr/wpcontent/uploads/sites/444/2018/05/02-03-b%c3%bclentay.pdf bressler, d. m., & bodzin, a. m. (2013). a mixed-methods assessment of students' flow experiences during a mobile augmented reality science game. journal of computer assisted learning, 29(6), 505-517. doi: 10.1111/jcal.12008 cepni, s. (2014). araştırma ve proje çalışmalarına giriş (7. baskı) [introduction to research and project work]. trabzon: celepler printing. chang, h. y., hsu, y. s., wu, h. k., & tsai, c. c. (2018). students’ development of socio-scientific reasoning in a mobile augmented reality learning environment. international journal of science education, 40(12), 1410-1431. doi: 10.1080/09500693.2018.1480075 chang, h. y., wu, h. k., & hsu, y. s. (2013). integrating a mobile augmented reality activity to contextualize student learning of a socio-scientific issue. british journal of educational technology, 44(3), e95-e99. https://doi.org/10.1111/j.1467-8535.2012.01379.x cheng, k. h., & tsai, c. c. (2013). affordances of augmented reality in science learning: suggestions for future research. journal of science education and technology, 22, 449–462. doi: 10.1007/s109560129405-9 cresswell, j. w., & plano clark, v. l. (2015). mixed method research: design and execution (y. dede & s. b. demir, eds.). ankara: anı publishing. di serio, á., ibáñez, b. m., & kloos, d. c. (2013). impact of an augmented reality system on students' motivation for a visual art course. computers & education, 68, 586-596. doi: 10.1016/j.compedu.2012.03.002 dunleavy, m., dede, c., & mitchell, r. (2009). affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. journal of science education and technology, 18, 7–22. doi: 10.1007/s10956-008-9119-1 ecevit, t., & kaptan, f. (2019). describing an argumentation aiming research questioning model for 21st century skills. hacettepe university journal of education faculty, 36(2), 470-488. http://www.efdergi.hacettepe.edu.tr/yonetim/icerik/makaleler/3 118-published.pdf efstathiou, i., kyza, e. a., & georgiou, y. (2018). an inquiry-based augmented reality mobile learning approach to fostering primary school students’ historical reasoning in non-formal settings. interactive learning environments, 26(1), 22-41. doi: 10.1080/10494820.2016.1276076 eursch, a. (2007). increased safety for manual tasks in the field of nuclear science using the technology of augmented reality. ieee nuclear science symposium conference record 3:2053–2059. fleck, s., & simon, g. (2013). an augmented reality environment for astronomy learning in elementary grades: an exploratory study. 25. conference francophone sur interaction homme-machine, bordeaux, france. doi: 10.1145/2534903.2534907 heinecke, w. f., milman, n. b., washington, l. a., & blasi, l. (2002). new directions in the evaluation of the effectiveness of educational technology. computers in the schools, 18(2-3), 97-110. doi: 10.1300/j025v18n02_07 icten, t., & güngör, b. a. l. (2017). examining the latest developments and applications on augmented reality. gazi university journal of science part c: design and technology, 5(2), 111-136. https://dergipark.org.tr/tr/download/article-file/840617 ifenthaler, d., & eseryel, d. (2013). facilitating complex learning by mobile augmented reality learning environments. in reshaping learning: frontiers of learning technologies in a global context, 415-438. springer, berlin, germany. doi: 10.1007/978-3-642-32301-0_18 jamali, s. s., shiratuddin, m. f., wong, k. w., & oskam, c. l. (2015). utilising mobile-augmented reality for learning human anatomy. procedia-social and behavioral sciences, 197, 659-668. https://doi.org/10.1016/j.sbspro.2015.07.054 keles, e., oksüz, b. d., & bahcekapili, t. (2013). teachers' views on the use of technology in education: example of fatih project. gaziantep university journal of social sciences, 12(2). kerawalla, l., luckin, r., seljeflot, s., & woolard, a. (2006). “making it real’’: exploring the potential of augmented reality for teaching primary school science. virtual real, 10(3–4), 136–174. doi: 10.1007/s10055-006-0036-4i: kozma, r., & anderson, r. (2002). qualitative case studies of innovative pedagogical practices using ict. journal of computer assisted learning, 18(4), 387-394. doi: 10.1046/j.02664909.2002.00250.doc.x https://doi.org/10.1016/j.sbspro.2015.01.450 file:///c:/users/sertan/desktop/tojet.docx%23ref1 file:///c:/users/sertan/desktop/tojet.docx%23ref2 http://bby2018kongre.ankara.edu.tr/wp-content/uploads/sites/444/2018/05/02-03-b%c3%bclent-ay.pdf http://bby2018kongre.ankara.edu.tr/wp-content/uploads/sites/444/2018/05/02-03-b%c3%bclent-ay.pdf http://bby2018kongre.ankara.edu.tr/wp-content/uploads/sites/444/2018/05/02-03-b%c3%bclent-ay.pdf file:///c:/users/sertan/desktop/tojet.docx%23ref3 file:///c:/users/sertan/desktop/tojet.docx%23ref4 file:///c:/users/sertan/desktop/tojet.docx%23ref14 file:///c:/users/sertan/desktop/tojet.docx%23ref5 file:///c:/users/sertan/desktop/tojet.docx%23ref6 file:///c:/users/sertan/desktop/tojet.docx%23ref8 http://www.efdergi.hacettepe.edu.tr/yonetim/icerik/makaleler/3118-published.pdf http://www.efdergi.hacettepe.edu.tr/yonetim/icerik/makaleler/3118-published.pdf file:///c:/users/sertan/desktop/tojet.docx%23ref9 file:///c:/users/sertan/desktop/tojet.docx%23ref11 file:///c:/users/sertan/desktop/tojet.docx%23ref12 file:///c:/users/sertan/desktop/tojet.docx%23ref13 https://dergipark.org.tr/tr/download/article-file/840617 file:///c:/users/sertan/desktop/tojet.docx%23ref16 file:///c:/users/sertan/desktop/tojet.docx%23ref17 file:///c:/users/sertan/desktop/tojet.docx%23ref18 journal of science learning article doi: 10.17509/jsl.v4i4.32310 336 j.sci.learn.2021.4(4).327-336 kücük, s. (2015). effects of learning anatomy via mobile augmented reality on medical students' academic achievement, cognitive load, and views toward implementation (phd dissertation). ataturk university, turkey. mcmahon, d., cihak, d. f., & wright, r. (2015). augmented reality as a navigation tool to employment opportunities for postsecondary education students with intellectual disabilities and autism. journal of research on technology in education, 47(3), 157-172. doi: 10.1080/15391523.2015.1047698 papagiannakis, g., schertenleib, s., o’kennedy, b., arevalo-poizat, m., magnenat-thalmann, n., stoddart, a., & thalmann, d. (2005). mixing virtual and real scenes in the site of ancient pompeii. computer animation and virtual worlds, 16(1), 11–24. doi: 10.1002/cav.53 rosenbaum, e., klopfer, e., & perry, j. (2007). on location learning: authentic applied science with networked augmented realities. journal of science education and technology, 16(1), 31-45. doi: 10.1007/s10956-006-9036-0 sentürk, m. (2018). the investigation with solomon four-group design on the effect of using mobile augmented reality (ar) applications in the unit titled solar system and beyond in the seventh-grade on the students' academic success, motivation, science and technology attitude (master’s thesis). kocaeli university, turkey. sirakaya, m. (2015). effects of augmented reality applications on students' achievement, misconceptions and course engagement (phd dissertation). gazi university, turkey. somyürek, s. (2014). attracting the attention of generation z in the teaching process: augmented reality. educational technology theory and practice, 4(1), 63-80. https://doi.org/10.17943/etku.88319 squire, k. d., & jan, m. (2007). mad city mystery: developing scientific argumentation skills with a place-based augmented reality game on handheld computers. journal of science education and technology, 16(1), 5-29. doi: 10.1007/s10956-006-9037-z turan, z., meral, e., & sahin, i. f. (2018). the impact of mobile augmented reality in geography education: achievements, cognitive loads and views of university students. journal of geography in higher education, 42(3), 427-441. doi: 10.1080/03098265.2018.1455174 wang, f., & hannafin, m. j. (2005). design-based research and technology-enhanced learning environments. educational technology research and development, 53(4), 523. doi: 10.1007/bf02504682 watson, d. m. (2001). pedagogy before technology: re-thinking the relationship between ict and teaching. education and information technologies, 6(4), 251-266. doi: 10.1023/a:1012976702296 yildirim, a., & simsek, h. (2013). qualitative research methods in the social sciences. ankara: seckin publishing. yilmaz, r. m. (2014). effects of three dimensional storytelling developed with augmented reality technology on narrative skill and creativity (phd dissertation). ataturk university, turkey. zhang, j., sung, y. t., hou, h. t., & chang, k. e. (2014). the development and evaluation of an augmented reality-based armillary sphere for astronomical observation instruction. computers & education, 73, 178–188. file:///c:/users/sertan/desktop/tojet.docx%23ref19 file:///c:/users/sertan/desktop/tojet.docx%23ref7 file:///c:/users/sertan/desktop/tojet.docx%23ref21 file:///c:/users/sertan/desktop/tojet.docx%23ref22 file:///c:/users/sertan/desktop/tojet.docx%23ref23 file:///c:/users/sertan/desktop/tojet.docx%23ref24 https://doi.org/10.17943/etku.88319 file:///c:/users/sertan/desktop/tojet.docx%23ref25 file:///c:/users/sertan/desktop/tojet.docx%23ref26 file:///c:/users/sertan/desktop/tojet.docx%23ref27 file:///c:/users/sertan/desktop/tojet.docx%23ref28 file:///c:/users/sertan/desktop/tojet.docx%23ref29 file:///c:/users/sertan/desktop/tojet.docx%23ref30 a © 2021 indonesian society for science educator 160 j.sci.learn.2021.4(2).160-172 received: 23 january 2021 revised: 13 february 2021 published: 24 march 2021 exploring preschool teachers' pedagogical content knowledge: the effect of professional experience ali yigit kutluca1 1primary school teacher education, faculty of education/department of elementary education, istanbul aydin university, istanbul, turkey *corresponding author. alikutluca@aydin.edu.tr abstract this study aimed to explore the pedagogical content knowledge (pck) of preschool teachers. this multiple case study was conducted with the participation of two preschool teachers. in this way, two preschool teachers' pck about science teaching was examined, and it was attempted to determine the underlying causes of their pedagogical conceptualizations about any subject. therefore, the participants were asked to create a lesson plan specific to the subject matter they selected, answer the interview questions about pck, and implement the lesson plan they created. while the participants' responses to the lesson plan and pck interview questions were analyzed by inductive content analysis based on the constant comparative method, classroom observations were evaluated with an analytical evaluation rubric. furthermore, pck maps showing the interaction between the subject-specific and unique pck components of preschool teachers were created. results revealed that teachers did not have sufficient knowledge about science teaching and that their pck tended to change according to professional experience. it was determined that experienced teachers had more teacher-centered orientations than less experienced teachers with more student-centered orientations. moreover, it was observed that professional experience increased the relationship between the pck components. keywords pedagogical content knowledge, preschool teachers, early science teaching, professional experience 1. introduction one of the most vital themes in the national science standards is that all children can learn science, and all children have the opportunity to be scientifically literate (national research council [nrc], 2013; mone, 2013). children need to make sense of what is going on in their environment from the moment they are born, and consequently, they develop some simple scientific skills, which constitutes the first science experiences of their lives (furman, luzuriaga, taylor, podestá, & jarvis, 2019). preschool education provides excellent opportunities to support children's curiosity, and effective science teaching in early education may lay the foundations of rigorous thinking and understanding about the nature of science. to achieve all of them, the importance of preschool science teaching should be well understood, and it should be aimed to provide children with the acquisition of basic science knowledge and skills (cowie & otrel-cass, 2011). science education provided at an early age enables children to recognize the events in their environment and nature, perceive the relationships, make observations, interpret the knowledge, and acquire scientific process skills (ravanis, 2017). furthermore, children start to acquire environmental awareness and take charge of plants or animals such as nutrition and cleaning by obtaining the necessary knowledge to protect the environment. they develop a sense of responsibility by performing these tasks (kurniah, andreswari, & kusumah, 2019). therefore, science education in early childhood supports all areas of development of children. a research-based approach provides children with opportunities to make sense of the world and environment and satisfy their curiosity and learn scientific processes (dejonckheere, wit, keere, & vervaet, 2016). it supports them to gain the necessary knowledge in daily life and provides them with the ability to solve universal problems. therefore, crucial importance should be given to science education in early childhood, and it should be integrated into the daily curriculum (eliason & jenkins, 2003). in this process, preschool teachers should mailto:alikutluca@aydin.edu.tr journal of science learning article doi: 10.17509/jsl.v4i2.31599 161 j.sci.learn.2021.4(2).160-172 create safe and risk-free study areas where children can make observations by doing research. furthermore, they should encourage children's active participation by giving them time to share their ideas to support their creativity and problem-solving skills (mclachlan, fleer, & edwards, 2018). with this aspect, preschool teachers play a crucial role in providing compelling science experiences and creating environments where children can perform these science experiences. andersson and gullberg (2014), who suggested that it would not be enough alone for preschool teachers to give children correct answers about any science subject, determined four skills that can be developed and used by preschool teachers: ⚫ paying attention to and using children's previous experiences, ⚫ capturing unexpected things that happen at the moment they occur, ⚫ asking questions that challenge the children and that stimulate further investigation, ⚫ listening to the children and their explanations. preschool teachers' levels of knowledge, views, attitudes, and self-efficacy about science subjects are significant predictors for a qualified science teaching (saçkes, 2014). furthermore, preschool teachers' limited pedagogical conceptualizations for science teaching will also affect their teaching quality (neuman & danielson, 2020). therefore, it is necessary to explore preschool teachers' aspects representing their professional knowledge about science teaching in-depth. in this study, preschool teachers' pedagogical conceptualizations while structuring and implementing a science subject were examined. 1.1 the role of teacher in early childhood science education teachers play a crucial role in developing scientific thinking from early childhood and their positive attitudes towards science courses during elementary school years (thulin & redfors, 2017). the integration of the diversity of opportunities provided with teachers with different methods with positive attitudes and behaviors during science activities directs children to research, investigation, and examination, which are the foundations of scientific thinking. therefore, it is necessary for preschool teachers to offer different options during science activities and to include children in the discussion processes related to science events (bustamante, white, & greenfield, 2018). furthermore, preschool teachers should include science concepts in play-based activities integrated with the curriculum's achievements (gerde, pierce, lee, & van egeren, 2018). therefore, preschool teachers should develop more creative and original pedagogical strategies from primary or secondary school context-specific to science teaching (blömeke, jenßen, grassmann, dunekacke, & wedekind, 2017). previous results indicated that preschool teachers' science-specific knowledge was relatively low compared to science-specific knowledge of primary or secondary school teachers (garbett, 2003). andersson and gullberg (2014), who conducted action research with preschool and primary school teachers' participation, problematized the objective of preschool science education and the competencies needed by preschool teachers to carry out science activities in the classroom. the researchers determined that preschool teachers failed to support children's conceptual understanding in science activities carried out with them, indicating that preschool teachers are inadequate in adapting science subjects to the preschool education context (kallery & psillos, 2001; tu, 2006). however, andersson and gullberg (2014) suggested that preschool teachers needed more than just subject matter knowledge. therefore, the factors affecting preschool teachers' pedagogical conceptualizations about science teaching should be evaluated from multi-perspectives. 1.2 pedagogical content knowledge teacher knowledge is one of the most critical factors affecting teachers' classroom behaviors and their students' success (shulman, 1987). grossman (1990) collected the types of knowledge that the teacher should have to perform effective teaching under four titles: subject matter knowledge, general pedagogical knowledge, pedagogical content knowledge, and context knowledge. accordingly, the most important qualifier of effective teaching is pedagogical content knowledge (pck). according to loughran, mulhall, & berry (2008), pck, which teachers develop and are a teacher-specific quality, is the subjectspecific type of knowledge they have developed through time experiences. furthermore, pck is based on a mix of an understanding of the content that allows students to understand any subject and teacher pedagogy better. pck is subject matter-specific, title-specific, teacher-specific, and content-specific (kind, 2009). magnusson, krajcik, & borko (1999), who contextualized the pck based on science teaching, suggested that a teacher should have the following components for qualified science teaching: ⚫ orientations to teaching science (ots): ots reflects teachers' perspectives on science teaching. ⚫ knowledge about students' understanding of science (ksu): ksu includes students' concepts on specific topics, learning difficulties, motivation, and diversity of talent, learning style, the field of interest, level of development, and knowledge of needs. ⚫ knowledge of science curriculum (ksc): ksc represents the knowledge about the curriculum and curriculum materials. ⚫ knowledge of instructional strategies for teaching science (kisr): kisr indicates how the teacher benefits from instructional strategies and representation. journal of science learning article doi: 10.17509/jsl.v4i2.31599 162 j.sci.learn.2021.4(2).160-172 ⚫ knowledge of science learning assessment (kas): kas represents the knowledge about the methods to evaluate students' content learning. a teacher has specific insights and knowledge about the components of pck before performing teaching and aims to use these knowledge structures during teaching within the framework of his/her plan. however, these knowledge structures and components undergo specific changes after teaching, which is his/her interactions with the students during teaching. these interactions lead to the emergence of teacher efficacy, which is a new affective component of pck (park & oliver, 2008). in the literature, it is observed that pck, which is structured as an implicit form of teachers' professional knowledge, was generally conceptualized specifically for science teacher education (e.g., loughran et al., 2008; nilsson & vikström, 2015). furthermore, these studies were generally conducted with science teachers' participation or preservice teachers (e.g., nilsson, 2013; luft & zhang, 2014). however, little is known about preschool teachers' pedagogical conceptualizations about science teaching and their professional knowledge level and structure. the studies indicate that qualified teaching support could encourage young children to learn (e.g., mclean, jones, & schaper, 2015; larimore, 2020). however, it was reported that preschool teachers resisted while including science subjects in their teaching (gerde et al., 2018). it was stated that this resistance was generally caused by the lack of material (greenfield et al., 2009), limited understanding of science (kallery & psillos, 2001), and the lack of self-efficacy (leon, 2014). andersson and gullberg (2014) argued that preschool teachers should have pck, which is a characteristic that would go beyond their subject matter knowledge in order to perform an effective science teaching. for this purpose, preschool teachers should have extensive repertoires on play-based pedagogical strategies and children's cognition. therefore, the way preschool teachers reflect their concepts for the subject matter to classroom practices is related to their pck (de jong, 2003). furthermore, the ability to recognize domain-specific content in everyday situations can also be considered a component of preschool teachers' pck (mccray & chen, 2012). accordingly, preschool teachers can improve their pck by applying scientific activities in teacher education and in-service courses. abell (2008) criticized the use of similar samples (e.g., science teachers) in the studies on the development of pck and emphasized that different samples (inexperienced – experienced teachers, etc.) should be compared within the same study. however, very few studies evaluate preschool teachers' science-specific conceptualizations based on their professional knowledge (e.g., andersson & gullberg, 2014; furman et al., 2019). therefore, following the recommendation of abell (2008), the primary aim of this study was to explore preschool teachers' pck about science teaching. the secondary aim was to determine how professional experience changed the structure of pck. 2. method the multiple case study approach was used in this research conducted to explore preschool teachers' pck about science teaching. the case examined in this study was to describe the pck of preschool teachers continuing their postgraduate education about science teaching and to find the underlying causes of their pedagogical conceptualizations about any subject. for this purpose, data were collected from two teachers with different professional experiences through lesson plans, interviews, and observations. the holistic multiple case study was used because the data collected from all these sources were compared according to professional experience (yin, 2017). this method includes comparing many situations to explain any case and provide insight into that case (creswell & poth, 2007). in the holistic multiple-case design, all cases are discussed as a whole within themselves and then compared. it was considered to reveal certain previously unknown situations related to pck structures for preschool science teaching with such a study. 2.1 participants the participants were selected by the criterion sampling method, one of the purposeful sampling methods (leedy & ormrod, 2001). the criteria were that participants continued their postgraduate education, had different professional experiences, and worked in public kindergartens. different names were used to keep the identity of the participants confidential (patton, 2014). detailed information about the participating teachers is presented below. barbara: barbara is 23 years old. barbara, who completed child development and education in high school, graduated from preschool teaching at a private university with a first. currently, she is actively continuing her postgraduate education in the department of preschool teaching. during her undergraduate education, barbara took a preschool science education course for only one semester concerning the study's subject. barbara has two years of professional experience and is working in a public kindergarten. barbara attended many seminars on preschool teaching during her student and professional life. however, there is no study on science education among these seminars. there is a total of 23 48-month-old children in the classroom of barbara. there is a playing house center, atatürk center, block center, science, and nature center where children can move quickly. many different activities can be performed concerning her classroom's physical conditions. gwen: gwen is 35 years old and the mother of three children. gwen, who states that she preferred this profession because she loved children very much, is a teacher with ten years of experience. after completing her journal of science learning article doi: 10.17509/jsl.v4i2.31599 163 j.sci.learn.2021.4(2).160-172 undergraduate education at a public university, she suspended her education for a while. she is currently continuing her postgraduate education in preschool teaching to improve herself in her field. gwen was not involved in any science education study and did not attend in-service training or seminar during her student and teaching life. during her university education, she took a preschool science education course for only one semester. therefore, she considered herself inadequate in science teaching. gwen, who worked in three different institutions during her ten years of teaching, claimed that physical conditions prevented her, although she attached importance to science activities. there are 25 48-monthold children in the classroom of gwen. for her classroom's physical conditions, it was observed that there was no place where children could easily play and perform their activities, and there were not adequate materials. 2.2 data collection three different data collection tools were used to describe the pck of the participants in-depth. data triangulation was achieved through these different data sources (flick, 2018). these data collection tools are lesson construction task (lct), pck interview protocol, and classroom observation. first, the participants were directed to lct and pck interview protocol. then, teachers' science teaching in the classroom was observed. each data collection tool was directed to teachers once. before administering the interviews and lcts, tools were externally audited by an expert specializing in preschool science teaching (thomas & magilvy, 2011). finally, teachers' science teaching in the classroom was observed. lct in this study, the (core) methodology was used for preschool teachers to create lesson plans for any science subject (loughran et al. 2008). in brief, a core reflects an explicit science teaching content based on the recognition of 'big ideas' for any subject mapped against pedagogical demands. therefore, core was designed to reveal teachers' knowledge of teaching a particular science concept/subject. the questions included in a typical core instrument were reorganized and integrated into interview questions by considering the five components in the pck model of magnusson et al. (1999) (appendix-a). this form has been named lct. teachers who filled out the lct statements were recorded with a voice recorder to obtain the data appropriately. the lesson plan's statements were adapted to the preschool context with the help of researchers who specialized in studies on pck and early science teaching. the final version of the lct consisted of six questions. it took approximately 30-40 minutes for each teacher to respond to the lct. preschool teachers used mone's (2013) preschool education curriculum while preparing lct. for example, barbara, who studies 48month-old children, determined two learning outcomes 'children sort the events in order of the occurrence' and 'children explain the concepts of time'. she chose the formation of night and day topics to reach these learning outcomes. also, she aimed to gain three scientific process skills as observing, classifying and predicting. on the other hand, gwen, who studies 48-month-old children, determined three learning outcomes' children pay attention to object/situation/event', 'children remember what they perceived', and 'children observe objects or entities'. she chose the formation of colors topic to reach these learning outcomes. besides, she aimed to gain two scientific process skills as observing and predicting. pck interview questions the interview protocol used to determine preschool teachers' pcks for science teaching was presented to the participants in two parts. part-i included two main questions and one probe question that help determine the teacher efficacy. part-ii consisted of questions representing the five-component pck structure proposed by magnusson et al. (1999). in these parts, each organized to represent a different pck component, there were 22 questions, including five main questions and 17 probe questions (see appendix-b). the order of five main questions according to the pck components is as follows: ots, ksu, ksc, kisr, kas. the interviews lasted for a total of 50-60 minutes. all interviews were conducted using a voice recorder. classroom observations classroom observations were conducted to obtain more perceptible and traceable knowledge about the pcks of preschool teachers included in the study. this way, it aimed to understand better teachers' processes of realizing this knowledge and the context they taught (park & oliver, 2008; andersson & gullberg, 2014). classroom observations were conducted using the observation protocol developed by newton, driver, and osborne (1999). according to this protocol, classroom observations are generally conducted by following three necessary frameworks. these frameworks are activities involving children (pa), how they are grouped during the activities (pwg), and teacher-children interaction styles (p&ti) (see appendix-c). following the objectives stated here, what teachers and students did during the courses and at which moment (at one-minute time intervals) of the courses they did them were determined. thus, both the practical form of the preschool teacher's pck achieved through the interview protocol and how teachers acted in their classrooms while applying scientific activities were revealed. before the teachings, teachers determined the learning outcomes suitable for early science teaching from the preschool education curriculum of the mone (2013), and they performed teaching according to them. each teacher was observed once. teaching observations were made by considering the teachers' time they were available. to do this, it has been negotiated with teachers, and the journal of science learning article doi: 10.17509/jsl.v4i2.31599 164 j.sci.learn.2021.4(2).160-172 process is planned together. all of the learning outcomes were determined by the participating teachers. for example, gwen conducted her instruction based on the 'children pay attention to object/situation/event', 'children remember what they perceived', and 'children observe objects or entities' learning outcome. her central theme was colored. the instructions lasted for approximately 1315 minutes. in this process, no other detail was mentioned to teachers. the researcher did not interfere with teaching during the observations. during the observations, these lessons were videotaped. in this way, field notes were reflected in the data analysis process. 2.3 data analysis in this study, the content analysis approach was adopted to reveal the uncertain themes and specific themes, as strauss and corbin (1990) suggested. the inductive content analysis method analyzed the participants' responses to the lct and pck interview questions (patton, 2002). in this process, the raw data obtained were first divided into meaningful parts and what each part meant conceptually was determined. secondly, the common aspects between the codings obtained were found and turned into conceptual categories. the coding framework's focus was the five components of the pck (ots, ksu, ksc, kisr, kas) and teacher efficacy for early science teaching. thirdly, the codes and themes obtained were processed. fourthly, the participants' responses to the lct and pck interview questions were comparatively analyzed in-depth under the study's aim by the constant comparative method (kolb, 2012), and the coding key containing analytical evaluation options was created. for example, for ots, which was one of the components of pck, if the teacher conceptualized her instructional goals as merely the transfer of knowledge on a specific topic, it was coded as teacher-centered. on the other hand, if the teacher made life-based conceptualizations instead of transferring knowledge, it was coded as child-centered. after an in-depth analysis, to portray the interactions among teachers' pck components in a quantitative sense, an enumerative approach and pck mapping were employed (park & chen, 2012). based on the assumption that there must be a connection between any two of the identified pck components within the selected teaching episodes, it was counted the number of connections. it was defined the directions of these connections. each connection within teaching episodes was given '1' to indicate its strength. thus, it was created a unit system to quantify teachers' pck's interconnections. after the enumeration process is completed, was drawn pck maps using the pentagon model as an analytic device. the interactions defined within the pentagon model were visualized through the pck mapping (park & chen, 2012). the identified answers of a teacher (gwen) during this process were sent to an expert researcher, and they were subjected to a separate analysis process. the expert independently analyzed these answers to draw a pck map and to make the content analysis. after that, we came together and compared our maps and codes. the intercoder reliability percentage was achieved by comparing the analyses performed at different times and places (kurasaki, 2000). this value, which was 89%, indicated that the analyses were proceeding reliably (miles & huberman, 1994). by coming together with the same expert again, the analyses' differences were discussed, and the remaining data were analyzed individually based on the criteria determined. finally, all analyses were sent to the same expert, opinions were received for external control, and the process was completed. as a result, the final code list was created, and other analyzes were carried out based on this code list (table 1). the second step of data analysis consisted of analyses of classroom observations. in the analysis process performed by using the rubric developed by newton et al. (1999), time sequences were made for each participant's instructional application according to pa, pwg, and p&ti. one more expert also participated in the analyses to ensure the reliability of evaluations. the rubric was first introduced to the relevant expertise, and a framework was created by performing theory-laden negotiations for each activity. here, if the teacher gives direct information about the subject, it has been coded as listening under this pa category. this activity has also been coded under the p&ti category as a teacher explaining science. if the teachers have their students do group work, it has been coded as small group activity under the pwg category. the intercoder reliability rate for these analyses was 92%. 3. results the results obtained by analyzing preschool teachers' responses to the lct and pck interview questions and their in-class teaching observations are presented in this section. first, the inductive content analysis results of teachers' answers to the lct and pck interview protocol questions are presented in table 1 and then in the subtitles. therefore, the teachers' views of teacher efficacy and pck were interpreted through the themes and concepts obtained after the inductive content analysis. then, two teachers' pck maps were created and interpreted. finally, the charts reflecting the time schedules of teachers' instructional practices were presented. 3.1 teacher efficacy at the beginning of the process, the questions in lct and pck interview questions for science teaching in preschool education were asked to determine preschool science teaching's instructional competence. accordingly, barbara referred to utilizing the opportunities in teaching, recognizing changing situations, and child-centered processes. on the other hand, gwen talked about the transfer of knowledge and teacher-centered processes. journal of science learning article doi: 10.17509/jsl.v4i2.31599 165 j.sci.learn.2021.4(2).160-172 both teachers, who had different professional experiences, indicated that the importance was not attached to science teaching courses during their undergraduate and postgraduate education. therefore, they considered themselves inadequate in science teaching. it was also determined that they referred to the curriculum's knowledge by indicating the curriculum's deficiencies. table 1 themes and concepts about teacher efficacy and pck categories themes and concepts barbara gwen teacher efficacy utilizing the opportunities in teaching classroom management enriching teaching utilizing the opportunities in teaching classroom management recognizing changing situations being aware of the differences being able to distinguish the differences raising awareness teacher-centered processes concept teaching direct transfer (knowledge) expository instruction child-centered processes associating knowledge with daily life using information in different activities integrated activities teacher efficacy lack of confidence lack of in-service training curriculum-material inadequacy the misconception of strategy teacher efficacy lack of confidence lack of in-service training curriculum inadequacy ots structuring and transferring information access to information meaningful learning associating with daily life child-centered processes using curriculum materials structuring and transferring information direct concept teaching teacher-centered processes ksu characteristic of the subject context time concept hour, second… characteristic of the subject context primary and intermediate color instructional diversity original activities enrich-integrate teaching child-centered processes cognitive skills concept knowledge teacher-centered processes developmental features cognitive readiness psychomotor development ksc orientations based on learning outcome suitable for age and developmental level insufficient focus of learning outcome orientations based on learning outcome integration with goals presenting different perspectives curriculum competence lack of acquisition knowledge inability to integrate learning outcome curriculum competence towards knowledge of learning outcome putting learning outcomes first kisr child-centered processes providing guidance in teaching guiding the child to teaching meaningful learning strategy variety transition between activities (integration) teacher-centered processes teaching method by experiment rhetorical question and answer (teacher-student) demonstration motivational strategies increasing motivation encouraging participation teacher efficacy classroom management self-criticism in science teaching autonomy in the context of the subject kas alternative assessment and evaluation observing affective features involving the child in the process achievement-oriented assessment and evaluation traditional assessment and evaluation question and answer method knowledge-oriented assessment and evaluation journal of science learning article doi: 10.17509/jsl.v4i2.31599 166 j.sci.learn.2021.4(2).160-172 gwen: i do not conduct science activities by myself in my classroom. every week, a child presents a science activity that he/she conducts with his/her family at home to his/her friends. i want them to know which primary colors merge and which intermediate colors they create (interview). i want them to know which colors will appear when they know the primary colors and the colors are mixed, they would make an effort to create intermediate colors in their activities, and they would keep the color combinations in mind and use them (lct). barbara: first of all, i work with a crowded group. we have 20-21 people. therefore, i implement my activity as a small group. i mean, according to the classroom size, the current situation of the children, or, depending on whether my previous activity was active or passive. i also structure my science activity according to my activity situation (interview). i cannot say that i consider it exactly enough. i cannot say because sometimes i feel that i am inadequate in explaining some concepts. i try to reduce to what children can understand. however, sometimes i have a problem with the concepts, especially in explaining concepts (lct). gwen's conceptualizations, who gave similar responses to the lct and pck interview questions, indicated that she imposed task and responsibility to the child and made knowledge a purpose rather than a tool. on the other hand, it was determined that barbara identified different strategies according to the changing situations in the classroom environment, cared about children being active in the teaching process, and ensured the acquisition of science education by enriching various activities related to the subject. however, she indicated that she sometimes had difficulty reducing some science concepts to a level that children could understand. she stated that she considered herself inadequate in teaching science during those times. 3.2 pck conceptualizations pck conceptualizations of teachers included in the study for preschool science teaching are presented as separate subtitles for each pck component. ots the results related to teachers' instructional goals for including children in science teaching specific to subject matter selected by them were gathered around the concepts of transfer of knowledge and transfer to daily life. nevertheless, it was observed that teachers generally knew the curriculum. however, they gave conflicting responses between what they knew and what they planned to do. on the other hand, while barbara referred to child-centered processes, gwen mostly referred to teacher-centered processes. gwen: because the subject of primary colors and intermediate colors is suitable for experimenting, for example, let us say i will teach a value, i will teach love, i cannot do it by experiment, but it is appropriate to teach the intermediate colors and primary colors by experimenting (interview). there are achievement and indicators in which they need to learn primary colors, there is a plan we should provide the acquisition of which, and if it is written in the plan that i should teach them, i apply them through experiment, which may contribute to them in primary school (lct). barbara: as i said in previous questions, i think that children should know that it is night or day, in order words, in order to understand and make sense of time-related concepts and to adapt themselves to daily life accordingly (interview). we, preschool teachers, have annual plans. accordingly, i arrange activities by myself and include them in my activity plan. i mean, i act according to my monthly plan (lct). gwen's responses specific to primary and intermediate color formation subject matter indicated that her instructional goal for preschool science teaching was to acquire achievements. however, although she had general knowledge about the curriculum, her responses revealed that she tended to use the subject as a purpose. furthermore, she stated that she would take an active role in the experiment during teaching indicated that she had a teacher-centered orientation. unlike gwen, barbara aimed to make knowledge quickly understood and impose it daily to guide children to education. nevertheless, it was observed that she referred to the interpretation of knowledge, knowledge of curriculum, and the use of different strategies, which also indicated that she had a child-centered orientation. ksu it is observed that teachers referred to cognitive maturity in this pck component, which includes their knowledge about children's learning needs for the subject matter they selected and how children had learning difficulties. in this process, it was revealed that gwen used traditional approaches, and barbara mentioned instructional diversity by thinking based on innovation. gwen: children should know the colors; they should know the primary colors before they can learn intermediate colors; they should be able to express themselves and have sufficient cognitive skills; for example, a child with special needs has difficulty in comprehending this experiment (interview). they may have difficulty using the materials while experimenting, and they have difficulty learning the colors, so the activity does not take place as desired (lct). barbara: they have to reach adequate maturity cognitively. it will not be adequate if i teach verbally. it does not provide permanence for the child as well. therefore, i will include the children in the process as far as possible, and i also think it would be more effective if i pay attention to using different sources (interview). it is enjoyable to integrate it with other activities. i use it myself as a science activity and as a turkish, language, drama, and art activity. i can conduct my activities by integrating based on day and night formation this year (lct). in sample quotations related to ksu, it was observed that gwen primarily focused on ensuring the readiness of children. gwen also thought that she could attract children's attention through experiments during the application. in this way, children could quickly obtain the knowledge, which indicated that gwen never lost her teacher-centered orientation, although she referred to journal of science learning article doi: 10.17509/jsl.v4i2.31599 167 j.sci.learn.2021.4(2).160-172 children's cognitive readiness. barbara, who referred to similar concepts with gwen, mentioned a teaching process enriched with different subject matters. ksc this component represents teacher knowledge about the curriculum specific to the subject matter selected by the participants. it was revealed that the teachers included in the study internalized the curriculum expectations, which enabled them to have achievement-based orientations. gwen: yes, i think that the achievement related to this activity is given from every matter. there is an indicator in the cognitive or motor area, and i think self-care skills are included sufficiently in the social field. focusing attention, self-expression, using small muscles, the ability to pour water from glass to glass, self-care skills come to my mind (interview). i get help from the internet in this regard. the practice of teaching science education regularly is also not right, yes, i applied it first to raise awareness, but then i gave parents and students responsibility. because it is tough for me to find, research and apply another science activity every week. there are also other activities we need to do, and it is not easy to spare time to investigate (lct). barbara: we can see in that list of achievements and indicators in our national education curriculum, the acquisitions that i have to teach are specific in the process. i organize my activities in a way that children can understand according to those achievements (interview). we have an achievement indicating that students will be able to sort the events in order of occurrence and explain time concepts. i can also find it from there. so, i think it is enough. the achievements that preschool children can understand are included (lct). gwen's responses related to ksc indicated that she had known about the curriculum and conceptualized the achievements and indicators. although she stated that the curriculum was a guide for teachers, she indicated that she mostly resorted to the internet, not to the curriculum, to save time and overcome her shortcomings in the subject matter. therefore, it was revealed that gwen had a general knowledge of the curriculum. however, she contradicted the responses she gave in some situations. she referred to teacher-centered processes since she applied the method of teaching by imposing the duty and responsibility to the child in the process. on the other hand, barbara could not make sufficient conceptualizations in terms of the curriculum. kisr two significant results can be mentioned in this component, which shows how teachers use their knowledge of instructional strategies and representations. the first is that teachers agreed that children's intrinsic motivation should be high and that their feelings of curiosity should be triggered before teaching. another result is that gwen referred to teacher-centered strategies while barbara referred to child-centered strategies. gwen: when i conduct it, i first start with a story or a play. i conduct integrated activities. teachers have many shortcomings related to science education and teaching. i also consider myself inadequate, which results from the fact that science activities were never given importance, including universities (interview). barbara: as i said, the group i address here is essential. large group or small group?. accordingly, i organize my activity process. the size of the model i will use is also important to me. if i address a large group, i make the model big so that all children can see the fields of view and participate. alternatively, if it is a small group, i prepare something more moderate so that they can see. i make such a preparation (interview). as i said before, i prepare in advance. i do not conduct the activity suddenly. by moderating in advance, i mean a passive then a dynamic. i try to attract attention. i can use visuals. alternatively, as i said, it can be a model (lct). gwen stated that she would include children in integrated science education with different activities specific to the subject matter she selected. however, by making a self-criticism, she argued that the vital importance was not given to science teaching during her undergraduate and postgraduate education. she mentioned that this situation made her feel inadequate, and she referred to teacher competence. on the other hand, barbara focused on using concrete materials for qualified teaching that may attract the attention of children. she also suggested that integrating science teaching activities with other activities would lead to more effective learning. therefore, barbara referred to child-centered processes, increasing intrinsic motivation in teaching, and using different teaching strategies. kas this component represents the measurement and evaluation approaches that the participants consider to include in their process specific to the subject matter they select. here, it was observed that gwen referred to traditional measurement and evaluation techniques while barbara referred to alternative measurement and evaluation techniques. gwen: questions and answers can be applied, observation, i mean we do not hold an examination. the best measurement and evaluation technique of a teacher in science teaching is observation and question and answer (interview). with the questions, i examine the activities they conduct. if i have not created any awareness possible, i may fail. i try to teach primary colors in another activity (lct). barbara: the main thing is to be able to teach the achievement. to ensure that they can sort the events in order of occurrence. i call children one by one, and i make them use that model. i expect them to act correctly to see whether they can remember and do again what they have learned and heard. i certainly try to make them do it (interview). i try to focus their attention by chatting, by giving questions in a more fun way rather than by compelling their knowledge (lct). according to sample explanations related to kas, gwen indicated that she would perform measurement and evaluation through question and answer and referred to the transfer of knowledge to daily life. on the other hand, barbara paid attention to children's feelings and thoughts by including them in the process and referred to childjournal of science learning article doi: 10.17509/jsl.v4i2.31599 168 j.sci.learn.2021.4(2).160-172 centered processes. she also tended to consider knowledge as a means to reach achievements. 3.3 interaction between pck components in this section, numbers of teaching episodes and dyad connections among components were identified for each teacher firstly were presented (table 2). then, pck maps of preschool teachers who participated in the study are comparatively summarized in figure 1. at first glance, it is observed that barbara's dyad connections between pck components were higher than those of gwen. accordingly, barbara's essential components that interacted with other pck components were kisr, ksu, and ots, respectively. furthermore, the interaction between ksu-kisr components was potent. there was no connection between ksu-ksc. this result indicated that barbara could not complete the pentagon model and was lacking, especially in ksc. gwen's most crucial component, a more experienced teacher than barbara that interacted with other pck components, was ots. ots-ksu and ots-kisr interaction appeared to be stronger than others. furthermore, unlike barbara, gwen completed the pentagon model and interacted with all components with each other. teachers' pck maps also confirm some of the distinctive features of pck. accordingly, the integration of components is unique and subject-specific (e.g., park & suh, 2015; smith & banilower, 2015). for instance, among the teachers who tended towards different matters, gwen interacted with each pck component. barbara tended to integrate strategy and student insights intensively. ksc was the least interactive component compared to other components. however, it can be said that gwen interacted with ksc more than barbara. 3.4 practical change of instructional content the analysis of the activities based on science teaching performed by preschool teachers in their classes was determined according to the activities within the framework type of activity, (pa) student and teacher interactions (p&ti), and student study group (pwg). the results were detailed with the help of figure 2. furthermore, observation notes on science teaching conducted by teachers in their classes were also shared. gwen: the activity was started by asking, shall we experiment with you today?. by showing the finger paints on the table, their names were asked one by one. after asking if it is colored by showing the water in the glasses, let us color this water by taking the blue finger paint with the help of a spoon, putting it in the water, and starting mixing. by stating that the glass next to the blue will remain colorless, it was told let us put the red color in the third glass next to it who wants to do it, and a child assigned randomly among the volunteers was given the task of mixing. by repeating the same process once more, yellow paint was put in the last glass. then, pieces of napkin were placed between the glasses, and it was waited for a while for the colors to blend into the napkin. after the colors began to blend on the napkin, it was told that the colors in napkins would be intertwined, and "what will happen after the colors are intertwined?" was asked, and after children responded colored napkin, "would you like that the name of our experiment is a colored napkin?" was asked. after receiving the response yes, children were called from the chairs they were sitting around the table where the experiment was applied. the colors formed in the middle of napkins and how they were formed and primary and intermediate color formation were discussed. figure 1 preschool teachers' pck map 1-8% 9-16% 17-24% 24% < 19/68 (28%) barbara gwen ots 26/110 (24%) ots 9/68 (13%) ksu 28/110 (26%) ksu 13/68 (19%) ksc 8/110 (7%) ksc 12/68 (18%) kisr 30/110 (27%) kisr 15/68 (22%) kas 18/110 (16%) kas ots kisr kasksc ksu 14 (25%) 2 (4%) 3 (5%) 26 30 8 (15%) 9 (16%) 6 (11%) 6 (11%) 7 (13%) 188 28 ots kisr kasksc ksu 1 (3%) 2 (6%) 1 (3%) 19 15 6 (18%) 8 (24%) 4 (12%) 5 (15%) 4 (12%) 129 13 1 (3%) 2 (6%) table 2 teaching episodes and connections episodes dyad connections among components barbara 19 55 gwen 12 34 journal of science learning article doi: 10.17509/jsl.v4i2.31599 169 j.sci.learn.2021.4(2).160-172 barbara: an intriguing introduction was made. first, children's estimates about what the model could be were obtained, children were directed to questioning by asking if they had previously seen the model figures. then, in line with the responses obtained, the subject was introduced by enabling children to establish a relationship between the model and the subject. it was talked about the earth and the sun. it was continued to inquire on the formation of water, day and night. then, during this inquiry, the explanation verbally expressed through the model was carried out concretely while explaining that day and night occur as the earth rotates around the sun. after explaining the formation of day and night, the children were asked questions linking the subject with everyday life to ensure permanent learning. after the questions of 'what do we do at night? when do we go to school?' were responded by thinking, each child was allowed to examine the model individually. after each child made the formation day and night through the model, the role distribution was made among the children determined in turn by the teacher. while one child was sun, the other child was asked to animate the earth. under the teacher's guidance, the earth revolved around the sun while the sun remained constant. thus, they knew how the day and night formation took place by using different methods. children remembered which friends were animating the sun and earth by discussing among themselves. they shared their ideas about why the sun does not revolve around the earth and how they will live every day or night by communicating. the teacher answered the questions and helped them think and question the activity by including their conversations. as shown in observation notes and figure 2, while participants were applying the lesson plans they created, they generally tended to include the children in individual and large groups in the teacher's teaching and questionanswer processes. furthermore, barbara conducted less structured activities. on the other hand, it is observed that barbara adopted a child-centered approach by including children in group discussion, a discourse-based dialogical form of interaction. therefore, it can be said that teachers were more active, and children were in a listener position in the science teaching process as experience increased. this result indicates that preschool teachers' theoretical conceptualizations about science teaching can also be confirmed in practice. figure 2 classroom observations 0 2 4 6 8 10 12 14 gwen barbara time t e a c h e r pa closed practical task observing demonstration preparing or clearing away group discussion 0 2 4 6 8 10 12 14 gwen barbara time t e a c h e r p&ti teacher giving instructions teacher explaining science question–answer interactions 0 2 4 6 8 10 12 14 gwen barbara time t e a c h e r pwg whole class activity small group activity individual activity journal of science learning article doi: 10.17509/jsl.v4i2.31599 170 j.sci.learn.2021.4(2).160-172 4. discussion this study aimed to explore the pck of two preschool teachers with different professional experiences in science teaching. the analyses performed on qualitative data obtained through lesson plans, pck interviews, and observations revealed some direct results specific to science teaching in preschool education. first, the knowledge and understanding of preschool teachers about science teaching were limited. this result confirms the results obtained in many studies (e.g., garbett, 2003; andersson & gullberg, 2014; furman et al., 2019). teachers argued that the lack of materials (greenfield et al., 2009) and instructional competence (saçkes, 2014) affected their teaching. secondly, gwen, who is a more experienced teacher, had a more teacher-centered orientation. this orientation affected her strategy and measurement and evaluation processes theoretically and practically (henze, van driel, & verloop, 2008; käpylä, heikkenen, & asunta 2009). children's learning in the preschool education processes should be mainly performed in play-based situations. here, preschool teachers' primary role is to create child-based science experiences for children (mccray & chen 2012). the primary expectation is that the experienced preschool teacher is better equipped in playbased pedagogical strategies than a less experienced teacher (andersson & gullberg, 2014). however, gwen's personal, educational trajectories, beliefs in science, difficulties in science content, and personal starting points about teaching experiences may have differentiated her pck for science teaching (arias, davis, marino, kademian, & palincsar, 2016). a teacher's pck for science teaching tends to be affected by variables such as contextual, cultural, and social limitations in the learning environment. furthermore, teachers learn to teach by being affected by the culture where they grew up (park & oliver, 2008). luft and zhang (2014) argued that this change took place mainly in the first three years. when teachers face real difficulties during the practice, they usually abandon new practices and return to their teachers' teaching methods. pck conceptualizations and teaching practices of barbara and gwen confirm this argument. barbara, who had a child-centered orientation, performed her teaching practices by following this orientation. however, ots-kisr interaction on barbara's pck map was not very strong. on the other hand, gwen also reflected her teacher-centered orientation to the selection of strategy. unlike barbara, this situation was reflected in the ots-kisr interactions in the pck map with the effect of gwen's experience. thirdly, although barbara tended to use more childcentered orientation and strategies than gwen, she was inadequate in contextualizing science teaching with the preschool education curriculum. this result confirms that professional experience helps to integrate the knowledge of the curriculum with other pck components. when teachers' pck maps were compared, gwen completed the pentagon model. barbara could not complete the connection between ksc and ksu. therefore, when it is considered that the quality of pck depends on consistency between components and power of individual components, it can be said that gwen's quality of pck was better (park & chen, 2012). furthermore, this result is consistent with the findings of studies conducted in other contexts (friedrichsen et al., 2009; kleickmann et al., 2013). the knowledge of curriculum that shulman (1987) excluded from pck was considered as a component that complemented the pck by grossman (1990) and magnusson et al. (1999). when it is considered in terms of the preschool education context, the teacher's main role is not to give the scientific concepts directly to the children but to give them scientific process skills with the help of achievement-based and play-based activities. therefore, knowledge of curriculum has extra importance in preschool education (cairney, 2002). on the other hand, faulkner-schneider (2005) emphasized that preschool teachers should not consider science an addition or a separate part of the early childhood program and should integrate science activities into a natural play-based curriculum. however, teachers' knowledge of the curriculum had the most limited connection with other components. this result indicates that preschool teachers could not adequately contextualize their knowledge of the curriculum regarding science teaching. teachers were able to say the mone (2013) preschool curriculum achievements when they were asked. however, they indicated that they had difficulty associating these achievements with appropriate science concepts, which confirms that teacher efficacy is an influential member of pck (park & oliver, 2008). finally, although two teachers' theoretical and practical conceptualizations about measurement and evaluation in preschool science teaching were generally limited, they tended to be affected by their orientation (park & chen, 2012). assessment in preschool education should be performed through individualized activities to support the functioning, learning, and thinking of children in cognitive, social, physical, and emotional development (brenneman, 2011). the teacher's role here is to include the child in the negotiation processes with a child-centered orientation. to overcome this difficulty requires teachers to understand child development and the expected learning sequences in multiple areas (gelman, brenneman, macdonald, & román, 2009). this study indicated that two teachers also tended towards traditional measurement and evaluation techniques and did not know complementary measurement and evaluation techniques. they could not go beyond asking questions during their practices. journal of science learning article doi: 10.17509/jsl.v4i2.31599 171 j.sci.learn.2021.4(2).160-172 5. conclusion results revealed that teachers did not have sufficient knowledge about science teaching and that their pck tended to change according to professional experience. it was determined that experienced teachers had more teacher-centered orientations than less experienced teachers who had more student-centered orientations. moreover, it was observed that professional experience increased the relationship between the pck components. in brief, the results obtained are similar to some results of the studies in similar or different contexts in the literature but different from some other results, which reveals that science teaching within preschool teacher education should be re-evaluated with its unique characteristics. this will only be possible by discussing teachers' pck, which is the implicit form of professional knowledge, from different aspects and specific to preschool education. therefore, the following suggestions can be made especially in terms of contributing to preschool education literature: (a) attention can be focused on practical applications for preschool science teaching in the preservice education process. (b) preservice science teaching courses can be conducted by contextualizing them with the curriculum. (c) new studies can be designed to determine the limits of teachers' pck for inquiry-based science activities in the preservice and in-service preschool teaching processes. (d) the number of studies to explore the individual components of the pck of teachers within preschool science education can be increased. through these suggestions, richer reasoning and future projections related to preschool teachers' pck structures can be developed. references abell, s. k. (2008). twenty years later: does pedagogical content knowledge remain a useful idea?. international journal of science education, 30(10), 1405-1416. arias, a. m., davis, e. a., marino, j. c., kademian, s. m., & palincsar, a. s. (2016). teachers' use of educative curriculum materials to engage students in science practices. international journal of science education, 38(9), 1504-1526. andersson, k., & gullberg, a. (2014). what is science in preschool and what do teachers have to know to empower children?. cultural studies of science education, 9(2), 275-296. blömeke, s., jenßen, l., grassmann, m., dunekacke, s., & wedekind, h. (2017). process mediates structure: the relation between preschool teacher education and preschool teachers' knowledge. journal of educational psychology, 109(3), 338. brenneman, k. (2011). assessment for preschool science learning and learning environments. early childhood research & practice, 13(1), 1-9. bustamante, a. s., white, l. j., & greenfield, d. b. (2018). approaches to learning and science education in head start: examining bidirectionality. early childhood research quarterly, 44, 34-42. cairney, t. (2002) bridging home and school literacy: in search of transformative approaches to curriculum. early child development and care, 172(2), 153–172. cowie, b., & otrel-cass, k. (2011). exploring the value of 'horizontal learning in early years science classrooms. early years, 31(3), 285295. creswell, j. w., & poth, c. n. (2007). qualitative inquiry and research method: choosing among five approaches. thousand oaks, ca: sage de jong, o. (2003). exploring science teachers' pedagogical content knowledge. science education research in the knowledge-based society (373-381). dordrecht, the netherlands: kluwer academic publishers. dejonckheere, p. j., de wit, n., van de keere, k., & vervaet, s. (2016). exploring the classroom: teaching science in early childhood. european journal of educational research, 5(3), 149-164. eliason, c., & jenkins, l. (2003). a practical guide to early childhood curriculum. prentice-hall upper: saddle river. faulkner-schneider, l. a. (2005). child care teachers' attitudes, beliefs, and knowledge regarding science and the impact on early childhood learning opportunities (doctoral dissertation), oklahoma state university, oklahoma. flick, u. (2018). triangulation in data collection. the sage handbook of qualitative data collection. london, uk: sage publications. friedrichsen, p. j., abell, s. k., pareja, e. m., brown, p. l., lankford, d. m., & volkmann, m. j. (2009). does teaching experience matter? examining biology teachers' prior knowledge for teaching in an alternative certification program. journal of research in science teaching, 46(4), 357-383. furman, m., luzuriaga, m., taylor, i., jarvis, d., dominguez prost, e., & podestá, m. e. (2019). the use of questions in early years science: a case study in argentine preschools. international journal of early years education, 27(3), 271-286. garbett, d. (2003). science education in early childhood teacher education: putting forward a case to enhance student teachers' confidence and competence. research in science education, 33(4), 467481. gelman, r., brenneman, k., macdonald, g., & román, m. (2009). preschool pathways to science (preps [tm]): facilitating scientific ways of thinking, talking, doing, and understanding. baltimore: brookes publishing company. gerde, h. k., pierce, s. j., lee, k., & van egeren, l. a. (2018). early childhood educators' self-efficacy in science, math, and literacy instruction and science practice in the classroom. early education and development, 29(1), 70-90. greenfield, d. b., jirout, j., dominguez, x., greenberg, a., maier, m., & fuccillo, j. (2009). science in the preschool classroom: a programmatic research agenda to improve science readiness. early education and development, 20(2), 238-264. grossman, p. l. (1990). the making of a teacher: teacher knowledge and teacher education. teachers college press, teachers college, columbia university. henze, i., van driel, j. h., & verloop, n. (2008). development of experienced science teachers' pedagogical content knowledge of models of the solar system and the universe. international journal of science education, 30(10), 1321-1342. kallery, m., & psillos, d. (2001). pre-school teachers' content knowledge in science: their understanding of elementary science concepts and of issues raised by children's questions. international journal of early years education, 9(3), 165-79. käpylä, m., heikkinen, j. p., & asunta, t. (2009). influence of content knowledge on pedagogical content knowledge: the case of teaching photosynthesis and plant growth. international journal of science education, 31(10), 1395-1415. kind, v. (2009). pedagogical content knowledge in science education: perspectives and potential for progress. studies in science education, 45(2), 169-204. kleickmann, t., richter, d., kunter, m., elsner, j., besser, m., krauss, s., ve baumert, j. (2013). teachers' content knowledge and pedagogical content knowledge: the role of structural differences in teacher education. journal of teacher education, 64(1), 90-106. kolb, s. m. (2012). grounded theory and the constant comparative method: valid research strategies for educators. journal of emerging trends in educational research and policy studies, 3(1), 83-86. journal of science learning article doi: 10.17509/jsl.v4i2.31599 172 j.sci.learn.2021.4(2).160-172 kurasaki, k. s. (2000). intercoder reliability for validating conclusions drawn from open-ended interview data. field methods, 12(3), 179194. kurniah, n., andreswari, d., & kusumah, r. g. t. (2019, april). achievement of development on early childhood based on national education standard. in international conference on educational sciences and teacher profession (icetep 2018). atlantis press. larimore, r. a. (2020). preschool science education: a vision for the future. early childhood education journal, 48, 703–714. leedy, p. d., ve ormrod, j. e. (2001). practical research: planning and design, merrill prentice hall. new jersey. leon, k. (2014). do you see what i see? exploring preschool teachers' science practices (doctoral dissertation), california state university, northridge. loughran, j., mulhall, p., & berry, a. (2008). exploring pedagogical content knowledge in science teacher education. international journal of science education, 30(10), 1301-1320. luft, j. a., & zhang, c. (2014). the pedagogical content knowledge and beliefs of newly hired secondary science teachers: the first three years. educación química, 25(3), 325-331. magnusson, s., krajcik, j., borko, h. (1999). nature, sources and development of pedagogical content knowledge for science teaching (95–132). dordrecht, the netherlands: kluwer academic publishers. mclachlan, c., fleer, m., & edwards, s. (2018). early childhood curriculum: planning, assessment and implementation. cambridge university press. mclean, k., jones, m., & schaper, c. (2015). children's literature as an invitation to science inquiry in early childhood education. australasian journal of early childhood, 40(4), 49-56. mccray, j. s., & chen, j. q. (2012). pedagogical content knowledge for preschool mathematics: construct validity of a new teacher interview. journal of research in childhood education, 26(3), 291-307. miles, m. b., & huberman, a. m. (1994). qualitative data analysis: an expanded sourcebook. sage. mone. (2013). republic of turkey, general directorate of preschool education, preschool education curriculum for 36-72 months old children. ministry of national education publishing, ankara. national research council. (2013). a framework for k-12 science education: practices, crosscutting concepts, and core ideas. washington: the national academies press. neuman, s. b., & danielson, k. (2020). enacting content-rich curriculum in early childhood: the role of teacher knowledge and pedagogy. early education and development, 1-16. https://doi.org/10.1080/10409289.2020.1753463 newton, p., driver, r., & osborne, j. (1999). the place of argumentation in the pedagogy of school science. international journal of science education, 21(5), 553-576. nilsson, p. (2013). what do we know and where do we go? formative assessment in developing student teachers' professional learning of teaching science. teachers and teaching, 19(2), 188-201. nilsson, p., & vikström, a. (2015). making pck explicit—capturing science teachers' pedagogical content knowledge (pck) in the science classroom. international journal of science education, 37(17), 2836-2857. park, s., & chen, y. c. (2012). mapping out the integration of the components of pedagogical content knowledge (pck): examples from high school biology classrooms. journal of research in science teaching, 49(7), 922-941. park, j., & oliver, j. s. (2008). national board certification (nbc) as a catalyst for teachers' learning about teaching: the effects of the nbc process on candidate teachers' pck development. journal of research in science teaching, 45(7), 812-834. park, s., & suh, j. k. (2015). from portraying toward assessing pck: drivers, dilemmas, and directions for future research. in reexamining pedagogical content knowledge in science education (pp. 114-129). routledge. patton, m. q. (2002). two decades of developments in qualitative inquiry: a personal, experiential perspective. qualitative social work, 1(3), 261-283. patton, m. q. (2014). qualitative research & evaluation methods: integrating theory and practice. sage publications. ravanis, k. (2017). early childhood science education: state of the art and perspectives. journal of baltic science education, 16(3), 284-288. saçkes, m. (2014). how often do early childhood teachers teach science concepts? determinants of the frequency of science teaching in kindergarten. european early childhood education research journal, 22(2), 169-184. shulman, l. (1987). knowledge and teaching: foundations of the new reform. harvard educational review, 57(1), 1-23. smith, p. s., & banilower, e. r. (2015). assessing pck: a new application of the uncertainty principle. in re-examining pedagogical content knowledge in science education (pp. 98-113). routledge. strauss, a., & corbin, j. (1990). basics of qualitative research. sage publications. thomas, e., & magilvy, j. k. (2011). qualitative rigor or research validity in qualitative research. journal for specialists in pediatric nursing, 16(2), 151-155. thulin, s., & redfors, a. (2017). student preschool teachers' experiences of science and its role in preschool. early childhood education journal, 45(4), 509-520. tu, t. (2006). preschool science environment: what is available in a preschool classroom?. early childhood education journal, 33(4), 245251. yin, r. k. (2017). case study research and applications: design and methods. sage publication. a © 2022 indonesian society for science educator 28 j.sci.learn.2022.5(1).28-41 received: 13 february 2021 revised: 21 may 2021 published: 1 march 2022 determining the mental images of fourth grade private and public school students for science learning environments by drawing technique sina şavlı1*, mustafa doğru2 1milli eğitim bakanlığı, turkey 2fen bilimleri eğitimi bölümü, eğitim fakültesi, akdeniz üniversitesi, turkey *corresponding author. syna.1@hotmail.com abstract the study aims to examine fourth-grade private and public-school students’ images of the science learning environment using their drawings. the survey was conducted in the 2017-2018 academic year, and a descriptive survey model was used. participant group of this study consist of 357 fourth-grade students. in this study, data were collected by drawing technique. content analysis, percentage, frequency, and chi-square test of independence were used in data analysis. regarding the study results, although the learning environment in both schools consists of traditional classrooms and laboratories, differences were observed in student drawings according to school type (public-private). the private school students use the laboratory in science lessons, whereas the students in the public school use the teacher's desk as the experiment table. another significant result of the study is that public-school students' priority is understanding the topic. in contrast, private school students assign more importance to the materials. regarding the independence test results, founding student behavior, teacher behavior, teacher position, and teaching method to differ in student images according to school type. but no found a significant difference between the place theme images of public school and private school students. keywords science, science learning environment, drawing, image 1. introduction science is the process of understanding the nature of science, thinking, and discovering new scientific knowledge (derman, 2019). therefore, scientific literacy should increase to develop these characteristics in individuals. many countries are making regulations in their science education programs to increase scientific and technological literacy (hastürk & sönmez, 2020). it is known that countries that attach importance to science and technology aim to carry out a qualified education to be at the top (yasin, prima, & sholihin, 2018). therefore, learning environments should be organized by the skills of the individuals, including researching, questioning, producing, and scientific process skills to achieve a qualified science education (candaş, kiryak, & ünal, 2021). it is also known in the literature that organizing the learning environments impacts student achievement (korkman & metin, 2021; salur & pehlivan, 2021). learning environments should be student-centered to discuss the topics, present their ideas comfortably and show their thoughts in practice in the process (ulu & ocak, 2018). besides, it has been found that learning environments that put students in the center increase academic achievement, allow them to develop a positive attitude towards science learning and improve the image, and make the learning permanent (karadeniz & doymuş, 2015). images are the schemes that occur in the mind about information, concept, or phenomenon. on the other hand, individual images are individual schemas about a subject, concept, or phenomenon, different from each other. these differences vary according to the individuals' lifestyle, prior knowledge, and interests (ergen, boyraz, batmaz, & çevik kansu, 2020). it is known from the literature that the science image that individuals carry in their minds is essential in science learning (bilir et al., 2020; dönmez, 2017). the ways students learn science style their images of the science learning environment. besides, students’ science learning environment images can affect their interests, attitudes, thoughts, and motivations towards learning (gökdaş & ak, journal of science learning article doi: 10.17509/jsl.v5i1.32116 29 j.sci.learn.2022.5(1).28-41 2019). for this reason, the image of science learning environments that individuals have should be known. it is thought that learning the mental images of individuals about a subject will be essential to reveal the strengths and weaknesses of that subject. for example, students' image of science plays an essential role in their attitude, thinking, motivation, and interest in science. therefore, it is also very effective in learning science (digilli baran & karaçam, 2020). they are examining mental images in the literature about the science course from many perspectives. for example, studies are available to investigate the lessons taught by teaching methods and techniques on students’ mental images (çavdar & doymuş, 2016; develi, 2017; parsa, 2016; zheng & spires, 2014). these studies' common point is to determine teaching methods and techniques on students’ mental images. in addition to these studies, studies examine the mental images brought by the individual from the past that is available. images about ideal learning environment (s. özdemir & akkaya, 2013), there is the investigation to compare the images for science learning environments according to education systems of countries (turkmen & unver, 2018). in addition to current investigations, (şahin akyüz, 2016) compared the actual science learning environments with the ideal learning environments according to the school's quality. as a result, it has been determined that students’ about the ideal science learning environments include intelligent boards, experiment tables, and visuals in the mental images. this result revealed how the science learning environment images of the students studying in the private school are. therefore, this study aims to determine whether there is a difference between the student images of the private school science environment and the public school science learning environment. based on this information, it is thought that this study will be essential to investigate the quality of the science learning environment in students’ science learning process. based on the literature's information, investigating the importance of the science learning environment's qualities is necessary for the students' science learning processes. in this context, this study's results will contribute to the institutions and organizations involved in regulating learning environments and researchers who want to work on this issue. furthermore, the study examines fourth-grade private and public-school students' images of science learning environments using their drawings. significant differences, if any, between students' images of science learning environment according to the type of their school (private or public) will also be revealed. in this context, the study's main question is " how is the image of the science learning environment of fourth-grade private and public school students?". the following problems have been addressed as the study's sub-questions: (1) how is the image of the science learning environment of fourth-grade publicschool students? (2) how is the image of the science learning environment of fourth-grade private-school students? (3) do students' images of the science learning environment differ significantly according to the type of their school (public-private)? 2. method 2.1 research model the survey model, one of the descriptive research methods, was preferred. the descriptive method is generally used to clarify a situation, an event, make evaluations within the framework of specific rules, and reveal the relationships between events. the primary purpose of a descriptive study is to provide a deep understanding of the case under investigation and explain it. in a descriptive study, the research field is directly present and evaluated (acar, 2000, p. 30; as cited in gay, 1996 ). on the other hand, the survey model serves to reveal the current situation. it deals with the current state of the situation or event under investigation and the position related to the problem (çepni, 2014). since the students' existing images will be revealed in this study, the survey model of the descriptive research method was preferred. 2.2 study group in this study, a simple random sampling method was used. the sample of the study consists of private and public school students studying in fourth grade. science course starts from the third grade in the turkish education system. students studying in the fourth grade have two school years of science learning backgrounds. therefore fourth-grade students were included in this study. one school from each of the konyaaltı, kepez, and muratpaşa districts, the three central districts of antalya, was included. still, permission was not granted from the schools in the muratpaşa district. therefore, the number of schools from kepez and konyaaltı districts willing to participate in the study increased. therefore, the study was conducted with three schools from each of the two districts. the drawing method, a data collection tool, is applied as a questionnaire, which allowed the number of participants to be 357. compared to other data collection methods (interview, observation), the mentioned questionnaire can be applied quickly to huge groups from different regions at a meager cost (büyüköztürk, kılıç çakmak, akgün, karadeniz, & demirel, 2018). so, the number of participants was kept high to get healthier and more reliable results in revealing students' images of the science learning environment. 2.3 data collection tool "actual science learning environment drawing test" (şahin akyüz, 2016) was used as the data collection tool. obtained the necessary permissions for the drawing test were from the researchers who developed the test. the test used consists of two sections. in the first part of the test, students are asked to draw following the instructions and journal of science learning article doi: 10.17509/jsl.v5i1.32116 30 j.sci.learn.2022.5(1).28-41 answer four open-ended questions in the second section of the test. there are 17 items about students’ learning preferences. in this study, students’ learning preferences will not determine the second part of the test has not been using. "actual science learning environment drawing test" administered on 8th graders by the developers overlap with fourth-grade skills, the validity and reliability measurements of the test did not perform again. as a result of the calculation, the reliability of the research was found to be 89%. since the reliability values above 70% are considered reliable (miles & huberman, 1994), the result obtained here indicates the research's reliability. the science course begins from the third grade in the turkish education system. students studying in the fourth grade have two school years of science learning backgrounds. therefore fourthgrade students were included in the study. various methods, including drawings, word associations, analogies, and metaphors, can reveal students' mental images (dikmenli, 2010). collecting data through drawings allows us to analyze social, emotional, cognitive, and motivational dimensions and evaluate them together. therefore, drawings are a type that can be preferred as a data collection tool (kıryak, candaş, karanisanoğlu, & özmen, 2019). in interviews and surveys, children may not feel comfortable due to their developmental characteristics and may fail to express themselves correctly. therefore, the drawing method is a better method for collecting data from children to identify the images' attitudes, interests, and beliefs (armstrong, 2007). in this study, since it was a desire to collect data about the images of 9-10-year-old children, preferred the drawing method was. 2.4 data collection process the study's data collection process took place in mayjune of the 2017-2018 academic year. in the process, participants filled the actual science learning environment drawing test with their drawings and answered as “what are you doing yourself in this drawing? explain” “what is your science teacher doing in this drawing? explain” “what are your friends doing in this drawing? explain” “what are the most important things for you in your science learning environment?” four questions just below the drawing area prepared for the researcher to better understand the drawing. 2.5 data analysis the content analysis method for the analysis was used of the data. content analysis is a method that allows working indirectly to determine human nature and behavior; it is a repeatable and systematic method in which some parts of an entity are divided into smaller units and summarized by coding according to specific rules. the primary purpose of content analysis is to explain the collected data. for this purpose, similar information is a group under certain concepts and themes (yıldırım & şimşek, 2018) regarding the above information, and it is seen that the most appropriate method to be used in analyzing the data collected in this study is content analysis. the chi-square test has determined a significant difference between the mental images of the science learning environment between private and public school students. therefore spss 21 package program was used in the analysis. the drawing tests of the students and the analysis of four open-ended questions have been carried out together with an expert in science education. 3. result and discussion this section examined whether there is a significant difference between the images of the fourth-grade students in public and private schools regarding the science learning environment and the images of the public and private school students studying in the fourth grade regarding the science learning environment. the resulting results are discussed. table 1 presents the findings of the 4th grade public and private school students about place sub-themes. table 1, the drawings of almost all fourth-grade public school students (96.3%) fall under the formal sub-theme in terms of the place. a very small portion of the students (2.8%) drew an informal environment. almost all fourthgrade private school students (93.7%) drew the formal environment for place sub-theme in private school. only one (0.7) of the students drew informal. table 2 presents the findings of the chi-square test of independence related to fourth-grade private and public-school students’ image of a place. table 1 findings of primary school fourth-grade students studying at public and private schools regarding the 'place' sub-theme theme sub-theme f % public place formal 206 96.3 informal 6 2.8 private place formal 134 93.7 informal 1 0.7 table 2 findings of the chi-square test of independence related to 'place' theme images of private and public school students studying in the fourth grade of primary school theme sub-theme public private sd calculated value critical value f % f % place formal 206 96.3 134 93.7 1 1.8 3.8 informal 6 2.8 1 0.7 journal of science learning article doi: 10.17509/jsl.v5i1.32116 31 j.sci.learn.2022.5(1).28-41 regarding table 2, calculated value (1.8) <critical value (3.8), therefore there is no significant difference between public and private school students’ images of a place. found no significant difference between teaching place images according to the public and private school education status. the science learning environment of fourth-grade public school and private school students consists of classrooms or laboratories. however, when students’ drawings examine, it is seen that traditional classroom drawings are familiar in both schools. this result is consistent with similar studies in the literature (şahin akyüz, 2016). when studies on out-of-school learning environments in science education examine, it is known that teachers approach these environments positively (bozdoğan, 2012; kubat, 2018; selanik ay & erbasan, 2016). however, formal teaching environments are generally included in students' drawings, revealing that informal environments are not preferred. it is thought that the reason for this situation may be because the financial and permit process can be complicated (özen orhan, 2013; şahin akyüz, 2016; yavuz, 2012). science is a field that contains many abstract concepts. therefore, students may be prejudiced towards learning abstract concepts. it is thought that supporting science lessons without school environments (museum, zoo, national park) will attract students’ interest and increase their motivation (laçin şimşek, 2011; liu & schunn, 2018; sontay, tutar, & karamustafaoğlu, 2016; turkmen, 2010). the location theme of public school is included in figure 1. in figure 1, there is a relative drawn by a public school student regarding the 'place' sub-theme. according to the visual, the teacher is the person presenting the subject, and the student is the person who listens to the teacher during the course. table 3 presents the findings of the 4th grade public and private school students about student behavior sub-themes. regarding table 3, the theme of student behavior, the drawings of 21.5% of fourth-grade public school students belong to the academic sub-theme, 67.8%, which constitute the majority, to the active sub-theme, 45.8%, which constitutes almost half of them, to the visual-spatial sub-theme. in contrast, very few students’ drawings fall in the sub-themes of using technology (1.4%) and indifferent passive (1.4%). regarding the theme of student behavior, the drawings of 16.1% of fourth-grade private school students belong to the academic sub-theme, the majority of the students (71.3%) to the active sub-theme, 23.1% to the visual-spatial sub-theme, 2.1% to the indifferent-passive sub-theme. whereas none of them drew a student using technology. table 4 presents the findings of the chi-square test of independence related to fourth-grade private and public-school students’ image of student behavior. regarding table 4, calculated value (12.0) > critical value (9.4). therefore there is a significant difference between public and private school students’ images of student behavior. a significant difference was found between the theme images of student behaviors according to the public and private school education status. academic student behavior has been drawn more of the students studying in public schools. the reason for this may be that students in public schools are usually listening or taking notes. in the theme of student behavior, the visual-spatial sub-theme was found to be the most differentiated sub-theme in terms of figure 1 formal sub-theme of the place theme, student drawing (public school) table 4 findings of primary school fourth-grade students studying at public and private schools regarding ‘student behavior’ sub-theme theme sub-theme f % public school student behavior academic 46 21.5 active 145 67.8 visual-spatial 98 45.8 using technology 3 1.4 indifferent-passive 3 1.4 private school student behavior academic 23 16.1 active 102 71.3 visual-spatial 33 23.1 using technology 0 0 indifferent-passive 3 2.1 table 3 findings of the chi-square test of independence related to ‘student behavior’ theme images of private and public school students studying in the fourth grade of primary school theme sub-theme public private sd calculated value critical value f % f % student behavior academic 46 21.5 23 16.1 4 12.0 9.4 active 145 67.8 102 71.3 visual-spatial 98 45.8 33 23.1 using technology 3 1.4 0 0 indifferentpassive 3 1.4 3 2.1 journal of science learning article doi: 10.17509/jsl.v5i1.32116 32 j.sci.learn.2022.5(1).28-41 two school types. visual-spatial student behavior has been drawn much more frequently in the images of students studying at public schools. the reason for this may be the high number of students in the classrooms in public schools, the lack of materials in the laboratory, and the inability of the teacher to do experiments and do experiments with her students. many studies yield similar results to this result (koç ünal & şeker, 2020; ürey & aydın, 2014). for example, güngör seyhan & okur (2020) determined that, as a result of their study, there are no laboratories in many schools and that teachers do not have a place where they can perform their experimental practices (güngör seyhan & okur, 2020). for this reason, they found that science teachers' experiments took place in the classroom. they found that branch teachers used the laboratory environment but mostly performed demonstrations or group experiments. while the image of using technology is not featured in the images of private school students, it has been included in the images of a tiny portion of the public school students. from this perspective, it can be concluded that students in both school types do not use technology sufficiently in science learning environments. this result is incomplete according to the requirements of the 21st century. because, as required by the century we live in, we use technology in almost every aspect of our lives. for this reason, importance should be given to the use of technology in science education. as a result, it is seen in the studies in the literature that the use of technology is effective in student achievement, teacher-student, and student-student communication (zhai, zhang, & li, 2018; zydney & warner, 2016). figure 2 contains a visual of the sub-theme of student behavior. figure 2 shows a drawing of a private school student regarding the theme of the place. according to the drawing, the student is experimenting with his student friends. the teacher follows the lesson away from the students. table 5 presents the findings of the fourth-grade public and private school students about teacher behavior sub-themes. regarding table 5, 29.9% of fourth-grade public school students drew the teacher as an interactive person, 38.3% as a person presenting the topic, 16.8% as a person directing the learning, 0.5% as a record keeper, 13.1% as a person watching/monitoring whereas 9.8% did nor drew a teacher. 28.0% of fourth-grade private school students drew the teacher as an interactive person, 30.1% as a person presenting the topic, 21.0% as a person directing the learning, 0.7% as a record keeper, 14.7% as a person watching/monitoring, and 2.1% did nor drew a teacher. table 6 presents the findings of the chi-square test of independence related to fourth-grade private and publicschool students’ image of teacher behavior. regarding table 6, calculated value (9.4) <critical value (11.0), therefore there is no significant difference between public and private school students’ images of teacher behavior. figure 2 formal sub-theme of the place theme, student drawing (private school) table 6 findings of the chi-square test of independence related to ‘teacher behavior’ theme images of private and public school students studying in the fourth grade of primary school theme sub-theme public private sd calculated value critical value f % f % teacher behavior interactive person 64 29.9 40 28.0 5 9.4 11.0 presenting the topic 82 38.3 43 30.1 driving learning 36 16.8 30 21.0 keeping record 1 0.5 1 0.7 no teacher 21 9.8 3 2.1 watching /monitoring 28 13.1 21 14.7 table 5 findings of primary school fourth-grade students studying at public and private schools regarding 'teacher behavior' sub-theme theme sub-theme f % public school teacher behavior interactive person 64 29.9 presenting the topic 82 38.3 driving learning 36 16.8 keeping record 1 0.5 no teacher 21 9.8 watching /monitoring 28 13.1 private school teacher behavior interactive person 40 28.0 presenting the topic 43 30.1 driving learning 30 21.0 keeping record 1 0.7 no teacher 3 2.1 watching /monitoring 21 14, 7 journal of science learning article doi: 10.17509/jsl.v5i1.32116 33 j.sci.learn.2022.5(1).28-41 the theme of ‘teacher behavior’ in science learning environments has similar images for both types of schools. however, the ‘no teacher’ sub-theme in this theme was used more by public school students. that may be that some of the public school students only boots themselves in the science learning e nvironment. in other words, the public school students who experimented with their friends at the table may have only drawn the table, the materials, themself, and their friends, and not the teacher who was elsewhere in that classroom. in both types of school, the teacher is generally drawn as the person presenting the subject. when the literatüre was examined, it was revealed that the teacher focused on presenting the subject (baltürk, 2006; bayındır & arıcı, 2015; duru, 2017; şahin akyüz, 2016; tatar & ceyhan, 2018; tezci, dilekli, yıldırım, kervan, & mehmeti, 2017). image of the person presenting the subject was followed by the images of the interactive person and the person who directed the learning. in learning environments, teachers have roles such as teacher, guide, student, and learner (çakıcı, 2008). in the research, drawing the teacher as the person presenting the subject in general, drawing the role of the teacher as the instructor, secondly drawing the role of the learner as an interactive person, and drawing the role of the learner as the third person directing the learning may also be a reflection of the guide role of teacher. it has been determined in the literature that teachers use the board frequently (bayındır & arıcı, 2015). however, it has been revealed that only one student per school sees the teacher as the record holder. the literature does not support this result of the study. that may be that even if the teacher keeps records during the lesson, it is not reflected in the student's image. figure 3 shows a drawing of a student studying at a public school on the theme of teacher behavior. according to the drawing, students follow the teacher and the experimental table from a distance. figure 4 shows a drawing of a student studying at a private school on teacher behavior. according to the drawing, the student performs an activity in the lesson with his friends. the teacher is giving the lesson on the board. table 7 presents the findings of the 4th grade public and private school students about the position of the teacher sub-themes. regarding table 7, 35.5% of fourth-grade public school students drew the teacher away from students, whereas approximately half (43.5%) drew them inside them. 47.6% of fourth-grade private school students drew the teacher away from students, whereas 35.5% drew them inside the students. table 8 presents the findings of the chi-square test of independence related to fourth-grade private and public-school students’ image of teacher position. figure 4 sub-theme student drawing presenting the subject of the theme of teacher behavior (public school) figure 3 sub-theme student drawing presenting the subject of the theme of teacher behavior (private school) table 7 findings of the fourth-grade primary school students studying at the public school regarding the 'position of the teacher' sub-theme theme sub-theme f % public school position of teacher away from students 76 35.5 inside the students 93 43.5 private school position of teacher away from students 48 47.6 inside the students 51 35.5 table 8 findings of the chi-square test of independence regarding the teacher position theme images of private and public school students studying in the fourth grade of primary school theme sub-theme public private f % f % sd calculated value critical value position of teacher away from students 76 35.5 68 47.6 1 4.1 3.8 inside the students 93 43.5 51 35.5 journal of science learning article doi: 10.17509/jsl.v5i1.32116 34 j.sci.learn.2022.5(1).28-41 regarding table 8, calculated value (4.1) > critical value (3.8); therefore, there is a significant difference between public and private school students’ images of teacher’s position. a significant difference was found between the theme images of teacher positions according to the public and private school education status. in the theme of classroom position of the teacher in science learning environments, it was revealed that the teacher was predominantly intertwined with the students in public schools. in contrast, the teacher was distant from the students in the students’ images in private schools. that may be that in private schools, teachers provide more opportunities for students to discover and construct information independently. ministry of education made a radical change in the science program in 2004 and brought the constructivist learning approach to the program. in 2005, this understanding was put into practice. in the constructivist learning theory, individuals structure the information they obtain through their efforts. the teacher is in the role of a guide in structuring knowledge by the student. it is the active student (akınoğlu, 2018; kaya & zengin, 2018). table 9 presents the findings of the fourth-grade public and private school students about teaching method sub-themes. regarding table 9, 66.8% of the fourth-grade public school students drew a student-centered teaching method, whereas 30.8% drew a teacher-centered method. fourthgrade private schools, most students (76.9%) drew the teaching method as student-centered and 18.2% as teachercentered. table 10 presents the findings of the chi-square test of independence related to fourth-grade private and public-school students’ image of teaching methods. regarding the above table 10, calculated value (6.5) > critical value (3.8); therefore, there is a significant difference between public and private school students’ images of teaching method. according to education in private and public schools, a significant difference is found in the 'teaching method' theme in science learning environments. however, it has been revealed that the teacher-centered teaching method is used more in public schools than private schools, and the student-centered teaching method is used less than in private schools. that may be because the constructivist approach introduced by the ministry of national education in 2005 and the methods suitable for this approach are used more in private schools than public schools. according to the constructivist learning theory, individuals construct their knowledge themselves. the teacher is a guide in the process of structuring the information. following this approach, teachers can use contemporary methods such as problem-based learning, project-based learning, learning through argumentation, and collaborative learning (acat, karadağ, & kaplan, 2012; mengi & schreglman, 2013; yılmaz & akkoyunlu, 2006). some studies reveal that teachers cannot fully adopt and apply constructivist learning methods that were put into practice throughout the country in 2005 (güneş, dilek, hoplan, & güneş, 2011; tatar & ceyhan, 2018; yılmazlar, çorapçıgil, & toplu, 2014) and those who say they apply it are inadequate (özdemir & köksal, 2015). the finding that teacher-centered teaching in public schools is made more than private schools and student-centered teaching less than private schools may be because state schools are not applied as much as these constructivist and individual teaching methods are applied in private schools. in the study, which investigated the constructivist features of the classroom environment in the secondary school science course, it was suggested that cooperative learning in learning environments, learning by doing and experiencing, student-centered activities that include different perspectives will lead learners to think (eroğlu, armağan, & bektaş, 2015). table 11 presents the findings of the fourth grade public and private school students about teaching environment elements sub-themes. regarding table 11, almost all fourth-grade public school students (92.1%) drew tools suitable for the topic, very few students (2.3%) drew technological equipment, 26.6% drew classic student desks, more than half (69.6%) drew experiment table, more than half (73.4%) drew positive experience, and 17.3% drew laboratory material. in table 9 findings of primary school fourth-grade students studying at public school regarding the sub-theme of teaching method theme sub-theme f % public school teaching method student-centered 143 66.8 teacher-centered 66 30.8 private school teaching method student-centered 110 76.9 teacher-centered 26 18.2 table 10 findings of the chi-square test of independence regarding the teaching method theme images of private and public school students studying in the fourth grade of primary school theme sub-theme public private sd calculated value critical value f % f % teaching method student-centered 143 66.8 110 76.9 1 6.5 3.8 teacher-centered 66 30.8 26 18.2 journal of science learning article doi: 10.17509/jsl.v5i1.32116 35 j.sci.learn.2022.5(1).28-41 contrast, none of the students drew negative experiences. the majority of fourth-grade private school students (86.0%) drew tools suitable for the topic, 14.4% drew technological equipment, 8.4% drew classic student desk, 37.8% drew experiment table, the majority (79.7%) drew positive experience, 60.8% positive experience and 64.3% drew laboratory material. in contrast, only one student drew negative experience (0.7%). table 12 presents the findings of the chi-square test of independence related to 4th grade private and public-school students’ image of the elements of the teaching environment. regarding table 12, calculated value (147.8) > critical value (14.0). therefore there is a significant difference between public and private school students’ images of the elements of the teaching environment. a significant difference was found in the images of ‘teaching environment elements’ of the students according to their education status in private and public schools. however, in this theme, in both types of schools, most of the students' images have a suitable tool in their images. based on this result, it can be interpreted that teachers want to concretize the subject according to the cognitive characteristics of the students in the concrete operational period. when the literature is examined, it has been revealed that teachers tend to teach their lessons with concrete material (baltürk, 2006; pişkin tunç, durmuş, & akkaya, 2012; şimşek, hırça, & coşkun, 2012; yazlık, 2018). for example, it has been revealing that the technological equipment in the science learning environment is much more in private schools. the reason for this may be that the financial means of private schools are better than public schools. while the classical student desk in the public school is more common in student images, the group table is very much drawn in private schools. while the experimental table is seen in the majority of student images in both school types, it can be concluded that while the real experiment table is used in private schools due to the details such as the faucet, the sink, the u-shaped table that covers the whole classroom, in the state schools the student desk is used as the experiment table 11 findings of fourth-grade students studying at public and private schools regarding the sub-theme of teaching environment elements theme sub-theme f % public school the elements of teaching environment tools suitable for the topic 197 92.1 technological equipment 5 2.3 classic student desk 57 26.6 group table 1 0.5 experiment table 149 69.6 positive experience 157 73.4 negative experience 0 0.0 laboratory material 37 17.3 private school the elements of teaching environment tools suitable for the topic 123 86.0 technological equipment 20 14.4 classic student desk 12 8.4 group table 54 37.8 experiment table 114 79.7 positive experience 87 60.8 negative experience 1 0.7 laboratory material 92 64.3 table 12 findings of the chi-square test of independence related to the theme images of teaching environment elements of private and public school students studying in the fourth grade of primary school theme sub-theme public private calculated value critical value f % f % sd the elements of teaching environment tools suitable for the topic 197 92.1 123 86.0 7 147.8 14.0 technological equipment 5 2.3 20 14.4 classic student desk 57 26.6 12 8.4 group table 1 0.5 54 37.8 experiment table 149 69.6 114 79.7 positive experience 157 73.4 87 60.8 negative experience 0 0.0 1 0.7 laboratory material 37 17.3 92 64.3 journal of science learning article doi: 10.17509/jsl.v5i1.32116 36 j.sci.learn.2022.5(1).28-41 table or generally the teacher table. in both types of schools, it was found that students’ images generally contain positive experiences. the student who had a negative experience was absent in the public schools subject to the study. a negative experience encounter in the image of a student in private schools. that may be due to the downbeat mood of the student before drawing or the negative attitude towards the science learning environment. when the two schools are compared, the sub-theme that makes the difference between them the most is the "laboratory material" sub-theme. students generally have a beaker, microscope, graduated cylinder, scaffold, test tube, magnifier, etc., in the science learning environment in private schools. while most of the laboratory materials were drawn, the number of students drawing these materials in public schools is deficient. the laboratory table 13 findings of the primary school fourth-grade students studying in public and private schools regarding the subtheme of the elements that are important to the student in the learning environment theme sub-theme f % in public school the elements of the learning environment that are important to the students cleanness 22 10.3 having a laboratory 6 2.8 materials 35 16.4 security 3 1.4 learning/understanding the topic 49 22.9 the topic itself 32 15.0 quiet place 29 13.6 being careful/doing it right 10 4.7 listening 8 3.7 interaction with the teacher 15 7.0 undertaking a task 6 2.8 interaction with friends 17 7.9 having fun 1 0.5 explanation of the station 7 3.3 experimenting 25 11.7 everything 1 0.5 my existence 5 2.3 seeing the experiment 5 2.3 health 0 0.0 being successful 8 3.7 in private school the elements of the learning environment that are important to the students cleanness 2 1.4 having a laboratory 1 0.7 materials 44 30.8 security 12 8.4 learning/understanding the topic 27 18.9 the topic itself 6 4.2 peace/silence 5 3.5 being careful/doing it right 13 9.1 listening 6 4.2 interaction with the teacher 10 7.0 undertaking a task 0 0.0 interaction with friends 9 6.3 having fun 5 3.5 getting an explanation 1 0.7 experimenting 22 15.4 everything 3 2.1 itself 3 2.1 seeing the experiment 3 2.1 health 6 4.2 being successful 4 2.8 journal of science learning article doi: 10.17509/jsl.v5i1.32116 37 j.sci.learn.2022.5(1).28-41 materials are drawn by very few students studying at the state school. the subject covered when the natural science learning environment drawing test is applied to the students is a subject that does not require laboratory materials. public school students generally teach science lessons in traditional classrooms. the reason why laboratory materials are included in the images of private school students may be because science learning environments in private schools take place in laboratories based on drawings. table 13 presents the findings of the fourth-grade public and private school students about the elements of the learning environment that are important to the students. according to table 13, 10.3% of the fourth-grade public school students answered as cleaning, 2.8% as laboratory, 16.4% as materials, 1.4% as security, 22.9% as learning/understanding the topic, 15% as the topic itself, 13.6 as peace and silence, 4.7% as being careful/doing right, 3.7% as listening to the teacher, 7.0% as interacting with the teacher, 2.8% as undertaking a task /explaining a topic, 7.9% as communicating with friends, 0.5% (one student) as having fun, 3.3% as getting an explanation, 11.7% as doing experiments, 0.5% (one student) as everything, 2.3% as the topic itself, 2.3% as seeing the experiment, and 3.7% as to be successful; nobody mentioned health in the answer. regarding the elements in table 4 that are important for the students related to the learning environment, which emerged with the question "what is the most important thing for you in the learning environment?", 1.4% of the fourth-grade private school students answered as cleaning, 0.7% as laboratory, 30.8% as materials, 8.4% as security, 18.9% as learning/understanding the topic, 4.2% as the topic itself, 3.5 as peace and silence, 9.1% as being careful/doing right, 4.2% as listening to the teacher, 7.0% as interacting with the teacher, 0.0% as undertaking a task, 6.3% as communicating with friends, 35.0% as having fun, 0.7% as getting an explanation, 15.4% as doing experiments, 2.1% as everything, 2.1% as the topic itself, 2.1% as seeing the experiment, 4.2% as health and 2.8% as to be successful. table 14 presents the findings of the chi-square test of independence related to 4th grade private and publicschool students’ image of the elements of the learning environment that are important to the students. regarding table 14, calculated value (75.9) > critical value (30.1); therefore, there is a significant difference between public and private school students’ images of the elements of the learning environment that are important to the students. a significant difference was found between the theme images of the essential elements in the science learning environment for the students between the two types of school. in the theme of the elements that students care most about in the science learning environment, learning / understanding the subject has been the most preferred element in public school students. also, the subject itself, table 14 findings of the chi-square test of independence related to the images of the theme of the things that are important in the learning environment for the students of private and public school students in the fourth grade of primary school theme sub-theme public private f % f % sd calculated value critical value the elements of the learning environment that are important to the students cleanness 22 10.3 2 1.4 19 75.9 30.1 having a laboratory 6 2.8 1 0.7 materials 35 16.4 44 30.8 security 3 1.4 12 8.4 learning/understanding the topic 49 22.9 27 18.9 the topic itself 32 15.0 6 4.2 peace/silence 29 13.6 5 3.5 being careful/doing it right 10 4.7 13 9.1 listening 8 3.7 6 4.2 interaction with the teacher 15 7.0 10 7.0 undertaking a task 6 2.8 0 0.0 interaction with friends 17 7.9 9 6.3 having fun 1 0.5 5 3.5 getting an explanation 7 3.3 1 0.7 experimenting 25 11.7 22 15.4 everything 1 0.5 3 2.1 itself 5 2.3 3 2.1 seeing the experiment 5 2.3 3 2.1 health 0 0.0 6 4.2 being successful 8 3.7 4 2.8 journal of science learning article doi: 10.17509/jsl.v5i1.32116 38 j.sci.learn.2022.5(1).28-41 peace/quiet, experimenting / being able to experiment, materials, and cleanliness stand out compared to other subthemes. it is seen that public school students find the cleaning element important (table 14). this situation makes us think that the students studying in public schools do not find cleaning sufficient. when the studies in the literature are examined, there are results for students to complain about cleaning problems in schools (göksoy, 2017; yüksel, 2019). the laboratory element has been given more in public schools than in private school students. students in public schools write it more. as understood from student drawings, science learning in public schools is not done in a laboratory environment, and students want a laboratory. when the studies in the literature on this subject are examined, it is observed that, similarly, public school students want to do laboratory activities from their teachers (kılıç & aydın, 2018). the laboratory element is less in private schools than in public schools because private schools have budgets allocated to laboratories (üzümlü, 2019). the abundant material element is written more in public schools than in private schools. students' missing material in the teaching environment may have arisen in their drawings. based on this, it can be deduced that the students do not find the instructional environment material sufficient (okuyucu, 2019). the security element was written by private school students more than public school students. this situation is closely related to how students interpret the concept of security. students in private schools may have answered security by referring to the strict safety rules in their schools. learning/understanding the subject has been written extensively by students of both school types. it can be considered normal that this element is essential in both types of school. the issue itself is written more often in public schools than in private school students. in this study, teachers took place more proportionally as the person presenting the subject in public schools' images. the reason for this situation is thought to be that the teacher wants to explain the subject herself due to reasons such as a large number of classrooms in the public school and the skill of the teacher in classroom management. the rate of writing the peace/silence element in public schools is higher than in private schools. it is thought that the crowded learning environments of the students and other reasons may create a peaceful learning environment in the student. being attentive / doing right, listening, interacting with the teacher, interacting with friends, having fun, experimenting, everything, the student himself, seeing the experiment, and the successful elements were written by the students studying in both types of school at close rates. this situation can be considered normal. the element of taking part is not written by any students in private schools but by a small group of public schools. the reason for this may be that students want to take more positions in public schools. the element of disclosure is written more frequently to students in public schools. that may be because the classes in public schools are more crowded. it is difficult for teachers to explain separately according to each student's learning style and the possibility of not choosing it. the health factor is written more frequently in private schools. there may be more attention to health issues in private schools or vice versa. that is also an issue that needs to be investigated. conclusion as a result, this study aims to investigate the learning environment images of public and private school students in fourth grade and determine whether there is a significant difference between these images. no significant difference was found between the place theme images of public and private school students. this result reveals that teachers working in public and private schools do not prefer informal environments. a significant difference was found between public and private school students' student behavior theme images. the most prominent in this theme of the study are the drawings showing the visual/spatial sub-theme. within the scope of visual-spatial intelligence, students follow an experiment and a situation according to their drawings. from this point of view, it can be said that the lessons are teacher-centered in public schools. there was no significant difference between public and private school students' teacher behavior images. however, when the drawings of the ‘no teacher’ subtheme of this theme are examined, it was determined that private school students stand out more than state school students. this situation shows that the training carried out in private schools is mostly student-centered. a significant difference was found between the state and private school students' images of the position of the teacher. teachers are intertwined with students in public schools, while in private schools, teachers are located away from students. this result is consistent with the 'no teacher' sub-theme drawings in the teacher behavior theme of private school students. found a significant difference between the teaching method images of public and private school students. there is a student-centered education in both school types. however, student-centered education is more common in private school student images. a significant difference was found between the images of teaching environment staff of public and private school students. according to this result of the study, it stands out that the classroom order in the public school is in the classical order. another element that stands out in this theme is the frequent use of laboratory materials in the drawings of private school students. based on this result, it can be said that private school students perceive science lessons more as teaching with experiments. a significant difference was found between public and private school students' images of things that are important to students in the teaching journal of science learning article doi: 10.17509/jsl.v5i1.32116 39 j.sci.learn.2022.5(1).28-41 environment. among these images, it has been determined that the state school students draw the most cleaning element in the school. based on this result, the drawings of the public school students made drawings due to cleaning problems. another element that emerges in this theme is that public school students attach importance to the element of peace. this situation may be the inability of both students and teachers to ensure the peace of the lesson. private school students often drew the materials element. according to this result, those students give importance to teaching environment materials. in addition, it is one of the prominent sub-themes that private school students give importance to being careful / doing right. this result may be due to the students' desire for the expected success from science education to be error-free. references acar, ç. (2000). zihin özürlü çocuklarla çalışan özel eğitim öğretmenlerinin sınıflarında karşılaştıkları problem davranışlarla ilgili görüş ve önerileri. [the opinions and suggestions of teachers who work with children with mental retardation about the problem behaviors in their classes] (master thesis). anadolu university institute of social sciences, eskişehir. acat, m. b., karadağ, e., & kaplan, m. (2012). kırsal bölgelerde fen ve teknoloji dersi öğrenme ortamları: yapılandırmacı öğrenme açısından bir değerlendirme çalışması. [learning environments of science and technology course in rural areas: an assessment study in terms of constructivist learning]. dicle university ziya gökalp eğitim fakültesi dergisi, (18), 106–119. akınoğlu, o. (2018). öğrenme öğretme kuram ve yaklaşımları (4.edition; b. oral, ed.). ankara: pegem akademi. armstrong, d. c. (2007). using drawn images to prepare teachers who can envision and actualize a world of educational quality. https://www.researchgate.net/publication/234620140_using_dr awn_images_to_prepare_teachers_who_can_envision_and_ac tualize_a_world_of_educational_quality baltürk, m. (2006). fen bilgisi öğretmen ve öğretmen adaylarının laboratuvar kullanımında karşılaştıkları zorluklar ve çözüm önerileri. [encountered problems and solution suggestions in laboratory studies of science teachers and teacher candidates] (master thesis). kafkas university, institute of science, kars, kars. bayındır, n., & arıcı, a. f. (2015). sınıf tahtalarının etkili kullanımı üzerine bir araştırma. [a study on the effective use of class board]. ana dili eğitimi dergisi, 3(4), 74–83. bilir, v., tatlı, a., yıldız, c., emiroğlu, b. b., ertuğrul, d., & sakmen, g. (2020). argümantasyon tabanlı öğrenme yaklaşımında kullanılan agümantasyon tekniklerinin ortaokul sekizinci sınıf öğrencilerinin bilim insanı imajları üzerine etkisi. [the effect of argumentation techniques used in argumentation based learning approach on 8th grade secondary school students’ image of scientist]. gazi eğitim fakültesi dergisi, 40(2), 481-510. bozdoğan, a. e. (2012). eğitim amaçlı gezilerin planlanmasına ilişkin fen bilgisi öğretmen adaylarının uygulamaları: altı farklı alan gezisinin değerlendirilmesi. educational sciences:theory and practice, 12(2), 1049–1072. büyüköztürk, ş., kılıç çakmak, e., akgün, ö. e., karadeniz, ş., & demirel, f. (2018). bilimsel araştırma yöntemleri. [scientific research methods] (26. edition). ankara: pegem yayınevi. çakıcı, y. (2008). fen ve teknoloji öğretiminde yapılandırmacı yaklaşım, fen ve teknoloji öğretiminde yeni yaklaşımlar (1. edition). ankara: pegem akademi. candaş, b., kiryak, z., & ünal, s. (2021). bilim tarihi temelli hikâyelerle fen öğretimi: tesla ve volta örneği. yüzüncü yıl üniversitesi eğitim fakültesi dergisi, 18(1), 405–435. https://doi.org/10.33711/yyuefd.867570 çavdar, o., & doymuş, k. (2016). i̇yi bir eğitim ortamı ı̇çin yedi ı̇lkenin ı̇şbirlikli öğrenme yöntemi ile kullanılmasının fen ve teknoloji dersinde başarıya etkisi. [the using of seven principles for good practice with cooperative learning method: effect on achievement in science course]. atatürk university journal of social sciences institue, 20(2), 441–466. çepni, s. (2014). araştırma ve proje çalışmalarına giriş. [introduction to research and project studies] (7.edition). trabzon: celepler matbaacılık. derman, i̇. (2019). eğitim bilimleri ana bilim dalı eğitim programları ve öğretim programı. [the level of association of science course with daily life] (doctoral thesis). hacettepe university, institute of education sciences, ankara, ankara. develi, g. (2017). teknolojik pedagojik destekli öğrenme ortamının öğrencilerin fen başarısına ve motivasyonuna etkisinin incelenmesi. [the investigation of the effect of the technological pedagogical based learning environment on students’ achievement and motivation in science] (master thesis). çanakkale onsekiz mart university, institute of educational sciences çanakkale. digilli baran, a., & karaçam, s. (2020). bilim ı̇nsanlarının sahip oldukları risklere yönelik ortaokul öğrencilerinin algıları. [perceptions of secondary school students about risks of scientists]. pau journal of education, (50), 269–290. dikmenli, m. (2010). undergraduate biology students’ representations of science and the scientist. college student journal, 44(2), 579.(2), 579–588. dönmez, g. (2017). ortaokul öğrencilerinin fen bilimleri dersine, bilime, fen bilimleri öğretmenine ve bilim insanına yönelik metaforik algıları ve imajları. [metaphorical perceptions and imaginations of secondary school students for science course, science, science teachers and scientists] (master thesis). adnan menderes university, institute of science, aydın, aydın. duru, s. (2017). pre-service elementary education teachers’ beliefs about teaching and learning in turkey türkiye’de sınıf öğretmenliği öğretmen adaylarının öğretme ve öğrenme ile ilgili inançları. journal of human sciences, 14(4), 4002-4014.. ergen, y., boyraz, c., batmaz, o., & çevik kansu, c. (2020). zihinsel imaj oluşturmaya dayalı öğretimin okuduğunu anlama becerisine etkisi.[the effect of teaching based on mental image formation on reading comprehension skills]. kastamonu university kastamonu education journal, 28(1), 72–81. eroğlu, s., armağan, f., & bektaş, o. (2015). fen bilimleri dersi öğrenme ortamlarının yapılandırmacı özellikler açısından değerlendirilmesi. [evaluation of learning environment of science course in terms of constructivist properties]. kırşehir eğitim fakültesi dergisi, 16(2), 293–312. gökdaş, i̇., & ak, ş. (2019). ortaokul öğrencilerinin geleceğin okullarına ilişkin hayalleri. [the dreams of elementary school students in terms of the schools of the future]. kastamonu university kastamonu education journal, 27(5), 2161–2172. göksoy, s. (2017). okulların altyapı yeterliliği. international journal of leadership trainig, 1(1), 9–15. güneş, t., dilek, n. ş., hoplan, m., & güneş, o. (2011, april). fen ve teknoloji dersinin öğretmenler tarafından uygulanması üzerine bir araştırma. presented at the 2nd international conference on new trends in education and theır implications, antalya, turkey. güngör seyhan, h., & okur, m. (2020). fen bilimleri laboratuarlarında mobil teknoloji desteğinin önemi hakkında öğretmen görüşlerinin ı̇ncelenmesi. [investigation of teachers’ opinions about the ımportance of mobile technology support in science laboratories]. yüzüncü yıl üniversitesi eğitim fakültesi dergisi, 17(1), 1242–1271. hastürk, g., & sönmez, d. (2020). türkiye’de fen eğitimi alanında doktora düzeyinde yapılan tez çalışmalarının bibliyografik analizi. [bibliographic analysis of doctoral level thesis studies in science education domain in turkey]. journal of humand and social science researches, 9(5), 3174–3194. parsa, m. k. (2016). i̇şbirlikli sorgulamaya dayalı öğrenme ortamının yaratıcı, düşünmeye, sorgulayıcı öğrenme becerilerine, fen ve teknoloji dersine yönelik tutuma etkisi. [effect of collaborative journal of science learning article doi: 10.17509/jsl.v5i1.32116 40 j.sci.learn.2022.5(1).28-41 inquirybased learning environment on creative thinking, inquiry learning skills, attitudes towards science and technology lesson] (doctoral thesis). unpublished doctorate dissertation, marmara üniversitesi eğitim bilimleri enstitüsü, i̇stanbul. karadeniz, y., & doymuş, k. (2015). fen ve teknoloji öğretmenlerinin işbirlikli öğrenme modeli sınıfta uygulamaları ve elde edilen sonuçların değerlendirilmesi: iğdır i̇l örneği. [informing of science and technology teachers applications of cooperative learning model, in the class and evaluating the obtained results: example of igdir city]. e-kafkas journal of educational research, 2(1), 1–12. kaya, e., & zengin, e. (2018). 5e modelinin hücre bölünmesinin öğretiminde öğrenci başarısına etkisi. ekev akademi dergisi, (73), 415–427. kılıç, m. s., & aydın, a. (2018). öğretmenlerin fen bilimleri dersi kapsamında laboratuvar uygulamaları hakkındaki görüşlerinin planlanmış davranış teorisi yardımıyla ı̇ncelenmesi. [investigating the opinions of theacher’s to perform laboratory practices in science lessons by means of the theory of planned behaviour]. kastamonu university kastamonu education journal, 26(1), 1–6. kıryak, z., candaş, b., karanisanoğlu, b., & özmen, h. (2019). yedinci sınıf öğrencilerinin enerji dönüşümlerine yönelik bilgi düzeylerinin çizimler aracılığıyla belirlenmesi. [determining the seventh grade students’ knowledge level on energy transformations through drawings]. journal of science mathematics entrepreneurship and technology education, 2(2), 79–92. koç ünal, i̇., & şeker, r. (2020). sanal laboratuvar uygulamalarının öğrenci akademik başarıları üzerine etkisinin i̇ncelenmesi: elektrik ünitesi. kırşehir eğitim fakültesi dergisi, 21(1). korkman, n., & metin, m. (2021). the effect of inquiry-based collaborative learning and inquiry-based online collaborative learning on success and permanent learning of students. journal of science learning, 4(2), 151–159. kubat, u. (2018). fen bilimleri öğretim programına yönelik öğretmenlere verilen hizmet içi eğitimin öğretmen görüşlerine göre değerlendirilmesi. [evaluation of the ın-service training program given about the science education curriculum on the basis of the teachers’ opinions]. msku journal of education, 4(2), 1–8. laçin şimşek, c. (ed.). (2011). okul dışı öğrenme ortamları ve fen eğitimi içinde, fen öğretiminde okul dışı öğrenme ortamları (2. edition). ankara: pegem akademi. liu, a. s., & schunn, c. d. (2018). the effects of school-related and home-related optional science experiences on science attitudes and knowledge. journal of educational psychology, 110(6), 798–810. mengi, f., & schreglman, s. (2013). yapılandırmacı sınıf öğrenme ortamı algısı. [constructivist class learning environment perception]. gümüşhane university sosyal bilimler elektronik dergisi, (7), 160–174. miles, m. b., & huberman, a. m. (1994). qualitative data analysis: an expanded sourcebook (2. edition). california: sage publications. okuyucu, ü. (2019). ortaokul düzeyinde hacim kavramına giriş: somut materyal destekli bir öğretim örneği / introduction to the concept of volume in the middle school level: a sample teaching with concrete material support (master thesis). anadolu university, institute of educational sciences, eskişehir, eskişehir. özdemir, i̇., & köksal, n. (2015). i̇lköğretim kinci kademe öğretmenlerinin yapılandırmacı yaklaşıma yönelik tutumları [attitudes of elementary school teachers towards constructivism]. atatürk university journal of social sciences institue, 19(2), 221–244. özdemir, s., & akkaya, e. (2013). genel lise öğrenci ve öğretmenlerinin okul ve ideal okul algılarının metafor yoluyla analizi [the analysis of secondary school students’ and teachers’ mental images of school and ideal school by using metaphor]. kuram ve uygulamada eğitim yönetimi dergisi, 19(2), 295–322. özen orhan, s. (2013). öğrenmeye yeni bir bakış: cihaz, yer ve zamandan bağımsız bir öğrenme ortamı geliştirme çalışması. [learning a new perspective: ubiquitous learning environment development study]. (master thesis). ege university, institute of science, i̇zmir, i̇zmir. pişkin tunç, m., durmuş, s., & akkaya, r. (2012). ilköğretim matematik öğretmen adaylarının matematik öğretiminde somut materyalleri ve sanal öğrenme nesenlerini kullanma yeterlikleri. matder mathematics education journal, 1(1), 13–20. şahin akyüz, s. (2016). sekizinci sınıf öğrencilerinin gerçek ve ideal fen öğrenme ortamlarına yönelik imajlarının çizimle tespiti (muğla ili örneği). [the determination by drawing of the images of eighth grade students, which are intended for actual and ideal science learning environment (muğla city example)]. (master thesis). muğla sıtkı koçman university institute of education sciences, muğla, muğla. salur, i̇., & pehlivan, m. (2021). sorgulamaya dayalı oöğretimin fen bilgisi öğretmen adaylarının erişi ve sorgulayıcı öğrenme becerilerine etkisi. [the effect of inquiry-based teaching on preservice science teachers’ academic achievement and skills of inquiry learning]. journal of ahmet keleşoğlu education faculty, 3(1), 101–116. selanik ay, t., & erbasan, ö. (2016). views of classroom teachers about the use of out of school learning environments. journal of education and future, (10), 35–50. şimşek, h., hırça, n., & coşkun, s. (2012). i̇lköğretim fen ve teknoloji öğretmenlerinin öğretim yöntem ve tekniklerini tercih ve uygulama düzeyleri: şanlıurfa ili örneği. [primary science and technology teachers’ selection of using teaching methods and techniques and the levels of theır applications: the sample of şanlıurfa city]. mustafa kemal üniversitesi sosyal bilimler enstitüsü dergisi, 9(18), 249– 268. sontay, g., tutar, m., & karamustafaoğlu, o. (2016). student views about “science teaching with outdoor learning environments: planetarium tour. journal of research in informal environments (jrinen), 1(1), 1–24. tatar, n., & ceyhan, n. (2018). fen bilgisi öğretmen adaylarının yapılandırmacı kurama dayalı öğretim uygulamalarının geliştirilmesi. [development of pre-service science teachers’ constructivistoriented teaching practices]. elementary education online, 17(1), 207–222. tezci, e., dilekli, y., yıldırım, s. y., kervan, s., & mehmeti, f. (2017). öğretmen adaylarının sahip olduğu öğretim anlayışları üzerine bir analiz [an anaylsıs on pre-servıce teachers’ teachıng conceptıon]. education sciences, 12(4), 163–176. turkmen, h. (2010). i̇nformal (sınıf-dışı) fen bilgisi eğitimine tarihsel bakış ve eğitimimize entegrasyonu. çukurova university faculty of education journal, 3(39), 46–59. turkmen, h., & unver, e. (2018). comparison of elementary students’ images of science teaching for turkish, dutch, scottish, and german science classrooms. universal journal of educational research, 6(11), 2624–2633. ulu, h., & ocak, i̇. (2018). i̇lköğretim öğrencilerinin fen öğretimine yönelik zihinsel modellerinin incelenmesi. [examination of primary school students' mental models for science teaching.]. electronic turkish studies, 13(11), 1367–1388. ürey, m., & aydın, m. (2014). genel biyoloji laboratuvarı kapsamında geliştirilen biyo-lab-web yönteminin etkililiği ve öğretmen adaylarının görüşleri. [pre-service primary teachers’ views on a method called bio-labweb (blw) in introductory biology laboratory course and its effectiveness]. dicle üniversitesi ziya gökalp eğitim fakültesi dergisi, (22), 150–167. üzümlü, s. (2019). özel okullarda ve devlet okullarındaki müzik eğitiminin farklı değişkenler açısından karşılaştırmalı olarak incelenmesi. [compared analysis of music education’s from different variables in private and state schools] (master thesis). pamukkale university, institute of education sciences, denizli, denizli. yasin, a. i., prima, e. c., & sholihin, h. (2018). learning electricity using arduino-android based game to improve stem literacy. journal of science learning, 1(3), 77–94. yavuz, m. (2012). fen eğitiminde hayvanat bahçelerinin kullanımının akademik başarı ve kaygıya etkisi ve öğretmen-öğrenci görüşleri. [the effect of using zoos in science education on students’ academic achievement and anxiety towards science and teachers-students’ conceptions]. (master thesis). sakarya university, institute of educational sciences, sakarya, sakarya. yazlık, d. ö. (2018). öğretmenlerin matematik öğretiminde somut öğretim materyali kullanımına yönelik görüşleri. [the views of journal of science learning article doi: 10.17509/jsl.v5i1.32116 41 j.sci.learn.2022.5(1).28-41 teachers about use of concrete teaching materials in mathematics teaching]. opus international journal of society researches, 8(15), 777–805. yıldırım, a., & şimşek, h. (2018). sosyal bilimlerde nitel araştırma yöntemleri. ankara: seçkin yayıncılık. yılmaz, m., & akkoyunlu, b. (2006). farklı öğrenme ortamlarının kalıcılığa etkisi. [the effect of different learning environments on retention]. eurasian journal of educational research, 7(3), 45–56. yılmazlar, m., çorapçıgil, a., & toplu, b. (2014). fen bilgisi öğretiminde programlı öğretimin öğrenci başarısı ve tutumu üzerindeki etkisi.[the effect of programmed instruction in science education on students’ achievements and attitudes]. necatibey eğitim fakültesi elektronik fen ve matematik eğitimi dergisi, 8(1), 45–67. yüksel, i̇. (2019). okul yönetimleri ve öğretmenlik mesleğine yönelik bimer’e konu olan şikayetlerin analizi (antalya örneği). [analysis of complaints that are subject to bi̇mer for school management and teaching profession (antalya case)]. international journal of karamanoglu mehmetbey educational research, 1(1), 19–30. zhai, x., zhang, m., & li, m. (2018). one‐to‐one mobile technology in high school physics classrooms: understanding its use and outcome. british journal of educational technology, 49(3), 516–532. zheng, m., & spires, h. a. (2014). fifth graders’ flow experience in a digital game-based science learning environment. international journal of virtual and personal learning environments (ijvple), 5(2), 69-86. zydney, j. m., & warner, z. (2016). mobile apps for science learning: review of research. computers & education, 94(march 2016), 1–17. a © 2022 indonesian society for science educator 165 j.sci.learn.2022.5(1).165-175 received: 31 march 2021 revised: 6 october 2021 published: 1 march 2022 comparing effects of two different explicit–reflective instructions on preschool prospective teachers’ view about nature of science and scientific knowledge mustafa metin1* 1department of mathematics and science education, faculty of education, erciyes university, turkey *corresponding author. mustafametin@erciyes.edu.tr abstract this study aims to compare the effect of formative assessment with explicit-reflective instruction and explicit-reflective on pre-school prospective teachers' views about the nature of science and scientific knowledge. in this study, it was used a pretestposttest nonequivalent control group design and the sample of the study consists of 66 pre-school prospective teachers in the 2nd grade. there are 33 of them were assigned control group and the others were assigned as the experimental group. since the students could not be randomly grouped in the study, a quasi-experimental design was used. in this study, the nature of science scale (noss) and the attitude scale towards scientific knowledge (sks) were used as pre-test, post-test and retention test. in order to teach the students by using nature of science (nos) and scientific knowledge (sk), open-reflective instruction was used in the control group, and open-reflective instruction and embedded formative assessment were applied in the experimental group. noss and sks were used as a pre-test, post-test and retention-test in this study. in order to teach the students noss and sk, while explicit-reflective instruction was applied in the control group, formative assessment embedded with explicit-reflective instruction was applied experimental group.as a result of the study, it was determined that the formative assessment embedded with explicit-reflective instruction was more positively and permanently changed on pre-school prospective teachers' view about nos and sk than the other method. keywords explicit-reflective instruction, formative assessment, nature of science, scientific knowledge, pre-school prospective teacher 1. introduction it is emphasized that every citizen should receive a good education in order for countries to create a strong future, especially to be educated as science and technology literate (eş & sarıkaya, 2010; güneş & karaşah, 2016). in recent years, necessary reforms have been made in science education. while developing these reforms, it was taken into account that students must be science literate to adapt to changing life (nrc, 1996; nsta, 2000). while there is a consensus in many areas about the importance of science literacy, scientific literacy has been attributed to different meanings by many researchers (deboer, 2000). as a result, various definitions of scientific literacy have been made (nrc, 1996). the most widely used of these definitions is that scientific literacy is the ability to access and use information (aaas, 1993). therefore, individuals who are having scientific literate are expected to have skills such as proposing concrete and rational solutions to problems and using scientific methods and techniques when they encounter problems in daily life (altındağ, 2010). in addition, scientifically literate individuals, it is stated that he can easily understand the scientific knowledge (sk), the nature of science (nos), the interrelationship between science and society, and between science and humanities (roberts, 2007). researchers agree on a subject that emphasizes students' understanding of the nos, which is an epistemological aspect of scientific literacy (bybee, 1997). the national science educational standards (nrc, 1996) emphasized that the nos concept is a component of scientific literacy, has become one of the priorities aims of teachers in science teaching. therefore, it is not enough to teach science concepts alone to gain students' scientific literacy. the nature of science, seen as a sub-dimension of mailto:mustafametin@erciyes.edu.tr journal of science learning article doi: 10.17509/jsl.v5i1.33190 166 j.sci.learn.2022.5(1).165-175 scientific literacy in science education, it is researched by science educators, science historians, sociologists, and philosophers in the last thirty years. there is no consensus about the definition of the nos among science historians and philosophers. however, it seems that students from pre-school to university (k-12) agree on what they should gain about the nature of science (abd-el-khalick, bell, & lederman, 1998). there is some basic agreement about the aspects of nos and features of sk among science educators. the first aspect of nos is "tentative nature", which expresses that principles, laws, and theories can change in line with new evidence, change in thinking, and developments in the cultural and social environment (mccomas, clough, & almazroa, 1998). the "empirical nature" aspect means that scientific knowledge arises from observations of the natural world. scientists need experimental evidence to produce sk. therefore, the availability of new evidence requires a review of scientific knowledge (mccomas, 1998). the other aspect, "observation and inference," have different meanings; while observations are descriptive facts about natural phenomena that we create through our senses, inferences are interpretations of those observations. this point of view explains the differences between these two concepts (lederman, abd-el-khalick, bell, & schwartz, 2002). for another aspect, "creative and imaginative nature", refers to the importance of imagination and creativity in producing scientific knowledge (mccomas, 1998). the aspect of "social and cultural embeddedness" dictated that scientific knowledge revealed by scientists is influenced by the culture and society of the scientist (mccomas et al., 1998). the aspect of "scientific theories and laws"; are different forms of scientific knowledge. according to this perspective, it is emphasized that there is no hierarchical relationship between laws and theories. scientific laws explain the relationships between visible phenomena, while scientific theories are well-organized and highly proven explanations. it is also emphasized that although scientific laws can be tested, scientific theories cannot be directly tested because they are based on certain assumptions and unobservable entities. the aspect "theory-laden nature of science" explains that scientists depending on a particular theory, his work shows that they accept some assumptions and principles. in his work, they sometimes believe that while evaluating their observations in the light of the theories they adhere to, theories constitute their expectations (mccomas, 1998). mccomas (1998) argued that expectations affect observations and conclusions. he stated that they are not impartial. in the researches, it is stated that the understanding of the nature of science that teachers have and their teaching of the nature of science have critical importance in terms of science education and teaching (deniz & adibelli, 2015; hanuscin, lee, & akerson, 2011; mıhladız & doğan, 2017; prima, utari, chandra, hasanah, & rusdiana, 2018). teaching the nos and aspects to students has long been a goal of science educators (kang, scharmann, & noh, 2005). however, students and teachers generally lack sufficient understanding of nos (pomeroy, 1993; ryan & aikenhead, 1992). it is thought that although pre-service teachers have taken the nature of science and history of science courses, they are still insufficient in establishing the relationship between theory and law (önen öztürk, 2019). for many years, scientists and science educators have agreed on the aim of helping students improve informed nos (abd-el-khalick et al., 1998). therefore, any reforms should be made in the curriculum and the pedagogical field to improve students' understanding of the nos in science education (aaas, 1993; nrc, 1996; nsta, 2000). however, the researchers state that students, teacher candidates, and teachers from kindergarten to 12th grade (k-12) did not reach the desired understanding about nos (abd-el-khalick & lederman, 2000; önen öztürk, 2019). therefore, it is seen that it is crucial to increase the number of studies aimed at gaining the perspective of the nos at all grades starting from the pre-school period to 12th grade (k-12). when pre-school age is considered the period when the nos is founded, pre-school teachers have significant responsibilities in having students acquire the elements of the nos, adapting the targeted dimensions into the instructional process, and implementing ways of them. moreover, in pre-school, children conduct preliminary scientific experiments, observe the experimental process, interpret the events, and use their imagination and creativity. therefore, the quality of teachers and the activities are essential for these children's quality of instruction. therefore, it is crucial to teach nos and its components to pre-school teachers and prospective teachers in this sense. in the literature, there are some approaches for teaching nos. 1.1. teaching approaches nature of science three approaches, such as historical, implicit, and explicit-reflective, were used to teach nos for students, pre-service, and in-service teachers (lederman, 1992). in the historical approach, to students understand the nos, it is discussed how and under what conditions scientists worked in history. in addition, a historical approach can be used to show the change of sk in the historical process and to understand the nos by using the history of science (khishfe & abd-el-khalick, 2002). the implicit approach argues that the nos does not require much effort to teach teachers or students. instead, they can learn nos spontaneously by including them in scientific activities (abd-el-khalick & lederman, 2000). finally, the explicitreflective approach considers that a direct effort is required to teach nos contrary to the implicit approach. according to this approach, while teaching the nos, activities involving scientific research skills, scientific journal of science learning article doi: 10.17509/jsl.v5i1.33190 167 j.sci.learn.2022.5(1).165-175 discussions, and models are carried out with teachers or students. then, the students or teachers tried to comprehend the nos by giving feedback on the activities (gess-newsome, 2002). in many recent studies, it is emphasized that the explicit-reflective teaching approach has positively changed the teachers' thoughts about the concept of nos, and this approach is more effective than other approaches (akerson & abd-el-khalick, 2005; akerson & hanuscin, 2007; akerson & volrich, 2006; bell, lederman, & abd-elkhalick, 2000; moss, abrams, & robb, 2001). in order to improve the effectiveness of the explicitreflective approach, assessment procedures can also be used in teaching nos and sk. for example, an assessment may be diagnostic, summative, and formative, but formative assessment is used during the instructional process. 1.2. formative assessment the formative assessment, which came to the forefront with the book working inside the black box by black & william (1998a), is expressed as an assessment for learning without any giving marks to students (black & william, 2004; black & william, 2009; keeley, eberle, & farrin, 2005). black & william (1998a), black & william (1998b), and bryant & timmins (2002) define formative assessment as a cycle in which the assessment data is used for improving learning and assessment is carried out again. the fundamental aim of this assessment is to have the students develop desired behaviors and guide students (black & wiliam, 1998b; bell & cowie, 2001; gipps, 1994; metin, 2014). for this aim, teachers give feedback to students about the deficient and insufficient points of the activities they conduct (black & wiliam, 1998a; bell & cowie, 2001). while students understand their deficient and insufficient aspects and direct their studies with the feedback, teachers decide how to continue instruction according to students' learning conditions (bell & cowie, 2001; brookhart, 2001). this formative assessment helps teachers have information about their students' learning, decide the following instruction step, and help students learn more effectively (clarke, 2001). the most critical matter that should be considered is that formative assessment should be integrated into instruction and continuous (torrance & pryor, 1998). a common conclusion in the literature is that the assessment type directs the learning (biggs & watkins, 1996). they effectively increase the level and quality of learning when used properly (black & william, 2002; clarke, 2001). many researchers state that formative assessment increases the quality and level of students' learning (ali & iqbal, 2013; black & wiliam, 1998a; 1998b; black, harrison, lee, marshall, & william, 2002; clarke, 2001; crooks, 1988; gallagher, 2000; harlen, brand, & brown, 2003; stiggins & conklin, 1992; torrance & pryor, 1998). an effective formative assessment involves; sharing the learning aim with students, giving feedback to students to correct their mistakes and complete their deficiencies, helping students investigating their own progress to find the deficiencies, and increasing the motivation and selfconfidence, which are rather important for effective learning and development, by using proper assessment techniques (alarg, 2002). therefore, it is possible to say that these activities can be used as a method in the context of formative assessment. thus, black and william (1998a), black and william (2002), brookhart (2001), clarke (2001) and harlen and james (1997) found that formative assessment; increase the learning quality and level of students, help them learn effectively, increase their motivation and have students learn self-assessment. besides, harlen et al. (2003) state that formative assessment is highly effective in realizing students’ research-based learning. llewellyn (2002) and bonner (2005) express that research-based learning teaches students scientific research, studies, and how to think like a scientist. it is essential that teachers who work in pre-school have enough information about nos and correctly explain nos to students in order for early childhood students to comprehend the nature of science fully. for prospective teachers who will teach in pre-school after graduation to comprehend nos correctly, it is necessary to provide practical training to explain nos. therefore, it is essential to conduct activities that clearly emphasize the nature of science lessons to pre-service teachers and provide them with feedback on applications to learn better. it is thought to become more effective by using formative assessment related to nos and sk teaching activities in the teaching process. in this context, this study aims to compare the effect of formative assessment with explicit-reflective instruction and explicit-reflective on pre-school prospective teachers' views about nos and sk. this study is thought to be significant for integrating formative assessment with explicit-reflective instruction of nos. furthermore, this aspect presents an example for researchers who conduct research on nos and teacher educators who train preschool and science teachers. 2. method as the prospective pre-school teachers could not be grouped randomly within the research, it was decided to use a quasi-experimental design. the quasi-experimental design has been used in this study because it allows randomly deciding whether one or more of the pre-formed groups are experimental and one or more of the control groups when random grouping is not possible (creswell, 2005; judd, smith, & kidder, 1991). this study used a pretest-posttest nonequivalent control group design (fraenkel & wallen, 2006). this design has an experimental and a control group, and the groups are determined randomly. in the design, a pre-test is implemented for both groups, journal of science learning article doi: 10.17509/jsl.v5i1.33190 168 j.sci.learn.2022.5(1).165-175 experimental intervention is carried out with an experimental group, no specific intervention is done for the control group, and a post-test is applied for both groups again (creswell, 2005; fraenkel & wallen, 2006). in this study, one group of prospective pre-school teachers in 2nd grade was assigned randomly as an experimental group and the other as a control group. 2.1. research sample the study sample consists of 66 prospective pre-school teachers in the 2nd grade. fifty-six of the pre-service teachers are female, and ten are male, and their average age is between 21 and 25 years old. thirty-three pre-school prospective teachers were assigned the control group among these students as the experimental group found by the other pre-school prospective teacher. pre-school prospective teachers in both groups come from similar educational and socio-economic backgrounds. the researcher applied an application related to the nature of science in experimental and control groups. the researcher had 11 years experience in teaching science, a doctorate in science education, and carried out some research regarding the nature of science and science process skills. 2.2. study context the study was carried out in science education course in the fifth semester of the pre-school education program. the course is instructed four hours a week, two theoretical and two practices, and the term has 14 weeks. the course covers; the subjects like the importance of science and nature, the instruction of basic science concepts and scientific thinking skills in the pre-school period, preparation of activities and materials for nos and scientific process skills, and the implementation of these materials. the seven essential components handled in science education course about nos and scientific process skills are as follows; scientific knowledge is; i) tentative, ii) experimental, iii) subjective, iv) a result of imagination and creativity, v) affected by social and cultural structure, vi) related with observation and implication, vii) related with law and theory. in this study, prospective pre-school teachers were informed about these seven components, and implementations were carried out. then, prospective pre-school teachers designed activities for pre-school children in order to have them gain these essential components. 2.3. data collection tool “nature of science scale (noss)”, which was developed by özgelen (2013), and “attitude scale towards scientific knowledge (sks)", developed by ayvacı (2007), were used in this study. nature of science scale the noss consisted of 30 items developed by özgelen (2013), and it was applied to 655 pre-service teachers in four different universities. exploratory factor analysis was made by using the spss program, the scale was found to have five factors, and the total number of items was found to be 19. the reliability of the scale as a result of all was .83. furthermore, a sample of 391 pre-service teachers, confirmatory factor analysis was conducted with the amos program. according to the confirmatory factor analysis results, χ2/df rate was 0.83. this ratio (0.83) shows that the measurement model fits the data well. some examples of the scale items are given in table 1. scientific knowledge scale this scale is designed to evaluate the students' thoughts about scientific knowledge quantitatively. the scale contains 25 items about sk, which are in five-point likert type. each item of the scale was developed by taking the components of science and nature of sks and the characteristics in the literature. after the analysis by a language expert, the scale was applied. then, the data were analyzed by the spss program, and factor load value (.515‒ .885) and total variance percent (42%) were calculated for each item. the internal consistency of the scale was found to be 0.73. some examples of the items in the scale are given in table 2. 2.4. instructional materials four activities such as i) the investigation of the cubes, ii) old teacher, iii) young or old?, iv) sheet roll performed in the class to table 1 examples of the nature of science scale items item characteristics of scientific knowledge 5 scientific knowledge changes in time. tentative 7 scientific knowledge is created only as a result of experiments and objective observations. empirical 9 the studies of scientists are influenced by their thought about the same subject. subjective 10 scientists use imagination while creating scientific knowledge. creative 13 scientific questions and methods change according to historic conditions. social-cultural 18 the two scientists observing the same room may have different inferences. relationship between observations and inferences 25 scientific theories turn into laws in time. relationship between theory and law journal of science learning article doi: 10.17509/jsl.v5i1.33190 169 j.sci.learn.2022.5(1).165-175 have prospective pre-school teachers in experimental and control groups comprehend the nos. i) the cubes investigation: in this activity, prospective preschool teachers were given a cube with names and numbers on their sides. they were asked to discover the relations between the names and numbers and explain them by presenting pieces of evidence. the question examined in this activity was, "what is there on the bottom of the cube?" the activity lasted nearly 90 minutes, and preservice teachers tried to comprehend the experimental, tentative, imaginative, and creative nature of science and the difference between observation and inference. ii) old teacher: this activity focuses on the fact that "the knowledge and expectation that scientist have while working may influence the way of inferencing it." the activity lasted 45 minutes, and some pictures were shown to students which were said to belong to a teacher who started teaching recently. by examining these pictures, prospective pre-school teachers were asked to recognize the changes on the teacher's face. at the end of the activity, prospective pre-school teachers were told that the picture belonged to a woman, and they were asked why they saw the teacher, not the woman. this activity aims to have preservice teachers learn the difference between observation and inference and gain the theoretical and socio-cultural nature of science. iii) old or young?: this activity focuses on the fact that scientists can see different things while looking at the same data or events. the activity took 45 minutes. first, pictures that show differences in different perspectives were shown to students, and they were asked to say the thing they saw. the discussions on the pictures stressed that two individuals looking at the same thing might see different things. this activity aims to have prospective pre-school teachers learn the difference between observation and inference and gain the subjective, theoretical and sociocultural nature of science. iv) sheet roll: this activity presents an experiment to students with a system prepared by using sheet rolls and how it works. then, prospective pre-school teachers were asked to think about the internal structure of the roll by working together and setting up a system similar to the activity by using the rolls given to them. next, the prospective pre-school teachers worked in groups, generated hypotheses about the internal structure of the roll by discussing their ideas, and designed models based on this hypothesis. after that, each team explains the model by presenting how it works. this activity aims to have pre-service teachers learn the difference between observation and inference and gain the tentative, experimental, imaginative, and creative nature of science. the researcher performed these activities with experimental and control groups in two weeks and 8 hours. in addition, the researcher had the role of facilitator during the activities and encouraged prospective pre-school teachers to discuss activities. 2.5. implementation process within the research context, an explicit-reflective teaching approach was applied to teach the control group the nature of science and the properties of scientific knowledge. the activities of "the investigation of the cubes", "old teacher", "old or young?", "sheet roll" were carried out by the academician at eight weeks following direct reflective teaching. the experimental group was explained all of the activities in the control group using the direct reflective learning method. in addition, the experimental group was asked to prepare activities to teach the nature of science at the end of each activity. pre-school prospective teachers were asked to present their activities in the classroom environment. during the presentation, the academician provided feedback on whether the activities reflected the nature of science. pre-school prospective teacher rearranged their activities according to this feedback. the activities of the experimental and control groups during 15 weeks are given in table 3. 2.6. data analysis the data obtained using achievement tests as the pretest, post-test, and retention tests have been analyzed using the spss program. because the groups have been randomly assigned before the implementation and the data collection tool is an interval scale, a t-test has been used in data analysis. in addition, the kolmogorov-simonov test was applied to determine whether the data had a normal distribution, and results showed that they presented a normal distribution (p > 0.05). levene test was used for testing the homogeneity of variances of experimental and control groups. because the result of the levene test was higher than 0.05, it was specified that both groups were equal. independent samples t-test was used to compare the experimental and control group's pre-test, post-test, and retention test results. paired samples t-test was used to analyze the pre-test, post-test, and post-test retention test average point differences of each group's test results. the table 2 examples of the scientific knowledge scale items item 2 science may testify something, solve a problem or find the answer to a question. 4 most scientists work on their own. 10 scientists have solved most of the great mysteries of nature. 15 science may research things and events even millions of years. 18 the scientist's race, gender, nationality, and religion may influence their work. 20 scientist mostly tries to disprove their thoughts. 25 the conflict between scientists is one of the weaknesses of science. journal of science learning article doi: 10.17509/jsl.v5i1.33190 170 j.sci.learn.2022.5(1).165-175 level of significance was considered as p = .05. in order to comment on a test result, considering only the significance level is not sufficient (yıldırım & yıldırım, 2011). the result may be meaningful, but its effect may be low. effect size is calculated in different ways in different tests (meline & wang, 2004), and cohen's d was calculated in this study. effect size is considered as; low between 0‒0.2, moderate around 0.5, and high 0.8 and above (cohen, 1988). the data obtained in this study have been interpreted by considering correlation, mean, standard deviation, p-value, and cohen's d. table 3 implementation process of learning activities weeks activities in experimental group activities in control group week 1 noss and sks were implemented as a pre-test in the first course. the implementation of each scale took 20 minutes. noss and sks were implemented as a pre-test in the first course. the implementation of each scale took 20 minutes. week 2 pre-school prospective teachers were informed about science and scientific knowledge and what type of characteristics science-literate people have. besides, they were informed about what nos means and characteristics of sk such as tentative, experimental, subjective, experimental, imaginative, creative, socio-cultural, and its relation with observation and inference, and law and theory. pre-school prospective teachers were informed about science and scientific knowledge and what type of characteristics science-literate people have. besides, brief information was given to them about nos and the characteristics of sk. week 3 the cubes activity analysis was performed to have prospective pre-school teachers comprehend the tentative, experimental, imaginative, and creative nos and the difference between observation and inference. the group members also had discussions during the activity. besides, prospective teachers were asked to determine a science subject at the pre-school level and perform an activity to have prospective pre-school teachers comprehend the tentative, experimental, imaginative, and creative nos and the difference between observation and inference. pre-school prospective teachers were informed about science's tentative, experimental, imaginative, and creative nature. week 4 the activities that prospective pre-school teachers designed were discussed in the class, and feedback was given to them about how to correct deficiencies and mistakes and make the activity better. in this process, the prospective pre-school teachers' activities were evaluated using formative assessment. pre-school prospective teachers were informed about the socio-cultural influences on scientific knowledge, its relations with observation and inference, theory, and law. week 5 old teacher activity was performed to have prospective pre-school teachers comprehend the theoretical and socio-cultural nos and the difference between observation and inference. the prospective pre-school teacher also had discussions on activities. besides, they were asked to design an activity for these characteristics of science. the cubes activity analysis was performed to have prospective pre-school teachers comprehend the tentative, experimental, imaginative, and creative nature of science and the difference between observation and inference. the group members also had discussions during the activity. week 6 the activities that prospective pre-school teachers designed were discussed in the class, and feedback was given to them about how to correct deficiencies and mistakes and make the activity better. old teacher activity was performed to have prospective pre-school teachers comprehend science's theoretical and socio-cultural nature and the difference between observation and inference. the students also had discussions on activities. week 7 "old or young?" activity was performed to have prospective pre-school teachers comprehend the subjective, theoretical and socio-cultural nature of science and the difference between observation and inference. the prospective pre-school teachers also had discussions on activities. besides, they were asked to design an activity for these characteristics of science. "old or young?" activity was performed to have prospective pre-school teachers comprehend the subjective, theoretical and socio-cultural nature of science and the difference between observation and inference. the prospective pre-school teachers also had discussions on activities. week 8 the pre-school prospective teacher-designed activities were discussed in the class, and feedback was given to them about how to correct deficiencies and mistakes and make the activity better. "sheet roll" activity was performed to have prospective pre-school teachers comprehend the tentative, experimental, imaginative, and creative nature of science and the difference between observation and inference. the prospective preschool teachers also had discussions on activities. week 9 "sheet roll" activity was performed to have prospective pre-school teachers comprehend the tentative, experimental, imaginative, and creative nos and the difference between observation and inference. the prospective preschool teachers also had discussions on activities. besides, they were asked to design an activity for these characteristics of science. pre-school prospective teachers continued sheet roll activity. week 10 the activities that prospective pre-school teachers designed were discussed in the class, and feedback was given to them about how to correct deficiencies and mistakes and make the activity better. the noss and sks were implemented as posttest. week 11 the noss and sks were implemented as post-test. week 15 the noss and sks were implemented as retention tests. the noss and sks were implemented as retention tests. journal of science learning article doi: 10.17509/jsl.v5i1.33190 171 j.sci.learn.2022.5(1).165-175 3. findings the findings of the study were analyzed in five steps. in the first one, the results of independent samples t-test applied to means of pre-tests of experimental and control groups have been given in table 4. table 4 shows that there is not a difference between the pre-test variances of experimental and control groups. while pre-test mean of experimental group is higher than the control group’s (x̄experimental = 87.09; x̄control = 86.53;) in noss, there is not statistically a significant difference between pre-test means of experimental and control groups (t = .446; p > 0.05). besides, the pre-test mean of the experimental group is higher than the control group's (x̄experimental = 99.09; x̄control = 97.97) in sks. however, this difference is not significant in statistical terms (t = .931; p > 0.05). the independent samples t-test applied to means of post-tests of experimental and control groups have been given in table 5. table 5 shows that there is not a difference between the post-test variances of experimental and control groups. however, the means of pre-service teachers in the experimental group in noss is higher than the control group's (x̄experimental = 101.68; x̄control = 92.47). the results of the t-test show that there is a significant difference (t = 6.964; p < 0.05) in favor of the experimental group. also, the means of pre-service teachers in the experimental group in sks is higher than the control group's (x̄experimental = 108.15; x̄control = 100.71). the results of the t-test show that there is a significant difference (t = 5.620; p < 0.05) in favor of the experimental group. the independent samples t-test applied to means of retention tests of experimental and control groups have been given in table 6. table 6 shows that there is not a difference between the post-test variances of experimental and control groups. however, the means of pre-service teachers in the experimental group in noss as a retention test is higher than the control group's (x̄experimental = 100.47; x̄retention = 90.59). the results of the t-test show that there is a significant difference (t = 17.286; p < 0.05) in favor of the experimental group. also, the means of pre-service teachers in the experimental group in sks as a retention test is higher than the control group's (x̄experimental = 106.50; x̄retention = 99.15). the results of the t-test show that there is a significant difference (t = 8.883; p < 0.05) in favor of the experimental group. the results of paired samples t-test applied to means of pre-tests and post-tests of experimental and control groups have been given in table 7. table 4 independent samples t-test results of experimental and control group’s pre-test pre-test results experimental group control group t p x ̄ sd x ̄ sd noss 87.09 5.817 86.53 4.433 .446 .657 sks 99.09 5.749 97.97 3.989 .931 .355 table 5 independent samples t-test results of experimental and control group’s post-test post-test results experimental group control group t p x ̄ sd x ̄ sd noss 101.68 4.013 92.47 6.964 6.678 .00 sks 108.15 5.930 100.71 4.945 5.620 .00 table 6 independent samples t-test results of experimental and control group’s post-test retention test results experimental group control group t p x ̄ sd x ̄ sd noss 100.47 2.957 90.59 1.540 17.286 .00 sks 106.50 4.406 99.15 1.971 8.883 .00 table 7 paired samples t-test results of experimental and control group’s pre-test and post-test groups tests noss r sks r x ̄ sd t p x ̄ sd t p experimental group pretest 87.09 5.817 12.036 .00 .83 99.09 5.749 6.396 .00 .61 posttest 101.68 4.013 108.15 5.930 control group pretest 86.53 4.433 4.196 .00 .41 98.56 3.791 2.009 .004 .24 posttest 92.47 6.964 100.71 4.945 journal of science learning article doi: 10.17509/jsl.v5i1.33190 172 j.sci.learn.2022.5(1).165-175 table 7 shows that post-test means of experimental group in noss and sks (x̄noss = 101.68; x̄sks = 108.15) are higher than its pre-test means (x̄noss = 87.09; x̄sks = 99.09). it has been found that experimental group has a significant increase in post-test results (tnoss = 12.036; p < 0.05; tsks = 6.396; p < 0.05) and the effect sizes (rnoss = .83; rsks = .61) of this increase are in high level. the table also shows post-test means of control group (x̄noss = 92.47; x̄sks = 100.71) are higher than its pre-test means (x̄noss = 86.53; x̄sks = 98.56). it has been found that control group has a significant increase in post-test results (tnoss = 4.196; p < 0.05; tsks = 2.009; p < 0.05) and the effect sizes (rnoss = .41; rsks = .24) of this increase are in high level. the research results show a statistically significant difference between both groups' pre-test and post-test results. however, it was determined that the effect size of the experimental group was more significant than the control group in the noss and sks tests. the results of paired samples t-test applied to means of post-tests and retention tests of experimental and control groups have been given in table 8. table 8 shows that post-test means of experimental group in noss and astsk (x̄noss = 101.68; x̄sks = 108.15) are higher than its retention test means (x̄noss = 100.47; x̄sks = 106.50). however, a significant difference has not been found (tnoss = 1.411; p > 0.05; tsks = 1.300; p > 0.05) between post-tests and retention tests. the table also shows post-test means of control group (x̄noss = 92.47; x̄sks = 100.71) are higher than its retention test means (x̄noss = 90.59; x̄sks = 99.15). however, these differences are not significant (tnoss = .1539; p > 0.05; tsks = 1.707; p > 0.05) in statistical terms. 4. results and discussion an extensive literature review shows that perceptions about nos and sk substantially affect science learning in students and teachers (abd-el-khalick & lederman, 2000; akerson & abd-el-khalick, 2005; altındağ, 2010; khishfe & abd-el-khalick, 2002; lederman, 1992; moss et al., 2001). this study aims to compare the effect of formative assessment embedded with explicit-reflective instruction and explicit-reflective on pre-school prospective teachers' views about the nature of science and scientific knowledge. when the results were obtained from the noss and sks, which were applied as a pre-test to prospective preschool teachers in the experimental and control groups, it was observed that there was a homogeneous distribution. furthermore, the pre-test found no statistically significant difference between the experimental and control groups in both scales. this means both groups had similar features before the implementation process. equivalence of the sample group is essential for the correct interpretation of the data obtained during the experimental research process (creswell, 2005; fraenkel & wallen, 2006). in this respect, the result obtained is essential in interpreting the study results. at the end of a 10-week experimental process, the noss and sks, applied as a pre-test, were re-applied as a post-test. as a result of this application, when the control group's pre-test and post-test results are compared, there is a statistically significant increase in favor of the post-test. according to this result, it is seen that the explicit-reflective learning approach contributes to a positive change in preschool prospective teachers' thoughts about the nos and sk. it supports the results obtained from many studies in the literature. in many studies using the explicit-reflective teaching method, it was concluded that students, prospective teachers, and teachers learned the nos better than other methods (abd-el-khalick & lederman, 2000; khisfe & abd-el-khalick, 2002). it is also stated that learning the nature of science has an important effect on understanding sk (abd-el-khalick & lederman, 2000; akerson & abd-el-khalick, 2005; akerson & hanuscin, 2007; akerson & volrich, 2006; bell, lederman, & abdel-khalick, 2000; khishfe & abd-el-khalick, 2002; moss et al., 2001). besides, when the experimental group's pre-test and post-test results are compared, there is a statistically significant increase in favor of the post-test. according to this result, it is seen that the formative assessment embedded with the explicit-reflective learning method contributes to a positive change in pre-school prospective teachers' thoughts about the nos and sk. in the literature, it is stated that formative assessment has a positive effect on students' learning, helps prospective pre-school teachers gain research skills, and students who gain scientific research skills reach scientific knowledge like scientists (bonner, 2005; harlen et al., 2003; llewellyn 2002; metin, 2014; metin & birişçi, 2009; metin & özmen, 2010). in this respect, it can be said that formative table 8 paired samples t-test results of experimental and control group’s post-test and retention test groups tests noss sks x ̄ sd t p x ̄ sd t p experimental group post-test 101.68 4.103 1.411 .163 108.15 5.930 1.30 .133 retention 100.47 2.957 106.50 4.406 control group post-test 92.47 6.964 1.539 .240 100.71 4.945 1.707 .095 retention 90.59 1.540 99.15 1.971 journal of science learning article doi: 10.17509/jsl.v5i1.33190 173 j.sci.learn.2022.5(1).165-175 assessment causes an increase in the pre-school prospective teachers' knowledge about the nature of science and scientific knowledge, and in this case, it positively changes their thoughts about nos and sk. in addition, based on the statements that formative assessment increases research and inquiry skills, it is likely that pre-school prospective teachers will better understand the nos, which is based on research and inquiry. it is inevitable that pre-school prospective teachers' acquisitions about the nos and sk will also affect their opinions on this subject. as a result of the research, it was determined that both the methods applied to the control group and experimental group caused a statistically significant change in the students' thoughts about the nos and sk. this result causes us to ask the question "which method is more effective in changing the nos and sk of pre-school prospective teachers?. when the post-test results of the experimental and control groups were compared, it was determined that there was a statistically significant difference between both groups and this difference was in favor of the experimental group. in addition, in order to test this result, the change between the pre-test and posttest results of both groups is examined in terms of effect size. a criterion that shows whether the difference between the results of the groups in the study is significant is the effect size (cohen, 1988). effect size can be expressed as the expected difference between two averages or two ratios according to the outcome variable of interest to reveal a clinically significant difference (meline & wang, 2004). in other words, the effect size is used to determine how much a new method makes a difference compared to the old one (yıldırım & yıldırım, 2011). when it was seen effect size score, it was determined that formative assessment embedded with explicit-reflective teaching instruction caused more positive changes in pre-school prospective teachers' thoughts about the nos and sk than explicitreflective teaching instruction (table 7). according to this result, it can be said that when formative assessment and explicit-reflective teaching instruction are used together, the positive aspects of both methods combine to affect better learning of concepts such as the nos science and sk. it is essential but not sufficient for prospective preschool teachers to learn the nature of science and scientific knowledge. for this, pre-school prospective teachers' thoughts on the nos and sk must be permanent. it is important that they do not forget what they learned, even after a certain period of experimental intervention. for this, it is necessary to test the persistence of pre-school prospective teachers' nos and sk. in order to determine whether the use of formative assessment with explicitreflective instruction has a permanent effect on pre-school prospective teachers' thought on nos and sk, the scales have been administered after four weeks. the results have shown a significant difference between experimental and control groups. the students in the experimental group have had higher means than the pre-school prospective teachers in the control group. besides, the comparison of retention tests with post-tests in both groups reveals that there is not a significant difference between the results of these tests in experimental and control groups. this result shows that formative assessment with explicit-reflective instruction has a permanent effect on pre-school prospective teachers' thought nos and sk. according to these results, it can be stated that the formative assessment embedded with explicit-reflective instruction has more positively and permanently changed pre-school prospective teachers' views about nos and sk than explicit-reflective teaching instruction. conclusion as a result of the study, it was determined that both formative assessments embedded with explicit-reflective instruction and explicit-reflective instruction changed preschool prospective teachers' thoughts about the nos and sk positively and permanently. however; it was seen that the formative assessment embedded with explicitreflective instruction was more positively and permanently changed on pre-school prospective teachers' view about nos and sk than explicit-reflective instruction. references abd-el-khalick, f., bell, r. l., & lederman, n. g. (1998). the nature of science and instructional practice: making the unnatural natural. science education, 82(4), 417-436. abd-el-khalick, f., & lederman, n. g. (2000). improving science teachers’ conceptions of the nature of science: a critical review of the literature. international journal of science education, 22, 665–701. akerson, v. l., & abd-el-khalick, f. (2005). how should i know what scientists do?—i am just a kid: fourth grade students’ conceptions of nature of science. journal of elementary science education, 17, 1–11. akerson, v. l., & hanuscin d. l. (2007). teaching nature of science through inquiry: results of a 3-year professional development program. journal of research in science teaching, 44(5), 653–680. akerson, v. l., & volrich, m. (2006). teaching nature of science explicitly in a first grade internship setting. journal of research in science teaching, 43, 377-394. altındağ, c., (2010). öğretmen adaylarına bilimin doğasını öğretmeye yönelik bir çalışma [a study to teach pre-service teachers the nature of science]. (unpublished master's thesis). pamukkale üniversitesi fen bilimler enstitütüsü, denizli. ali, i., & iqbal, m. h. (2013). effect of formative assessment on students’ achievement in science. world applied sciences journal, 26(5), 677-687. american association for the advancement of science (aaas). (1993). benchmarks for science literacy. new york: oxford university press. assessment for learning assessment reform group (alarg). (2002). pamphlet assessment for learning–beyond the black box. retrieved from http://www.assessment-reform-group.org.uk ayvacı, h. ş. (2007). bilimin doğasının sınıf öğretmeni adaylarına kütle çekim konusu içerisinde farklı yaklaşımlarla öğretilmesine yönelik bir çalışma [a study on teaching the nature of science to prospective classroom teachers with different approaches within the subject of gravity]. (unpublished doctoral thesis). karadeniz teknik üniversitesi, fen bilimleri enstitüsü, trabzon. http://www.assessment-reform-group.org.uk/ journal of science learning article doi: 10.17509/jsl.v5i1.33190 174 j.sci.learn.2022.5(1).165-175 black, p., harrison, c., lee, c., marshall, b., & wiliam, d. (2004). working inside the black box: assessment for learning in the classroom. phi delta kappan, 86(1), 8-21. black, p., & william, d. (1998a). inside the black box, raising standards through classroom assessment. phi delta kappan, 80(2), 139-148. black, p., & william, d. (1998b). assessment and classroom learning, assessment in education, 5(1), 7-75. black, p., & william, d. (2002). improved standards achieved by transforming assessment for learning. news archive: kings college london. black, p., & william, d. (2004). the formative purpose: assessment must first promote learning. in m. wilson (ed.), towards coherence between classroom assessment and accountability (pp. 20–50). chicago, il: national society for the study of education. black, p., & wiliam, d. (2009). developing the theory of formative assessment. educational assessment. evaluation and accountability, 21, 5–31. bell, b., & cowie, b. (2001). the characteristics of formative assessment in science education. science education, 85(5), 536-553. bell, r. l., lederman, n. g., & abd-el khalick, f. (2000). developing and acting upon one’s conception of the nature of science: a follow-up study. journal of research in science teaching, 37(6), 563–581. biggs, j. b., & watkins, d. (1996). classroom learning: educational psychology for the asian teacher. new york: prentice-hall. bonner, j. j. (2005). which scientific method should we teach & when?. the american biology teacher, 67(5): 262-264. brookhart, m. s. (2001). successful students’ formative and summative uses of assessment information. assessment in education: principles, policy & practice, 8(2), 153-169. bryant, s., & timmins, a. (2002). portfolio assessment: an instructional guide (2nd edition). hong kong: department of curriculum & instruction, hkied, childers. bybee, r. w. (1997). achieving scientific literacy: from purposed to practices. portsmouth, nh: heinemann. clarke s. (2001). unlocking formative assessment: practical strategies for enhancing pupil’s learning in the primary classroom. london: hodder & stoughton educational. cohen, j. (1988). statistical power analysis for the behavioral sciences (2nd ed.). hillsdale, nj: lawrence earlbaum associates. creswell, j. w. (2005). educational research: planning, conducting, and evaluating quantitative and qualitative research (2nd ed.). upper saddle river, nj: pearson education. crooks, t. j. (1988). the impact of classroom evaluation practices on students. review of educational research, 58(4): 438-481. deboer, g. e. (2000). scientific literacy: another look at its historical and contemporary meanings and its relationship to science education reform. journal of research in science teaching, 37, 582–601. deniz, h., & adibelli, e. (2015). exploring how second grade elementary teachers translate their nature of science views into classroom practice after a graduate a level nature of science course. research in science education, 45(6): 867-888. eş, h., & sarıkaya, m. (2010). i̇lköğretim 6.sınıf fen ve teknoloji dersi “yaşamımızdaki elektrik” ünitesi kazanımları ile ilgili öğretmen görüşlerinin değerlendirilmesi [evaluation of teachers' opinions about the achievements of the unit "electricity in our lives" in the 6th grade science and technology lesson in primary education.]. ejournal of new world sciences academy education sciences, 6(1), 32-45. fraenkel, j. r., & wallen, n. e. (2006). how to design and evaluate research in education. newyork: mcgraw-hill. gallagher, j. j. (2000). teaching for understanding and application of science knowledge. school science and mathematics, 100(6), 310-318. gess-newsome, j. (2002). the use and ımpact of explicit ınstruction about the nature of science and science ınquiry in an elementary science methods course. science & education, 11, 55–67. gipps, c. (1994). beyond testing: towards a theory of educational assessment. london: the falmer press. güneş, m. h., & karaşah, ş. (2016). geçmişten günümüze fen eğitiminin önemi ve fen eğitiminde son yıllarda yapılan çalışmalar [the importance of science education from past to present and studies in science education in recent years]. journal of research in education and teaching, 5(3), 122-136. hanuscin, d., lee, m. h., & akerson, v. l. (2011). elementary teachers’ pedagogical content knowledge for teaching the nature of science. science education. harlen, w., brand, j., & brown, r. (2003). enhancing inquiry through formative assessment. san francisco, ca: exploratorium. harlen, w. & james, m. (1997), assessment and learning: differences and relationships between formative and summative assessment. assessment in education: principles, policy & practice, 4(3): 365 –379. judd, c. m., smith, e. r., & kidder, l. h. (1991). research methods in social relations (international edition). fort worth, tx: holt, rinehart, & winston. kang, s., scharmann, l. c. & noh, t. (2005). examining students' views on the nature of science: results from korean 6th, 8th, and 10th graders. science education, 89(2005, 314-334. keeley, p., eberle, f., & farrin, l. (2005). uncovering student ıdeas in science, 25 formative assessment probes. california: corwin & nsta press. khishfe, r., & abd-el-khalick. (2002). influence of explicit and reflective versus implicit inquiry-oriented instruction on sixth graders' views of nature of science. journal of research in science teaching, 39(7), 551–578. lederman, n. g. (1992). students’ and teachers’ conceptions of the nature of science: a review of the research. journal of research in science teaching, 29(4),331–359. lederman, n. g., abd-el-khalick, f., bell, r. l., & schwartz, r. s. (2002). views of nature of science questionnaire: toward valid and meaningful assessment of learners’ conceptions of nature of science. journal of research in science teaching, 39(6), 497-521. llewellyn, d. (2002). inquiry within: implementing inquiry-based science standards. usa: corwinn pres, inc. a sage publications company. mccomas, w. f. (1998). the principal elements of the nature of science: dispelling the myths. in w. f. mccomas (eds.). the nature of science in science education: rationales and strategies (pp. 53–70). dordrecht, the netherlands: kluwer academic publishers. mccomas, w. f., clough, m. p., & almazora, h. (1998).the role and character of the nature of science in science education. in w. f. mccomas (eds.). the nature of science in science education: rationales and strategies (pp. 3–39). dordrecht, the netherlands: kluwer academic publishers. meline, t., & wang, b. (2004). effect-size reporting practices in ajslp and other asha journals, 1999-2003. american journal of speechlanguage pathology, 13, 202-207. metin, m. (2014). okul öncesi dönemde fen eğitiminde ölçme değerlendirme [assessment and evaluation in pre-school science education]. in m. metin, & ç. şahin (ed.) örnek uygulamalarla okul öncesi dönemde fen eğitimi [science education in pre-school period with sample applications], pp. 321-350. ankara: pegema publishing. metin, m., & birişçi, s. (2009). biçimlendirici değerlendirmenin öğretmen adaylarının bilimsel süreç becerilerini geliştirmeye etkisi ve adayların değerlendirme hakkındaki düşünceleri [the effect of formative assessment on improving pre-service science teachers' scientific process skills and their thoughts on assessment.]. çağdaş eğitim dergisi, 34(370), 31-39. metin, m., & özmen, h. (2010). biçimlendirici değerlendirmeye yönelik öğretmen adaylarının düşünceleri [pre-service teachers' thoughts on formative assessment]. milli eğitim dergisi, 187, 293-310. mıhladız, g., & doğan, a. (2017). fen bilgisi öğretmen adaylarının bilimin doğası konusundaki pedagojik alan bilgilerinin araştırılması [investigation of science teacher candidates' pedagogical content knowledge on the nature of science]. hacettepe üniversitesi eğitim fakültesi dergisi, 32(2), 380-395. moss, d. m., abrams, e. d., & robb, j. (2001). examining student conception of the nature of science. international journal of science education, 23(8), 771-790. national research council (nrc). (1996). in m. s. e. board (ed.), mathematics and science education around the world: what can we learn survey of mathematics and science opportunities (smso) and the third international journal of science learning article doi: 10.17509/jsl.v5i1.33190 175 j.sci.learn.2022.5(1).165-175 mathematics and science study (timss)?. washington, dc: national academy press. national science teachers association (nsta). (2000). nsta position statement: the nature of science. retrieved march 18, 2003, from www.nsta.org/159&psid=22 önen öztürk, f. (2019). fen bilimleri öğretmen adaylarının hazırladığı bilim tarihi temelli dramaların i̇ncelenmesi [analysis of science history-based dramas prepared by science teacher candidates]. uludağ üniversitesi eğitim fakültesi dergisi , 32(2). özgelen, s. (2013). bilimin doğası ölçeğinin geliştirilmesi [development of the nature of science scale]. kastamonu eğitim dergisi, 21(2), 711736. prima, e. c., utari, s., chandra, d. t., hasanah, l., & rusdiana, d. (2018). heat and temperature experiment designs to support students' conception of the nature of science. journal of technology and science education, 8(4), 453-472. https://doi.org/10.3926/jotse pomeroy, d. (1993). implications of teachers’ beliefs about the nature of science: comparison of the beliefs of scientists, secondary science teachers, and elementary teachers. science education, 77(3), 261–278. roberts, d. a. (2007). scientific literacy/science literacy. in s. k. abell & n. g. lederman (eds.), handbook of research in science education. englewood cliffs, nj: erlbaum publishers. ryan, a. g., & aikenhead, g. s. (1992). students’ preconceptions about the epistemology of science. science education, 76, 559–580. stiggins, r. j. & conklin, n. (1992). in teachers’ hands: investigating the practices of classroom assessment. state university of new york press. torrance, h., & pryor, j. (1998). investigating formative assessment: teaching, learning and assessment in the classroom. philadelphia: open university press. yıldırım, h. h. & yıldırım, s. (2011). hipotez testi, güven aralığı, etki büyüklüğü ve merkezi olmayan olasılık dağılımları üzerine [on hypothesis testing, confidence interval, effect size and decentralized probability distributions]. i̇lköğretim online, 10(3), 1112-1123. http://www.nsta.org/159&psid=22 https://doi.org/10.3926/jotse a © 2021 indonesian society for science educator 275 j.sci.learn.2021.4(3).275-287 received: 7 february 2021 revised: 21 april 2021 published: 05 july 2021 gifted students’ beliefs about knowledge and learning oğuzhan nacaroğlu1, oktay kızkapan2* 1science teacher, science and art center, turkey 2department of educational sciences, faculty of education, nevşehi̇r haci bektaş veli̇ university, turkey corresponding author: okizkapan@nevsehir.edu.tr abstract epistemological beliefs can be defined shortly as beliefs about the source, certainty, organization of knowledge, and beliefs on ability and speed of learning. word association tests (wat) are practical alternative assessment and evaluation tools that can reveal students' thoughts on different concepts. in this regard, this research aims to investigate the gifted students' beliefs about "knowledge" and "learning" concepts by using wat. phenomenology design was utilized in the research. the study was carried out with 118 gifted students studying at science and art center in turkey's central east anatolia region in the 2018-2019 academic year. the word association test was used as a data collection tool. in this context, participants were given the concepts "knowledge" and "learning" and asked to write their associations about these concepts. they were then asked to make a sentence about each concept. deductive content analysis was used to analyze the data. the research findings revealed that most students have sophisticated epistemological beliefs in specific knowledge, simple knowledge, source of knowledge, and quick learning dimensions. however, it has been found that the number of students in the sophisticated and naive categories of innate ability dimension is approximately equal. keywords gifted students, epistemological beliefs, word association test, science education 1. introduction defining knowledge is a complicated issue for philosophers and scientists since the classical greek era (alavi & leidner, 2001; eroğlu & güven, 2006). since then, knowledge has been viewed from various views: a state of mind, an object, a process, a condition of accessing information, or a capability. for example, plato defined knowledge as "justified true beliefs". although some other researchers criticize this definition (e.g., gettier, 1963; nozick, 1981; popper, 1979), it is commonly used in philosophical and scientific environments (chisholm, 1982). on the other hand, another term to define is learning. unfortunately, the word "learning" is rarely described explicitly by researchers. this situation could be due to the lack of consensus on what constitutes learning. learning may be defined as the process of bringing about behavioral changes in an individual (ertürk, 1993), and there are multiple theories of how these changes occur. among these, piaget, bruner, gagné, and ausubel's learning theories are frequently referred to in science education (özmen, 2004). an individual's perception of knowledge and learning and his/her beliefs in gaining knowledge are of great importance in defining an individual (chan, 2003). in this regard, the concept of epistemology comes to the fore as a discipline trying to explain what knowledge is and how it is acquired (palmer & marra, 2004; schommer, 1994). epistemological belief is expressed as philosophical assumptions regarding the source, scope, limits, acquisition, and knowledge criteria (hofer & pintrich, 1997; mason & bromme, 2010). epistemological beliefs are subjective belief systems about the source, certainty, creativity, and learning of knowledge. in literature, there are competing theories on epistemological/ epistemic beliefs. in one strand, william perry and other researchers coming after him (magolda, 1992; king & kitchener, 1994; kuhn, cheney & weinstock, 2000) claim that epistemological beliefs are one-dimensional. on the other strand, some researchers claim that epistemological beliefs are multi-dimensional, and these dimensions are more or less independent from each other (schommer, 1990; hofer & pintrich, 1997). there are also some discussions between the researchers in the second strand. schommer (1990) defines an epistemological belief system consisting of five dimensions: simple knowledge, specific knowledge, innate ability, omniscient authority, and quick learning. however, hofer and pintrich (1997) criticize schommer's model mailto:okizkapan@nevsehir.edu.tr journal of science learning article doi: 10.17509/jsl.v4i3.31995 276 j.sci.learn.2021.4(3).275-287 claiming that two of schommer's dimensions are not about epistemological beliefs. they are rather about beliefs about learning. hofer and pintrich (1997) suggest their epistemic belief model consisting of four dimensions under two categories based on this criticism. they named the categories as nature of knowledge (simplicity of knowledge and certainty of knowledge) and knowing (source of knowledge and justification). another group of researchers (greene, azevedo & torney-purta, 2008) criticizes both strands models. green et al. (2008) explained a new theory named "epistemic and ontological cognition." this theory claims that justification constitutes the core of epistemic beliefs. unlike hofer and pintrich's (1997) model, justification of knowing cannot be considered as one dimension. thus, this model's dimensions are personal justification, justification by authority, simple knowledge, and certain knowledge. among all these models, schommer's model is adopted in this study for pragmatic reasons. schommer's model has dimensions about the nature of learning, which is essential in science education settings. educational processes and epistemological beliefs should not be considered independently from each other (tickle, brownlee & nailon, 2005). as the epistemological beliefs of individuals develop, their learning sensitivity and achievement also increase (hofer & pintrich, 1997; cheng, chan, tang, & cheng, 2009). studies show that epistemological beliefs are closely related to learning, teaching processes, construction of knowledge, transfer of knowledge to daily life, and academic achievement (atasoy, 2020; aydın & geçici, 2017; belet & güven, 2011; conley, pintrich, vekiri, & harrison, 2004; hammer, 1997; harteis, gruber & hertramph, 2010; hofer, 2000; qian & alvermann, 2000). individuals with sophisticated epistemological beliefs use effective learning strategies for complex academic processes, insist on solving problems and adopt a deep learning approach (deryakulu, 2004; trautwein & lüdtke, 2007). similarly, dahl, bals, and turi (2005) reported that students' beliefs in simple knowledge and innate ability dimension affect their preferences of cognitive and metacognitive learning styles. indeed, while beliefs about the simple knowledge dimension are essential in selecting rehearsal and organizational strategies, beliefs about innate ability affect selecting detailed and critical thinking strategies. beliefs in both simple knowledge and innate ability dimension are essential for metacognitive monitoring and self-regulation strategies. besides its importance in general pedagogy, epistemological beliefs are also an essential component of science education. researches in science education showed that epistemological beliefs play an essential role in learners' informal (sadler & chambers, 2004; schommer-aikins & hutter, 2002) and scientific reasoning skills (zeineddin & abd-el-khalick, 2010), nature of science views (akerson & buzzelli, 2007; akerson, morrison & roth mcduffie, 2006; deng, chen, tsai, & chai, 2011), argument construction (öztürk yılmaz-tüzün, 2017), constructivist learning environment perceptions (saylan, öner-armağan & bektaş, 2016), acceptance of evolution (deniz & donnelly, 2011; hokayem & boujaoude, 2008; sinatra, southerland, mcconaughy, & demastes, 2003) and cognitive understanding in physics (gök, 2018). students who hold dualist epistemological beliefs are said to view scientific laws and knowledge as certain and proven, not understand the role of inference in science, and believe that scientists are not creative or subjective (akerson & buzzelli, 2007; akerson et al., 2006). hence, when the relationship between students' epistemological beliefs and these critical aspects of science education is accepted, evaluating these beliefs becomes critical. epistemological beliefs are affected by individual and socio-cultural factors (khine, 2008). in other words, when determining the epistemological beliefs of individuals, many factors such as their age, developmental characteristics, interests, and cultural development should be taken into account (wang, zhou & shen, 2016). in this regard, due to the difficulties in determining the epistemological beliefs of young students, epistemological belief studies in educational sciences are primarily conducted with teachers and teacher candidates (aslan, 2017; dorsah, shahadu & kpemuonye, 2020; mardiha & alibakhshi, 2020; lee & jhun, 2020; rott, 2020; tanrıverdi, 2012). however, there has been an increase in studies aimed at determining the epistemological beliefs of younger students in recent years (atasoy, 2020; feucht, 2017; sung, shin & kim, 2020; üztemur & dinç, 2018). nevertheless, there is still a lot to learn about what children believe regarding knowledge and knowing. for example, a group of students with few studies on their epistemological beliefs is gifted students. gifted students are defined as individuals who learn faster than their peers, are prominent in creativity, art, and leadership, have special academic ability, can understand abstract ideas, love to act independently in their interests, and show high performance (ministry of national education [mone], 2015). in other words, they differ from their peers in cognitive and affective aspects. gifted students attend science and art centers (sacs) in their out-of-school times in turkey. sacs aim to educate gifted students attending pre-school, primary, elementary, and post-secondary schools to become aware of their abilities and use their capacities at the highest level. students go through five different programs in sacs. these are orientation, support education, recognizing individual skills (ris), developing special skills (dss), and project production and management programs (project). in the orientation program, students are introduced to the school, teachers, education program. in support programs, students diagnosed with general mental ability receive enriched education in all fields/disciplines. ris is a journal of science learning article doi: 10.17509/jsl.v4i3.31995 277 j.sci.learn.2021.4(3).275-287 program conducted for students diagnosed with general mental ability to help them realize their abilities. dss is a program conducted to develop the special skills of students who have completed the ris program. finally, in project production and management programs, students carry out individual or group projects in a field/discipline in line with their interests, wishes, and abilities under the guidance of an advisor. there is a hierarchy among these programs, and students complete support, ris, dss, and project programs, respectively, starting from the orientation program. the duration of these programs may differ among sacs (mone, 2015). as of 2020, the number of sacs has reached 182 in turkey, and 63.000 students attend these centers (mone, 2020). regarding the epistemological beliefs of gifted students, there are numerous researches in the literature. these researches are primarily conducted in western countries (gallagher, 2019; schommer, 1993a, 1993b, 1998; kalman, sobhanzadeh, thompson, ibrahim, & wang, 2015; chen & pajares, 2010; schommer & dunnell, 1992, 1997). in turkish literature, although there is a significant body of research on epistemological beliefs, these researches are conducted mostly with students who were not gifted (aşut & köksal, 2015; atasoy, 2020; atasoy & küçük, 2020; aydemir, aydemir & boz, 2013; aydın & geçici, 2017; balantekin, 2013; boz, aydemir & aydemir, 2011). however, the number of research focusing on the gifted students as a sample quite limited (dönmez & yalmancıyücel, 2020; uçar, 2018). considering each student have epistemological baggage which supports or hinders his/her performance (schommer, 1993a), gifted students may have different epistemological baggage than their peers. in this direction, schommer and dunnell (1992) report that gifted students' epistemological beliefs become more sophisticated compared to non-gifted in the later stages of their high school education. therefore, this study is expected to contribute to the literature on gifted students' epistemological beliefs. learning about the epistemological beliefs of gifted students is essential in guiding these students correctly, understanding their skills and needs, and using appropriate methods and techniques in their education (hammer, 1997). another concern to be considered in examining the epistemological beliefs is the measurement tools. according to duell and schommer-aikins (2001), initially lengthy and in-depth interviews were mainly used to determine the epistemological beliefs. however, due to the challenges regarding the interpretation, grading, validity, reliability, affordability of the interviews, in later studies, paper and pencil scales and questioners were developed by different researchers based on different epistemological beliefs models (see duell & schommer-aikins, 2001). although there are specific measurement tools found in the literature, scales can be open to misunderstandings and misinterpretations when used with children. thus, measuring children's epistemological beliefs is difficult (brownlee, curtis, spooner-lane, & feucht, 2017). also, some researchers claim that due to cultural differences, standardized scales are not reliable tools to identify epistemological beliefs (chan & elliott, 2002). in this manner, it can be said that "no single instrument may be the definitive measure of epistemological beliefs" (schommer, 1993a). therefore, alternative measurement tools are also used in determining epistemological beliefs (briell, elen, depaepe, & clarebout, 2010; brownlee et al., 2017). for example, üztemur and dinç (2018) examined the epistemological beliefs of middle school students using an alternative measurement tool and used the draw-writetell technique. similarly, atasoy (2020) examined middle school students' epistemological beliefs by using concept cartoons. in this study, as another alternative measurement tool, word association test (wat) is used to examine the epistemological beliefs of gifted students. wat is seen as an alternative assessment and evaluation tool (taşdere, özsevgeç & türkmen, 2014) that reveals students' thoughts (bahar, johnstone & sutcliffe, 1999). being used to examine emotional connections in psychology, wat is an effective technique to reveal information systematically organized in individuals' minds (yun, 2020). wat is also used in determining the relationships between concepts (atasoy, 2004) and highly preferred in science and social studies education (cebesoy & karisan, 2020; du, wu & lan, 2019; ekici & kurt, 2014; kalaycı, 2020; mahror & mahmut, 2020). while using wat, students are asked to write the word or group of words that come to their mind within the framework of a stimulus word in a very short time and then form a sentence (çetinkaya, sönmez & topçam, 2020; kostova & radoynovska, 2010). when the relevant studies are examined, it is seen that the wat is used to determine students' cognitive structures (çetin & timur, 2020; ozer, 2020), conceptual changes (hovardas & korfiatis, 2006), and perceptions (uluçınar sağır, 2017). for example, wat was used to determine the perceptions of students from different levels on the concepts of environmental pollution (kalaycı, 2020), technology (çetin & timur, 2020), aids (ekici & kurt, 2014), social studies, and social sciences (deveci, köse & bayır, 2014) and current, resistance and voltage concepts (balbağ & karademir, 2020). based on these studies, it can be concluded that the wat is an effective tool in revealing the cognitive structures of the students. therefore, it can be said that wat is influential in determining the misconceptions, cognitive structures, and perceptions of individuals. considering the developmental levels and ages of the students, it is vital to use an alternative technique to reveal the epistemological beliefs of gifted students. in the light of information given, in the present study, the aim is to determine the epistemological beliefs of the gifted students studying at the science and art center journal of science learning article doi: 10.17509/jsl.v4i3.31995 278 j.sci.learn.2021.4(3).275-287 (sac) through revealing their thoughts about the concept of "knowledge" and "learning" by using wat. for this purpose, the research question is determined as "what are the epistemological beliefs of gifted students studying at sac in a city in the eastern anatolia region of turkey?" 2. method 2.1 research design this study is designed as phenomenological research. phenomenology is one of the qualitative research method designs intended to make sense of people's views (creswell, 2013). it is used to investigate events, situations, and facts that we do not understand in-depth (cropley, 2002). in addition, it focuses on revealing how individuals remember and make sense of a concept or phenomenon from their experiences (patton, 2018). this study aimed to determine their epistemological beliefs by examining the thoughts of the gifted students studying at sac on the concept of "knowledge" and "learning" and the meanings they attribute to these concepts. for this reason, phenomenology design was chosen as a qualitative research design. 2.2 study group this study was carried out with 118 gifted students studying at sac located in a province in the eastern anatolia region of turkey in the 2018-2019 academic year. in phenomenology research, small groups of participants are preferred to have a deeper understanding of the participants' opinions about the facts and events (smith & osborn, 2009). however, the number of participants has been kept high in the study since it is aimed to determine the cognitive structures of the students who continue their education in different education programs at sac by using wat. therefore, when studies using wat as a data collection tool are examined, it is seen that the number of participants is kept relatively high (bahar, johnstone & sutcliffe, 1999; ekici & kurt, 2014; yun, 2020). in this direction, information about the gender, the age of the participants, and the program they studied at sac are given in table 1. according to table 1, 37.28% of the participants are girls, and 62.72% are boys. likewise, there are 41 participants between the ages of 6-10, 70 participants between the ages of 11-15, and 7 participants between the ages of 16-20. among these participants, 34.74% receive support education, 36.44% ris, 22.88% dss and 5.94% project production and management program. 2.3 data collection process in this study, wat was used as a data collection tool. in this context, the participants were asked to write the connotations that come to their minds about the concepts of "knowledge" and "learning" as a stimulus word, and then they were asked to form a sentence about these concepts. before using wat, the participants were asked, "what do you think about the concepts of knowledge and learning?", "what are the characteristics of knowledge and learning for you?" it was ensured that the participants thought about 10 minutes before the application and became aware of their cognitive structures regarding the concepts of knowledge and learning. before the research data was collected, explanations were made about the wat to increase the participants' familiarity with the test. a preliminary study was conducted with the participants using the term "sac" as a stimulus word. then, participants were asked to complete the test in 1 minute for each concept in the data collection. the sample page layout of the test is given in figure 1. 2.4 data analysis the data were analyzed by using content analysis. content analysis is based on the interpretation of the codes created under themes and categories in a cause-effect relationship (yıldırım & şimşek, 2013; wimmer & dominick, 2000). content analysis may be used in an inductive or deductive manner. the inductive approach is recommended, especially when there is no entirely constructed theory and when little is known about the phenomenon you study (lauri & kyngäs, 2005). on the other hand, deductive content analysis is based on an earlier theory or model; therefore, it progresses from general to particular (burns & grove 2005). in this study, the participants' associations for the concepts of "knowledge" and "learning" in the wat were coded. the codes were collected under categories named as naive or sophisticated under themes. schommer's (1990) model uses names of the themes as simple knowledge, certain knowledge, innate ability, omniscient authority, and quick learning. in this table 1 demographic information about participants demographic information f % gender girls 44 37.28 boys 74 62.72 age 6-10 41 34.74 11-15 70 59.32 16-20 7 5.94 program being studied support education 41 34.74 ris (recognizing individual skills) 43 36.44 dss (developing special skills) 27 22.88 project 7 5.94 figure 1 sample page layout journal of science learning article doi: 10.17509/jsl.v4i3.31995 279 j.sci.learn.2021.4(3).275-287 regard, deductive content analysis was used in data analysis. the answers given to the "knowledge" concept were coded under certain knowledge, simple knowledge, and omniscient authority themes; the answers given to the "learning" concepts were coded under quick learning and innate ability themes. in the data analysis process, papers were examined, and inappropriate papers were removed from the analysis, participants' answer sheets were numbered, categories were determined, validity and reliability were checked, and the data were interpreted by calculating the frequencies of the concepts. in addition, concept maps were created for each dimension according to the cut points determined in the data analysis. for any key concept in wat, 3-5 numbers below the most frequent answer are used as the cut-point. the answer frequency above the cut-point is placed in the concept map, and concept maps are drawn by decreasing the cut-point to certain intervals (bahar, johnstone & sutcliffe, 1999). 2.5 validity and reliability study many aspects were considered to ensure validity in this research. in this context, coding of data and categorization of the data analysis processes were explained in detail (daymon & holloway, 2003). opinions of students that best reflect that category in each category were directly reported in the findings (wiersma & jurs, 2005). the findings were discussed concerning the previous studies and compared to the previous findings. (ratcliff, 1995). in order to ensure the reliability of the study, the codes and categories specified in the study were evaluated by two expert researchers who carried out studies in the field of educational sciences (miles & huberman, 1994). as a result of the analysis, miles and huberman's (1994) formula [reliability = consensus / (consensus + disagreement) x 100] was used to check the inter-rater reliability, and the reliability coefficient was calculated as 89%, which ensured the reliability of the study. all rules within the scope of publication ethics were followed in the research. in addition, the ethics committee approval was obtained from nevşehir hacı bektaş veli university ethics committee. (date: 02/03/2020, no: 28.02.2020/e.6052) 3. findings in the research, wat was used to reveal the cognitive structures related to "knowledge" and "learning" concepts. in this section, themes, categories, and codes for the concepts of "knowledge" and "learning" are presented in table 2. the frequency of each association is given in table 2, and student associations unrelated to knowledge and learning concepts are not included in the table (torkar & bajd, 2006). association frequencies in each category are listed from highest to lowest. in addition, each theme table 2 themes, categories, and codes certain knowledge sophisticated naive tentative (34), socially embedded (14), can be improved (9) limitless (5), development (4), unchangeable (27), limited (6), certain (4) omniscient authority sophisticated naive experts are not needed (32), created by individuals (27), thinking (24), connected with learning (22), working (22), it is research (18), it is learning (18), it is obtained through research (12), concrete and provable (12), being realized (9), begins with an idea (7), curiosity (6), infinite (6), imagination (5), designing (5), provable (4), obtained by thinking (4), experience (3), obtained by various methods (3), discovery (2), experiment (2), invention (1)hypothesis (1), it happens as a result of events (1) transferred by experts (14), experts are needed (12), books (7), transference (5), teacher (3), objective thinking (2), scientist (2), internet (1), constitution (1) simple knowledge sophisticated naive complex (28), intertwined (32), interrelated (4) concise (10), easy (9), simple (9), abstract (9), easily forgotten (1) quick learning sophisticated naive it is a process (33), it requires much work (28), endeavor (17), gradual (14), time is required (13), it is an action (8), repeat is important (8), it is research (7), it is curiosity (7) it is obtained by living (5), it is difficult to learn (4), patience is required (3), it is learned gradually (1), there are many methods (1), it is patience (1), it is complex (1) happens instantly (32), memorization (9) innate ability sophisticated naive it is necessary to work (24), it depends on the person (9), experience required (4) intelligence is important (24), it is mind (7), logic (4), talent (2), putting knowledge into the brain (1), ambition (1) journal of science learning article doi: 10.17509/jsl.v4i3.31995 280 j.sci.learn.2021.4(3).275-287 belonging to the concept of "knowledge" has been categorized as "advanced" and "novice" with reference to schommer (1990). in the "certain knowledge" regarding the "knowledge" concept, most of the participants fall under the "sophisticated" category (f = 66). while the participants focus on the concept of tentativeness, they stated that the knowledge is improvable, unlimited, and open to development. on the other hand, in the "naive" category, they focused on certainty. for this category, it was observed that participants state that knowledge is limited and certain. the concept map of a certain knowledge theme is given in figure 2. sample sentences of the participants on the theme of "certain knowledge" are as follows: knowledge does not change for me (p2) in my opinion, knowledge is something we can have if we try (p1) knowledge is based on discovery and experimentation (k14) to me, knowledge is a resource that can develop and be produced (k56) knowledge is the most important part of life (k101) in my opinion, knowledge is essential and can change (p59) knowledge is a concept that can change and develop (k73) knowledge is something that comes later and changes (k79) in the second theme of the "knowledge" concept, which is determined as the "omniscient authority", most of the participants fall under the "sophisticated" category (f = 256). sophisticated participants focus on the idea that there is no need for an expert for knowledge, while the participants in the naive category adopt the idea that experts hand down the knowledge. the sophisticated participants expressed their thoughts as experts are not needed, knowledge created by the individuals, thinking, connected with learning, working, knowledge is research, knowledge is learning, knowledge is obtained by research, concrete and provable, learning later, begins with an idea, curiosity, infinite, imagination, designing, provable, obtained by thinking, experience, obtained by various methods, discovery, experiment, figure 2 students' associations about particular knowledge dimension figure 3 students’ associations about omniscient authority dimension journal of science learning article doi: 10.17509/jsl.v4i3.31995 281 j.sci.learn.2021.4(3).275-287 invention, hypothesis, it happens as a result of events. in the naive category of omniscient authority, the participants stated their thoughts with the expressions such as experts are needed, books, transference, teacher, objective thinking, and scientist. the concept map of the theme of omniscient authority is given in figure 3. sample sentences of the participants on the theme of "omniscient authority" are as follows: knowledge depends on experience in life (k71) curiosity is important to access knowledge (k88) the horizons of the individual expand as new knowledge is learned (k94) knowledge and learning are directly proportional (k96) for me, knowledge starts with thinking (k104) knowledge is, in my opinion, an important thing that experts give us (k102) most of the participants in the third theme, which is named "simple knowledge" belonging to the concept of "knowledge", fall under the "sophisticated" category (f = 64). while the participants in the "sophisticated" category of simple knowledge theme focus on the complexity of the knowledge, participants in the "naive" category express that the information is concise. the sophisticated participants expressed their thoughts with the phrases such as complex, intertwined, and interrelated. in the naive category of simple knowledge, the participants stated their thoughts with concise, easy, simple, abstract, and easily forgotten expressions. the concept map of the simple knowledge theme is given in figure 4. the sample sentences of the participants on the "simple knowledge" theme are as follows: knowledge is a part of life and is related to each other (k70) the knowledge we gain involuntarily is linked to each other (k115) knowledge is the most important thing we need in our lives (k72) in my opinion, knowledge is the development of the feeling of wondering about something through education (p6) knowledge is an effort to learn what is not learned (k9) knowledge starts from a specific part of our lives and shapes our lives (k108) most of the participants in the fourth theme, which is determined as "quick learning", fall under the category of "sophisticated" (f: 152). sophisticated participants focus on the idea that learning takes place slowly and gradually, while the participants in the naive category adopt the idea that learning is a gradual process. the sophisticated participants expressed their thoughts as learning is a process, learning requires much work, tentative, gradual, time is essential, learning is an action, repeat is important, learning is research, learning is curiosity, learning is achieved through living, it is difficult to learn, patience is required, learning is learned gradually, there are many methods, learning is patience, learning is complex. on the other hand, in the figure 4 students’ associations about simple knowledge dimension figure 5 students' associations about quick learning dimension journal of science learning article doi: 10.17509/jsl.v4i3.31995 282 j.sci.learn.2021.4(3).275-287 naive category of quick learning, the participants stated their thoughts with the expressions such as memorization and learning happens instantly. the concept map of the theme of quick learning is given in figure 5. in the fifth theme determined as "innate ability", most of the participants are under the "naive" category (f: 39). for this theme, while the sophisticated participants focus on the idea that hard work is essential, naive participants emphasize intelligence. the sophisticated participants expressed their thoughts as working is needed, experience is required, and learning depends on the person. in the naive category of innate ability, the participants stated their thoughts with the expressions such as intelligence, mind logic, talent, brain, and ambition. the concept map of the innate ability theme is given in figure 6. the sample sentences of the participants on the theme of "innate ability" are as follows: success is achieved by trying and effort (k80) intelligence is important to be knowledgeable (k62) for me, knowledge is research (k66) in my opinion, we access to knowledge ourselves (p27) 4. results and discussion this research aims to examine the epistemological beliefs of gifted students using wat. content analysis was employed for the associations obtained to reveal the cognitive structures of the students' "knowledge" and "learning" concepts. as a result of the content analysis, five themes, in consistence with the adopted theory (i.e., schommer's (1990) epistemological belief system), were determined as "certain knowledge", "omniscient authority", "simple knowledge", "quick learning" and "innate ability" (table 2). furthermore, when the data were analyzed, it was seen that categories related to the five dimensions of schommer's (1990) epistemological belief model were formed. likert-type scales are the most frequently used data collection tools to determine students' epistemological beliefs (boz, aydemir & aydemir, 2011; elder, 2002; evcim, 2010; duell & schommer-aikins, 2001, kızkapan & bektaş, 2020; schommer-aikins, mau, brookhart, & hutter, 2000; yilmaz-tüzün. & topcu, 2010). in addition, it is seen in the literature that scales prepared for different grade levels, especially students in older classes, are also used in younger age groups (başer-gülsoy, erol & akbay, 2015; yeşilyurt, 2013). also, data collection tools such as interview (feucht, 2017; saylan-kırmızıgül & bektaş, 2019), sketch (mansfield & clinchy, 2002), draw-write-tell (brownlee et al., 2017; üztemur & dinç, 2018), and concept cartoons (atasoy, 2020) were used to determine the epistemological beliefs of students from different grade levels. in the present study, it was seen that wat could be used to determine the students' epistemological beliefs as an alternative measurement tool. at the end of the research, the epistemological beliefs of the students were grouped based on schommer's (1990) model as naive or sophisticated. in using wat, participants were given a stimulus word and then asked to respond verbally or in writing with the first word that comes to their minds. researchers may conclude from the participant's cognitive system structure by examining the relationship between responses and classifying them (suzuki-parker & higginbotham, 2019). the words that evoke in the students' minds about the stimulating words and their explanations afterward reflect their epistemological beliefs. as a result of the study, the epistemological beliefs of the gifted students could be determined inductively, without using a scale prepared following a predetermined theoretical background. moreover, the beliefs determined were compatible with schommer's epistemological beliefs model. therefore, wat is at least as applicable as the widely used scales in determining the epistemological beliefs of gifted students. 4.1 specific knowledge and omniscient authority within the scope of the research, each theme is explained and compared with the relevant literature and discussed. for example, it has been observed that under the "certain knowledge" theme, there are many associations regarding the changeability of knowledge. according to this result, it can be said that participants have sophisticated figure 6 students’ associations about innate ability dimension journal of science learning article doi: 10.17509/jsl.v4i3.31995 283 j.sci.learn.2021.4(3).275-287 beliefs about the tentative nature of knowledge (schommer, 1990). in addition, under the "omniscient authority" theme, many associations show that students think experts do not have authority on knowledge, and individuals can produce knowledge through experience and inquiry. in literature, while some of the previous researches support our findings (başer-gülsoy, erol & akbay, 2015; feucht, 2017; sadıç, 2013; boz, aydemir & aydemir, 2011), some other studies reported that students hold naive or moderate beliefs in these dimensions (aşut & köksal, 2015; atasoy, 2020; gök, 2018; üztemur & dinç, 2018). research that has opposite results state standardized exams as the reason for the naive beliefs (atasoy, 2020). on the other hand, it is stated in studies supporting our findings that students become less dependent on authority as age, experience, education, and family support increase (schommer, 1990; 1993a, 1998; schommer & dunnell, 1992). also, students who do not prefer external sources such as books and teachers as the source of scientific knowledge hold beliefs that experimentation and observation play a role in forming knowledge. that proof and inquiry play a role in the justification process (sadıç (2013). based on this, it can be said that the reason for gifted students' sophisticated epistemological beliefs in specific knowledge and omniscient authority dimension may be the use of student-centered methods such as project-based, inquiry-based, problem-based teaching at sacs. because research revealed that students' epistemological beliefs could be developed by using instructional strategies in which students are active, information is constructed by them, and learning is dependent on students' participation (deryakulu, 2004). researchers also report that students do workshops, experiments, hands-on activities, and rich material and activity opportunities at sacs (epçaçan & oral, 2019). 4.2 simple knowledge in the simple knowledge dimension, it can be interpreted that the majority of the participants have sophisticated epistemological beliefs. this conclusion can be drawn from the students' associations that knowledge is complex and interrelated. it can be said that gifted students have developed their beliefs in this dimension through their education at sac. conley et al. (2004) stated that students learning in constructivist environments are more likely to have more sophisticated epistemological beliefs than those in teacher-centered classrooms. in a supportive manner, schommer and dunnell (1992) reported that gifted students' epistemological beliefs in the simple knowledge dimension are sophisticated. on the other hand, it is noteworthy that some studies in literature examining the epistemological beliefs of participants at different class levels generally did not address the simple knowledge dimension (boz, aydemir & aydemir, 2011; kurt, 2009; olgun, 2018; sadıç, 2013; tüken, 2010; yenice & özden, 2013). however, having sophisticated beliefs in the simple knowledge dimension is quite noteworthy for science education because it is argued that individuals who believe that knowledge is organized in mind as isolated pieces tend to use rote learning (schommer, crouse & rhodes, 1992). also, students who believe knowledge is simple will have difficulty comprehending the ambiguous aspects of tasks that require analytical decisions, perseverance, and appropriate selfregulated learning. moreover, in some academic contexts, seeing knowledge as simple can restrict subsequent conceptual change (lodewyk, 2007). thus, as seen in the study, gifted students have sophisticated beliefs in the simple knowledge dimension. therefore, it can be said that they do not prefer rote learning; instead, they tend to use organizational learning strategies and construct connections between what they learn and what they already knew. also, they will probably be more likely to use metacognition and self-regulation strategies (dahl, bals & turi, 2005; tortop, 2015). 4.3 quick learning and innate ability based on the associations of the participants in terms of quick learning and the innate ability on learning, it can be said that the participants have sophisticated epistemological beliefs in the quick learning dimension but have moderate beliefs in the innate ability dimension. while some of the participants had sophisticated beliefs that learning is a process, it happens gradually. it is necessary to learn; an equal number of participants stated that intelligence is essential in learning. in one of the studies in the related literature, olgun (2018) found that middle school teachers had beliefs that learning depends more on effort than on ability. on the other hand, köse and dinç (2012) stated that the participants believe learning depends on ability. studies on this subject show that as students' beliefs in the effect of innate ability and quick learning decrease, their grade point averages increase (schommer, 1993b). this means students who hold sophisticated epistemological beliefs in the innate ability and quick learning become more successful. in addition, in a study conducted with university students, researchers determined that students who have sophisticated beliefs in the dimension of quick learning better understand complex academic texts (schommer, 1990; schommer, crouse & rhodes, 1992). also, dahl, bals, and turi (2005) reported that students with naive epistemological beliefs in the innate ability dimension do not tend to use elaboration, critical thinking, metacognitive, and self-regulation strategies. on the one hand, the current study participants frequently stated that learning takes place by working hard in the process; on the other hand, they emphasized the importance of intelligence in learning. the researcher, teachers, parents, and all other stakeholders should consider the findings of the present study because the participants were the students studying at the sac and journal of science learning article doi: 10.17509/jsl.v4i3.31995 284 j.sci.learn.2021.4(3).275-287 were identified as "gifted" students. this definition may have caused them to make associations with a belief in this direction. therefore, while defining these students, we should consider the risk that these students may ignore working by paying too much attention to intelligence. 5. conclusion and suggestions to sum up, the research results have shown that wat can be used as an alternative measurement tool to determine the epistemological beliefs of gifted students. based on the result obtained by using wat, it can be concluded that the gifted students studying at sac have sophisticated epistemological beliefs in the dimensions of certain knowledge, simple knowledge, omniscient authority, and quick learning dimensions, medium level beliefs in the dimension of innate ability. therefore, these students are expected to have the desired approach to work and develop metacognitive knowledge (schommer, crouse & rhodes, 1992). in addition, it can be interpreted that gifted students with sophisticated beliefs in the dimensions of quick learning and innate ability will be more willing to accept difficulties, learn from their mistakes and provide continuity in their learning (dweck & leggett, 1988). therefore, it is expected that gifted students with sophisticated epistemological beliefs can overcome difficulties and learn from mistakes. however, further studies should be conducted if gifted students having sophisticated beliefs adopt these desired competencies in their academic lives. when the studies examining the epistemological beliefs of gifted students in the literature are examined, it is seen that the students' development differs in the subdimensions of epistemological beliefs (aşut & köksal, 2015; schommer & dunnell, 1997). therefore, it is thought that there is a need for new researches to be carried out with gifted students from different age groups using different types of measurement tools. in addition, studies can be carried out to determine how the epistemological beliefs of the students reflect on their behaviors that can be observed in the classroom; in other words, what kind of differences are seen between the affective characteristics of students with sophisticated and naive epistemological beliefs such as class participation, attitude, motivation, and self-efficacy. references akerson, v. l., & buzzelli, c. a. (2007). relationships of preservice early childhood teachers’ cultural values, ethical and cognitive developmental levels, and views of nature of science. journal of elementary science education, 19, 15–24. akerson, v. l., morrison, j. a., & roth mcduffie, a. (2006). one course is not enough: preservice elementary teachers’ retention of improved views of nature of science. journal of research in science teaching, 43, 194–213. alavi, m., & leidner, d. e. (2001). knowledge management and knowledge management systems: conceptual foundations and research issues. mis quarterly, 25(1), 107-136. aslan, c. (2017). examining epistemological beliefs of teacher candidates according to various variables. eurasian journal of educational research, 67, 37-50. http://dx.doi.org/10.14689/ejer.2017.67.3 aşut, n., & köksal, m. (2015). relationship of gifted students’ epistemological beliefs wıth achievement and motivation towards science learning. journal of düzce university institute of social sciences, (2), 22-44. https://dergipark.org.tr/en/download/articlefile/210214 atasoy, b. (2004). science learning and teaching [fen öğrenimi ve öğretimi]. ankara: asil publishing. atasoy, s. (2020). using concept cartoons to ıdentify the epistemological beliefs of middle school students. journal of science learning, 3(3), 165-173. https://files.eric.ed.gov/fulltext/ej1267613.pdf atasoy, s., & küçük, o. (2020). development of eighth grade students' epistemological beliefs through writing-to-learn activities. journal of science learning, 3(2), 57-66. aydemir, n., aydemir, m., & boz, y. (2013). high school students’ epistemological beliefs [lise öğrencilerinin epistemolojik inançları]. kastamonu education journal, 21(4), 1305-1316. https://dergipark.org.tr/en/pub/kefdergi/issue/22604/241549 aydın, m., & geçici, m. e. (2017). investigation of 6th grade students’ epistemological beliefs with respect to some variables [6. sınıf öğrencilerinin epistemolojik inançlarının bazı değişkenler açısından incelenmesi]. journal of ahi evran university kırşehir faculty of education, 18(1), 213-229. https://dergipark.org.tr/en/download/article-file/1481453 bahar, m., johnstone, a.h., & sutcliffe, r. (1999). investigation of students’ cognitive structure in elementary genetics through word association tests. journal of biological education, 33(3), 134-141. https://doi.org/10.1080/00219266.1999.9655653 balantekin, y. (2013). epistemological beliefs of primary school students’ intended for scientific knowledge [i̇lköğretim öğrencilerinin bilimsel bilgiye yönelik epistemolojik inançları]. bartın university journal of faculty of education, 2(2), 312-328. balbağ, m. z. & karademir, e. (2020). examination of the cognitive structures of the secondary school eighth-grade students regarding some concepts in electricity through the word association test. osmangazi journal of educational research, 7(1), 50-64. https://ojer.ogu.edu.tr/storage/osmangazijournalofeducational research/uploads/ojer-v7-n1-3rd-article.pdf başer-gülsoy, v. g., erol, o., & akbay, t. (2015). the investigation of middle school students' epistemological belief based upon varied factors [ortaokul öğrencilerinin bilimsel epistemolojik inançlarının farklı değişkenlere göre incelenmesi]. mehmet akif ersoy university journal of education faculty, 1(35), 1-28. https://dergipark.org.tr/en/pub/maeuefd/issue/19408/206386 belet, ş. d., & güven, m. (2011). examination of epistemological beliefs and metacognitive strategies of classroom teacher candidates [sınıf öğretmeni adaylarının epistemolojik inançlarının ve bilişüstü stratejilerinin incelenmesi]. educational sciences: theory & practice, 11(1), 31-57. http://earsiv.anadolu.edu.tr/xmlui/bitstream/handle/11421/153 21/15321.pdf?sequence=1&isallowed=y boz, y., aydemir, m., & aydemir, n. (2011). 4th, 6th, and 8th grade turkish elementary students’ epistemological beliefs. elementary education online, 10(3), 1191-1201. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.857.7 280&rep=rep1&type=pdf briell, j., elen, j., depaepe, f., & clarebout, g. (2010). the exploration of drawings as a tool to gain entry to students’ epistemological beliefs. electronic journal of research in educational psychology, 8(2), 655688. https://www.redalyc.org/pdf/2931/293122002011.pdf brownlee, j. curtis, e., spooner-lane, r., & feucht, f. (2017). understanding children's epistemic beliefs in elementary education. education 3(13), 191-208. https://doi.org/10.1080/03004279.2015.1069369 http://dx.doi.org/10.14689/ejer.2017.67.3 https://files.eric.ed.gov/fulltext/ej1267613.pdf https://dergipark.org.tr/en/pub/kefdergi/issue/22604/241549 https://dergipark.org.tr/en/download/article-file/1481453 https://doi.org/10.1080/00219266.1999.9655653 https://ojer.ogu.edu.tr/storage/osmangazijournalofeducationalresearch/uploads/ojer-v7-n1-3rd-article.pdf https://ojer.ogu.edu.tr/storage/osmangazijournalofeducationalresearch/uploads/ojer-v7-n1-3rd-article.pdf https://dergipark.org.tr/en/pub/maeuefd/issue/19408/206386 http://earsiv.anadolu.edu.tr/xmlui/bitstream/handle/11421/15321/15321.pdf?sequence=1&isallowed=y http://earsiv.anadolu.edu.tr/xmlui/bitstream/handle/11421/15321/15321.pdf?sequence=1&isallowed=y http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.857.7280&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.857.7280&rep=rep1&type=pdf https://www.redalyc.org/pdf/2931/293122002011.pdf https://doi.org/10.1080/03004279.2015.1069369 journal of science learning article doi: 10.17509/jsl.v4i3.31995 285 j.sci.learn.2021.4(3).275-287 burns n., & grove s.k. (2005) the practice of nursing research: conduct, critique & utilization. elsevier saunders, st louis. cebesoy, u.b., & karisan, d. (2020). teaching the role of forests in mitigating the effects of climate change using outdoor educational workshop. research in science & technological education, 1-23. https://doi.org/10.1080/02635143.2020.1799777 chan, k. (2003). hong kong teacher education students' epistemological beliefs and approaches to learning. research in education, 69(1), 36-50. https://doi.org/10.7227%2frie.69.4 chan, k.-w., & elliott, r. g. (2002). exploratory study of hong kong teacher education students’ epistemological beliefs: cultural perspectives and implications on beliefs research. contemporary educational psychology, 27, 392–414. chen, j. a. & pajares, f. (2010). implicit theories of ability of grade 6 science students: relation to epistemological beliefs and academic motivation and achievement in science. contemporary educational psychology, 35(1), 75-87. https://doi.org/10.1016/j.cedpsych.2009.10.003 cheng, m. m. h., chan, k.w., tang, s. y. f., & cheng, a. y. n. (2009). pre-service teacher education students’ epistemological beliefs and their conceptions of teaching, teaching and teacher education, 25, 319-327. https://doi.org/10.1016/j.tate.2008.09.018 chisholm, r. m. (1982). the foundations of knowing. minneapolis: university of minnesota press conley, a. m., pintrich, r. p., vekiri, i., & harrison, d. (2004). changes in epistemological beliefs in elementary science students. contemporary educational psychology, 29, 186-204. https://doi.org/10.1016/j.cedpsych.2004.01.004 creswell, j. w. (2013). qualitative inquiry & research design: choosing among five approaches (bütün, m. & demir, s. b. trans.). ankara: siyasal publishing. cropley, a. (2002). qualitative research methods: an introduction for students of psychology and education. university of latvia: zinatne. çetin, n. i̇., & timur, b. (2020). conceptual analysis of middle school students’ cognitive structure on the concept of “technology” through word association test. cukurova university faculty of education journal, 49(2), 1098-1125. https://dergipark.org.tr/en/pub/cuefd/issue/57389/548259 çetinkaya, f. ç., sönmez, m., & topçam, a. b. (2020). a formative assessment example: word association test. international education studies, 13(8), 103-117. https://doi.org/10.5539/ies.v13n8p103 dahl, t. i., bals, m., & turi, a. l. (2005). are students' beliefs about knowledge and learning associated with their reported use of learning strategies? british journal of educational psychology, 75(2), 257273. daymon, c., & holloway, i. (2003). qualitative research methods in public relations and marketing communications. london: rout ledge. http://fe.unj.ac.id/wp-content/uploads/2019/08/qualitativeresearch-methods-in-public-relations-and-marketingcommunications.pdf deng, f., chen, d., tsai, c., & chai, c. s. (2011). students’ views of the nature of science: a critical review of research. science education, 95(6), 961–999. https://doi.org/10.1002/sce.20460 deniz, h., & donnelly, l.a. (2011). preservice secondary science teachers’ acceptance of evolutionary theory and factors related to acceptance. national center for science education, 31, 12–19. deryakulu, d. (2004). epistemological beliefs [epistemolojik inançlar]. (editors: y. kuzgun & d. deryakulu). individual differences in education [eğitimde bireysel farklılıklar] (pp. 259-288). ankara: nobel publishing deveci, h., köse, t.ç., & bayır, ö.g. (2014). investigation of preservice teachers’ cognitive structures on the concepts of social sciences and social studies through word associatıon tests [öğretmen adaylarının sosyal bilimler ve sosyal bilgiler kavramlarına ilişkin bilişsel yapıları: kelime ilişkilendirme testi uygulaması.] adiyaman university journal of social sciences, 7(16), 101124. https://doi.org/10.14520/adyusbd.732 dorsah, p., shahadu, i., & kpemuonye, a.k.n. (2020). pre-service teachers’ scientific epistemological beliefs and science teaching efficacy beliefs: a correlational study. european journal of education studies, 7(9), 37-55. https://www.oapub.org/edu/index.php/ejes/article/view/3227 dönmez, i̇., & yalmancı-yücel, s. (2020). analysis of scientific epistemological beliefs and stem attitudes of the gifted students. bartın university journal of faculty of education, 9(3), 515-526 du, y., wu, y., & lan, m. (2019, november). exploring human gender stereotypes with word association test. in proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (emnlp-ijcnlp) (pp. 6135-6145). https://www.aclweb.org/anthology/d19-1635.pdf duell, o. k., & schommer-aikins, m. (2001). measures of people's beliefs about knowledge and learning. educational psychology review, 13(4), 419-449. dweck, c. s., & leggett, e. l. (1988). a social-cognitive approach to motivation and personality. psychological review, 95, 256-273. https://doi.org/10.1037/0033-295x.95.2.256 ekici, g., & kurt, h. (2014). student teachers’ cognıtıve structure on the concept of “aıds”: the sample of free word assocıatıon test [öğretmen adaylarının “aids” kavramı konusundaki bilişsel yapıları: bağımsız kelime ilişkilendirme testi örneği]. the journal of turkish social research,, 18(3), 267-306. https://dergipark.org.tr/en/pub/tsadergisi/issue/21495/230454 elder, a., d. (2002). characterizing fifth grade students’ epistemological beliefs in science. in b. hofer & p. r. pintrich (eds.). personal epistemology: the psychology of beliefs about knowledge and knowing (pp. 347-364). new jersey: lawrence erlbaum. epçaçan, u., & oral, b. (2019). the perceptions of talented students about teaching practices in sac. [özel yetenekli öğrencilerin bilsem’deki öğretim uygulamalarina ilişkin görüşleri]. siirt university journal of social sciences institute, 7(13), 139-166. eroğlu, s.e., & güven, k. (2006). examining the epistemological beliefs of university students in terms of some variables. [üniversite öğrencilerinin epistemolojik inançlarının bazı değişkenler açısından incelenmesi]. the journal of institute of social sciences, 16, 295-312. ertürk, s. (1993). program development in education. [eğitimde program geliştirme]. meteksan publishing, ankara evcim, i̇. (2010). the relatıon between 8th grade primary school students’ epistemological beliefs, science lesson acquisition, level of use of science in daily life and academic success (unpublished master thesis). marmara university institute of educational sciences, i̇stanbul. feucht, f. (2017). the epistemic beliefs of fourth-graders about the verification of second-hand knowledge and ıts knowledge sources. journal of education and human development, 6(1), 7-26. https://doi.org/10.15640/jehd.v6n1a2 gettier, e. (1963). is justified true belief knowledge? analysis, 23(6), 121– 123. gallagher, s. a. (2019). epistemological differences between gifted and typically developing middle school students. journal for the education of the gifted, 42(2), 164-184. gök, t. (2018). the evaluation of conceptual learning and epistemological beliefs on physics learning by think-pair-share. journal of education in science, environment and health (jeseh), 4(1), 69-80. doi:10.21891/jeseh.387489 greene, j. a., azevedo, r., & torney-purta, j. (2008). modeling epistemic and ontological cognition: philosophical perspectives and methodological directions. educational psychologist, 43(3), 142160. https://doi.org/10.1080/00461520802178458 hammer, d. (1997). discovery learning and discovery teaching. cognition and instruction, 15, 485-529. https://doi.org/10.1207/s1532690xci1504_2 harteis, c., gruber, h., & hertramph, h. (2010). how epistemic beliefs influence e-learning in daily work-life. educational technology & society, 13(3), 201-211. https://www.learntechlib.org/p/74915/ https://doi.org/10.1080/02635143.2020.1799777 https://doi.org/10.7227%2frie.69.4 https://doi.org/10.1016/j.cedpsych.2009.10.003 https://doi.org/10.1016/j.tate.2008.09.018 https://doi.org/10.1016/j.cedpsych.2004.01.004 https://dergipark.org.tr/en/pub/cuefd/issue/57389/548259 https://doi.org/10.5539/ies.v13n8p103 http://fe.unj.ac.id/wp-content/uploads/2019/08/qualitative-research-methods-in-public-relations-and-marketing-communications.pdf http://fe.unj.ac.id/wp-content/uploads/2019/08/qualitative-research-methods-in-public-relations-and-marketing-communications.pdf http://fe.unj.ac.id/wp-content/uploads/2019/08/qualitative-research-methods-in-public-relations-and-marketing-communications.pdf https://doi.org/10.1002/sce.20460 https://doi.org/10.14520/adyusbd.732 https://www.oapub.org/edu/index.php/ejes/article/view/3227 https://www.aclweb.org/anthology/d19-1635.pdf https://doi.org/10.1037/0033-295x.95.2.256 https://dergipark.org.tr/en/pub/tsadergisi/issue/21495/230454 https://doi.org/10.15640/jehd.v6n1a2 https://doi.org/10.1080/00461520802178458 https://doi.org/10.1207/s1532690xci1504_2 https://www.learntechlib.org/p/74915/ journal of science learning article doi: 10.17509/jsl.v4i3.31995 286 j.sci.learn.2021.4(3).275-287 hofer, b. k. (2000). dimensionality and disciplinary differences in personal epistemology. contemporary educational psychology, 25, 378405. https://doi.org/10.1006/ceps.1999.1026 hofer, b. k., & pintrich, p. r. (1997). the development of epistemological theories: beliefs about knowledge and knowing and their relation to learning. review of educational research, 67(1), 88-140. https://doi.org/10.3102/00346543067001088 hokayem, h., & boujaoude, s. (2008). college students perceptions of the theory of evolution. journal of research in science teaching, 45, 395– 419. hovardas, t., & korfiatis, k. j. (2006). word associations as a tool for assessing conceptual change in science education. learning and instruction, 16(5), 416-432. https://doi.org/10.1016/j.learninstruc.2006.09.003 kalaycı, s. (2020). cognitive perceptions of pre-service science teacher for environmental pollution. journal of baltic science education, 19(3), 415-428. https://doi.org/10.33225/jbse/20.19.415 kalman, c. s., sobhanzadeh, m., thompson, r., ibrahim, a., & wang, x. (2015). a combination of interventions can change students' epistemological beliefs. physical review special topics-physics education research, 11(2), 020136. khine, m. s. (2008). knowing, knowledge and beliefs-epistemological studies across diverse cultures. new york, ny: springer. https://link.springer.com/book/10.1007%2f978-1-4020-6596-5 kızkapan, o., & bektaş, o. (2020) developing a science-oriented epistemological belief scale for secondary school students: a validity and reliability study [ortaokul öğrencileri için fene yönelik epistemolojik inanç ölçeği geliştirilmesi: geçerlik ve güvenirlik çalışması]. manas journal of social studies, 9(4), 2173-2185. king, p. m., & kitchener, k. s. (1994). developing reflective judgment: understanding and promoting intellectual growth and critical thinking in adolescents and adults. jossey-bass higher and adult education series and jossey-bass social and behavioral science series. jossey-bass, 350 sansome street, san francisco, ca 94104-1310. kostova, z., & radoynovska, b. (2010). motivating students’ learning using word association test and concept maps. bulgarian journal of science and education policy, 4(1), 62-98. köse, s., & dinç, s. (2012). the relationships among pre-service science and technology teachers’ biology self-efficacy perceptions and epistemological beliefs [fen ve teknoloji öğretmen adaylarının biyoloji özyeterlilik algıları ile epistemolojik inançları arasındaki ilişki]. mustafa kemal university journal of social sciences institute, 9(18), 121-141. https://dergipark.org.tr/en/pub/mkusbed/issue/19552/208327 kuhn, d., cheney, r., & weinstock, m. (2000). the development of epistemological understanding. cognitive development, 15(3), 309-328. https://doi.org/10.1016/s0885-2014(00)00030-7 kurt, f. (2009). investigating students’ epistemological beliefs through gender, grade level, and fields of the study (unpublished master thesis). middle east technical university, ankara. lauri, s., & kyngäs, h. (2005) developing nursing theories. werner soderstrom, dark oy, vantaa. lee, s., & jhun, y. (2020). elementary teachers' epistemological beliefs and practice on convergent science teaching: survey and selfstudy. journal of the korean association for science education, 40(4), 359-374. https://doi.org/10.14697/jkase.2020.40.4.359 lodewyk, k. r. (2007) relations among epistemological beliefs, academic achievement, and task performance in secondary school students, educational psychology, 27(3), 307-327, doi: 10.1080/01443410601104080 magolda, m. b. b. (1992). students' epistemologies and academic experiences: implications for pedagogy. the review of higher education, 15(3), 265-287. https://doi.org/10.1353/rhe.1992.0013 mahror, n., & mahmud, s. n. d. (2020). secondary school students’ cognitive structures and misconceptions in respiration topic. humanities & social sciences reviews, 8(3), 1272-1284. https://doi.org/10.18510/hssr.2020.83130 mansfield, a. f., & b. clinchy. (2002). toward the integration of objectivity and subjectivity: epistemological development from 10 to 16. new ideas in psychology 20, 225-262. https://doi.org/10.1016/s0732-118x(02)00008-9 mason, l., & bromme, r. (2010). situating and relating epistemological beliefs into metacognition: studies on beliefs about knowledge and knowing. metacognition learning, 5, 1-6. https://doi.org/10.1007/s11409-009-9050-8 mardiha, s. m., & alibakhshi, g. (2020). teachers' personal epistemological beliefs and their conceptions of teaching and learning: a correlational study. cogent education, 7(1), 1-14. https://doi.org/10.1080/2331186x.2020.1763230 miles, m.b., & huberman, a.m. (1994). qualitative data analysis. thousand oaks, ca: sage. mone. (2015). ministry of national education, science and art centers directive [milli eğitim bakanlığı bilim ve sanat merkezleri yönergesi]. retrieved from https://orgm.meb.gov.tr/meb_iys_dosyalar/2016_10/07031350_ bilsem_yonergesi.pdf on 14.04.2021. mone. (2020). science and art centers are strengthening [bilim ve sanat merkezleri güçleniyor], retrieved from https://www.meb.gov.tr/bilim-ve-sanat-merkezlerigucleniyor/haber/21827/tr#:~:text=halen%2081%20ilde%2018 2%20b%c4%b0lsem,bin%20%c3%b6%c4%9frenci%20bu% 20merkezlerden%20yararlan%c4%b1yor. on 14.04.2021 nozick, r. (1981). philosophical explanations, harvard university press olgun, m. (2018). analysis of secondary school teachers' epistemological beliefs and teaching styles according to various variables (unpulished master thesis). kırıkkale university, kırıkkale. ozer, b. (2020). analysis of high school students' cognitive structures related to musical genre via word association test. cypriot journal of educational sciences, 15(1), 127-143. https://doi.org/10.18844/cjes.v15i1.4557 özmen, h. (2004). learning theories in science teaching and technology supported constructivist learning [fen öğretiminde öğrenme teorileri ve teknoloji destekli yapılandırmacı (constructivist) öğrenmek]. the turkish online journal of educational technology, 3(1), 100-111. özturk, n., & yılmaz-tüzün, o. (2017). preservice science teachers’ epistemological beliefs and informal reasoning regarding socioscientific issues. research in science education, 47(6), 1275-1304. palmer, b., & marra, r. m. (2004). college student epistemological perspectives across knowledge domains: a proposed grounded theory. higher education, 47(3), 311-335. https://link.springer.com/article/10.1023/b:high.0000016445. 92289.f1 patton, m. q. (2018). quallitative research and evaluation methods [nitel araştırma ve değerlendirme yöntemleri]. (m. bütün & s. b. demir, trans.). ankara: pegem publishing. (original work published in 2014). popper, k. r. (1979). objective knowledge: an evolutionary approach, clarendon press qian, g., & alvermann, d. (2000). relationship between epistemological beliefs and conceptual change learning. reading writing quarterly, 16(1), 59-74. https://doi.org/10.1080/105735600278060 ratcliff, d. (1995). validity and reliability in qualitative research. retrieved from http://qualitativeresearch.ratcliffs.net/validity.pdf. rott, b. (2020). teachers’ behaviors, epistemological beliefs, and their interplay in lessons on the topic of problem solving. international journal of science and mathematics education, 18(5), 903-924. https://doi.org/10.1007/s10763-019-09993-0 sadıç, a. (2013). 8th grade students epistemological beliefs for pisa performances and science and technology literacy (unpublished master thesis). muğla sıtkı koçman university, muğla. sadler, t. d., & chambers, f. w. (2004). student conceptualizations of the nature of science in response to a socioscientific issue. international journal of science education, 26(4), 387–409 https://doi.org/10.1006/ceps.1999.1026 https://doi.org/10.3102/00346543067001088 https://doi.org/10.1016/j.learninstruc.2006.09.003 https://doi.org/10.33225/jbse/20.19.415 https://link.springer.com/book/10.1007%2f978-1-4020-6596-5 https://dergipark.org.tr/en/pub/mkusbed/issue/19552/208327 https://doi.org/10.1016/s0885-2014(00)00030-7 https://doi.org/10.14697/jkase.2020.40.4.359 https://doi.org/10.1353/rhe.1992.0013 https://doi.org/10.18510/hssr.2020.83130 https://doi.org/10.1016/s0732-118x(02)00008-9 https://doi.org/10.1007/s11409-009-9050-8 https://doi.org/10.1080/2331186x.2020.1763230 https://www.meb.gov.tr/bilim-ve-sanat-merkezleri-gucleniyor/haber/21827/tr#:~:text=halen%2081%20ilde%20182%20b%c4%b0lsem,bin%20%c3%b6%c4%9frenci%20bu%20merkezlerden%20yararlan%c4%b1yor https://www.meb.gov.tr/bilim-ve-sanat-merkezleri-gucleniyor/haber/21827/tr#:~:text=halen%2081%20ilde%20182%20b%c4%b0lsem,bin%20%c3%b6%c4%9frenci%20bu%20merkezlerden%20yararlan%c4%b1yor https://www.meb.gov.tr/bilim-ve-sanat-merkezleri-gucleniyor/haber/21827/tr#:~:text=halen%2081%20ilde%20182%20b%c4%b0lsem,bin%20%c3%b6%c4%9frenci%20bu%20merkezlerden%20yararlan%c4%b1yor https://www.meb.gov.tr/bilim-ve-sanat-merkezleri-gucleniyor/haber/21827/tr#:~:text=halen%2081%20ilde%20182%20b%c4%b0lsem,bin%20%c3%b6%c4%9frenci%20bu%20merkezlerden%20yararlan%c4%b1yor https://doi.org/10.18844/cjes.v15i1.4557 https://link.springer.com/article/10.1023/b:high.0000016445.92289.f1 https://link.springer.com/article/10.1023/b:high.0000016445.92289.f1 https://doi.org/10.1080/105735600278060 http://qualitativeresearch.ratcliffs.net/validity.pdf https://doi.org/10.1007/s10763-019-09993-0 journal of science learning article doi: 10.17509/jsl.v4i3.31995 287 j.sci.learn.2021.4(3).275-287 saylan, a., öner-armağan, f, & bektaş, o. (2016). the relationship between pre-service science teachers' epistemological beliefs and preferences for creating a constructivist learning environment. european journal of science and mathematics education, 4(2), 251-267. saylan-kırmızıgul, a., & bektaş, o. (2019). investigation of pre-service science teachers' epistemological beliefs. cypriot journal of educational sciences, 14(1), 146-157. schommer, m. (1990). effects of beliefs about the nature of knowledge on comprehension. journal of educational psychology, 82(3), 498-504. https://psycnet.apa.org/doi/10.1037/0022-0663.82.3.498 schommer, m. (1993a). comparisons of beliefs about the nature of knowledge and learning among post-secondary students. research in higher education, 34(3), 355-370. schommer, m. (1993b). epistemological development and academic performance among secondary schools. journal of educational psychology, 85(3), 406411. schommer, m. (1994). synthesizing epistemological belief research: tentative understandings and provocative confusions. educational psychology review, 6(4), 293-319. https://link.springer.com/article/10.1007%252fbf02213418 schommer, m. (1998). the influence of age and education on epistemological beliefs. british journal of educational psychology, 68(4), 551-562. https://doi.org/10.1111/j.2044-8279.1998.tb01311.x schommer, m., & dunnell, p. a. (1992). epistemological beliefs among gifted and non-gifted students. paper presented at the annual meeting of the american educational research association. san francisco, california. schommer, m., & dunnell, p. a. (1997). epistemological beliefs of gifted high school students. roeper review, 19(3), 153-156. https://doi.org/10.1080/02783199709553812 schommer, m., crouse, a., & rhodes, n. (1992). epistemological beliefs and mathematical text comprehension: believing it is simple does not make it so. journal of educational psychology, 84(4), 435. schommer-aikins, m., & hutter, r. (2002). epistemological beliefs and thinking about everyday controversial issues. the journal of psychology, 136(1), 5–20. schommer-aikins, m., mau, w.c., brookhart, s., & hutter, r. (2000). understanding middle students' beliefs about knowledge and learning using a multi-dimensional paradigm. the journal of educational research, 94(2), 120-127. https://doi.org/10.1080/00220670009598750 sinatra, g. m., southerland, s. a., mcconaughy, f., & demastes, j.w. (2003). intentions and beliefs in students’ understanding and acceptance of biological evolution. journal of research in science teaching, 40, 510–528. smith, j. a., & osborn, m. (2009). interpretative phenomenological analysis. in j. a. smith, (ed.). qualitative psychology: a practical guide to research methods (pp. 53-80). london: sage. sung, g. s., shin, m. k., & kim, e. j. (2020). analyzing epistemological beliefs in science as perceived by elementary school students. journal of the korean earth science society, 41(2), 147-154. https://doi.org/10.5467/jkess.2020.41.2.147 suzuki-parker, j., & higginbotham, g. (2019). does method of administration influence word association test responses?. vocabulary education research bulletin, 8(1), 11-16. tanrıverdi, b. (2012). pre-service teachers’ epistemological beliefs and approaches to learning. procedia-social and behavioral sciences, 46, 2635-2642. https://doi.org/10.1016/j.sbspro.2012.05.538 taşdere, a., özsevgeç, t., & türkmen, l. (2014). a complementary measuring tool for the nature of science: word association test [bilimin doğasına yönelik tamamlayıcı bir ölçme aracı: kelime ilişkilendirme testi]. science education and research association journal of science education, 2(2), 129-144. https://dergi.fead.org.tr/wpcontent/uploads/v-fenteke_d1811-ahmet-tesdere.pdf tickle, e. i., brownlee, j., & nailon, d. (2005). personal epistemological beliefs and transformational leadership behaviours, the journal of management development, 24, 1-15. https://eprints.qut.edu.au/1056/1/brownlee_eaef3033.pdf torkar, g., & bajd, b. (2006).trainee teachers’ ideas about endangered birds. journal of biological education, 41(1), 5-8. https://doi.org/10.1080/00219266.2006.9656049 tortop, h. s. (2015). a comparison of gifted and non-gifted students’ self-regulation skills for science learning. journal for the education of gifted young scientists, 3(1), 42-57. doi: http://dx.doi.org/10.17478/jegys.2015112017 trautwein, u., & lüdtke, o. (2007). epistemological beliefs, school achievement, and college major: a large-scale longitudinal study on the impact of certainty beliefs. contemporary educational psychology, 32(3), 348-366. https://doi.org/10.1016/j.cedpsych.2005.11.003 tüken, g. (2010). determining scientific epistemological beliefs of students in urban and rural places (unpublished master thesis). eskişehir osmangazi university, eskişehir. uçar, f. m. (2018). investigation of gifted students' epistemological beliefs, self-efficacy beliefs and use of metacognition. journal for the education of gifted young scientists, 6(3), 1-10. doi: http://dx.doi.org/10.17478/jegys.2018.77 uluçınar sağır, s. (2017). science perceptions of prospective class teachers. journal of education and practice, 8(15), 91-102. https://files.eric.ed.gov/fulltext/ej1143822.pdf üztemur, s., & dinç, e. (2018). a student-centered approach to explore middle school students’ epistemological beliefs: draw-write-tell technique [ortaokul öğrencilerinin epistemolojik inançlarının keşfedilmesinde öğrenci merkezli bir yaklaşım: çiz-yaz-anlat tekniği]. journal of history culture and art research, 7(3), 566-592. http://dx.doi.org/10.7596/taksad.v7i3.1579 wang, x., zhou, j., & shen, j. (2016) personal epistemology across different judgement domains: effects of grade level and school curriculum. educational psychology, 36(1), 159-175. https://doi.org/10.1080/01443410.2014.915932 wiersma, w., & jurs, s.g. (2005). research methods in education: an introduction. boston: ally and bacon. wimmer, r.d., & dominick, j.r. (2000). mass media research: an introduction. belmont: wadsworth publishing company. http://www.rogerwimmer.com/mmr/mediaeffectsresearch.htm yenice, n., & özden, b. (2013). analysis of scientific epistemological beliefs of eighth-graders. international journal of education in mathematics, science and technology, 1(2), 107-115 yeşilyurt, e. (2013). scientific epistemological beliefs of primary school students [i̇lköğretim okulu öğrencilerinin bilimsel epistemolojik inançları]. international journal of social science, 6(1), 1587-1609. yıldırım, a., & şimşek, h. (2013). qualitative research methods in social sciences [sosyal bilimlerde nitel araştırma yöntemleri]. ankara: seçkin publishing. yilmaz-tüzün, ö., & topcu, m. s. (2010). investigating the relationships among elementary school students’ epistemological beliefs, metacognition, and constructivist science learning environment. journal of science teacher education, 21(2), 255-273. https://doi.org/10.1007/s10972-009-9163-6 yun, e. (2020). correlation between concept comprehension and mental semantic networks for scientific terms. research in science & technological education, 38(3), 329-354. https://doi.org/10.1080/02635143.2020.1777095 zeineddin, a., & abd-el-khalick, f. (2010). scientific reasoning and epistemological commitments. coordination of theory and evidence among college science students. journal of research in science teaching, 47(9), 1064–1093. https://doi.org/10.1002/tea.20368 https://psycnet.apa.org/doi/10.1037/0022-0663.82.3.498 https://link.springer.com/article/10.1007%252fbf02213418 https://doi.org/10.1111/j.2044-8279.1998.tb01311.x https://doi.org/10.1080/02783199709553812 https://doi.org/10.1080/00220670009598750 https://doi.org/10.5467/jkess.2020.41.2.147 https://doi.org/10.1016/j.sbspro.2012.05.538 https://dergi.fead.org.tr/wp-content/uploads/v-fenteke_d1811-ahmet-tesdere.pdf https://dergi.fead.org.tr/wp-content/uploads/v-fenteke_d1811-ahmet-tesdere.pdf https://eprints.qut.edu.au/1056/1/brownlee_eaef3033.pdf https://doi.org/10.1080/00219266.2006.9656049 http://dx.doi.org/10.17478/jegys.2015112017 https://doi.org/10.1016/j.cedpsych.2005.11.003 https://files.eric.ed.gov/fulltext/ej1143822.pdf http://dx.doi.org/10.7596/taksad.v7i3.1579 https://doi.org/10.1080/01443410.2014.915932 http://www.rogerwimmer.com/mmr/mediaeffectsresearch.htm https://doi.org/10.1007/s10972-009-9163-6 https://doi.org/10.1080/02635143.2020.1777095 a © 2022 indonesian society for science educator 14 j.sci.learn.2022.5(1).14-27 received: 10 february 2021 revised: 19 october 2021 published: 1 march 2022 the effects of the problem-based learnıng supported by experiments in science course: students' inquiry learning and reflective thinking skills huriye deniş-çeliker1*, seda dere2 1department of mathematics and science education, faculty of education, burdur mehmet akif ersoy university, burdur, turkey 2hamidiye secondary school, afyonkarahisar, turkey *corresponding author. huriyedenis@mehmetakif.edu.tr abstract this study aims to investigate the effect of the problem-based learning method supported by experiments on the inquiry learning skills of secondary school students and their reflective thinking skills for problem-solving. this research uses a quasi-experimental design with a pretest-posttest control group. this study consists of 21 students in the experimental group and 22 students in the control group from the sixth grade of a public secondary school in turkey. the inquiry learning skills and reflective thinking skills scale for problem-solving were used as data collection tools in the research. the electricity transmission unit was carried out in the experimental group with problem-based learning scenarios supported by the experiment. activities based on the science course curriculum were carried out in the control group. spss-21 was used to analyze the data. as a result of the study, it was concluded that the post-test scores of the inquiry learning skills scale of the students in the experimental group differed statistically significantly from those of the control group students, and this difference was in favor of the experimental group. no significant difference was found between the experimental and control groups in post-test scores regarding reflective thinking skills for problem-solving. keywords problem-based learning supported with experiment, inquiring learning skills, reflective thinking skills, science education 1. introduction there are skills that individuals living in the 21st century must have to survive in business life. 21st-century skills express the characteristics that enable individuals to become good citizens and qualified workers in the information society of this century (cansoy, 2018). individuals who are today's students will be adults of the future. some professions that exist today will disappear, and new jobs will emerge. in this respect, it is stated that it is essential for students to prepare for new professions and acquire these skills by creating quality learning times rather than transferring a lot of information in schools (organisation for economic co-operation and development [oecd], 2018). one of the aims of science education is to raise individuals with critical, creative, questioning, reflective, and high problem-solving skills among 21st-century skills. it is necessary to comprehend the importance of science by developing their inquiry skills to provide students with 21st-century skills. science course is an inquiry-based course by its nature (okumuş & yetkil, 2020). therefore, it is necessary to use science courses effectively to develop inquiry skills (yaşar & duban, 2009). inquiry skills are defined as asking questions about the learned subject, searching for responses, generating and creating the latest information while gathering information about any topic, and reflecting on experiences (taşkoyan, 2008). inquiry learning is also a lifelong learning skill that allows students to learn meaningfully and permanently by researching and questioning (i̇nel-ekici, 2017). it has been determined that students who make inquisitorial reflection spend more time on problem-solving processes and control and organize their problem-solving strategies more (wopereis, brand-gruwel, & vermetten, 2007). raising students with such skills requires their close relationship with inquiry-based learning (ibl) (howe, 2002). mailto:huriyedenis@mehmetakif.edu.tr journal of science learning article doi: 10.17509/jsl.v5i1.32076 15 j.sci.learn.2022.5(1).14-27 the questions asked in international exams such as the program for international student assessment (pisa) and trends in international mathematics and science study (timss) require students to use their inquiry skills and establish the context of knowledge and daily life. countries that are successful in these exams include these skills in their teaching processes (firman, ertikanto & abdurrahman, 2018). in these international exams, students in turkey are below the average (ministry of national education, 2009, 2011a, 2011b, 2012, 2015a, 2015b, 2018). according to the results of this exam, students do not have enough critical thinking and inquiry skills (dolapçıoğlu, 2020; baran, baran & maskan, 2018). inquiry learning requires the active participation of students in the process (balım, i̇nel & evrekli, 2008). at the heart of an inquiry-based class is questioning (stotter & gillon, 2011). inquiry-based learning can be applied with different methods like case-based learning, project-based learning, and problem-based learning (bezen & bayrak, 2020). for the pbl process to be carried out effectively, students should have the ability to learn by inquiry or develop these skills of students over time (uden & beaumont, 2006). while working towards the problem's solution, students build content knowledge and problemsolving skills (hung, jonassen, & liu, 2008). inquiry learning skills (ils) are one of these skills. scientific inquiry involves researching natural phenomena through higher-order thinking skills or experimentation (lee, hart, cuevas, & enders, 2004). for this reason, inquiry learning appears in research on science teaching (howes, lim, & campos, 2009). ibl is the process of problem-solving by asking questions, researching and analyzing information, and transforming learned information into useful information (zion & sadeh, 2007). inquiry learning in science improves students' perception skills by using their mental and physical skills to explain the results and explanations about events in nature and the world (harlen, 2004). therefore, laboratory applications are essential in science lessons (hofstein & lunetta, 2004). it has been proven by many studies that laboratory activities carried out on an interrogative basis are more effective in the development of many cognitive and affective skills (hofstein, shore, & kipnis, 2004; akpınar & yıldız, 2006; duru, demir, önen, & benzer, 2011). the inquiry learning process begins with questions that can answer using observation or experimentation data (yıldırım & can, 2018). the laboratory application, in which students investigate the teacher's problem through a procedure they designed, is problem-based (bell, smetana, & binns, 2005). it requires reflective thinking to conduct an inquiry and understand the entire inquiry process (van der schaaf, baartman, prins, oosterbaan & schaap, 2013). as a result of confronting students with problem situations, critical awareness will be created, and reflective thinking skills (rts) will develop. in this respect, the problem-solving process is directly related to the deliberative thinking process (demirel, derman, & karagedik, 2015; albayrak, şimsek, & yazıcı, 2018), and it encourages students to maintain their interests (epstein, 2003). reflective thinking enables students to show highlevel thinking skills to understand and solve a problem (song, grabowski, koszalka, & harkness, 2006). in the problem-solving process, reflection can be seen (wilson & jan, 1993). therefore, using rts in problem-solving helps to provide more in-depth learning and increase awareness in these processes (junsay, 2016). rts is not a spontaneous type of thinking (alp & taşkın, 2008). reflective thinking is a cognitive feature that is learned and developed consciously (durdukoca & demir, 2012). therefore, acquiring this skill in the school environment (demiralp, 2010). rts emerge most often when encountering a non-routine problem, in learnercentered environments, or when students are active and involved in collaborative problem solving (mezirow, 1997). in the reflective thinking process, inquiry, reasoning, and evaluation actions are taken (mansvelder-longayroux, beijaard, verloop & vermun, 2007). reflection is considered a fundamental way of thinking for teaching and learning and encourages thinking of problems from multiple angles. thus, it includes rethinking unexpected issues to see the experience differently (ekiz, 2006). educational theorists suggested that reflective thinking and similar reflection methods should improve students' decision-making skills in all classes (kuhn, 1990). rts; the individual's ability to put forward many thoughts for the problems s/he faces, question these thoughts, and make evaluations by using their past and present experiences (kember et al., 2000). based on the idea that reflective thinking will occur when a problem is encountered, the researchers discussed the rts for problem-solving in their studies (bayrak & usluer, 2011; yenilmez & turgut, 2016). rts for problem-solving is an effective high-level thinking style of reflective thinking that is performed from understanding the problem to solving the problem in problem-solving (saygılı & atahan, 2014). there is a significant relationship between elementary school students' rts and their academic achievement in science and mathematics (baş, 2013; baş & kıvılcım, 2013). however, the students' general reflective thinking skills were low in turkey (erdoğan, 2019). similarly, kholid, sa’dijah, hidayanto, & permadi (2020) determined that 21 out of 140 mathematics students used reflective thinking skills. the primary school teacher candidates have moderate levels of reflective thinking in science concepts and do not have high-level reflective thinking skills (bozan, 2021). it requires reflective thinking to conduct an inquiry and understand the entire inquiry process (van der schaaf, baartman, prins, oosterbaan & schaap, 2013). based on the relevant literature, inquiry learning skills and reflective thinking processes are related to problemjournal of science learning article doi: 10.17509/jsl.v5i1.32076 16 j.sci.learn.2022.5(1).14-27 solving. in addition, it is expected that the experimental courses will be practical on inquiry skills. one of the methods that will enable problem-solving processes in educational environments is problem-based learning (pbl). pbl is an educational method that allows students to be active participants in the learning process and advocates learning subjects and concepts related to learning areas by solving their problems (smith & hung, 2017). pbl is a method that is organized around the solution of open-ended real-life issues in a subject that students have not learned before and is based on learning experiences that enable students to learn by practicing and experiencing (hmelo & ferrari, 1997; marklin-reynolds & hancock, 2010; lu, lajoie, & wiseman, 2010). pbl assumes that the learner learned how to determine their knowledge about a perceived problem, identified their learning needs, and determined how to obtain best the relevant information (dahlgren & öberg, 2001). in this process, the analysis of the problem in the scenarios, problem-solving, cooperative learning with group interaction, reflection, and evaluation of learning results are carried out by students (huang, huang, wu, chen, & chang, 2016). the purpose of the pbl is to help students to construct their knowledge by their knowledge base and to develop the high-level thinking skills needed today (hmelo-silver, 2004; cantürk-günhan & başer, 2009; usta & mirasyedioğlu, 2017). in the pbl method, scenarios are designed to enable students to learn meaningfully, the permanence and transfer of information are supported, and at the same time encouraged to reflect (onan, 2013). inquiry learning and reflective thinking processes are expected to be developed with the pbl process supported by experiments. in the literature, there are studies on the development of a measuring tool for reflective thinking and applied at different education levels (badger, 2010; başol & evingencel, 2013; taşkın-can & yıldırım, 2014) and studies on improving reflective thinking for problem-solving (hoffman & spatariu, 2011; rieger, radcliffe, & doepker, 2013; chee-choy & san oo, 2012). there is a significant relationship between secondary school students' ils towards mathematics and rts (katrancı & şengül, 2020). some inquiry-based studies are related to the development of measurement tools (wu & hsieh, 2006; aldankarademir & saracaloğlu, 2013). studies support the pbl method with web, computer, and simulation (belland, 2010; hwang, kuo, chen, & ho, 2014; ioannou, brown, hannafin, & boyer, 2009). in addition, there are studies at different learning levels supported by concept maps (johnstone & otis, 2006) and concept cartoons (balım, deniş-çeliker, türkoğuz, evrekli & i̇nel-ekici, 2015). yıldız (2010) investigated the effects of practical applications on different variables in solving problembased learning scenarios. it has been determined that the use of pbl supported by experiments in pre-service science teachers has a positive effect on pre-service science teachers' critical thinking skills (deniş-çeliker, 2020). although studies are examining the impact of pbl practices on different variables in science teaching in the related literature, there is no research examining the perception of experimental-supported pbl practices on middle school 6th-grade students' ils and rts. both skills are skills that need to be developed in science teaching. for this reason, science education researchers should research which methods, techniques, and strategies can develop these skills. for this reason, the effect of the experimentally supported pbl method was investigated in this study. this study aimed to determine the impact of the pbl method supported by experiments on students' ils and rts towards problem-solving. for this purpose, the following questions will be answered: 1. how is the effect of the pbl method supported by the experiments on secondary school 6th-grade students’ ils scores and rts for problem-solving scores? 2. how is the effect of lessons taught with the content and activities of the science curriculum on secondary school 6th-grade students’ ils scores and rts for problem-solving scores? 3. do the effects of the pbl method supported by the experiments and lessons were taught with the content and activities of the science curriculum on the ils scores and rts for problem-solving scores of secondary school 6th-grade students differ? 4. is there a significant difference between the experimental and control groups according to the posttest scores of the ils scale sub-dimensions? 5. is there a significant difference between the experimental and control groups according to the posttest scores of the rts sub-dimensions for the problem-solving scale? 6. is there any relationship between ils scores and rts for problem-solving scores? 2. method 2.1. research design a quasi-experimental design with pre-test, post-test control groups (karasar, 2003) was used in the research. creswell (2003) states that whether the subjects will be in the experimental or control group should be determined by a completely random assignment method in the experimental process. the testing method is summarized in table 1. 2.2. participants in the 2015-2016 academic year, the research participants at a secondary school constitute grade 6 students located in the burdur province in turkey. the experimental group consists of 21 students studying in the 6d class of this secondary school, while the control group consists of 22 students studying in the 6a class of the same school. in addition, 12 of the students in the experimental journal of science learning article doi: 10.17509/jsl.v5i1.32076 17 j.sci.learn.2022.5(1).14-27 group were female, and nine were male (see in figure 1). of the students in the control group, 14 were female, and 8 were male. in turkey, the end-of-term grade of the courses in secondary school is scored as a maximum of 5 and a minimum of 1. the participants' distribution according to the previous semester's science course averages is presented in figure 2. all participants are 12 years old. 2.3. data collection tools 2.3.1. inquiry learning skills scale the research used the "inquiry learning skills perception scale for science" developed by balım & taşkoyan (2007). the scale consists of 22 perception items. the scale consists of three sub-factors. these are the negative perception items dimension with a cronbach’s alpha reliability coefficient of 0.73, the dimension of positive perception items with a reliability coefficient of 0.67, and the sense of questioning the accuracy with a reliability coefficient of 0.71. cronbach’s alpha coefficient for the whole scale is 0.84. examples of scale items are given below: -i argue with my friends to decide the accuracy of my experimental results. experiment, which is one of the ways scientists work, seems boring to me. -i think i have to experiment to get scientific results. 2.3.2. reflective thinking skills scale for problemsolving the study used the “reflective thinking skill scale for problem-solving” developed by kızılkaya & aşkar (2009), consisting of 14 questions. the scale consists of 14 items and three sub-factors. the cronbach’s alpha value of the inquiry factor was 0.73, the value of the reason factor was 0.71, and the value of the element was 0.69. this value is 0.83 for all scale items. examples of the things of the scale are given below: when i cannot solve a problem, i ask myself questions to understand why i cannot solve it. when i read a problem, i think about what information i need to solve it. after solving the problem, i compare it with the solutions of my friends and evaluate my result. table 1 quasi-experimental research pattern of the study group pre-test method time period post-test experimental ilss rtssps problem-based learning supported by an experiment in conduction of electricity unit. 4–5 weeks (16 class hours) ilss rtssps control lessons carried out activities based on the science lesson curriculum. ills inquiry learning skills scale rtssps reflective thinking skills scale for problem-solving figure 1 distribution of participants by gender male female 0 10 20 experimental group control group 9 8 12 14 p a rt ic ip a n ts male female figure 2 the distribution of the participants according to the science course averages of the previous semester experimental group control group 0 5 10 15 1 point 2 points 3 points 4 points 5 points 1 0 3 6 11 0 2 6 4 10 p a rt ic ip a n ts experimental group control group journal of science learning article doi: 10.17509/jsl.v5i1.32076 18 j.sci.learn.2022.5(1).14-27 2.4. the experimental group implementation the research was carried out in the "conduction of electricity" unit in the 6th-grade science lesson. the tasks were carried out with the pbl method supported by experiments during the experimental group unit. for the experimental group to adapt to the problem-based learning process and to eliminate possible malfunctions, the last topic of the previous unit was covered with a module consisting of three scenarios. thus, the team is completed in two modules, seven sessions. the first module, conductors / insulators, consists of 4 procedures. the second module consists of 3 scenarios on resistance. based on the scenarios, the students were asked to design seven experiments. since problem-based scenarios were applied together with the experiments in the science lesson, the experiment phase was added to the process. the application was made according to the steps specified by deniş-çeliker (2021), which is indicated in figure 3. as seen in figure 3, the scenario was prepared first. then the students read the plan and identified the problem. next, they designed experiments to solve the problem and reveal their solutions. finally, observation form, peer and self-assessment, and process assessment were made at the last stage. one of the scenarios used in the study is presented below: aylin and her family boarded a fully equipped passenger plane to visit her aunt in america. while the plane was underway, it had to make an emergency landing on an island in the pacific ocean due to a malfunction in its engine. passengers thought about how the planes flying over the island to get off the island might get their attention. aylin: “let's set up a simple electrical circuit. noticing the light, the planes can save us. " she said. passengers brought a 100-watt light bulb and 50-meter cable from the plane. 1. passenger, copper, and short cable; 2. passenger, copper, and extended cable; 3. passenger brings nickel-chrome and cable as long as the 1st passenger. aylin was surprised to see that the brightness of the bulbs in the prepared circuits was not the same. the questions asked about the scenario are given below: ● what is the problem to be addressed in the system? ● what information can we research to solve the situation in the design? ● what do we know? ● what could be the reason for the different brightness of the bulbs? ● are there other factors affecting the brightness of the bulb? ● why are copper and aluminum wires preferred in electrical circuits? ● design 3 different experiments in which you can observe the effect of the length of the wire, the cross-section of the wire, and the type of wire on the resistance. describe your experimental setups by drawing. during the experimental studies, the photographs taken while the students were working in groups, designing their experiments, and filling out the worksheets are presented below in figure 4. 2.5. the control group implementation the lessons carried out activities based on the science lesson curriculum in the control group. since 2005, the science curriculum in turkey has been prepared based on the student-centered constructivist approach. activities in figure 3 the stages phases of problem-based scenarios with experiments preparing scenario scenarios based on true or possible stories are created in the direction o learning targets . reading scenario the text of the scenario is read in order to understand the content of scenarios and subjects. determining the problem students may take some notes for determining the problem after reading scenarios students may analyze the suject by doing group discussion. so, the problem may be determined and defined. solving the problem within the scope of reference and sources, the solution is searched, examined and the knowledge is described. students needed various soruce and materials to research. so, soruces should be provided such as text books, results of computer simulations, results of laboratory and field research, professional magazines, wesites, articles, brochures. designing an experiment students apply experiments by designing those based on the data obtained as a result of research. the problem situation in the scenario is explained. assessment the teacher and students evaluate the process together. in this process, group observation form, peer, selfevaluatioin form may be benefitted from. journal of science learning article doi: 10.17509/jsl.v5i1.32076 19 j.sci.learn.2022.5(1).14-27 this group included activity exercises in official textbooks, smartboard exercises, listening to the teacher, and answering questions. students in this group did an experiment based on the book. they didn’t design the experiment themselves. often observed by the students who made teacher experiment in demonstration experiments. 2.6. students outcomes of the unit the students are expected to achieve the following achievements with this unit (ministry of national education, 2013): 2.6.1. conductive and insulating materials recommended duration: 6 lesson hours subject / concepts: conductive materials, insulating materials, conductive and insulating materials usage areas 2.6.1.1. the student classifies materials according to whether they conduct electricity or not by using the electrical circuit s/he designed. 2.6.1.2. explain with examples from daily life for which purposes the electrical conductivity and insulating properties of materials are used. 2.6.2. electrical resistance and related factors recommended duration: 10 lesson hours subject / concepts: electrical resistance, factors on which electrical resistance depends (cross-sectional area, length, type of conductor) 2.6.2.1. estimates the variables on which the light bulb's brightness in an electrical circuit depends and tests its predictions by experimenting. 2.6.2.2. it measures the resistance of a conductor by expressing electrical resistance and specifies its unit. 2.6.2.3. s/he realizes that the bulb is also composed of a conductive wire and has resistance. 2.7. data analysis the research was conducted with two groups, an experiment, and a control. the data collected from these groups were analyzed. the statistics package for social sciences 21.0 (spss) was used to evaluate the study's data. since the data didn’t deviate significantly (excessively) from the normal distribution, for comparison of the pre-test and post-test scores, independent samples t-test, pairedsamples t-test, and manova were used. in addition, whether there is a relationship between the students’ ils scores and rts for problem-solving scores has been determined by using pearson product-moment correlation. 3. result and discussion 3.1. the result and discussion for the first research question a comparison of the experimental group's pre-test and post-tests of ils is presented in table 2, and the figure 4 some photos of students during experimental studies journal of science learning article doi: 10.17509/jsl.v5i1.32076 20 j.sci.learn.2022.5(1).14-27 comparison of rts's pre-test and post-tests is shown in table 3. table 2 the paired samples t-test results regarding the comparison of the pre-test post-test ils scores of the students in the experimental group. a significant increase was found in students' ils after pbl practices supported by experiments t(20)=-4.88, p < 0.05. this finding shows that pbl application supported by experiments had a significant effect on improving students' ils were shown in table 2. the results revealed that the impact of the pbl method supported by experiments on the development of students' perceptions of ils was statistically significant. pbl prepares the ground for developing students' inquiry thinking skills through problems (barrows, 1996). kamin, o'sullivan, younger, & deterding (2001) stated in their study that students' defining issues and logically evaluating solutions in the process of problem-solving could improve their inquiry skills. in the pbl process, students use high-level thinking skills such as inquiry learning and problem-solving (i̇nel, 2012). as seen in table 3, when the pre-test post-test rts scores of the experimental group were compared, there was a significant difference in favor of the post-tests. thus, developmentally and age-appropriate problem-based learning environments that support reflective thinking should be designed. furthermore, they argued that problem-based learning includes a mechanism that increases reflective thinking (song, grabowski, koszalka, & harkness, 2006). in this study, a significant difference was found between the pre-test and post-test scores of the experimental group in which pbl was applied. methods such as learning diaries, concept maps, asking questions, collaborative learning, self-evaluation, interview, portfolio (student product file), peer review, discussion, observation, seminar studies, oral presentations, autobiography, drama, visual presentations used to develop reflective thinking (kazu & demiralp, 2012). the experimental group conducted selfassessment, peer evaluation, questions, and discussion during the pbl process. that can explain the increase in the experimental group itself. 3.2. the result and discussion for the second research question a comparison of the control group's pre-test and posttests of ils is presented in table 4, and the comparison of rts's pre-test and post-tests is shown in table 5. table 4 the paired samples t-test results regarding comparing the pre-and post-test ils scores of the students in the control group. when the pre-test-post-test inquiry learning skill scores of the students in the control group were compared, it was concluded that there was no statistically significant difference seen in tablo 4. likewise, there was no statistically significant difference between the experimental and control group students' pre-tests ils scores. when the pre-test-post-test reflective thinking scale scores of the students in the control group were compared, table 2 the paired samples t-test results regarding the comparison of the pre-test post-test ils scores of the students in the experimental group measurement n x̄ sd df t value p pre-test 21 87.66 13.72 20 -4.88 0.00* post-test 21 98.52 8.28 *p <0.05 difference is significant. cohen d =0.95 table 3 the paired samples t-test results regarding the comparison of the pre-test post-test rts scores of the students in the experimental group measurement n x̄ sd df t value p pre-test 21 50.61 12.94 20 -3.04 0.006* post-test 21 59.80 7.21 *p <0.05 difference is significant. cohen d =0.88 table 4 the paired samples t-test results regarding the comparison of the pre-test post-test ils scores of the students in the control group measurement n x̄ sd df t value p pre-test 22 83.04 8.84 21 0.423 0.67 post-test 22 82.09 7.06 p> 0.05 table 5 the paired samples t-test results regarding the comparison of the pre-test post-test rts scores of the students in the control group measurement n x̄ sd df t value p pre-test 22 54.63 5.07 21 -1.206 0.24 post-test 22 57.09 6.92 p> 0.05 journal of science learning article doi: 10.17509/jsl.v5i1.32076 21 j.sci.learn.2022.5(1).14-27 it was concluded that there was no statistically significant difference were shown in table 5. teachers believe that the curriculum's learning environment and evaluation process, in general, positively affect the development of students' reflective thinking (demiralp & kazu, 2012). since 2005, the science curriculum in turkey has been prepared based on the student-centered constructivist approach. for this reason, there are activities based on developing reflective thinking in the textbook. although not statistically significant, there was an increase in the reflective thinking skill scores of the control group students in this study. 3.3. the result and discussion for the third research question the comparison of the pre-test ils scale of the experimental and control groups is presented in table 6, and the post-tests are shown in table 7. as seen in table 6, students' pre-test inquiry learning skill scores didn’t differ according to the experimental and control groups. students' post-test inquiry learning skill scores differed significantly according to the group t(41)=7.01, p < 0.05. experimental group students' post-test inquiry learning skill scores were higher than control group students were shown in table 7. after the practical application, there was a statistically significant difference in favor of the experimental group. the teachers used the textbook as the primary source in the control group. textbooks have traditionally focused on memory and learning responses rather than inquiry skills (reaume, 2011). however, the students in the experimental group determined the problem based on the scenario and designed an experiment to solve this problem may have impacted their inquiry learning skills. the students in the experimental group had to use their inquiry learning skills. it has been revealed that the effect of problem-based learning on inquiry thinking skills is similar (kang, dechenne & smith, 2012). i̇nel (2009) reached a similar conclusion. inquiry activities improve students' inquiry skills, especially identifying a method and getting a result (cuevas, lee, hart, & deaktor, 2005). ibl can be expressed as a student-centered process in which students identify problems, create problems, and try to solve problems (wood, 2003; maaß, & artigue, 2013). the students had to use these skills in the experimentsupported problem-solving process. karamustafaoğlu & havuz (2016), who worked with prospective teachers, reached a similar result. they concluded that the laboratory activities based on ibl increased the future teachers' research inquiry skills in favor of the experimental group. in this study, secondary school students identified the problem themselves, found a solution to the problem they identified, solved it by doing and experiencing, and developed their ibl skills by designing a problem-based experiment. also, ibl was a practical way to link previous knowledge with definitions of the natural world (panasan & nuangchalerm, 2010). the comparison of the pre-test rts scores of the experimental and control groups are presented in table 8, and the post-tests are shown in table 9. as seen in table 8, pre-test rts scores of the students didn’t differ between the experimental and control groups. post-test rts scores of students didn’t differ between the experimental and control groups were shown in table 9. table 6 independent t-test results comparing the groups' pre-test ils scale scores group n x ̄ sd df t value p experiment 21 87.66 13.72 41 1.31 0.19 control 22 83.04 8.84 p> 0.05 table 7 independent t-test results comparing the groups' post-test ils scale scores group n x ̄ sd df t value p experiment 21 98.52 8.28 41 7.01 0.00 control 22 82.09 7.06 p < 0.05 cohen d =2.14 table 8 independent t-test results comparing the groups' pre-test rts scores group n x̄ sd df t value p experiment 21 50.61 12.94 25.77 -1.328 0.19 control 22 54.63 5.07 p > 0.05 table 9 independent t-test results comparing the groups' post-test rts scores group n x̄ sd df t value p experiment 21 59.80 7.21 41 1.26 0.21 control 22 57.09 6.92 p > 0.05 journal of science learning article doi: 10.17509/jsl.v5i1.32076 22 j.sci.learn.2022.5(1).14-27 it was determined that the effect of the pbl method supported by experiments on the development of students' rts for problem-solving was not statistically significant. furthermore, there is no significant difference between the experimental and control groups' rts post-test scores for problem-solving. the lack of difference in the post-tests of the experimental and control groups may be since the textbook activities applied in the control group also focused on developing reflective thinking. recently updated teaching programs at our schools positively affect students to acquire reflective thinking skills and inquiry learning (katrancı & şengül, 2020). this finding was in line with the study results (dadlı, 2017), which examined the effect of authentic pbl activities on secondary school students' rts. pusmaz & tavşan (2019) stated that students who are successful in problem-solving could show reflections in the themes of feelings/emotions and groupmatesalthough. however, they were successful in general knowledge, experience, and context pieces. nevertheless, they still have difficulty reflecting within the determined indicators, they can remember incompletely, or it was concluded that they could not make a reflection. in developing reflective thinking from preschool to 12th grade, teaching strategy, materials, student independence, collaboration efforts, individual attention of the teacher, and encouraging the student to the lesson are essential (king & kitchener, 2004). despite emphasizing student independence, collaboration, and material used in experiment-supported pbl, it was insufficient to develop students' rts. in a study, namvar, naderi, shariatmadari, & seifnaraghi (2009) found that a problem-solving weblog environment positively affects students' rts. 3.4. the result and discussion for the fourth research question the comparison of the ils scale pre-test and post-tests of the experimental group is presented in table 10. examining the findings in table 10, there was a difference in favor of the experimental group in the post-tests according to the students' positive perceptions in the experimental and control groups and the scores they received in the factors of questioning accuracy. there was also a significant difference in the negative perceptions factor in favor of the control group. with inquiry-type experiments, students develop many skills, such as asking better questions (hofstein, navon, kipnis, & mamlok‐ naaman, 2005). in a study investigating the effect of argumentationbased pbl on ils, it was found that the scores of the experimental groups in the positive perception subdimension and the perception of questioning accuracy subdimension were higher than the control groups. however, this difference is insignificant (yıldırım & can, 2018). with this difference between the results, it can be concluded that pbl supported by experiments was appropriate in developing ils sub-factors. 3.5. the result and discussion for the fifth research question the comparison of post-test scores of the sub-factors of the rts scale with the experimental and control groups is presented in table 11. when the post-test scores of the experimental and control groups were compared in terms of the questioning, assessment, and reasoning sub-factors of the scale, it was found that there was no statistical difference were shown in table 11. table 10 post-test manova results according to the factors of the experimental and control group students' ils scale factor group n x̄ sd f p positive perceptions experiment 21 41.28 3.59 18.04 0.00* control 22 37.00 3.00 negative perceptions experiment 21 10.66 2.92 47.90 0.00* control 22 20.22 5.64 questioning the accuracy experiment 21 31.90 3.03 9.30 0.04* control 22 29.31 2.51 * p <0.05 difference is significant. positive perceptions cohen d = 1.29; negative perceptions cohen d =, 2.13; questioning the truth of cohen d = 0.93 table 11 post-test manova results according to the factors of the experimental and control group students' rts scale factor group n x̄ sd f p questioning experiment 21 21.80 2.73 2.44 0.12 control 22 20.36 3.28 assessment experiment 21 21.14 2.53 1.92 0.17 control 22 19.95 3.04 reasoning experiment 21 15.19 1.53 0.22 0.79 control 22 15.04 2.01 journal of science learning article doi: 10.17509/jsl.v5i1.32076 23 j.sci.learn.2022.5(1).14-27 3.6. the result and discussion for the sixth research question the relationship between ils and rts and the subfactors of the scales are shown in table 12 and table 13. when the relationship between ils scores and rts for the problem-solving scores in table 12 over the pre-test and post-test total scores of the students is examined, it is seen that there is a moderately significant positive relationship between them. similarly, a moderate positive correlation was found between middle school students' inquiry learning skills and reflective thinking skills for problemsolving (katrancı & şengül, 2020). in a different study, a positive, significant, and moderate relationship was found between pre-service teachers' critical thinking and reflective thinking skills (aşkın-tekkol & bozdemir, 2018; erdoğan, 2020). according to the sub-factors of the scales is examined in table 13, the correlation of the pre-test and post-test scores is a moderate positive correlation between the assessment sub-dimension of the rts for problem-solving scale and the positive perceptions sub-dimension and questioning accuracy sub-dimension of the ils scale. the relation between the final ils scores and final rts scores in experimental and control groups is shown in table 14. table 14 points out a moderate, meaningful, and positive difference between the post-ils scores and rts scores in the experimental group. accordingly, it can be said that students with high final ils scores also have high rts scores. developing students' reflective thinking skills will also enable them to inquiry about their behavior (karaoğlan-yılmaz & keser, 2016). there was no relationship between the post-test ils scores of the control group students and the post-test rts scores. that can explain by the fact that the ils scores did not change much, even though the post-test rts scores of the students in the control group increased. conclusion although studies examine the effects of the pbl practices on different science teaching variables in the relevant literature, there is no study examining the perception of the pbl practices supported by experiments on the ils and rts of secondary school 6th-grade students. in addition, although studies integrate the pbl approach with different methods and techniques such as concept cartoons, argumentation, mobile applications, and computer technologies, a limited number of studies supported by experiments have been found. as a result of the research, there was a significant difference in the ils of secondary school 6th-grade students, in the positive perceptions, negative perceptions, and questioning the accuracy factors, which were subfactors of the ils scale. based on this, pbl activities supported by experiments can develop students' ils in science lessons. there was no significant difference between the experimental group in which pbl activities were carried out and the control group. the lessons were taught with the science lesson curriculum using the reflective thinking scale post-tests for problem-solving. after the practical application, there was no significant difference in the rts scale's questioning, assessment, and reasoning sub-factors for problem-solving. although it is stated in the theoretical table 12 the relationship between ils scores and rts for problem-solving scores measurement variables n r p pre-test ils scores rts for problem-solving scores 43 0.351* 0.021 post-test 0.311* 0.043 *p <0.05 table 13 correlation matrix according to subfactors of ils and rts for problem-solving scales measurement factor questioning assessment reasoning pre-test positive perceptions 0.29 0.36* 0.31* post-test 0.43* 0.45** 0.21 pre-test negative perceptions 0.11 0.12 0.01 post-test 0.54 0.08 0.07 pre-test questioning accuracy 0.30 0.39** 0.39** post-test 0.40** 0.34* 0.30 *p<0.05 **p< 0.01 table 14 the relation between the post-ils scores and post-rts scores in experimental and control groups group r p experiment post ils scores post rts scores 0.47* 0.032 control post ils scores post rts scores 0.006 0.98 *p< 0.05 journal of science learning article doi: 10.17509/jsl.v5i1.32076 24 j.sci.learn.2022.5(1).14-27 explanations of the relevant literature that pbl will improve rts, no significant difference was found between the post-test scores of the experimental and control groups in this exploratory study. longer practical practice time may be required for the development of rts. the reasons for this situation can be investigated in different researches. rts scale for problem-solving was used within the scope of the study. results can be compared using different rts scales. the research is limited to the transmission of the electricity unit of the 6th-grade science course. however, similar applications can be made in the departments related to electricity in the 5th, 7th, and 8th grades, and the results can be compared. different researchers and their validity developed the scales used within the scope of the study, and reliability studies were conducted. as a result, different scales of inquiry skills and reflective thinking skills could be used. within the scope of the study, data were collected with quantitative data collection tools. data triangulation can be achieved by incorporating qualitative data collection tools such as interviews and observation into the process. it can be replicated using the mixed research method. . references akpınar, e., & yıldız, e. (2006). investigation of the effect of openended experimental technique on students' attitudes towards the laboratory. dokuz eylül university buca faculty of education journal, 20, 69-76. albayrak, m., şimşek, m., & yazıcı, n. (2018). the predictive power to mathematical success of belief and reflective thinking for problem solving. journal of human sciences, 15(2), 807-815. aldan-karademir, ç., & saracaloğlu, a. s. (2013). developing the inquiry skills scale: validity and reliability study. asian journal of instruction, 1(2), 56-65. alp, s., & taşkın, ş. ç. (2008). importance of reflective thinking in education and developing reflective thinking. milli eğitim dergisi, 178, 311-320. aşkın-tekkol, i̇., & bozdemir, h. (2018). an investigation of reflective thinking tendencies and critical thinking skills of teacher candidates. kastamonu education journal, 26(6), 1897-1907. doi: 10.24106/kefdergi.2211. badger, j. (2010). assessing reflective thinking: pre‐service teachers’ and professors’ perceptions of an oral examination. assessment in education: principles, policy & practice, 17(1), 77-89. balım a.g., deniş-çeliker, h., türkoğuz, s., evrekli, e. & i̇nel-ekici d. (2015). the effect of concept cartoons supported problem based learning method on students' conceptual understanding levels and perceptions of problem solving skills. journal of turkish science education, 12(4), 53-76. doi:10.12973/tused.10151a balım, a. g., i̇nel, d., & evrekli, e. (2008). the effect of using concept cartoons in science teaching on students' academic achievement and perceptions of inquiry learning skills. ilkogretim online, 7(1), 188-202. balım, a., g., & taşkoyan, n. (2007). developing an inquiry learning skills perception scale for science. dokuz eylül university buca faculty of education journal, 21, 58-63. baran, m., baran, m., & maskan, a. (2018). evaluation of science pisa test results of students in turkey by physics teacher candidates. yyu journal of education faculty, 15(1), 1517-1539. doi:10.23891/efdyyu.2018.114 barrows, h., s. (1996). problem-based learning in medicine and beyond: a brief overview. new directions for teaching and learning, 68, 3-12. doi: 10.1002/tl.37219966804 başol, g., & evin-gencel, i. (2013). reflective thinking scale: a validity and reliability study. educational sciences: theory and practice, 13(2), 941-946. bayrak f., & usluer y., k. (2011). the effect of blogging on reflective thinking skills. hacettepe university journal of education faculty, 40, 93104. bell, r. l., smetana, l., & binns, i. (2005). simplifying inquiry instruction. the science teacher, 72(7), 30-33. belland, b., r., (2010). portraits of middle school students constructing evidence-based arguments during problem-based learning: the impact of computer-based scaffolds. educational technology research and development, 58(3), 285-309. doi: 10.1007/s11423-009-9139-4 bezen, s., & bayrak, c. (2020). determining students' attitudes and views using an inquiry-based learning approach. cukurova university faculty of education journal, 49(2), 555-599. doi: 10.14812/cufej.676679 bozan, s. (2021). determining students' reflective thinking levels and examining their reflections on science concepts. african educational research journal, 9(2), 544-550. doi: 10.30918/aerj.92.21.070 cansoy, r. (2018). uluslararası çerçevelere göre 21.yüzyıl becerileri ve eğitim sisteminde kazandırılması. i̇nsan ve toplum bilimleri araştırmaları dergisi, 7 (4), 3112-3134. doi: 10.15869/itobiad.494286 cantürk-günhan, b., & başer, n. (2009). the effect of problem-based learning on students' critical thinking skills. journal of turkish educational sciences, 7(2), 451-482. https://dergipark.org.tr/tr/pub/tebd/issue/26107/275071 chee-choy, s., & san oo, p. (2012). reflective thinking and teaching practices: a precursor for incorporating critical thinking into the classroom? international journal of instruction, 5(1), 167-181. https://files.eric.ed.gov/fulltext/ed529110.pdf creswell, j. w. (2003). qualitative, quantitative and mixed methods approaches. thousand oaks, ca: sage publications. cuevas, p., lee, o., hart, j. and deaktor, r. (2005). improving science inquiry with elementary students of diverse backgrounds. journal of research in science teaching, 42(3), 337–357. doi: 10.1002/tea.20053 dadlı, g. (2017). the effect of learning activities based on authentic problems in the human and environmental relations unit on the reflective thinking skills, academic success, environmental attitude and awareness of 7th grade students. (unpublished master’s thesis), kahramanmaraş sütçü i̇mam university, kahramanmaraş. dahlgren, m., a., & öberg, g. (2001). questioning to learn and learning to question: structure and function of problem-based learning scenarios in environmental science education. higher education, 41(3), 263-282. doi: 10.1023/a:1004138810465 demiralp, d. (2010). teachers' views on the effect of primary education programs on improving students' reflective thinking (example of elazig province). (unpublished master's thesis), fırat university institute of social sciences, elazig. demiralp, d., & kazu, h. (2012). ilkogretim birinci kademe programlarının ogrencilerin yansıtıcı dusunmelerini gelistirmedeki katkısına yonelik ogretmen gorusleri. pegem egitim ve ogretim dergisi, 2(2), 29-38. https://dergipark.org.tr/tr/pub/pegegog/issue/22589/241276 demirel, m., derman, i., & karagedik, e. (2015). a study on the relationship between reflective thinking skills towards problem solving and attitudes towards mathematics. procedia-social and behavioral sciences, 197, 2086-2096. doi: 10.1016/j.sbspro.2015.07.326 deniş-çeliker, h. (2021). problem-based scenario method with experiments: determining the prospective science teachers' biology self-efficacy and critical thinking tendency. science education international, 32(1), 23-33. doi: 10.33828/sei.v32.i1.3 duru, m., k., demir, s., önen, f., & benzer, e. (2011). the effect of inquiry-based laboratory applications on pre-service teachers' http://dx.doi.org/10.12973/tused.10151a https://doi.org/10.1002/tl.37219966804 https://doi.org/10.1002/tl.37219966804 https://doi.org/10.30918/aerj.92.21.070 http://dx.doi.org/10.15869/itobiad.494286 http://dx.doi.org/10.1002/tea.20053 http://dx.doi.org/10.1023/a:1004138810465 https://doi.org/10.1016/j.sbspro.2015.07.326 journal of science learning article doi: 10.17509/jsl.v5i1.32076 25 j.sci.learn.2022.5(1).14-27 attitude towards laboratory perception and scientific process skills. atatürk faculty of education journal of educational sciences, 33, 25-44. http://dspace.marmara.edu.tr/bitstream/handle/11424/853/904 -1756-1-sm.pdf?sequence=1&isallowed=y dolapçıoğlu, s. (2020). the effect of thinking classroom materials (dsm) on pisa reading skills. ana dili eğitimi dergisi, 8(1), 196210. doi: 10.16916/aded.658510 durdukoca, f. ş., & demir, m. (2012). the reflective thinking levels of primary school teachers according to some variables and the suitability of teacher qualifications in their thoughts with reflective teacher qualifications. mustafa kemal university institute of social sciences, 9(20), 357-374. https://dergipark.org.tr/tr/pub/mkusbed/issue/19549/208417 ekiz, d. (2006). self-observation and peer-observation: reflective diaries of primary student-teachers. elementary education online, 5(1), 45-57. https://dergipark.org.tr/en/download/articlefile/91061 epstein, a. s. (2003). how planning and reflection develop young children's thinking skills. young children, 58(5), 28-36. https://eric.ed.gov/?id=ej679112 erdoğan, a. (2019). examination of secondary school students' reflective thinking skills for problem solving in terms of some variables (unpublished master's thesis). necmettin erbakan university, konya. erdoğan, f. (2020). the relationship between prospective middle school mathematics teachers’ critical thinking skills and reflective thinking skills. participatory educational research, 7(1), 220-241. doi: 10.17275/per.20.13.7.1 firman, m., & ertikanto, c. & abdurrahman, a. (2019). description of meta-analysis of inquiry-based learning of science in improving students’ inquiry skills. journal of physics: conference series, 1157, 1-7. doi:10.1088/1742-6596/1157/2/022018 harlen, w. (2004). evaluating inquiry-based science developments: a paper commisisoned by the national reasearch council in preparation for a meeting on the status of evaluation of ınquirybased science education. cambridge: national academy of sciences. education, 26(1), 14-17. http://www7.nationalacademies.org/bose/wharlen_inquiry_mt g_paper.pdf. hmelo, c. e., & ferrari, m. (1997). the problem-based learning tutorial: cultivating higher order thinking skills. journal for the education of the gifted, 20(4), 401-422. doi: 10.1177/016235329702000405 hmelo-silver, c. e. (2004). problem-based learning: what and how do students learn?. educational psychology review, 16(3), 235-266. doi: 10.1023/b:edpr.0000034022.16470.f3 hoffman, b., & spatariu, a. (2011). metacognitive prompts and mental multiplication: analyzing strategies with a qualitative lens. journal of interactive learning research, 22(4), 607635. https://www.learntechlib.org/primary/p/35341/. hofstein, a., & lunetta, v. n. (2004). the laboratory in science education: foundations for the twenty‐first century. science education, 88(1), 28-54. doi: 10.1002/sce.10106 hofstein, a., shore, r., & kipnis, m. (2004). providing high school chemistry students with opportunities to develop learning skills in an inquiry-type laboratory: a case study. international journal of science education, 26(1), 47-62. doi: 10.1080/0950069032000070342 hofstein, a., navon, o., kipnis, m., & mamlok‐ naaman, r. (2005). developing students’ ability to ask more and better questions resulting from inquiry‐type chemistry laboratories. journal of research in science teaching, 42(7), 791–806. doi:10.1002/tea.20072 howe, a., c. (2002). engaging children in science (3rd ed.). upper saddle river, nj: werrill prentice hall. howes, e., v., lim, m., & campos, j. (2009). journeys into inquiry‐ based elementary science: literacy practices, questioning, and empirical study. science education, 93(2), 189-217. doi: 10.1002/sce.20297 huang, s., h., huang, y., m., wu, t., t., chen, h., r., & chang, s., m. (2016). problem-based learning effectiveness on micro-blog and blog for students: a case study. interactive learning environments, 24(6), 1334-1354. doi: 10.1080/10494820.2015.1004353 hung, w., jonassen, d. h., & liu, r. (2008). problem-based learning. in j. m. spector, j. g. van merriënboer, m. d., merrill, & m. driscoll (eds.), handbook of research on educational communications and technology (3rd, pp. 485–506). mahwah: erlbaum. hwang, g. j., kuo, f. r., chen, n. s., & ho, h. j. (2014). effects of an integrated concept mapping and web-based problem-solving approach on students' learning achievements, perceptions and cognitive loads. computers & education, 71, 77-86. doi: 10.1016/j.compedu.2013.09.013 ioannou, a., brown, s. w., hannafin, r. d., & boyer, m. a. (2009). can multimedia make kids care about social studies? the globaled problem-based learning simulation. computers in the schools, 26(1), 63-81. doi: 10.1080/07380560802688299 i̇nel, d. (2009). the effects of the using of problem based learning method in science and technology course on students’ the levels of constructing concepts, academic achievement and enquiry learning skill perceptions. master of thesis, dokuz eylül university, i̇zmir. i̇nel, d. (2012). the effects of problem-based learning method supported by concept cartoons on students' perceptions of problem solving skills, motivation for science learning and conceptual understanding. phd thesis, dokuz eylül university, i̇zmir. i̇nel-ekici, d. (2017). ortaokul öğrencilerinin bilimsel sorgulama becerileri algılarını etkileyen faktörlerin incelenmesi. kastamonu eğitim dergisi, 25 (2), 497-516. https://dergipark.org.tr/en/download/article-file/302519 johnstone, a., h., & otis, k., h. (2006). concept mapping in problem based learning: a cautionary tale. chemistry education research and practice, 7(2), 84-95. doi: 10.1039/b5rp90017d junsay, m., l. (2016). reflective learning and prospective teachers’ conceptual understanding, critical thinking, problem solving, and mathematical communication skills. research in pedagogy, 6(2), 43‐58. kamin, c. s., o'sullivan, p. s., younger, m., & deterding, r. (2001). measuring critical thinking in problem-based learning discourse. teaching and learning in medicine, 13(1), 27-35. doi: 10.1207/s15328015tlm1301_6 kang, n. h., dechenne, s. e., & smith, g. (2012). inquiry learning of high school students through a problem‐based environmental health science curriculum. school science and mathematics, 112(3), 147-158. doi: 10.1111/j.1949-8594.2011.00128.x karamustafaoğlu, s., & havuz, a. c. (2016). inquiry based learning and its effectiveness. international journal of assessment tools in education, 3(1), 40-54. https://dergipark.org.tr/en/pub/ijate/issue/22371/239553 karaoğlan-yilmaz, f. g., & keser, h. (2016). the impact of reflective thinking activities in e-learning: a critical review of the empirical research. computers & education, 95, 163-173. doi: 10.1016/j.compedu.2016.01.006 karasar, n. (2003). bilimsel araştırma yöntemi: kavramlar, ilkeler, teknikler.[scientific research method: concepts, principles, techniques]. ankara, turkey: nobel yayın dağıtım. katrancı, y., & şengül, s. (2020). evaluating the inquiry learning skills towards math of middle school students in terms of questioning, evaluating, reasoning and reflective thinking skills towards problem solving. education and science, 45(201), 55-78. doi: 10.15390/eb.2020.7765 kazu, h., & demiralp, d. (2012). usage status of methods that enhance reflective thinking in primary level programs (elazığ city example). international online journal of educational sciences, 4(1), 131145. kember, d., leung, d., jones, a., loke, a., mckay, j., sinclair, k., tse, h., webb, c., wong, f., wong, m., & yeung, e. (2000). development of a questionnaire to measure the level of reflective https://doi.org/10.16916/aded.658510 https://doi.org/10.17275/per.20.13.7.1 https://doi.org/10.1177%2f016235329702000405 https://www.learntechlib.org/primary/p/35341/ https://doi.org/10.1002/sce.10106 https://doi.org/10.1080/0950069032000070342 https://doi.org/10.1002/sce.20297 https://doi.org/10.1080/10494820.2015.1004353 https://doi.org/10.1016/j.compedu.2013.09.013 https://doi.org/10.1080/07380560802688299 https://doi.org/10.1039/b5rp90017d https://doi.org/10.1207/s15328015tlm1301_6 https://doi.org/10.1111/j.1949-8594.2011.00128.x https://dergipark.org.tr/en/pub/ijate/issue/22371/239553 https://doi.org/10.1016/j.compedu.2016.01.006 http://dx.doi.org/10.15390/eb.2020.7765 journal of science learning article doi: 10.17509/jsl.v5i1.32076 26 j.sci.learn.2022.5(1).14-27 thinking. assessment and evaluation in higher education, 25, 381–395. doi: 10.1080/713611442 kholid, m. n., sa’dijah, c., hidayanto, e., & permadi, h. (2020). how are students’ reflective thinking for problem solving? journal for the education of gifted young scientists, 8(3), 1135-1146. doi: 10.17478/jegys.688210 kızılkaya, g., & aşkar, p. (2009). the development of a reflective thinking skill scale towards problem solving. education and science, 34(154), 82-92. king, p. m., & kitchener, k. s. (2004). reflective judgment: theory and research on the development of epistemic assumptions through adulthood. educational psychologist, 39(1), 5-18. doi: 10.1207/s15326985ep3901_2 kuhn, d. (1990). developmental perspectives on teaching and learning thinking skills. new york: jossey-bass. lee, o., hart, j. e., cuevas, p., & enders, c. (2004). professional development in inquiry‐based science for elementary teachers of diverse student groups. journal of research in science teaching, 41(10), 1021-1043. doi: 10.1002/tea.20037 lu, j., lajoie, s. p., & wiseman, j. (2010). scaffolding problem-based learning with cscl tools. international journal of computer-supported collaborative learning, 5(3), 283-298. doi: 10.1007/s11412-0109092-6 maaß, k., & artigue, m. (2013). implementation of inquiry-based learning in day-to-day teaching: a synthesis. zdm mathematics education, 45(6), 779–795. doi: 10.1007/s11858-013-0528-0 mansvelder-longayroux, d., beijaard, d., verloop, n., & vermunt, j. (2007). functions of the learning portfolio in student teachers’ learning process. teachers college record, 109(1), 126-159. https://www.tcrecord.org/printcontent.asp?contentid=12738 marklin-reynolds, j., & hancock, d. r. (2010). problem‐based learning in a higher education environmental biotechnology course. innovations in education and teaching international, 47(2), 175186. doi: 10.1080/14703291003718919 mezirow, j. (1997). transformative learning: theory to practice. new directions for adult and continuing education, 74, 5-12. doi: 10.1002/ace.7401 ministry of national education. (2009). international student assessment program pisa 2009 national preliminary report. http://pisa.meb.gov.tr/?page_id=22 ministry of national education. (2011a). timss 2011 national math and science report 4th graders. http://timss.meb.gov.tr/www/raporlar/icerik/3 ministry of national education. (2011b). timss 2011 national math and science report 8th grade. http://timss.meb.gov.tr/www/raporlar/icerik/3 ministry of national education. (2012). pisa international student assessment program pisa 2012 research national final report. http://pisa.meb.gov.tr/?page_id=22 ministry of national education (2013). primary education institutions science lesson curriculum. ankara: board of education and discipline. ministry of national education. (2015a). international student assessment program pisa 2015 national report. http://pisa.meb.gov.tr/?page_id=22. ministry of national education. (2015b). timss 2015 national math and science preliminary report 4th and 8th grades. http://timss.meb.gov.tr/www/raporlar/icerik/3 ministry of national education. (2018). pisa 2018 turkey preliminary report. http://pisa.meb.gov.tr/?page_id=22 namvar, y., naderi, e., shariatmadari, a., & seifnaraghi, m. (2009). studying the impact of web-based learning (weblog) with a problem solving approach on student's reflective thinking. international journal of emerging technologies in learning, 4(2), 33-38. https://www.learntechlib.org/p/45012/. okumuş, s., & yetkil, k. (2020). ortaokul öğrencilerinin sorgulama becerilerinin değerlendirilmesi. bayburt eğitim fakültesi dergisi, 15(30), 508-526. doi: 10.35675/befdergi.740348. onan, a. (2013). the effect of problem based networked learning on the medical students’ selfefficacy perception. educational sciences and practice, 12 (24), 77-94. organisation for economic co-operation and development (oecd). (2018). the future of education and skills: education 2030. https://www.oecd.org/education/2030/e2030%20position%20 paper%20 panasan, m., & nuangchalerm, p. (2010). learning outcomes of project-based and inquiry-based learning activities. journal of social sciences, 6(2), 252-255. https://files.eric.ed.gov/fulltext/ed509723.pdf pusmaz, a., & tavşan, s. (2019). the analysis of reflective thinking skill toward problem solving of students with good at solving problems. kastamonu education journal, 27(2), 837-852. reaume, r. (2011). pre-service teacher perceptions of and experiences with the implementation of inquiry based science teaching (order no. mr76264). available from proquest dissertations & theses global. (916791964). https://www.proquest.com/dissertationstheses/pre-service-teacher-perceptions-experienceswith/docview/916791964/se-2?accountid=37161 rieger, a., radcliffe, b. j., & doepker, g. m. (2013). practices for developing reflective thinking skills among teachers. kappa delta pi record, 49(4), 184-189. doi:10.1080/00228958.2013.845510 saygılı, g., & atahan, r. (2014). examining reflective thinking skills of gifted children for problem solving in terms of various variables. süleyman demirel university faculty of arts and sciences journal of social sciences, 31, 181-192. https://dergipark.org.tr/en/pub/sufesosbil/issue/11406/136178 stotter, j., & gillon, k. (2011). inquiry learning with senior secondary students: yes it can be done!. access, 25(3), 14-19. doi: 10.3316/informit.341090467276495 smith, c. s., & hung, l. c. (2017). using problem-based learning to increase computer self-efficacy in taiwanese students. interactive learning environments, 25(3), 329-342. doi: 10.1080/10494820.2015.1127818 song, h. d., grabowski, b. l., koszalka, t. a., & harkness, w. l. (2006). patterns of instructional-design factors prompting reflective thinking in middle-school and college level problembased learning environments. instructional science, 34(1), 63-87. doi: 10.1007/s11251-005-6922-4 taşkın-can, b., & yıldırım, c. (2014). the instrument for determining the levels of reflective thinking among elementary school students. educational research and reviews, 9(1), 9-16. doi: 10.5897/err12.093 taşkoyan, s. n. (2008). the effect of inquiry learning strategies in science and technology teaching on students' inquiry learning skills, academic achievements and attitudes. master of thesis, dokuz eylül university, i̇zmir. uden, l., & beaumont, c. (2006). what is problem-based learning? in l. uden & c. beaumont (eds.), technology and problem-based learning (pp. 25-43). london: igi global. usta, n., mirasyedioğlu, ş., & tarihi, m. g. (2017). the effect of mathematics teaching with problem-based learning approach on 7th grade students' mathematics achievement and self-efficacy. kastamonu education journal, 25(6), 2263-2282. van der schaaf, m., baartman, l., prins, f., oosterbaan, a., & schaap, h. (2013). feedback dialogues that stimulate students’ reflective thinking. scandinavian journal of educational research, 57(3), 227–245. doi: 10.1080/00313831.2011.628693 wilson j. & jan w. l. (1993). thinking for themselves developing strategies for reflective learning, australia: eleanor curtain publishing. wood, w. b. (2003). inquiry-based undergraduate teaching in the life sciences at large research universities: a perspective on the boyer commission report. cell biology education, 2(2), 112-116. doi: 10.1187/cbe.03-02-0004 wopereis, i., brand-gruwel, s., & vermetten, y. (2007). the effect of embedded instruction on solving information problems. computers in human behavior, 24(3), 738-752. doi: 10.1016/j.chb.2007.01.024 wu, h. k., & hsieh, c. e. (2006). developing sixth graders’ inquiry skills to construct explanations in inquiry‐based learning https://doi.org/10.1080/713611442 https://doi.org/10.1207/s15326985ep3901_2 https://doi.org/10.1002/tea.20037 https://doi.org/10.1080/14703291003718919 https://doi.org/10.1002/ace.7401 http://timss.meb.gov.tr/www/raporlar/icerik/3 https://www.learntechlib.org/p/45012/ https://doi.org/10.1080/00228958.2013.845510 https://doi.org/10.1080/10494820.2015.1127818 https://doi.org/10.5897/err12.093 https://doi.org/10.1080/00313831.2011.628693 https://doi.org/10.1187/cbe.03-02-0004 https://doi.org/10.1016/j.chb.2007.01.024 journal of science learning article doi: 10.17509/jsl.v5i1.32076 27 j.sci.learn.2022.5(1).14-27 environments. international journal of science education, 28(11), 12891313. doi: 10.1080/09500690600621035 yaşar, ş., & duban, n. (2009). students’ opinions regarding to the inquiry-based learning approach. i̇lköğretim online, 8(2), 457-475. yenilmez, k. & turgut, m. (2016). relationship between prospective middle school mathematics teachers' logical and reflective thinking skills. journal of educational and instructional studies in the world, 6(4), 15–20. yıldırım, c., & can, b. (2018). the effect of argumentation supported problem-based learning on students' inquiry learning skill perceptions. pamukkale university journal of education, 44, 251-277. doi: 10.9779/puje.2018.217 yıldız, n. (2010). the effects of experimental practices in solving problem-based learning scenarios in science education on students' success, attitudes and scientific process skills. (unpublished master thesis). istanbul: marmara university institute of educational sciences. zion, m. i., & sadeh, i. (2007). curiosity and open inquiry learning. journal of biological education, 41(4), 162-169. doi: 10.1080/00219266.2007.9656092 https://doi.org/10.1080/09500690600621035 https://doi.org/10.1080/00219266.2007.9656092 a © 2022 indonesian society for science educator 55 j.sci.learn.2022.5(1).55-68 received: 28 february 2021 revised: 13 june 2021 published: 1 march 2022 facilitating 7th grade students’ food literacy through science activities: a qualitative study sevil kurt1*, nagihan yildirim1, bahar aksu sandikç2 1department of mathematics and science education, faculty of education, recep tayyip erdogan university, rize, turkey 2science teacher and job advisor in employment institution, ümraniye, i̇stanbul, turkey *corresponding author. sevilkurt@erdogan.edu.tr abstract this study aims to discuss the development of secondary 7th-grade students' opinions on food literacy within the scope of a series of teaching activities. through a case study, the study sample comprised 14 grade seven students in a state secondary school in the province of rize in türkiye. the data was collected through semi-structured interviews before and after the implementation and the reflective diaries written by the students during the implementation. the collected data were analyzed using content analysis and descriptive analysis techniques. as a result, it was found out that the students’ knowledge structures and understandings of food literacy changed positively in many aspects during the implementation. besides, it was seen that the implementation helped the students improve their knowledge and behaviors of food literacy. in the study, it was suggested to organize similar intervention studies for different age groups and to conduct studies in which parents are participants. keywords food literacy, 7th-grade students, science activities, qualitative 1. introduction the changing life conditions and comprehensive options offered by technological developments have gradually transformed humans’ eating habits (keser & çıracıoğlu, 2015; steils & obaidalahe, 2020). today, many foodstuffs which consume for nutrition have become less natural in terms of ingredients and become more complex (çalışır & çalışkan, 2003). the leaping sector of convenience food has particularly accelerated this process. convenience food is foods that undergo industrial processes for preparation, cooking, and packaging, containing food additives for various purposes and usually sold after being packaged. during the production processes of such foods, many food additives are added to extend shelf life, give flavor and color, increase consistency, regulate acidity, prevent clumping, sweeten, and prevent oxidation. the primary purpose of using additives in functional food seems to increase consumer preferences and preferences for products. however, conventional foods’ benefits and risks for human health should consider as an essential criterion as well as physical properties of convenience food such as color, odor, consistency, flavor, and durability (akbulut, 2011; çalışır & çalışkan, 2003; durmaz & keser, 2018). researches suggest that specific health problems in our age, such as heart diseases, diabetes, cancer, obesity, autism, hyperactivity, attention deficit disorder, and allergic diseases, may be related to environmental factors, and the most important of these factors are poor eating habits and food additives (erışık, 2012; malik et al., 2010; pepino, 2015; roberts, 2015; ustaahmetoğlu & toklu, 2015). in this respect, the foods consumed and the food ingredients deserve as much attention as the amount of food consumed daily. among all, convenience food targeting growing children as an essential audience group shows due diligence. childhood is a period when individuals’ psychological, cognitive and physical structures start to be built. thus, it becomes even more crucial to consume healthy and safe food during this period (black, 2018; mattei & pietrobelli, 2019; şanlıer & ersoy, 2005). school-age, in which parental control is reduced, and access to individual preferences becomes easier, is particularly important in this respect. in this connection, it would be helpful to teach individuals and children mainly food ingredients, food additives, food preparation methods, and food selection and consumption mailto:sevilkurt@erdogan.edu.tr journal of science learning article doi: 10.17509/jsl.v5i1.32422 56 j.sci.learn.2022.5(1).55-68 with a scientific approach rather than through common everyday knowledge for the ultimate aim of establishing reasonable and sustainable nutrition attitudes (doustmohammadian, omidvar & shakibazadeh, 2020; truman et al., 2017). the mentioned concepts translate into food and nutrition literacy in the literature. these concepts cannot be clearly distinguished in the literature as they have a lot in common. bellotti (2010) states that food and nutrition literacy is a concept that covers three main areas of knowledge, which are food, nutrition, and health; agriculture, environment, and ecology; and social development and equality. vidgen & gallegos (2014) define food literacy by emphasizing the interaction of knowledge, skills, and behaviors in strengthening dietary resilience by maintaining healthy nutrition, planning, managing, choosing, and preparing food intake. on the other hand, pendergast & dewhurst (2012) argue that the concept concerns various components, including sustainable environment, global food system, healthrelated behaviors, and food and beverage culture and skills. zoellner, connell, bounds, crook, & yadrick (2009) define nutritional literacy as the capacity of an individual to reach, process, and understand basic nutritional information. cimbaro (2008) defines nutrition literacy as generating information about the relationship between food systems and biological, social, and ecological systems using the language and conveying this knowledge to society. aktaş and özdoğan (2016) put forward a definition that emphasizes the shared properties of nutrition and food literacy and addresses both terms together. this definition addresses food and nutrition literacy more comprehensively by including access to, analyzing, evaluating the information on food and nutrition literacy and making and applying right decisions; developing and maintaining healthy nutrition, selecting and consuming an appropriate amount of healthy foods, evaluating the functioning of food the food system; and a combination of willingness, knowledge, skills, attitudes, skills, and behaviors to ensure food security. due to its suitability with the nature of the study, this study focused on the concept of food literacy as the ability to understand the nature of food, how important it is for individuals to obtain information about food, the ability to process, analyze, and use food. food literacy is often discussed with its components as, access; planning and management; selection; knowing where food comes from; preparation; eating; nutrition, and language (vidgen & gallegos, 2014). these components are also in harmony with the main elements in this study. in this context, it suggests that individuals are aware of the importance of food in their diet for themselves, care about where the food comes from, have skills in preparing, cooking, preserving food, and have an opinion on food and nutrition issues. they can analyze and use the information on food labels such as calories and ingredients and interpret written. visual messages in food and nutrition guides to act appropriately for a healthy diet. in parallel with the importance of the matter, and awareness on food and nutrition has started to develop, and the number of researches on food and nutrition literacy has increased in recent years (büyükkoyuncu, 2010; esmaeilpour, hanzee, mansouriani & khounsiavash, 2018; folkvord, anschütz, & buijzen, 2016; günlü, & derin, 2012; i̇ncedal-sonkaya, balcı & ayar, 2018; kurt & altun, 2014; malan et al., 2020; rhea, cater, mccarter & tuuri 2020; ünver & ünüsan, 2004; watson, kwon, nichols & rew 2009; velardo, 2015; vidgen & gallegos, 2012). however, the literature presents only a few studies examining how individuals accurately use certain concepts such as food additives, organic foods, natural foods, genetically modified foods, and additive-free foods. to what extent are they aware of this, and their buying and consumption behavior is guided by this (bekar, 2013; arıca, güreş & arslan, 2016; çalık & bayçelebi, 2020; ustaahmetoğlu & toklu, 2015; yurtbakan, çalık & güler, 2020). most of the related literature is descriptive studies that expose the current situation concerning individuals’ behaviors, opinions, attitudes, and food awareness in purchasing food (bekar, 2013; dalan, 2010; demir özdenk & özcebe, 2018; demirezen & cosansu, 2005; günlü & derin, 2012; kılınç & çağdaş, 2012; lee, jin & kim, 2013; raiha, tossavainen, turunen, enkenberg & halonen, 2006; sezek, kaya & doğan, 2008; thomas, 2005). it has been reported that the participants lack adequate knowledge and skills of food and nutrition, and the importance of training has been stressed for overcoming such deficiency. it has also been reported that such training should be planned by determining the objectives, acting based on scientific theories, approaches, and models, using appropriate teaching methods, and monitoring and evaluating activities. as another common finding of the previous studies, the researchers argue that should teach knowledge and skills about nutrition and foods early. in this respect, attention has been paid to the importance of increasing the number and continuity of applied studies aimed at creating social awareness about the matter in question (ronto, ball, pendergast & harris, 2016; van lippevelde et al., 2016). likewise, only a few national studies investigate student-centered teaching materials on the subject, implementing these materials, and the individual results. çalık & bayçelebi (2020) developed predictionobservation-explanation (poe) worksheets of healthy foods, and they investigated their effects on grade 3 students’ conceptions. they found that poe worksheets, which engaged the students in hands-on and minds-on activities, seem to have enhanced their learning capacities of healthy foods and supported students’ conceptual understanding/conceptual growth. yurtbakan, çalık, and journal of science learning article doi: 10.17509/jsl.v5i1.32422 57 j.sci.learn.2022.5(1).55-68 güler (2020) researched the impact of the standard knowledge construction model (ckcm)-oriented instructional treatment on the fourth-grade students’ conceptual growth of organic and non-organic foods. they utilized a word association test (wat) and worksheets to collect data. their study determined a significant difference between mean scores of the pre and post-wat in favor of the post-wat scores, and they concluded that ckcmoriented instructional treatment had facilitated students’ conceptual growth of the organic and non-organic foods. however, secondary school intervention studies are scarce, and the topic has little coverage in the secondary school science curriculum (bulut, nalbant & çokar, 2002; meb, 2018; vergi, 2018). particularly, adolescence, which overlaps with the onset of puberty, is accepted as a transition period when rapid physical and mental changes occur, environmental factors are influential, risky behaviors are likely, psychosocial needs exist, and various health problems come to the fore (adams, 1997; ayer, 2018). children become more willing to make their own decisions on various matters during this developmental stage by getting away from parental control. nutrition is one of these fragile subjects (van lippevelde et al., 2016). due to the reasons mentioned above, it looks significant to growingly maintain secondary schoolers’ knowledge levels, attitudes, and awareness about nutrition and foods, to design and implement comprehensive and applied education activities to this end, and to evaluate the results for bringing up food and nutrition literate individuals and attaining a sustainable healthy society. this study aimed to implement science activities to improve the food literacy levels of 7th-grade students and assess their developmental levels. 2. method this study followed a case study (sample case study) to sincerely handle the results within a particular situation. qualitative case studies allow in-depth study of the factors comprising a specific case with a holistic approach, focusing on how they affect the relevant case and how they are affected by the same (yıldırım & şimşek, 2008). therefore, the method of the present study is labeled as a qualitative case study as it attempts to reveal in-depth the opinions of 7th-grade students regarding the activities implemented for food literacy over a specific period. in line with the research objectives, five teaching materials were prepared to develop healthy diets, functional foods, natural foods, organic foods, genetically modified foods, food preservation methods, food additives and varieties, food additives, and the concept of e-coding. the study lasted for five weeks, 4 hours a week. the constructivist learning approach prepared the materials. the activities enrich various teaching strategies and techniques such as collaborative teaching strategy, group work, argumentation, experiment, observation, out-ofschool learning, and inquiry-based learning methods. in addition, three worksheets develop to supplement the implementation of some activities. teaching materials (activities, worksheets, and lesson plans) examine two chemistry educators, a science educator, and two science teachers in detail. some revisions were suggested to improve their readability, understandability, and instructional effectiveness. also, a pilot study was made for activities. after experts’ reviewing and pilot the study, some revisions were made. for example, some phrases were made more explicit in worksheets, some steps were rearranged in activities to better implementation, and some figure 1 summary of teaching environment journal of science learning article doi: 10.17509/jsl.v5i1.32422 58 j.sci.learn.2022.5(1).55-68 images change in worksheets. also, a video and an animation were shortened by taking experts’ suggestions into account. in this way, the teaching environment and materials have been finalized. in figure 1 teaching environment was summarized. 2.1. participants the sample was chosen using the convenience sampling method as it speeds up research and brings in practicality. in this method, the researcher picks up a specific case nearby and is easy to access. convenience sampling is often preferred when other sampling methods are not reasonable or available. this study consisted of 14 female students enrolled in one section of the 7th grade in a vocational secondary school in rize. 2.2. data collection tools the data were collected by using semi-structured interviews conducted with students and reflective diaries kept by the students. a set of questions was used to describe students' opinions about the concept of food literacy before and after implementing the activity. the semi-structured interviews were completed by addressing six basic questions. in addition, seven questions were included in determining students' thoughts about the activities and the overall implementation of this study. after each teaching material was applied, students filled out a reflective diary to express their opinion about the implementation under several headings. headings to be included in reflective diaries and interview questions were examined by a chemistry educator, a science educator, and a science teacher. a total of 70 reflective diary recordings were obtained, five from each of the 14 students. 2.3. data analysis the data obtained through interviews and diaries were both subjected to content analysis. in the analysis of the interview data, the students’ remarks were first taperecorded with their permission and were played back several times and transcribed. once transcribing was completed, content analysis was applied to the scripts. the point of content analysis is to reach the concepts and relationships that can explain the collected data and to organize them logically (yıldırım & şimşek, 2013). since this procedure requires a more in-depth study of the data, the transcribed data is reread several times to elicit study codes and the resulting themes. the themes were then tabulated after being placed under specific categories. the reflective diaries of the students were also analyzed to derive categories and themes. the analysis showed that the same themes could be applied to all of the diary results. therefore, the findings obtained from the diaries were presented in one single table by excluding the codes to ensure integrity and save space. apart from this, the activities referred to by the students were given in a column titled "activity codes". finally, the codes, categories, and themes obtained from the interview and reflective diary data analysis were examined and evaluated by two researchers and analyzed in line with common opinions. in this way, it contributed to the reliability of the data obtained. the suitability was performed considering the research problems, descriptive analysis on the data interview data regarding the teaching activities and the overall implementation process. first of all, the data recorded in digital media were put into writing. meaningless phrases or parts that are irrelevant to the research questions were omitted to simplify the data. next, explanations made by respondents to the interview questions were grouped on a similar basis to obtain a table. direct quotations from the respondents were also included in the table. in the current study, data triangulation was exploited to ensure reliability and validity of the data collection tools (pre and post-conceptual interviews, interviews of implementation process, and reflective diaries). thus, it is aimed to gain a deeper understanding of students’ food literacy development. table 1 students' opinions on food selection criteria in the purchasing process in pre and post-interview pre-interview post-interview theme category code f theme category code f food properties label information expiry date 11 food properties label information food additives 10 tse label 4 ingredients 9 ingredients 4 expiry date 9 food additives 3 e-coding 5 production place 1 tse label 4 label information 1 organic food certificate emblem 1 unmodified food (foods) being natural or not 1 health (fruit and vegetables) being in their season 2 packaging elements package 1 (foods) containing hormones or not 1 price 1 (foods) being rotten or not 1 physical properties appearance 1 packaging elements appearance 1 pleasing to eyes 4 price 3 journal of science learning article doi: 10.17509/jsl.v5i1.32422 59 j.sci.learn.2022.5(1).55-68 3. result and discussion 3.1 results from the pre and post-interview regarding the concept of food literacy in this section, firstly, the findings obtained from the interviews conducted using the same questions before and after the activities to identify the students' opinions of the concepts of food literacy are presented in combined tables. this was done to make sure that readers can evaluate the findings from the two interviews with ease. table 1 shows the pre and post-interview results regarding what factors students pay attention to when buying any food product. the respondents paid the most attention to label information such as expiry date, tse label and ingredients, and physical properties such as good eyes while buying food in the pre-interview. other studies reported that consumers often observe the production and expiry date among the details on the package (gözener, büyükbay & sayılı, 2009; grunert & wills, 2007; i̇ncedalsonkaya, balcı & ayar, 2018; karabiber & hazer, 2010; özgül & aksulu, 2006). after the implementation process was completed, the students were addressed the same questions, and their responses were analyzed. the majority stated that they read the label information while purchasing a foodstuff. in addition to the physical properties of price and appearance referred to during the pre-interview, the respondents introduced a new type of label information, organic food certification emblem, in post-interview. however, students frequently mentioned other label information such as food additives, expiry date, ingredients, and e-coding in the final interview. they also implied a new property of foods, which is “health”, by particularly relating it to (fruit and vegetables) being in their season and (food’s) containing hormones or not. it is understood that students pay more attention to a more significant number of criteria for food selection due to the implementation process. that implies that this implementation could help students select food consciously, which is an essential component of food literacy. food literacy is interpreted as the ability to understand the nature of the food and how important it is to learn about the food and process, analyze and use such information. this definition requires conscious selection of food and knowing its source (velardo, 2015). table 2 students' opinions on definition and examples of food additives in pre and post-interviews pre-interview post-interview theme category code f theme category code f definition food additive substances added to food 5 definition food additive substances added to food 8 harmful things added to food 2 substances added as a colorant 1 substances added as a sweetener 1 substances added as a sweetener 1 substances added as a colorant 1 substances that should not be in food 1 auxilary food substances 1 substances added to extend shelf life 1 i don't know no idea 4 substances added for durability 1 purpose shelf life foods' not going bad 1 purpose shelf life foods' not going bad 3 physical properties sweeteners 7 food preservatives 3 colorants 2 physical properties sweeteners 7 changing food's appearance 1 flavor enhancer 2 pleasing to eyes 2 colorants 4 making foods look beautiful 1 changing food's appearance 5 changing food's shape 1 to make it look beautiful 3 i don't know no idea 1 example sweeteners glucose syrup 11 example sweeteners glucose syrup 3 aspartame 3 paraffin 1 colorants food dyes 1 colorants food dyes 3 tartrazine 3 i don't know no idea 9 trans fats trans fats 5 flavor enhancers chinese salt (msg) 6 flavor enhancers 1 flavors flavors 2 preservatives paraffin 7 sodium benzoate 2 journal of science learning article doi: 10.17509/jsl.v5i1.32422 60 j.sci.learn.2022.5(1).55-68 during the interviews, the students were asked to describe and exemplify food additives. findings on this subject are given in table 2 table 2 shows that some students defined food additives as substances added to food, harmful things added to food, auxiliary food substances, sweeteners, and substances added as a colorant in the pre-interview. in contrast, some students had no idea at all about what a food additive is. regarding the purpose of using such chemicals, the students mentioned physical properties at the highest frequency. however, when it comes to examples of food additives, it was seen that the majority of the respondents did not have an idea. at the same time, only a tiny number mentioned glucose syrup, food dyes, and paraffin. in this case, the participants knew that food additive is added to foods, but they could not explain its purpose or examples. this finding is in line with studies in the literature reporting that children have poor food literacy (adams, 1997; ronto, ball, pendergast & harris, 2016). after the implementation, as seen in table 2, all of the students described food additives. in addition, to substances, similar opinions were noted in the postinterview with increased frequency this time. again, the post-interview revealed that the majority of the students exemplified food additives like sweeteners, preservatives, and flavor enhancers with the addition of colorants, trans fats, and flavorings. the most frequently mentioned food additives were glucose syrup, paraffin, and monosodium glutamate, also known as chinese salt, in the postinterview. contrarily, the fact that almost all of the students who could not adequately explain what food additives are could give explanations and examples in the post-interview is an indicator that the student's level of knowledge on food contents improved. therefore, this subject is essential in table 3 students' opinions about convenience food in pre and postinterviews pre-interview post-interview theme category code f theme category code f definition processed foods foods produced in factories 4 definition foods passing through the food industry process food processed in factories 8 packaged foods 2 foods with additives 4 foods produced with machines 2 packaged foods 4 foods with additives 1 foods that are not used as obtained from the source ready-to-eat foods 2 foods that are not made at home foods we buy ready to eat 2 unnatural foods 2 foods we buy effortlessly 1 grocery food 2 foods that are not made at home 1 foods we don't make ourselves 1 table 4 students' opinions about natural food in pre and postinterviews pre-interview post-interview theme category code f theme category code f definition unprocessed foods foods without food additives 5 definition foods used as obtained from the source foods without food additives 12 healthy unharmful foods 1 self-growing foods 2 foods used as obtained from the source food obtained from animals raised in the village 4 nature-grown foods 1 foods we grow gardens ourselves 2 fruit and vegetables picked from the branch 1 self-growing foods 1 foods picked from the garden 1 fruit picked from the tree 1 foods we grow ourselves 1 foods are grown in nature 1 foods picked from source 1 journal of science learning article doi: 10.17509/jsl.v5i1.32422 61 j.sci.learn.2022.5(1).55-68 acquiring food and nutrition literacy (aktaş & özdoğan, 2016; ayer, 2018). food literacy necessitates students’ ability to read and analyze the label and packaging information on foods, question the source of the food, and make the right food choice by judging all these at once (vidgen & gallegos, 2014). next, findings were reached regarding the students’ understanding of convenience food before and after the implementation. the findings are given in table 3. it was seen in the pre-interview that the students defined convenience food by relating with “processed foods” and “foods that are not made at home”. in this connection, they considered packaged foods, foods produced in factories, or foods produced with machines as convenience food. in the post-interview, most students were found to define convenience food as food processed in factories, packaged foods, and foods with additives. in other wo rds, they referred to foods passing through the food industry process to define convenience food. it determines that a small number of them made statements indicating the source of the food. table 4 shows the findings on what participants understand from the concept of natural food in the pre and postinterviews. it sees in table 4 that the students described natural food in categories of “unprocessed food”, “foods used as obtained from the source”, and “healthy”. in this connection, the most frequently stated natural foods were “foods without food additives,” “food obtained from animals raised in the village,” and “foods we grow ourselves in gardens”. after the interview, all respondents described natural food as food obtained from the source and used without food additives. another point of consideration in the interviews was what the students know about organic food and genetically modified (g.m.) food. findings revealing the students' opinions on this subject are given in table 5. in the pre-interview, it was seen that the students defined organic food as foods grown without chemicals and unprocessed foods, and some others held no idea about the concept. those who could provide a definition explained the concept as “additive-free foods”, “unprocessed foods”, “foods that do not contain pesticides”, “foods harvested from the field”, “foods grown in villages”, “foods obtained from animals we raise ourselves”, and “hormone-free foods”. it was found that none of the students had an idea in the pre-interviews about genetically modified foods. in the post-interview results, it was found out that the majority of the students described organic food about the certificate, production process, and production place. the most frequent codes appeared as “foods with organic food certificate emblem”, “foods grown with organic fertilizers,” and “foods without food additives”. according to table 5, a large number of the participants defined g.m. food in the post-interview by using the term “genetically modified” or “genes modified”. when examined together with the findings in tables 3, 4, and 5, it can be said that students were confused about the concepts of convenience food, natural food, organic food before the implementation process. while defining natural food, the students mostly used remarks like "foods used as obtained from the source", and they explained “the source” with “village”, “garden”, or “field”. a few of them mentioned the unprocessed quality of food. when the students were asked for the definition of organic food, most of the students used expressions similar to those used to describe natural food. these concepts are related to table 5 students' opinions about organic food and gm food in pre and postinterviews. pre-interview post-interview theme category code f theme category code f definition of organic food food is grown without chemicals additive-free foods 2 definition of organic food production process foods grew with organic fertilizers 5 unprocessed foods 1 foods without food additives 4 foods that do not contain pesticides 1 certificate foods with organic food certificate emblem 8 hormone-free foods 1 production place foods grew onfield themselves 1 unprocessed foods foods harvested from the field 1 foods are grown in the garden 1 foods obtained from animals we raise ourselves 2 food is grown in the village 1 foods are grown in villages 2 definition of gm food intervention to genetics genetically modified 12 i don't know no idea 4 genes modified 2 definition of gm food i don't know no idea 14 journal of science learning article doi: 10.17509/jsl.v5i1.32422 62 j.sci.learn.2022.5(1).55-68 knowing the food source and, of course, food literacy components (aktaş & özdoğan, 2016; vidgen & gallegos, 2014). in the final interviews on the concepts, most students could accurately define convenience food and natural food and explain their respective properties. in addition, most of the students could explain the production process and certification issues regarding organic food. finally, most of them talked about the presence of an organic food certificate emblem. however, none of the students could explain g.m.'s food in the first interviews, whereas all of them could define g.m.'s food as genetically modified food in the last interviews (table 5). demir & düzleyen (2012) also interviewed students who did not know anything about genetically modified organisms. thus, it can be argued that the activity implementation process proved effective in which the students collaboratively learned the topic through group work. the literature also reports the positive impact of learning environments where learners’ ideas are questioned and shared collaboratively, and they help each other (sun, looi & xie, 2017). an activity based on small group argumentation was conducted on hormones and genetically modified organisms used in foods. it can thus be claimed that the activity was beneficial in understanding and learning the importance of the topic. many studies show that activities based on small group arguments are influential and should be used primarily in teaching socio-scientific matters. the teaching process based on small group argumentation exposes students' thinking methods and improves their reasoning mechanisms (duschl & osborne, 2002; kelly & takao, 2002). the benefits of argumentation activities for students include understanding science concepts, participating in discussions, changing their views on science, and improving decision-making skills on socioscientific matters (chinn & clark, 2013; herawati & ardianto, 2017). the interviews also searched what ideas students have for naturally increasing the durability of foods without using additives. their suggestions before and after the implementation are summarized in table 6. the students mentioned several preservation methods which can be grouped as physical and chemical methods in the first and last interviews. in the pre-interview, they referred to putting (foods) in the fridge, putting in cold storage, putting in the freezer, putting in cool places, putting in tupperware boxes, putting in warm places, and putting in the storeroom, which means physical methods. among others, “putting in the fridge” was seen to match the highest frequency. at the other end, “fermenting” was matched with the lowest frequency (1) as a chemical preservation method. it can be said that before the implementation, students’ knowledge and experiences concerning food preservation ways were limited levels. the most frequently mentioned methods were physical methods such as putting in the fridge or putting in cold places. in reality, storing food in the fridge can prevent spoilage, but it is not a longterm and economical preservation method. however, traditional methods such as fermenting, canning, salting, pickling, and drying have been practiced from past to present; thanks to these, foods are kept for a long time without going wrong in an economical way and also provide health benefits with prebiotics and probiotics (amit, uddin, rizwanur rahman, islam & khan, 2017; kumar,2019). in the post-interview, it was found that most of the respondents listed both physical and chemical preservation methods. it can be inferred from table 6 that the students regarded canning, drying, salting, and heating up to be physical methods, whereas everybody saw pickling as a table 6 students' opinions about food preservation methods in pre and post-interviews pre-interview post-interview theme category code f theme category code f food preservation methods physical preservation methods putting in the fridge 7 food preservation methods physical preservation methods canning 10 putting in cold storage 1 drying 10 put in the freezer 1 salting 2 put in cool places 1 heating up 1 putting in tupperware boxes 1 chemical preservation methods pickling 14 put in warm places 1 fermenting 3 put in storeroom 1 brining 1 chemical preservation methods fermenting 1 journal of science learning article doi: 10.17509/jsl.v5i1.32422 63 j.sci.learn.2022.5(1).55-68 chemical preservation method. in those interviews, two other chemical methods were also put up, fermenting and brining namely. in this context, it can be said that the students' knowledge about food preservation methods has improved in the implementation process. before the implementation stage ended, the students examined natural (no food additives) preservation methods and learned one of them by experiencing it in person. likewise, previous studies noted that students learn by being active, cooperating, doing, and experiencing make more sense to them and contribute to permanent learning (chinn & clark, 2013; chiu ve cheng, 2017; ünal & ergin, 2006). lastly, findings were obtained on students’ knowledge about e-coding on food packages according to pre and postinterviews, as seen in table 7. it was found out that while none of the students had an idea or example about the topic in the pre-interview, all of them could describe ecoding in the post-interview. of those, 8 explained ecoding as numbers are given to food additives, and four thought it was an international figure. also, in the last interview, the students could exemplify e-codings for food additives used as colorants, preservatives, sweeteners, and flavor enhancers. it was observed that most of the students gave specific examples such as 'e102', a colorant, 'e221', a preservative, and 'e21', a flavor enhancer. as a part of this study, the students made a supermarket trip, an out-of-school teaching activity. in this way, they learn about what to pay attention to when buying groceries and understand what food labels mean, place and experience. it can thus be suggested that this active learning table 7 students' opinions about e-coding in pre and post-interviews pre-interview post-interview theme category code f theme category code f definition i don't know no idea 14 definition e-coding numbers are given to food additives 8 example i don't know no idea 14 an international figure 4 closed representation of food additives 2 a scientific number 1 example colorant e102 8 preservative e221 4 sweetener e951 2 e924 2 flavor enhancer e621 4 table 8 students' opinions obtained from reflective diaries theme category activity codes frequency learnt concepts food selection 1 14 food additives 2 14 food additive usage purposes 2 6 gm food 3 14 foods grew with hormones 3 14 e-coding 4 14 e-coding examples 4 14 things to consider in food shopping 4 14 food preservation methods 5 14 aspects different from other lessons collaborative learning 1,2,3,4 35 taking an active role 1,2,3,4 37 out-of-school activity 4 14 design work 5 14 challenging parts of the activity none 1,2,3,4,5 14 pleasing parts of the activity collaborative learning 1,2,3,5 26 taking an active role 1,2,3,5 42 learning food preservation methods 5 14 out-of-school activity 4 14 things to add to the activity none 1,2,3,4,5 67 various activities 1 3 journal of science learning article doi: 10.17509/jsl.v5i1.32422 64 j.sci.learn.2022.5(1).55-68 process outside of school improved students’ opinions on foodstuffs and e-coding (chin, 2004). in addition, the literature also reports several favorable effects of out-ofschool learning environments such as drawing students’ attention, being fun and more motivating and encouraging meaningful and lasting learning (bakioğlu & karamustafaoğlu, 2014; balkan kıyıcı, 2011). 3. 2. results from the reflective diaries the results obtained from the reflective diaries kept by the students during the experimental stage were summarized in a single table by leaving the study codes outside to maintain the semantic integrity and use the space sparingly. for this reason, the frequency figures in the relevant table indicate the total frequency of the thoughts of 14 students in 5 diaries. in addition, student activities in the diary are entered into a table with the name "activity code". according to table 8, all students stated that they learned about food selection, food additives, g.m. food, foods grown with hormones, e-coding, e-coding examples, things to consider in food shopping, and food preservation methods thanks to the carrying out of the activities. it was seen that the students listed aspects of the practical activities different from other lessons, such as taking an active role, collaborative learning, out-of-school activity, and design work. the students stated that they faced no challenging situations during the five activities they took part in. as for the good parts of the activities, the students said that they enjoyed collaborative learning, taking an active role, learning food preservation methods, and out-ofschool activity in decreasing order of frequency. it was understood from table 8 that there were no points the students wanted to add or revise in the teaching activities. only three students stated that different activities such as origami, doing sports activities, and watching videos could be added to activity 1. in the results obtained from the students' reflective diaries, the categories of collaborative learning and taking an active role emerged as extraordinary aspects compared to other lessons about all of the activities. at the same time, cooperative learning and taking an active role emerged as two distinct categories under the theme of aspects liked. it can be said that students enjoy the collaborative learning method and being active in the process. the researcher also shows that during collaborative group work, students can experience extraordinary learning experiences such as asking questions, criticizing different thoughts, expressing ideas, and providing examples thanks to interactions with their peers which are not possible when they study alone. (slavin, 1991; zimmerman ve gallagher, 2006). table 9 students' opinions about the implementation process opinion students' direct remarks overall judgment fun i think it was enjoyable. we went grocery shopping, watched videos, and did many food activities. (ö14) instructive the activities were instructive. i learned a lot. i learned e-coding, i learned food additives, i learned gm food. i learned the health hazards of glucose syrup and trans fats. (ö5) it went beautifully. we learned things that we did not know (before), i learned which additives are in foods. we learned what to consider while shopping. we learned healthy and balanced eating. (ö8) consciousnessraising yes. for example, we just ate food. we were not paying attention to the ingredients. now i pay attention to the expiry date, whether additives are used or not. (ö3) benefits it helps to associate with everyday life yes, there were. we learned new things. it was instrumental because i will apply it in daily life. (ö10) we went grocery shopping, bought food after examining it in detail, bought food, and to apply it in daily life. (ö14) awarenessincreasing yes. for example, we just ate food. we were not paying attention to the ingredients. now i pay attention to the expiry date, whether additives are used or not. (ö3) yes. we made pickles, we were going to store them, and now i know why we put salt and vinegar. (ö4) for example, i became more aware of e-coding; i pay attention when buying, and i will continue to do so. (ö6) i pay attention not to eat junk food and try not to consume unhealthy products that contain food additives. (ö8) for example, when i go shopping, i look at its appearance, expiry date, and whether or not food additives are used. (ö3) behaviorchanging i was buying it without knowing what glucose syrup is. i will not buy anymore. i will consume fruit and vegetables in the season. i will store foods after canning, salting, or drying. (ö5) for example, when i go to the supermarket, i no longer buy because the price is low. i look at the ingredients. i look at the e-codings. (ö4) i didn't use to look at the ingredients, but now i read the ingredients, look at the e-coding, and did not use to pay attention because i did not know food additives. (ö13) journal of science learning article doi: 10.17509/jsl.v5i1.32422 65 j.sci.learn.2022.5(1).55-68 3.3. results from the interviews regarding the implementation process this section is devoted to the findings obtained from the descriptive analysis of the interviews about the student's opinions on the implementation. the findings are displayed in table 9. students stated that the learning process was fun, instructive, and consciousness-raising. they stated the benefits of the implementation process as it helps to associate with everyday life, awarenessincreasing, behavior-changing. students explained aspects as videos watched, activities carried out, extracurricular learning (grocery shopping), watching movies, worksheets, table 9 students' opinions about the implementation process (continued) opinion students' direct remarks aspects liked videos watched we watched many different videos and learned from them. you gave us worksheets, and we filled them out. then you handed out images, and we analyzed them. there are no aspects that i don't like. (ö4) we created slogans and designed t-shirts. it was fun. (ö7) we watched videos, designed t-shirts, went to the supermarket, made pickles. it was fun, and i wasn't bored. i enjoyed it. (ö1) activities carried out extracurricular learning (grocery shopping) watching movies worksheets t-shirt designing activity aspects disliked favorite activity pickle making i liked the pickle-making very much. we also do this in daily life. it was beneficial and fun. (ö4) designing t-shirts. because we worked as a group, we decided together, and we exchanged ideas. (ö8) going grocery shopping, examining the food on-site, we read the ingredients section. while purchasing the fruit, we looked at whether an organic food certification emblem or not, and we learned better. (ö14) creating slogans and designing tshirts grocery shopping thoughts on group work joint decision making when one of us said wrong, we had a chance to correct it. when we form a common idea together, we think more versatile. (ö4) there was no downside. i think it's nice to take everyone's opinion and come to a joint decision. (ö5) it was positive. we helped each other. my friends did something that i could not do. we decided together. (ö14) sharing ideas helping each other versatile thinking correcting mistakes reflections of the activities on students' daily lives stopping to consume a type of convenience food consumed before yes, i mostly consumed chips, but i learned the ingredients, so i will not consume them anymore. (ö11) teacher, you brought ready-made crisps and homemade crisps. there have been differences between them. homemade crisps did not burn when we burnt the two. but ready-made crisps are burnt because they contain additives. after observing this phenomenon, i no longer consume ready-made crisps. (ö1) starting to read the product packaging information in more detail during the purchasing process usually, when i went to the supermarket, i used to look at the expiry date, but i did not look at the ingredients. i look now. (ö2) for example, when i go to the supermarket, i no longer buy because the price is low. i look at their ingredients. i look at the e-codings. (ö4) yes, there was. for example, when i go shopping, i look at its appearance, expiry date, and whether food additives have. (ö3) i didn't use to look at the ingredients, but now i read the ingredients, look at the e-coding, and did not use to pay attention because i did not know food additives. (ö13) while purchasing food, i was buying my loved ones, i did not use to look at the expiry date and the ingredients, but now i learned, and i pay attention a lot. (ö12) adding criteria for product selection that was not considered before challenges faced journal of science learning article doi: 10.17509/jsl.v5i1.32422 66 j.sci.learn.2022.5(1).55-68 and t-shirt designing activities. their favorite activities were pickle making, creating slogans and designing their t-shirts, grocery shopping. also, students stated thoughts on group work as joint decision making, sharing ideas, helping each other, versatile thinking, and correcting mistakes. on the whole, the students disliked no particular thing or found it challenging at the implementation stage. however, a significant result is taken from the study is reflections of the activities on students' daily lives. they explained these issues as stopping to consume a type of convenience food consumed before, starting to read the product packaging information in more detail during the purchasing process, and adding criteria for product selection that was not considered before. students’ improvement in the implementation process is revealed once again by the conceptual interview findings and the findings related to the theme “learned concepts” in the reflective diaries and the findings obtained from interviews about the teaching activities. furthermore, the students’ remarks suggest that the intended improvement was achieved in behavior beyond perception. the experimental stage was carried out with five teaching materials based on the constructivist teaching approach. various teaching methods, techniques, and materials were used in those materials such as argumentation, out-of-school learning, inquiry-based learning, experimentation, producing slogans, watching videos and movies, story, t-shirt design, which are suitable for modern learning approaches and attract students' attention (ünal & ergin, 2006; erdem, 2018). in this context, according to the participants in the present study, those activities were distinguished from other lessons for several reasons, as reported in the interviews and diaries. the students’ positive thoughts might be accounted for by the probability that many rich learning experiences were offered by the teaching materials developed in line with their needs to teach a topic closely related to their real-life (jack & lin, 2017). and another reason may be the opportunity to experience and observe the topic on-site by moving the teaching process out of the familiar classroom environment. additionally, the students learned by doing these activities and had the chance to reveal their creativity. in the literature, that such activities are practical for actively engaging students in learning and bringing them meaningful learning (bakioğlu & karamustafaoğlu, 2014; balkan kıyıcı, 2011). 4. conclusion the current study aimed to discuss the development of 7th-grade students' opinions on food literacy within the scope of a series of teaching activities. the findings showed that the student's knowledge and experience about the subject were very limited before the implementation. the students' limited knowledge about food literacy shows that the current science curriculum is insufficient to improve food literacy. however, after implementation, the students' opinions on food selection and consumption particulars such as convenience food, natural food, organic food, food additives, hormones, gm food, e-coding and food preservation methods developed significantly. improvement after implementation indicates that the intervention is effective in facilitating students' learning. also it can be concluded that the intervention was raised students’ awareness. in this context, the following suggestions can be made. it is pivotal to bring up school-age children as food literate individuals for a healthy society. therefore, food literacy subjects such as healthy foods, additives, choosing and consuming safe food items, nutritional content, and food label reading must be wider in the science curriculum. therefore, the number of learning outcomes and the course hours included in the curriculum should be increased. in the current study, the components of food literacy such as food selection, knowing where food comes from, food prevention, and preparation were handled together and tried to be improved by hands-on learning activities by a qualitative approach. different components of food literacy such as planning, management, and consumption can also be included in the study and examined by experimental design. because eating habits are inculcated at young ages within the family, the current study can adapt to contexts involving parents as participants. also, considering that the teaching materials and methods developed and applied in this study were effective in learning food literacy and were liked by the students in many senses, they can be adjusted to other grade levels. references adams, l.b.(1997). an overview of adolescent eating behavior barriers to implementing dietary guidelines. annals of the new york academy of sciences, 817 (1), 36-48. amit, s.k., uddin, m. m., rahman, r.,, islam, r.s.m,., & khan, m.s. (2017). a review on mechanisms and commercial aspects of food preservation and processing. agriculture & food security, 6(51). doi 10.1186/s40066-017-0130-8. akbulut, m., 2011. gıda katkı maddeleri: fonksiyonları ve kaynakları, 1. ulusal helal ve sağlıklı gıda kongresi bildiriler kitabı, pp:5968, 19-20 kasım, ankara. aktaş, n. & özdoğan, y. (2016). gıda ve beslenme okuryazarlığı. harran tarım ve gıda bilimleri dergisi, 20 (2), 146-153. ayer, ç. (2018). çivril yöresindeki adolesanlarda beslenme okuryazarlığının mevcut durumu ve etkileyen faktörler. yüksek lisans tezi. pamukkale üniversitesi, sağlık bilimleri enstitüsü, denizli. bakioğlu, b., & karamustafaoğlu, o. (2014). okul dışı ortamlarda fen eğitimi: diyaliz merkezine teknik bir gezi. turkish journal of teacher education, 3(2), 15-26. balkan kıyıcı, f. (2011). fen öğretiminde okul dışı öğrenme ortamları. c. laçin şimşek (ed.), hayvanat bahçeleri içinde (s.51-64). ankara: pegem a. bekar, a. (2013). tüketicilerin gıda güvenliğine yönelik tutumları. yüzüncü yıl üniversitesi tarım bilimleri dergisi, 23 (2), 90–101. bellotti, b. (2010). food literacy: reconnecting the city with the country. agricultural science, 22, 29-34. journal of science learning article doi: 10.17509/jsl.v5i1.32422 67 j.sci.learn.2022.5(1).55-68 black mm., 2018. impact of nutrition on growth, brain, and cognition. nestle nutr inst workshop ser. 89:185-195. doi: 10.1159/000486502. epub 2018 jul 10. pmid: 29991042. bulut, a., nalbant, h. & çokar, m. (2002). ergenlerin sağlık bilincinin geliştirilmesi projesi: ergenler ve sağlık durum raporu. i̇stanbul: kadın ve çocuk sağlığı eğitim ve araştırma birimi, i̇stanbul üniversitesi tıp fakültesi. büyükkoyuncu, n. (2010). gaziantep il merkezindeki ilköğretim ikinci kademe öğrencilerinin okul kantininden besin seçimleri ve annelerinin besin güvenliği bilgi düzeylerinin saptanması. yüksek lisans tezi, selçuk üniversitesi sosyal bilimler enstitüsü, konya. chin, c.c. (2004). museum experiencea resource for science teacher education. international journal of science and mathematics education, 2, 63-90. chinn, c. a., & clark, d. b. (2013). learning through collaborative argumentation. in c. e. hmelo-silver, c. a. chinn, c. k. k. chan, & a. m. o’donnell (eds.) international handbook of collaborative learning (pp. 314-332). new york: taylor & francis. chiu, p. h. p., & cheng, s. h. (2017). effects of active learning classrooms on student learning: a two-year empirical investigation on student perceptions and academic performance. higher education research & development, 36(2), 269-279, doi: 10.1080/07294360.2016.1196475. cimbaro, m.a., 2008. nutrition literacy: towards a new conception for home economics education. the university of british columbia, vancouver, 83: 4-44. çakır arıca, ş., güreş, n., & arslan, s. (2016). tüketicilerin sağlıklı beslenmeye yönelik tutumlarının incelenmesine yönelik bir araştırma. çukurova üniversitesi i̇i̇bf dergisi, 20, 99-110. çalık, m., bayçelebi, z. (2020) developing and implementing prediction-observation-explanation worksheets of healthy foods. elementary education online, 19 (3), 18-39. çalışır, z.e., & çalışkan, d. (2003). gıda katkı maddeleri ve insan sağlığı üzerine etkileri, ankara eczacılık fakültesi dergisi, 32 (3), 207-206. dalan, h. (2010). liselerde obezite ve sağlıklı beslenme konularında öğrencilerin bilgi düzeylerinin değerlendirilmesi (yayınlanmamış yüksek lisans tezi). gazi üniversitesi, fen bilimleri enstitüsü, biyoloji eğitimi ana bilim dalı, ankara. demir, b., & düzleyen, e. (2012). i̇lköğretim 8. sınıf öğrencilerinin gdo bilgi düzeylerinin incelenmesi, x. ulusal fen bilimleri ve matemnatik eğitimi kongresi, 27-30 haziran, niğde. demirezen, e. & cosansu, g. (2005). adölesan çağı öğrencilerde beslenme alışkanlıklarının değerlendirilmesi. sürekli tıp eğitimi dergisi, 14(8), 174-178. demir özdenk, g., & özcebe, l.h. (2018). bir üniversite çalışanlarının beslenme okuryazarlığı, beslenme davranışları ve ilişkili faktörler. turkish journal of public health , 16 (3) , 178-189 . doi: 10.20518/tjph.499902 doustmohammadian, a., omidvar, n., & shakibazadeh, e. (2020). school-based interventions for promoting food and nutrition literacy (fnlit) in elementary school children: a systematic review protocol. systematic reviews, 9(1), 87. https://doi.org/10.1186/s13643-020-01339-0. duschl, r.a., & osborne, j.(2002). supporting and promoting argumentation discourse in science education. studies in science education, 38, 1, 39-72. erdem, b. (2018). ortaokulda görev yapan okul yöneticisi ve öğretmenlerin öğrenci merkezli eğitime ilişkin görüşleri. yüksek lisans tezi. eskişehir osmangazi üniversitesi, eğitim bilimleri enstitüsü, eskişehir. esmaeilpour, f., hanzee, h. k., mansouriani, y. & khounsiavash, m. (2018). children’s food choice: advertised food type, health knowledge and entertainment. journal of food products marketing, 24(4), 476-494. https://doi.org/10.1080/10454446.2017.1315843. folkvord, f., anschütz, j. d. & buijzen, m. (2016). the association between bmi development among young children and (un)healthy food choices in response to food advertisements: a longitudinal study. international journal of behavioral nutrition and physical activity, 13(16), 1-7. http://dx.doi.org/10.1186/s12966-0160340-7. günlü, z. & derin, o. (2012). televizyon reklamlarının okul çağı çocuklarının besin seçimi üzerine etkisinin bir i̇ncelemesi. selçuk i̇letişim, 7(3), 62-77. herawati d., & ardianto, d. (2017). socioscientific argumentation of pre-service teachers about genetically modified organisms., journal of physics: conference series 895 012023 doi :10.1088/17426596/895/1/012023 erışık, e. (2012). 1-12 yaş arası çocuğu olan annelerin çocuk beslenmesi ve gıda katkıları konusunda bilgi ve davranışa yönelik tutumlarının belirlenmesi. yüksek lisans tezi, atatürk üniversitesi sağlık bilimleri enstitüsü, erzurum. eşer durmaz, s., & keser, a . (2018). yapay tatlandırıcıların vücut ağırlığı ve insülin direnci üzerine etkileri . gazi sağlık bilimleri dergisi, 3 (2), 8-15 . retrieved from https://dergipark.org.tr/tr/pub/gsbdergi/issue/37812/432966 grunert, k.g., & wills, j.m. (2007). gıda etiketlerinde beslenme bilgisine tüketici tepkisi üzerine avrupa araştırması. halk sağlığı dergisi, 15, 385-399. gözener, b., büyükbay, e.o., & sayılı, m. (2009). gıda güvenliği konusunda öğrencilerin bilgi düzeylerinin incelenmesi, goü. ziraat fakültesi dergisi, 26 (2), 45-53. i̇ncedal-sonkaya, z., balcı, z., & ayar, a. (2018). üniversite öğrencilerinin gıda okuryazarlığı ve gıda güvenliği konusunda bilgi, tutum ve davranışları “amasya üniversitesi sabuncuoğlu şerefeddin sağlık hizmetleri meslek yüksekokulu örneği”. türk hijyen ve deneysel biyoloji dergisi, 75(1), 53 – 64. jack, b.m., & lin, h.s. (2017) making learning interesting and its application to the science classroom. studies in science education, 53(2), 137-164, doi:10.1080/03057267.2017.1305543. karabiber, c., & hazer, o. (2010). tüketicilerin bilgi kaynağı olarak gıda ürünlerindeki etiketi okuma ve anlamada karşılaştıkları sorunların incelenmesi. hacettepe üniversitesi tüketici-pazar-araştırmadanışma test ve eğitim merkezi tüketici yazıları ii. 253-270. kelly, g.j., & takao, a. (2002). epistemic levels in argument: an analysis of university oceanography students’ use of evidence in writing. science education, 314–342. keser, a. & çıracıoğlu, e.d. (2015). sağlık ve beslenme okuryazarlığı. sağlık okuryazarlığı, (ed.) yıldırım, f ve keser, a., ankara üniversitesi basımevi, ankara. kılınç, f. & çağdaş, d. (2012). sağlık meslek lisesi öğrencilerinin beslenme alışkanlıklarının, beslenme bilgi düzeylerinin ve vücut bileşimlerinin değerlendirilmesi. türk pediatri arşivi, 47, 181188.http://dx.doi.org/10.4274/tpa.842. kumar, a. (2019). food preservation: traditional and modern techniques, acta scientific nutritional health, 3(12), 45-49. kurt, e. & altun, t. (2014). televizyon reklamlarının ilkokul öğrencilerinin beslenme alışkanlıklarına etkisi üzerine inceleme. akademik sosyal araştırmalar dergisi, 2(7), 393-408. lee, s., jin, n. & kim, h. (2013). relationships among knowledge of healthy food, health concern, and behavioral intention: evidence from the united states and south korea. journal of quality assurance in hospitality & tourism, 14, 344–363. malan h, watson td, slusser w, glik d, rowat ac., & prelip m. (2020). challenges, opportunities, and motivators for developing and applying food literacy in a university setting: a qualitative study. journal of the academy of nutrition and dietetics, 120(1), 33-44. doi: 10.1016/j.jand.2019.06.003. malik v., popkin b., bray g., despres j., willett w., & hu f.(2010). sugar-sweetened beverages and risk of metabolic syndrome and type 2 diabetes. diabetes care, 33, 2477–2483. mattei d., & pietrobelli a. (2019). micronutrients and brain development. current nutrition reports. 8(2):99-107. doi: 10.1007/s13668-019-0268-z. meb, 2018. i̇lköğretim fen bilimleri dersi (3, 4, 5, 6, 7 ve 8. sınıflar) öğretim programı, talim ve terbiye kurulu başkanlığı, ankara. https://doi.org/10.1080/10454446.2017.1315843 http://dx.doi.org/10.1186/s12966-016-0340-7 http://dx.doi.org/10.1186/s12966-016-0340-7 journal of science learning article doi: 10.17509/jsl.v5i1.32422 68 j.sci.learn.2022.5(1).55-68 özgül, e., & aksulu, i̇. (2006). ambalajlı gıda ürünlerinde tüketicilerin etiket duyarlılığındaki değişimler. ege akademik bakış, 6, 1-10. pendergast, d., & dewhurst, y. (2012). home economics and food literacy: an international investigation. international journal of home economics, 5(2), 245–263. pepino, my. (2015). metabolic effects of non-nutritive sweeteners. physiology & behavior, 152 (pt,b) 450-455. doi: 10.1016/j.physbeh.2015.06.024. raiha, t., tossavainen, t., turunen, h., enkenberg, j. & halonen, p. (2006). adolescents’ nutrition health issues: opinions of finnish seventh-graders. health education, 106(2), 114-132. https://doi.org/10.1108/09654280610650954 rhea, kc., cate,r mw., mccarter k., & tuuri g. (2020). psychometric analyses of the eating and food literacy behaviors questionnaire with university students. journal of nutrition education and behavior, 52(11), 1008-1017. doi: 10.1016/j.jneb.2020.05.002. roberts j. r. (2015). the paradox of artificial sweeteners in managing obesity. current gastroenterology reports, 17(1), 423. https://doi.org/10.1007/s11894-014-0423-z ronto, r., ball, l., pendergast, d., & harris, n.d. (2016). food literacy at secondary schools in australia. journal of school health, 86(11), 823-831. doi: 10.1111/josh.12440. pmid: 27714873. sezek, f., kaya, e. & doğan, s. (2008). üniversite öğrencilerinin genel beslenme alışkanlıkları, katkılı besinler hakkındaki bilgi, görüş ve tutumları. çankaya üniversitesi fen –edebiyat fakültesi sanat ve bilim dergisi, 1(10), 117-134. slavin, r. e. (1991). synthesis of research on cooperative learning. educational leadership, 48(5), 71-82. steils, n., obaidahale, z. (2020). “social food”: food literacy coconstruction and distortion on social media. food policy, elsevier, 95, pp.101932. doi:10.1016/j.foodpol.2020.101932. sun, d., looi, c.k., & xie, w. (2017). learning with collaborative inquiry. a science learning environment for secondary students. technology, pedagogy and education, 26(3), 241-263, doi: 10.1080/1475939x.2016.1205509 şanlıer, n., & ersoy, y. (2005). okul çağı çocuklarının beslenmesi beslenme prensipleri. i̇stanbul: morpa yayınları. thomas, m. (2005). post-16 students’ perceptions to health and healthy eating in welsh secondary schools. health education, 105(2), 89102. truman, e., raine, k., mrklas, k., prowse, r., hoed ,rcd., watsonjarvis, k., loewen, j., gorham, m., ricciardi, c., tyminski, s., & elliott c. (2017). toward a consensus statement on food literacy. canadian journal of public health revue canadienne de sante publique, 108(2), 211-e213. doi: 10.17269/cjph.108.5909. ustaahmetoğlu, e., & toklu, i̇.t. (2015). organik gıda satın alma niyetinde tutum, sağlık bilinci ve gıda güvenliğinin etkisi üzerine bir araştırma. ekonomik ve sosyal araştırmalar dergisi, 11, 198-211. ünal, g., & ergin, ö. (2006). buluş yoluyla fen öğretiminin öğrencilerin akademik başarılarına, öğrenme yaklaşımlarına ve tutumlarına etkisi. türk fen eğitimi dergisi, 3(1), 36-52. ünver, y. & ünüsan, n. (2004). okul öncesinde beslenme eğitimi üzerine bir araştırma. selçuk üniversitesi sosyal bilimler enstitüsü dergisi, 14, 529-551. watson, l. c., kwon, j., nichols, d. & rew, m. (2009). evaluation of the nutrition knowledge, attitudes and food consumption behaviors of high school students before and after completion of a nutrition course. family and consumer sciences research journal, 37(4), 523-534. https://doi.org/10.1177/1077727x08329002. van lippevelde, w., vangeel, j., de cock, n., lachat, c., goossens, l., beullens, k., vervoort, l., braet, c., maes, l., eggermont, s., deforche, b., & van camp, j. (2016). using a gamified monitoring app to change adolescents’ snack intake: the development of the reward app and evaluation design. bmc public health 16, 725. doi 10.1186/s12889-016-3286-4. velardo, s. (2015). the nuances of health literacy, nutrition literacy, and food literacy. journal of nutrition education behavior, 47, 385-389. vergi, y. (2018). besin etiketlerinin sağlık algısı üzerine etkisi. yüksek lisans tezi. marmara üniversitesi, sağlık bilimleri enstitüsü, i̇stanbul. vidgen, h.a., & gallegos, d. (2012). defining food literacy and its components. appetite, 76 (1), 50-59. vidgen, ha., gallegos, d. (2014). defining food literacy and its components. appetite, 76:50-9. doi: 10.1016/j.appet.2014.01.010. yıldırım, a. & şimşek, h. (2008). sosyal bilimlerde nitel araştırma yöntemleri. seçkin yayıncılık, geliştirilmiş 6. basım, 8-10. yurtbakan, e., çalık, m., & güler, t. (2020). investigating fourth grade students’ conceptual growth of the ‘organic and nonorganic foods’ subject: a case of common knowledge construction model. hacettepe university journal of education. advance online publication. doi: 10.16986/huje.2020058881. zimmerman, d.k., & gallagher, s.r. (2006). creavity and team environment: an exercise illustrating how much one member can metter. journal of management education, 30 (4), 617-625. zoellner, j., connell, c., bounds, w., crook, l. &yadrick, k. (2009). nutrition literacy status and preferred nutrition communication channels among adults in the lower mississippi delta. preventing chronic disease, 6, 128. a © 2022 indonesian society for science educator 42 j.sci.learn.2022.5(1).42-54 received: 15 april 2021 revised: 23 september 2021 published: 1 march 2022 innovativeness in science education: an examination of secondary school students’ perceptions bahar muradoğlu1, nevzat yiğit1, ebru mazlum güven1* 1department of mathematics and science education, trabzon university, turkey *corresponding author: eumazlum@gmail.com abstract as innovation has gained importance worldwide, educating students as individuals with innovative qualities has become imperative. therefore, identifying students' perceptions of innovativeness in science lessons has become an issue of concern. the aim of this study was to determine secondary school students' perceptions of innovativeness. the research was conducted according to the survey method. 'perceptions of innovative thinking scale,' was revised, and necessary scale development steps were followed. accordingly, secondary school students' general innovative thinking perceptions and the relationships of the scale subdimension s with the variables were determined with single and correlational survey models. after the responses that 831 students gave to the scale were analysed, it was determined that the students' perceptions of innovativeness were high and that gender excepted, grade level, achievement in the subject of science, participation in the tübi̇tak 4006 science fair, and the case of receiving programming training created significant differences in the subdimensions. the research findings were discussed according to the literature. it was recommended that stem and programming be included in science courses and that teachers guide these processes. keywords science education, innovativeness in science education, perceptions of innovativeness 1. introduction as the world's resources decrease, countries' ability to innovate to gain an advantage in a competitive environment and their labor force increases indicates that adopting innovations is important (yılmaz-öztürk, 2015). therefore, training individuals who possess innovative qualities have also become state policy (açıkgöz-ersoy & muter-şengül, 2008; öğüt, aygen, & demirsel, 2007). innovativeness, which is defined as 'the state of being open to innovations,' is the ability to take risks, renew oneself, use new technologies, produce new ideas, cooperate, think creatively, and contribute to the change or development of existing situations (demirel & seçkin, 2008). besides keeping pace with changes by using innovations, individuals themselves must also be able to contribute to the changes (kılıçer & odabaşı, 2010). with the constant drive for innovation in the world's economy and the increasing demand for graduating students who are more innovative contributors to society, interest in defining and measuring individual innovativeness is growing (menold, jablokow, purzer, ferguson, & ohland, 2014; yenice & alpak tunç, 2019; weis, scharf, & gryl, 2017). accordingly, teaching programs must be prepared in such a way as to serve to educate individuals who can investigate, inquire, solve the problems they encounter, and benefit themselves and society; in short, who are qualified to respond to the needs of contemporary society (menold et al., 2014). for this purpose, it is seen that to keep up in the innovation race, the subject of 'engineering and design skills' was added to the 2018 turkish science curriculum. these skills allow students to examine problems from an interdisciplinary perspective, attain a level at which they can invent and innovate, and create products by using the knowledge and skills they have acquired (mone, 2018). therefore it is considered the most important means for students to acquire perceptions towards innovative thinking first, and later innovativeness itself. it is expected that individuals be open to innovations, adopt innovations, and closely follow developments in technology. moreover, individuals differ in terms of adopting innovations, and it is seen that they are separated into five categories, namely innovative, traditionalist, mailto:eumazlum@gmail.com journal of science learning article doi: 10.17509/jsl.v5i1.33533 43 j.sci.learn.2022.5(1).42-54 pioneering, inquiring, and sceptical (rogers, 1995). it is also known that individuals' socioeconomic levels and different demographic characteristics influence their adoption of innovations (daghfous, petrof, & pons, 1999). in the literature, although in some studies it was expected that creativity, which is stated to be closely related to innovativeness, would not differ according to gender (baysal, kaya, & üçüncü, 2013; midilli, 2019; polat, 2017; ulusoy-yılmaz & yıldız, 2019). in other studies, it is seen that a difference was revealed in favor of women (barışık, 2019; gök & erdoğan, 2011; rıza, 1999). studies conducted concerning innovativeness determined that total mean scores increased from 6th grade to 8th grade. moreover, it was revealed that 5th-grade students' total means innovativeness scores were higher than 6th-grade students' scores (deveci & kavak, 2020). it is known that students who are high achievers in science also have high levels of creativity (kılıç & tezel, 2012) and that students with high general academic achievement also have high levels of innovativeness (deveci & kavak, 2020). when considering students' creativity scores, differences were determined between students with high end-of-term grades in science and those with low grades, favoring those with high achievement (kılıç & tezel, 2012). regarding innovativeness, differences were determined between students with high academic achievement and those with low achievement in favor of the high achievers (deveci & kavak, 2020). it is known that states of participation in the project preparation process, which allows students to acquire several skills and to develop these skills, enables their creativity to develop positively and allows them to generate new ideas (atalmış, selçuk, & ataç, 2018; seechaliao, 2017; siew & ambo, 2018). furthermore, it has been determined that giving students the chance to develop their creativity and the ability to produce innovations has benefits such as gaining self-confidence and learning to cooperate (avcı, su-özenir, & yücel, 2016) and that at the end of the process, students present different project ideas (soyuçok, 2018). in programming, which is one of the skills expected from students in the 21st century, students can solve problems by figuring them out and using their creativity. at this point, it is stated that in students who learn programming, the development of their problem solving, logical reasoning, creativity, and innovativeness will also be affected positively (aytekin, çakır, yücel, & kulaözü, 2018; yoon, 2018). by this means, in programming, students will have the opportunity to put their ideas into practice by thinking creatively and innovatively. they will also be able to develop their innovativeness in a technological sense. literature indicates that many factors can influence students' innovative thinking as mentioned above. educational policies and schools need to create learning environments to foster innovative thinking. therefore, this contribution seeks to give insight into kinds of innovative thinkers and provide teachers to evaluate students' perception of innovativeness levels, and enable them to improve their teaching to promote students' innovativeness. by revealing which variables affect students' perceptions of innovative thinking and to what extent, it will be possible to offer students innovative thinking and learning environments. 1.1. aim and importance of the study considering the literature, it can be thought that secondary school students' gender, grade level, success grades in the subject of science, project preparation process, and programming training may influence their perceptions of innovative thinking. consequently, it is necessary to determine students' perceptions of innovativeness and the variables that positively affect them. it has been seen that innovativeness studies gained importance worldwide. it is stated that students at all stages of education students need to possess this skill, and measures are taken for this purpose. as the concept of innovativeness has gained so much importance and it is also clearly stated in the science curriculum in turkey (mone, 2018), the position of students attending secondary schools has become an issue of concern. accordingly, the fact that there are an inadequate number of studies at the secondary school level in the literature is regarded as a deficiency. when the literature is examined, the limited number of studies, and the fact that they have generally been conducted on innovativeness in teachers, preservice teachers, school administrators, and academicians in different branches, is striking (aldahdouh, nokelainen, & korhonen, 2020; atamanova & bogomaz, 2021; bayrakçı & eraslan, 2014; demir-başaran & keleş, 2015; kasapoğlu, 2018; kinay & suer, 2020; sarı & kartal, 2018; webster et al., 2020). it is striking that studies related to innovativeness mainly collect around teachers and in higher education (bautista, 2021; mikhailova, 2019; kinay & suer, 2020; nguyen, pietsch, & gümüş, 2021; öztürk, önder, & güven-yıldırım, 2019; parlar, polatcan, & cansoy, 2019; polat, 2017). almost no studies have been conducted about secondary school students' innovativeness (akkaya, 2016; deveci & kavak, 2020; kavacık, yanpar-yelken, & sürmeli, 2015;). among these studies, very few have been carried out on perceptions of innovative thinking, while in one study conducted according to a mixed method (deveci & kavak, 2020), students' general innovativeness was examined. therefore, this study in which secondary school students' innovativeness is examined is thought to contribute to the science education. the current study differs from studies in the literature in that it examines the effect of different variables on students' perceptions of innovative thinking based on subfactors and includes a detailed research process for revealing the existing state of their innovativeness. moreover, it is considered that the study journal of science learning article doi: 10.17509/jsl.v5i1.33533 44 j.sci.learn.2022.5(1).42-54 will serve as a guide for other researchers who will research in the field of science education. this study attempts to determine secondary school students' perceptions of innovativeness. in line with this primary aim, the study's research questions are as follows. 1. what is the level of secondary school students’ perceptions of innovativeness? 2. is there any differences between the secondary school students' 'innovator,' 'traditionalist,' and 'open to inquiry' scores according to their gender? 3. is there any differences between the secondary school students' 'innovator,' 'traditionalist,' and 'open to inquiry' scores according to grade level? 4. is there any differences between the secondary school students' 'innovator,' 'traditionalist,' and 'open to inquiry' scores according to success in science? table 1 distribution of demographic characteristics of secondary school students variables groups frequency (f) percentage (%) gender female 432 52 male 399 48 grade level 6th grade 255 30.7 7th grade 290 34.9 8th grade 286 34.4 achievement in science 0-49.99 (fail) 24 2.9 50-59.99 (pass) 40 4.8 60-69.99 (average) 70 8.4 70-84.99 (good) 198 23.8 85-100 (excellent) 499 60 mother’s education level primary and secondary 245 29.5 high school 326 39.2 bachelor’s and above 260 31.3 mother’s occupation housewife 500 60.2 teacher/lecturer 94 11.3 engineer 14 1.7 healthcare employee 71 8.5 civil servant 67 8.1 worker 85 10.2 father’s education level primary and secondary 190 22.9 high school 304 36.6 bachelor’s and above 322 40 monthly income (tl) 0-1500 64 7.7 1501-3000 331 39.8 3001 and over 436 52.5 area of residence city 601 72.3 village or town 210 27.7 level of liking of science yes 613 73.8 partial 194 23.3 no 24 2.9 state of reading scientific journals yes 295 35.5 no 536 64.5 participation in tübi̇tak yes 206 24.8 no 625 75.2 use of smartboard in class yes 743 89.4 no 88 10.6 state of receiving step education yes 42 5.1 no 789 94.9 length of step education none received 789 94.9 less than one semester 18 2.2 one semester-two semester 13 1.6 more than one year 11 1.3 state of receiving programming training yes 295 35.5 no 536 64.5 length of programming training none received 536 64.5 less than one year 135 15 more than one year 170 20.5 more than one year 170 20.5 journal of science learning article doi: 10.17509/jsl.v5i1.33533 45 j.sci.learn.2022.5(1).42-54 5. is there any difference between the secondary school students' 'innovator,' 'traditionalist,' and 'open to inquiry' scores according to participation in the tübi̇tak (the scientific and technological research council of turkey)? 6. is there any differences between the secondary school students' 'innovator,' 'traditionalist,' and 'open to inquiry' scores according to receiving programming training? 2. method 2.1. research model in this study survey method was applied to determine students' perceptions of innovativeness. quantitative data were collected to reveal secondary school students' perceptions of innovative thinking and the relationship with different variables. by determining the secondary school students’ perceptions towards innovative thinking and the relationship of these with various variables according to the survey method of quantitative research methods, an attempt was made to reveal the general states of the students in terms of their perceptions of innovative thinking. a single survey model revealed a general situation related to the participants' perceptions of innovative thinking. in contrast, a correlational survey model was used to determine whether their perceptions of innovative thinking differed significantly regarding different variables. 2.2. study group it is known that the project preparation process allows students to generate new ideas, develops their creativity positively, and has benefits for students such as gaining self-confidence by producing innovations and learning how to cooperate (atalmış et al., 2018; avcı et al., 2016). therefore, the study's quantitative data were collected from the participants consisting of 6th, 7th, and 8th-grade students who prepared projects for the tübi̇tak 4006 science fair and attended secondary schools participating in the fair during the first semester of the 2018-2019 academic year. table 2 rotated factor loading values and item-total correlation values item no. rotated factor loading values item-total correlation values innovative individual traditional individual inquiring individual 2 .74 .63 1 .73 .59 29 .65 .55 20 .62 .56 13 .60 .57 18 .60 .43 31 .60 .49 28 .58 .59 22 .56 .56 3 .55 .45 14 .54 .53 23 .45 .53 27 .67 .52 16 .62 .52 30 .59 .44 24 .57 .50 15 .55 .40 21 .53 .41 32 .51 .47 9 .68 .55 11 .65 .53 7 .62 .25 8 .62 .37 10 .56 .40 19 .54 .43 eigenvalues 7.74 1.93 1.49 percentage of explained variance 30.96 7.74 5.97 cronbach’s alpha .88 .76 .74 explained variance for total scale 44.68 cronbach’s alpha for total scale .90 https://www.tubitak.gov.tr/en https://www.tubitak.gov.tr/en journal of science learning article doi: 10.17509/jsl.v5i1.33533 46 j.sci.learn.2022.5(1).42-54 accordingly, to collect the quantitative data of the research, the scale was first administered to a total of 1190 students at five different secondary schools in the center of a city and its districts located in the black sea region. the responses given by 176 students to the scale were considered inconsistent and formed a pattern, which was excluded from the analysis. moreover, after outliers were also excluded to ensure normal distribution, the data obtained from 831 students were analyzed, and as a result, the study's quantitative findings were obtained as tabulated in table 1. 2.3. data collection tools development of perceptions of innovative thinking scale within the scope of the research, the 32-item 'perceptions of innovative thinking scale,' developed aimed at determining secondary school students' perceptions of innovativeness, was used. however, the large number of items in this scale may decrease students' motivation, and this situation may prevent the likelihood of obtaining valid and correct responses. considering the age group for whom the scale was developed and the response time of the scale, it was decided to reduce the number of scale items to obtain correct answers (büyüköztürk, 2005; erkuş, 2016). an attempt was made to make the item density more readable without impairing the integrity of the items included in the subdimensions by reducing the number of items from 32 to 25. as it is recommended that in scale development, the implementation should be made with a number of participants between 5 and 10 times the number of items (büyüköztürk, 2002), the scale with the reduced number of items was administered to 320 students outside the scope of the actual study. as a result of the exploratory factor analysis, it was determined that the scale, which was reduced to 25 items, met the conditions of validity and reliability. as in its original form, it was made up of three subdimensions. the kaiser-meyer-olkin (kmo) value and bartlett's test were considered to determine whether the data were suitable for factor analysis. a kmo value of 0.90 was found. this shows that the sample size was excellent. the result of bartlett's test (p < 0.05) showed that factor analysis could be performed with the items in the data set (pallant, 2020). the factor loading values and item-total correlation values obtained from the exploratory factor analysis are presented in table 2. as can be seen in table 2, items 1, 2, 3, 13, 14, 18, 20, 22, 23, 28, 29 and 31 are grouped under factor 1, items 15, 16, 21, 24, 27, 30 and 32 are grouped under factor 2, and items 7, 8, 9, 10, 11 and 19 are grouped under factor 3, respectively. item-total correlation values range between 0.25 and 0.63. values greater than 0.30 show that the items are differentiated, while cases where values are between 0.20 and 0.30 indicate that the items need to be found based on a requirement in the test or that they need to be revised (bursal, 2017; büyüköztürk, 2018). since the total correlation value of item 7 ('i am afraid of taking risks') was below 0.30, by obtaining expert opinion, it was changed in such a way as to bear the same meaning to 'i do not feel the need to continually seek different ways to solve a problem.' the rotated factor loading values ranged between 0.47 and 0.68. seven items were removed when giving the scale its final form (4, 5, 6, 12, 17, 25, and 26). according to table 2, the cronbach alpha values of the total scale and its subdimensions were 0.90, 0.88, 0.76, and 0.74, respectively. based on these findings, it can be said that the reliability level of the broad-scale and its subfactors are high (büyüköztürk, 2018). the scale's subfactors that were reduced to 25 items were revised as 'innovator,' 'traditionalist,' and 'open to enquiry.' the rating statements and their equivalent scores are as follows: strongly disagree 1, disagree 2, somewhat agree 3, agree 4, and strongly agree 5. there are 12 positive items (1, 2, 3, 9, 10, 13, 15, 17, 18, 21, 22, 24) and 13 negative items (4, 5, 6, 7, 8, 11, 12, 14, 16, 19, 20, 23, 25) in all. since the first factor contains statements with which an individual can be characterized as innovative, such as being open to innovations, being able to generate new ideas, being self-confident, being able to use new technologies, and considering social benefit and the national economy, it is given the name 'innovator'. examining the second factor, it is considered to recall an individual who can be characterized as a traditionalist. traditionalist individuals show considerable resistance to innovation and change and regard change, renewal, and innovation as unnecessary. they are characterized by being content with their present situation. they are very uncomfortable with disrupting their habits or conventions, and they display an attitude of indifference to innovations in particular. considering that its items evoke the characteristics of a traditional individual, the second factor is given the name 'traditionalist'. when the items belonging to the third subdimension are considered, it is seen that they contain statements such as 'i am undecided about using innovations and new technologies, or 'i worry about trying out new ideas.' it can be seen that these items contain anxiety, indecision, and fear towards innovation and that there is worry and caution regarding innovations. therefore, the third factor representing these items is 'open to inquiry.' worry and indecision towards an innovation indicate the necessity to question or ponder that innovation. this situation recalls individuals with inquiring characteristics in terms of thinking for a long time and feeling the need for other people's ideas when encountering an innovation. 2.4. data analysis the quantitative data obtained in the scope of the research were analyzed on a computer. the data were coded and computerized, and care was taken to code the negative items in reverse. to analyze the data obtained from the 'perceptions of innovative thinking scale' for journal of science learning article doi: 10.17509/jsl.v5i1.33533 47 j.sci.learn.2022.5(1).42-54 secondary school students, a normality test was performed to determine whether the data showed a normal distribution. tabachnick and fidell (2013) stated that variables taking skewness and kurtosis values between -1.5 and +1.5 could be accepted as showing normal distribution. in order to determine whether or not the secondary school students' perceptions of innovative thinking differed significantly according to the different demographic variables, the parametric manova test was applied since the scale contains three subfactors. to enable the manova test to be performed, multivariate normality wa s tested in line with the general normality analysis results. by examining the skewness and kurtosis values for normality of the distributions of the dependent variables according to the independent variable categories, the distributions were determined to be normal. the' mahalanobis distance ' was examined to ensure the condition of multivariate normality. the threshold value was set as 7.815 (pallant, 2020). the process was repeated by excluding data sources taking values above this value. after the assumption of multivariate normality was met, it was determined that the condition was enabled by examining the appropriateness of the correlation between the dependent variables (< 0.90). equality of covariance was ensured for each independent variable. finally, as a result of levene's test, it was determined that the variances for each independent variable were homogeneous. a oneway analysis of variance was performed to determine the source of differences for variables, including more than two groups for which significant differences were determined due to manova analysis. scheffe's test was used to determine which paired groups there were differences. in order to determine the effect size of the relationship established for the variables, the eta-squared values were examined. according to cohen's recommendation, effect sizes of 0.01 are evaluated as small, 0.06 as medium, and 0.14 as large. an attempt was made to explain with tables the mean scores and standard deviation values obtained by the secondary school students participating in the research from the general 'perceptions of innovative thinking scale' and its subfactors. moreover, the relationship of the scores obtained by the secondary school students from the 'innovator', 'traditionalist', and 'open to inquiry' subfactors of the scale with the determined independent variables were examined. the independent variables are gender (1: female, 2: male), grade level (1: sixth grade, 2: seventh grade, 3: eighth grade), success grade in science (1: fail, 2: pass, 3: average, 4: good, 5: excellent), participation in the tübi̇tak science fair (1: participated, 2: did not participate), and state of receiving programming training (1: received, 2: not received). the total innovativeness score was obtained from the scale, and in turn, the scores related to the subdimensions were determined. the lowest score that can be obtained from the scale was determined to be 25, while the highest obtainable score was determined as 125. to specify intervals in the name of determining the total score obtained from the scale, a standard unit was calculated by dividing the sequence width of the highest and lowest scores obtainable from the scale by the number of options [(125-25)/5 = 20], and the intervals were determined approximately according to this unit. scores obtained from a scale of 86 and over were accepted as high perceptions of innovation by the secondary school students. in contrast, scores of 85 and below were considered low perceptions of innovation by the students. in terms of the subdimensions of the scale, scores obtained for the 12item 'innovator' dimension of 40.7 and below were evaluated as negative, while scores of 40.8 and above were assessed as positive. concerning the 7-item 'traditionalist' dimensions, scores of 23.7 and under were assessed as negative, while scores of 23.8 and over were evaluated as positive. finally, for the 'open to inquiry' dimension consisting of 6 items, scores of 20.3 and below were negative, while scores of 20.4 and above were regarded as positive. 3. findings 3.1. findings related to levels of secondary school students’ perceptions of innovative thinking the mean scores and standard deviation values obtained by the students participating in the study from the perceptions of innovative thinking scale in general and its subfactors were examined. descriptive statistical information related to the general distribution of the scores obtained by the students from the ‘perceptions of innovative thinking scale’ and its subdimensions is provided in table 3. in light of the findings, it was determined that the mean scores obtained by the secondary school students from the perceptions of innovative thinking scale and its subdimensions were 105.41, 49.85, 30.87, and 24.59, table 3 descriptive statistical information related to perceptions of innovative thinking scale mean scores no. of items no. of participants mean standard deviation min. max. perceptions of innovative thinking 25 831 105.41 11.01 77 125 innovator 12 831 49.95 6.58 33 60 traditionalist 7 831 30.87 3.26 22 35 open to inquiry 6 831 24.59 2.99 17 30 journal of science learning article doi: 10.17509/jsl.v5i1.33533 48 j.sci.learn.2022.5(1).42-54 respectively, and that the scores were high. moreover, while the lowest score obtained from the whole scale was 77, the highest was 125. the correlation matrix, which presents the correlations of the factors and the factor total, is given in table 4. examination of table 4 shows that the subfactors are correlated with the total score in amounts ranging between .74 and .93. in the related literature, in determining interfactor correlations, a correlation coefficient between .70 and 1 indicates a high correlation, while a coefficient between .70 and .30 shows moderate correlation (büyüköztürk, 2018). accordingly, each factor has a high positive correlation with the factor total, while the subfactors are moderately correlated with each other. 3.2. findings related to examination of secondary school students’ perceptions of innovativeness according to gender multivariate analysis of variance (manova) was performed to examine differences between the secondary school students' 'innovator', 'traditionalist' and 'open to inquiry' scores according to their gender. the findings obtained are presented in table 5. the assumption of multivariate normality, which is the precondition of the manova test, was met (box m test = 4.933, p = 555). when table 5 is examined, it is seen that the group effect of both the ‘innovator’ scores [wilks’𝜆 = 0.988, f(1.829) = 1.411, p > 0.05] and the ‘traditionalist’ scores [wilks’𝜆 = 0.988, f(1.829) = 0.607, p > 0.05] of the secondary school students is not significant. in other words, a significant difference was not found between boys' and girls' 'innovator' or 'traditionalist' scores. on the other hand, it is seen that the group effect of the secondary school students 'open to inquiry' scores is significant [wilks'𝜆 = 0.988, f(1.829) = 4.686, p < 0.05]. however, the effect size value (η2 = 0.006) was found to be very low. when the mean scores are examined for the source of the difference, it can be seen that males’ ‘open to inquiry’ scores (𝑋 = 24.83) are higher than those of females (𝑋 = 24.38), albeit to a minimal extent. 3.3. findings related to examination of secondary school students’ perceptions of innovativeness according to grade level multivariate analysis of variance (manova) was carried out to examine differences between the 'innovator', 'traditionalist', and 'open to inquiry' scores of the secondary school students at different grade levels. the findings obtained are shown in table 6. the assumption of multivariate normality, which is the precondition of the manova test, was met (box m test = 0.859, p = 0.589). examination of table 6 reveals that the group effect of both the ‘innovator’ scores [wilks’𝜆 = 0.978, f(2.828) = 6.726, p < 0.05] and the ‘traditionalist’ scores [wilks’𝜆 = 0.978, f(2.828) = 6.473, p < 0.05] of the secondary school students is significant. however, the group effect of the students’ ‘open to inquiry’ scores was not found to be significant [wilks’𝜆 = 0.978, f(2.828) = 2.333, p > 0.05]. the effect size value of both the ‘innovator’ (η2 = 0.016) and the ‘traditionalist’ (η2 = 0.015) scores was found to be low. when the mean scores are examined for the source of the difference, 6th-grade secondary school students' 'innovator' scores (𝑋 = 51.01) do not differ significantly from 7th-grade students' scores (𝑋 = 50.01). however, they differ significantly from 8th-grade secondary table 4 correlation matrix for factors and factor total factor innovator traditionalist open to inquiry innovator 1 .62 .55 traditionalist .62 1 .47 open to inquiry .55 .47 1 total .93 .80 .74 table 5 manova results for ‘innovator’, ‘traditionalist’, and ‘open to inquiry’ scores according to gender variable kt sd ko f p η2 innovator 61.176 1 61.176 1.411 0.235 0.002 traditionalist 6.467 1 6.467 0.607 0.436 0.001 open to inquiry 41.957 1 41.957 4.686 0.031* 0.006 *p < 0.05 table 6 manova results for ‘innovator’, ‘traditionalist’, and ‘open to inquiry’ scores according to secondary school students’ grade level kt sd ko f p η2 innovator 575.700 2 287.850 6.726 0.001* 0.016 traditionalist 136.175 2 60.088 6.473 0.002* 0.015 open to inquiry 41.830 2 20.915 2.333 0.098 0.006 *p < 0.05 journal of science learning article doi: 10.17509/jsl.v5i1.33533 49 j.sci.learn.2022.5(1).42-54 school students' 'innovator' scores (𝑋 = 48.94). the reason for this is that 6th-grade secondary school students' 'innovator' scores were found to be higher than 8th-grade secondary school students' 'innovator' scores. when examined in terms of the 'traditionalist' scores, it is seen that 6th-grade secondary school students' scores (𝑋 = 31.32) do not differ from 7th-grade students' scores (𝑋 = 31.00), whereas they differ significantly from 8th-grade students' scores (𝑋 = 30.34). it was determined that 6thgrade students' 'traditionalist' scores are higher than 8thgrade students' 'traditionalist' scores. 3.4. findings related to examination of secondary school students’ perceptions of innovativeness according to success grades in the subject of science multivariate analysis of variance (manova) was performed to examine whether there were differences between the ‘innovator’, ‘traditionalist’ and ‘open to inquiry’ scores of the secondary school students who had different success grades in the subject of science. the findings obtained are presented in table 7. the assumption of multivariate normality, which is the precondition of the manova test, was met (box m test = 0.621, p = 0.923). when table 7 is examined, it is seen that the group effect of all the ‘innovator’ [wilks’𝜆 = 0.922, f(4.826) = 10.564, p < 0.05], ‘traditionalist’ [wilks’𝜆 = 0.922, f(4.826) = 13.565, p < 0.05] and ‘open to inquiry’ [wilks’𝜆 = 0.922, f(4.826) = 9.156, p < 0.05] scores is significant. the effect size value of both the ‘innovator’ (η2 = 0.049) and ‘open to inquiry’ (η2 = 0.041) scores were found to be low, while that of the 'traditionalist' (η2 = 0.062) scores was found to be moderate. when the mean scores are examined in terms of the source of the difference, the ‘innovator’ scores of students with average grades in science (𝑋= 48.82) differ significantly from scores of students who failed (𝑋 = 43.75). the ‘innovator’ scores of students with good grades in science (𝑋 = 48.97) differ significantly from scores of students who failed (𝑋 = 43.75). finally, the ‘innovator’ scores of students with excellent grades in science (𝑋= 50.91) also differ significantly from scores of students who failed (𝑋 = 43.75). examination of the ‘traditionalist’ scores reveals that the ‘traditionalist’ scores of secondary school students with average grades in science (𝑋 = 30.15) differ significantly from students who failed (𝑋= 27.58). the ‘traditionalist’ scores of secondary school students with good grades in science (𝑋 = 30.30) also differ significantly from scores of students who failed (𝑋= 27.58). the ‘traditionalist’ scores of secondary school students with excellent grades in science (𝑋 = 31.42) differ significantly from scores of students who failed (𝑋 = 27.58), scores of those with average grades (𝑋 = 30.15), and scores of those with good grades (𝑋 = 30.30). when the ‘open to inquiry’ scores are examined, it can be seen that the ‘open to inquiry’ scores of secondary school students with excellent grades in science (𝑋= 25.04) differ significantly from scores of students with good grades (𝑋 = 24.14), scores of those with pass grades (𝑋 = 23.35), and scores of those who failed (𝑋 = 22.70). it was determined that secondary school students with excellent grades in science have higher ‘open to inquiry’ scores than secondary school students with good and pass grades and students who failed. 3.5. findings related to examination of secondary school students’ perceptions of innovativeness according to state of participation in tübi̇tak science fair multivariate analysis of variance (manova) was carried out to examine whether there were differences between the ‘innovator’, ‘traditionalist’ and ‘open to inquiry’ scores of the secondary school students according to whether or not they had participated in the tübi̇tak 4006 science fair. the findings obtained are shown in table 8. the assumption of multivariate normality, which is the precondition of the manova test, was met (box m test = 1.431, p = 0.198). examination of table 8 shows that the table 7 manova results for ‘innovator’, ‘traditionalist’, and ‘open to inquiry’ scores according to science success grades variable kt sd ko f p η2 innovator 1752.494 4 438.123 10.564 0.001* 0.049 traditionalist 545.245 4 136.311 13.565 0.000* 0.062 open to inquiry 316.923 4 79.231 9.156 0.000* 0.041 *p < 0.05 table 8 manova results for ‘innovator’, ‘traditionalist’, and ‘open to inquiry’ scores according to variable of participation in tübitak 4006 science fair kt sd ko f p η2 innovator 1334.216 1 1334.216 31.898 0.000* 0.037 traditionalist 201.124 1 201.124 19.289 0.000* 0.023 open to inquiry 108.724 1 108.724 12.253 0.000* 0.015 *p < 0.05 journal of science learning article doi: 10.17509/jsl.v5i1.33533 50 j.sci.learn.2022.5(1).42-54 group effect of all the ‘innovator’ [wilks’𝜆 = 0.961, f(1.829) = 31.898, p < 0.05], ‘traditionalist’ [wilks’𝜆 = 0.961, f(1.829) = 19.289, p < 0.05] and ‘open to inquiry’ [wilks’𝜆 = 0.961, f(1.829) = 12.253, p < 0.05] scores is significant. the effect size value of all the ‘innovator’ (η2 = 0.037), ‘traditionalist’ (η2 = 0.023) and ‘open to inquiry’ scores (η2 = 0.015) was found to be low. when the mean scores are examined in terms of the source of the difference, it is seen that the scores of the students who took part in the science fair are higher than scores of those who did not participate with regard to their ‘innovator’ scores (𝑋= 52.16, 𝑋= 49.22, respectively), ‘traditionalist’ scores (𝑋 = 31.73, 𝑋 = 30.59, respectively), and ‘open to inquiry’ scores (𝑋= 25.22, 𝑋= 24.39, respectively). 3.6. findings related to examination of secondary school students’ perceptions of innovativeness according to state of receiving programming training multivariate analysis of variance (manova) was carried out to examine differences between the 'innovator', 'traditionalist' and 'open to inquiry' scores of the secondary school students according to whether or not they had received programming training. the findings obtained are presented in table 9. the assumption of multivariate normality, which is the precondition of the manova test, was met (box m test = 0.347, p = 0.912). when table 9 is examined, it is seen that the group effect of all the ‘innovator’ [wilks’𝜆 = 0.984, f(1.829) = 12.440, p < 0.05], ‘traditionalist’ [wilks’𝜆 = 0.984, f(1.829) = 6.223, p < 0.05] and ‘open to inquiry’ [wilks’𝜆 = 0.984, f(1.829) = 6.609, p < 0.05] scores is significant. the effect size value of the ‘innovator’ (η2 = 0.015) scores was found to be low, while in terms of the ‘traditionalist’ (η2 = 0.007) and ‘open to inquiry’ scores (η2 = 0.008), it was found to be very low. when the mean scores are examined in terms of the source of the difference, is can be seen that the scores of the students who received programming training are higher than scores of those who did not receive it in terms of their ‘innovator’ scores (𝑋= 51.03, 𝑋= 49.35, respectively), ‘traditionalist’ scores (𝑋 = 31.25, 𝑋 = 30.66, respectively), and ‘open to inquiry’ scores (𝑋= 24.95, 𝑋= 24.40, respectively). 4. discussion in this study, which was conducted to determine the innovativeness of secondary school students in their science lessons, it was determined that the secondary school students' perceptions towards innovative thinking were high and that in terms of the subdimensions, their mean scores in the 'innovator', 'traditionalist' and 'open to inquiry' subdimensions were also positive and high. this finding corresponds with the findings of the study by deveci and kavak (2020), in which 46% of students showed a high innovative thinking tendency. accordingly, it can be said that the students in the sample were generally open to innovation and change. the increase in students' interest in technology nowadays can be given as a reason for this. therefore, it is considered that teachers should have innovative characteristics and that they can frequently include innovative teaching methods-techniques and technology in their classes. moreover, factors such as parents' knowledgeableness and high-income levels due to their professions may also come into play. when examined in terms of the gender factor, it was determined that while there was no significant difference for the 'innovator' and 'traditionalist' factors, there was an effect on the 'open to inquiry' factor in favor of boys albeit at a very low level. similar to these findings, it was revealed that gender did not make a significant difference to innovativeness (deveci & kavak, 2020), creativity (dilek, 2013; kanlı, 2017; midilli, 2019), entrepreneurship (deveci, 2018), problem-solving skills (özbulak, aypay, & aypay, 2011), or critical thinking (akar & kara, 2016). however, the fact that significant differences were determined in secondary school students’ creativity (barışık, 2019), 21stcentury learning skills (bozkurt & çakır, 2016), scientific inquiry perceptions (i̇nel-ekici, 2016), and critical thinking skills (köksal & çöğmen, 2018), in favor of females, conflicts with the findings of this study. öztürk et al. (2019) stated the reason why the entrepreneurial characteristic does not differ significantly according to gender, that together with the increase in their education level, females are claiming their place in the world of entrepreneurship, and that in the 21st century, families in our country are offering similar opportunities without gender discrimination. the fact that gender did not significantly differ from the 'innovator' and 'traditionalist' scores may be because the students had more or less the same opportunities in the areas where they grew up. moreover, rather than gender, the family's education level, school facilities, and teachers' and administrators' innovative characteristics may have affected the 'innovator' and 'traditionalist' scores. therefore, it can be thought that families with a high table 9 manova results for ‘innovator’, ‘traditionalist’, and ‘open to inquiry’ scores according to variable of receiving programming training kt sd ko f p η2 innovator 532.343 1 532.343 12.440 0.000* 0.015 traditionalist 65.907 1 65.907 6.223 0.013* 0.007 open to inquiry 59.037 1 59.037 6.609 0.010* 0.008 *p < 0.05 journal of science learning article doi: 10.17509/jsl.v5i1.33533 51 j.sci.learn.2022.5(1).42-54 education level will also have high levels of innovativeness, and their children will also have high innovativeness levels. when the grade level variable was examined, it was determined that while there was a significant difference concerning the 'innovator' and 'traditionalist' subdimensions, a significant difference did not occur regarding the 'open to inquiry' subfactor. sixth-grade students' 'innovator' and 'traditionalist' scores were determined to be higher than eighth-grade students' scores. in the literature, the finding that the use of 21st-century skills decreased as grade level increased (bozkurt & çakır, 2016) corresponds with the finding of this study. however, the fact that creativity (barışık, 2019), perceptions towards problem-solving skills (tunç & taşgın, 2018), and critical thinking skills (çakırlar-altuntaş, yılmaz, & turan, 2017) did not vary according to grade level conflicts with this finding. one of the reasons why the secondary school students’ ‘innovator’ scores decreased as grade level increased may be the fact that secondary school students focus on solving tests rather than activities and experiments in their lessons in order to prepare for the high school placement examinations (bozkurt & çakır, 2016). it can be thought that teachers and parents also encourage students mainly to prepare for the examinations. when considered in terms of the ‘innovator’ dimension, it was determined that the ‘innovator’ scores of students with high success grades in the subject of science differed significantly from the scores of those with other success grades. when the literature is examined, it was stated in the literature that creativity levels were high in students with good science grades (erdoğdu & şirin, 2018; kılıç & tezel, 2012; baysal et al., 2013), entrepreneurship tendencies were high in students with high academic achievement (deveci, 2018), problem-solving skills were high in students with high achievement in science (durgun & önder, 2019), and innovativeness tendencies were high in students with high perceptions of scientific inquiry skills (i̇nel-ekici, 2016) and in academically successful students (deveci & kavak, 2020). these findings correspond with the findings of the present study. deveci and kavak (2020) reported that successful students asked more questions than others. since they generated creative ideas, the fact that their tendencies towards innovative thinking were also high was an expected result. i̇nel-ekici (2016) stated that perceptions towards innovativeness were high since teachers included inquiry-based activities in their lessons and that students who were highly successful in science also showed active participation in this process. when examined in terms of participation in the tübi̇tak 4006 science fair, a significant difference was found in students' 'innovator', 'traditionalist' and 'open to inquiry' mean scores in favor of those who took part in the science fair. similar to this finding, yıldırım (2018) revealed that problem-solving skills developed in students who participated in science festivals, çavuş, balçın, and yılmaz (2018) reported that science fair activities increased secondary school students' perceptions of problem-solving skills, and soyuçok (2018) stated that communication and creativity skills developed in students who created projects by taking part in science fairs. it was determined that science fairs reduce students' anxiety towards the subject of science and also have a positive effect on their motivation (keskin, 2019), that they have positive benefits for students' inquiry skills, and that they increase students' interest and achievement in the subject of science (soyuçok, 2018). project-based learning methods will support the education of students as individuals who are curious, investigate, inquire, solve the problems they encounter, and think critically and creatively (avcı & suözenir, 2018; seechaliao, 2017, siew & ambo, 2018). accordingly, the fact that 'innovator' and 'open to inquiry' scores were high is an expected result. the reason why the 'traditionalist' scores of students who participated in the science fair were also high maybe because innovative thinking skills had only recently been added to the curriculum. the 'innovator', 'traditionalist' and 'open to inquiry' scores of students who had received programming training were higher than those who had not received it. this finding shows similarity with the findings that for students receiving programming training, their problem-solving, creative, and innovative thinking skills developed (başarmak & hamutoğlu, 2019). in addition, the training enabled their creativity and digital thinking skills, ability to identify problems and solve the problems they identified, design skills, and ability to think multilaterally (göksoy & yılmaz, 2018). considering the interest in technology shown by students nowadays, it can be thought that the programming training they received also attracted their interest. therefore, the process was experienced productively. on the other hand, the reason why the 'traditionalist' scores of students who received programming training were also high maybe because some teachers were not sufficiently equipped (mıhcı-türker & pala, 2018), that some school principals did not have adequate knowledge (ünsal, 2019), and that reasons such as these led students to regard programming as a simple game. conclusion it was determined that the students generally possessed innovative personality traits, paid regard to social benefit, and were open to innovation and development. findings reveal that the secondary school students were open to innovation and change, possessed innovative traits, and gave importance to social benefit; in short, they were innovators. moreover, it was concluded that the increasing importance given to innovativeness nowadays, projectbased learning aimed at developing innovativeness in the journal of science learning article doi: 10.17509/jsl.v5i1.33533 52 j.sci.learn.2022.5(1).42-54 education process, different teaching practices such as stem, and the inclusion of technology in the process, have positive effects on students’ innovativeness. it was revealed that sixth-grade students' 'innovator ‘scores were higher than those of eighth grade students. it can be said that because students prepared for high school placement examinations and focused on answering test questions, their innovativeness decreased as grade level increased. considering the positive effect on innovativeness of involvement in project work, practices towards finding solutions to everyday problems can be included in eighth grade students' lessons to contribute positively to their innovativeness. although the students who took part in the study were at ages when abstract thinking skills began to develop, concrete products are essential for their better interpretation. consequently, experiencing the process of including stem or programming activities and project work is important for contributing to students' innovativeness. therefore, it can be recommended that science teachers provide the necessary opportunities. teachers should serve as a guide to students in their activities such as project preparation and should not help them achieve results by gravitating towards highly successful students in lessons and offering them readymade project ideas. instead, more encouragement should be given, especially to students with low achievement in lessons and whose creative and innovative thinking skills are not sufficiently developed. moreover, in the process, students should be given opportunities to generate ideas such as identifying the problem, developing solution suggestions, and putting these ideas into practice. acknowledgment this article is generated from the master thesis of the first author. references açıkgöz-ersoy, b., & muter-şengül, c. (2008). yenilikçiliğe yönelik devlet uygulamaları ve ab karşılaştırması [government applications towards ınnovation and comparison with european union]. journal of management and economics, 15(1), 59-74. akar, c., & kara, m. (2016). i̇lkokul 4. sınıf öğrencilerinin eleştirel düşünme becerilerinin bazı değişkenlere göre değerlendirilmesi [the evaluation of 4ht elementary school students' critical thinking skills in terms of some variables]. international journal of turkish literature culture education, 5(3), 1339-1355. akkaya, d. (2016). i̇lköğretim 7. sınıf öğrencilerinin inovasyon becerilerinin değerlendirilmesi [evaluation of innovation skills of 7th grade primary school students]. (unpublished master’s thesis). university of adnan menderes. aldahdouh, t. z., nokelainen, p., & korhonen, v. (2020). technology and social media usage in higher education: the influence of individual innovativeness. sage open, 10(1) https://doi.org/10.1177/2158244019899441 atalmış, e. h., selçuk, g., & ataç, a. (2018). tübi̇tak 4006 projelerine ilişkin yönetici, yürütücü ve öğrenci görüşleri [manager, executive and student views on tubitak 4006 projects]. kırşehir journal of education, 19(3), 1999-2020. http://dx.doi.org/10.29299/kefad.2018.19.03.006 atamanova, i., & bogomaz, s. (2021). value and activity-based orientations of university students: the choice between safety and innovativeness. science, 11(1), 59-74 http://dx.doi.org/10.15293/2658-6762.2101.04 avcı, e., su-özenir, ö., & yücel, e. (2016). tübi̇tak ortaöğretim öğrencileri araştırma projeleri yarışmasına katılan öğrencilerin yarışma sonrası kazanımlarının incelenmesi [students' experiences during tubitak secondary education students' research projects competition and its contribution to their university life]. uşak university journal of social sciences, 9(27/3), 1-21. avcı, e., & su-özenir, s. (2018). bilim fuarları sürecinin yürütücü öğretmenler gözünden değerlendirilmesi [evaluation of science fair process from project coordinator teachers’ point of view]. elementary education online, 17(3), 1672-1690. https://doi.org/10.17051/ilkonline.2018.466417 aytekin, a., sönmez-çakır, f., yücel, y. b., & kulaözü, i̇. (2018). geleceğe yön veren kodlama bilimi ve kodlama öğrenmede kullanılabilecek bazı yöntemler [coding science directed to future and some methods to be available, and coding learned]. eurasian journal of social and economic research, 5(5), 24-41. barışık, a. (2019). ortaokul öğrencilerinin benlik saygıları ve yaratıcı düşünme becerilerinin çeşitli değişkenler açısından incelenmesi i̇stanbul ili örneği [examination of secondary school students' self-esteem and creative thinking skills in terms of various variables the sample of i̇stanbul province]. (unpublished master’s thesis). university of çağ. başarmak, u., & hamutoğlu, n. b. (2019). ortaokul öğrencilerinin “kod adı 2023” projesi eğitimine yönelik görüşleri [secondary school students' opinions on “kod adı 2023” project education]. gazi journal of educational sciences, 5(special issue), 55-66. https://doi.org/10.30855/gjes.2019.os.01.004 bautista, r. g. (2021). project kinang: a gem in advancing the individual innovativeness of pre-service secondary school teachers. turkish journal of computer and mathematics education (turcomat), 12(3), 5265-5278. bayrakçı, m., & eraslan, f. (2014). ortaöğretim okul yöneticilerinin inovasyon yeterlilikleri [innovation competencies of highschool administrator]. sakarya university journal of education faculty, (28), 96-135. baysal, z. n., kaya, n. b., & üçüncü, g. (2013). i̇lkokul dördüncü sınıf öğrencilerinde bilimsel yaratıcılık düzeyinin çeşitli değişkenler açısından incelenmesi [examining the level of scientific creativity in primary school fourth grade students in terms of various variables]. eğitim bilimleri dergisi, 38, 55-64. bozkurt, ş. b., & çakır, h. (2016). ortaokul öğrencilerinin 21. yüzyıl öğrenme beceri düzeylerinin cinsiyet ve sınıf seviyesine göre incelenmesi [21st century learner skills: an investigation of middle school students based on grade level and gender]. pamukkale university journal of education, 39(39), 69-82. https://doi.org/10.9779/puje757 bursal, m. (2017). spss ile temel veri analizleri [basic data analysis with spss]. anı yayıncılık. büyüköztürk, ş. (2002). faktör analizi: temel kavramlar ve ölçek geliştirmede kullanımı [factor analysis: basic concepts and its use in scale development]. educational administration – theory and practice, 8(4), 470-483. büyüköztürk, ş. (2005). anket geliştirme [survey development]. türk eğitim bilimleri dergisi, 3(2), 133-151. büyüköztürk, ş. (2018). sosyal bilimler için veri analizi el kitabı, istatistik, araştırma deseni spss uygulamaları ve yorum [data analysis handbook for social sciences, statistics, research design spss applications and interpretation]. pegem akademi. çakırlar-altuntaş, e., yılmaz, m., & turan, s. l. (2017). biyoloji öğretmen adaylarının eleştirel düşünme eğilimleri üzerine bir inceleme [an investigation on critical thinking tendencies of preservice biology teachers]. ege journal of education, 19(1), 34-45. https://doi.org/10.12984/egeefd.306019 https://doi.org/10.1177%2f2158244019899441 http://dx.doi.org/10.29299/kefad.2018.19.03.006 http://dx.doi.org/10.15293/2658-6762.2101.04 https://doi.org/10.17051/ilkonline.2018.466417 https://doi.org/10.30855/gjes.2019.os.01.004 https://doi.org/10.9779/puje757 https://doi.org/10.12984/egeefd.306019 journal of science learning article doi: 10.17509/jsl.v5i1.33533 53 j.sci.learn.2022.5(1).42-54 çavuş, r., balçın, m. d., & yılmaz, m. m. (2018). bilim fuarı etkinliklerinin ortaokul öğrencilerinin fen ve problem çözme becerilerine yönelik algılarına etkisi [the effect of science fair activities on secondary school students' perceptions of science and problem-solving skills]. inonu university journal of the graduate school of education, 5(10), 1-17. https://doi.org/10.29129/inujgse.395132 daghfous n., petrof j.v., & pons f. (1999). values and adoption of ınnovations: a cross cultural study. journal of consumer marketing, 16(4), 314-331. demir-başaran, s., & keleş, s. (2015). yenilikçi kimdir? öğretmenlerin yenilikçilik düzeylerinin incelenmesi [who is innovative? examination of teachers’ innovativeness level]. hacettepe university journal of education, 30(4), 106-118. demirel, y., & seçkin, z. (2008). bilgi ve bilgi paylaşımının yenilikçilik üzerine etkileri [the impact of knowledge and knowledge sharing on innovation]. çukurova university journal of institute of social sciences, 17(1), 189-202. deveci, i̇. (2018). ortaokul öğrencilerine yönelik fen tabanlı girişimcilik ölçeği: geçerlik ve güvenirlik çalışması [science-based entrepreneurship scale for middle school students: a validity and reliability study]. journal of multidisciplinary studies in education, 2(1), 1-15. deveci, i̇., & kavak, s. (2020). ortaokul öğrencilerinin yenilikçilik algıları ve yenilikçi düşünme eğilimleri: bir keşfedici ardışık desen [innovativeness perceptions and innovative thinking tendencies of middle school students: an exploratory sequential design]. journal of qualitative research in education, 8(1), 346-378. https://doi.org/10.14689/issn.2148-2624.1.8c.1s.15m dilek, a. n. (2013). sosyo-kültürel özelliklerin yaratıcı düşünmeye etkisi [the impact of socio-cultural features on creative thinking]. (unpublished master’s thesis). university of osmangazi. durgun, e., & önder, i̇. (2019). ortaokul 7. sınıf öğrencilerinin fen bilimleri başarıları ile okuduğunu anlama, grafik okuma ve sorun çözme becerileri arasındaki ilişki [the relationship of science achievement with reading comprehension, graphic reading, problem solving skills in middle school seventh grade students]. journal of individual differences in education, 1(1), 1-13. erdoğdu, m. y., & şirin, t. (2018). i̇lkokul öğrencilerinin yaratıcılıkları ile ahlaki olgunluk ve akademik başarı ilişkisi [the relationshıp of empathy moral maturity and academic achievement with the development of primary school students’ creativeness]. electronic turkish studies, 13(19), 681-696. https://doi.org/10.7827/turkishstudies.13746 erkuş, a. (2016). psikoloji ölçme ve ölçek geliştirme-i: temel kavramlar ve işlemler [psychology measurement and scale development-i: basic concepts and operations]. pegem akademi. gök, b., & erdoğan, t. (2011). sınıf öğretmeni adaylarının yaratıcı düşünme düzeyleri ve eleştirel düşünme eğilimlerinin incelenmesi [the investigation of the creative thinking levels and the critical thinking disposition of pre-service elementary teachers]. ankara university journal of faculty of educational sciences, 44(2), 29-51. göksoy, s., & yılmaz, i̇. (2018). bilişim teknolojileri öğretmenleri ve öğrencilerinin robotik ve kodlama dersine ilişkin görüşleri [the opinions of informatıon relations teacher and their students with regard to lessons of robots and decoding]. journal of düzce university institute of social sciences, 8(1), 178-196. i̇nel-ekici, d. (2016). ortaokul öğrencilerinin bilimsel sorgulama becerileri algılarını etkileyen faktörlerin incelenmesi [a study on the factors affecting secondary school students' scientific research skills perceptions]. kastamonu education journal, 25(2), 497-516. kanlı, e. (2017). üstün yetenekli öğrencilerin bilimsel yaratıcılık düzeyleri, cinsiyet ve bilimsel tutumları arasındaki ilişkilerin incelenmesi [investigating the relations between scientific creativity, gender and scientific attitudes of gifted learners]. elementary education online, 16(4), 1792-1802. https://doi.org/ 10.17051/ilkonline.2017.342992 kasapoğlu, h. (2018). üniversite yöneticilerinin yenilikçilik ve risk alma davranışlarına ilişkin yönetici ve öğretim elemanlarının görüşleri [the views of administrators and academic staff about the innovativeness and risk-taking behaviors of university administrators]. journal of higher education, 8(1), 45-54. https://doi.org/10.2399/yod.18.001 kavacık, l., yanpar-yelken, t. & sürmeli, h. (2015). i̇lköğretim fen ve teknoloji dersinde inovasyon (yenilikçi) proje uygulamaları ve öğrenciler üzerindeki etkileri [innovation (innovative) project applications in primary science and technology lesson and its effects on students]. education and science, 40(180), 247-263. http://dx.doi.org/10.15390/eb.2015.2613 keskin, d. (2019). bilim fuarlarının ortaokul öğrencilerinin bilimsel süreç becerileri, fen dersine karşı motivasyonları ve kaygı düzeyleri üzerinde etkisi [the effect of science fairs on science process skills, motivation and anxiety levels of secondary school students]. (unpublished master’s thesis). university of pamukkale. kılıç, b., & tezel, ö. (2012). i̇lköğretim sekizinci sınıf öğrencilerinin bilimsel yaratıcılık düzeylerinin belirlenmesi [determining scientific creativity levels of 8th grade students]. türk fen eğitimi dergisi, 9(4), 84-101. kılıçer, k., & odabaşı, h. f. (2010). bireysel yenilikçilik ölçeği (byö): türkçeye uyarlama, geçerlik ve güvenirlik çalışması [individual innovativeness scale (is): the study of adaptation to turkish, validity and reliability]. hacettepe university journal of education, 38(38), 150-164. kinay, i., & suer, s. (2020). investigation of relationship between prospective teachers’ learning beliefs and state of individual innovativeness. cypriot journal of educational sciences, 15(3), 604-618 https://doi.org/10.18844/cjes.v15i3.4642 köksal, n., & çöğmen, s. (2018). ortaokul öğrencilerinin eleştirel düşünme ve iletişim becerileri [critical thinking and communication skills of secondary school students]. pamukkale university journal of education, 44, 278-296. https://doi.org/ 10.9779/puje.2018.218 menold, j., jablokow, k., purzer, s., ferguson, d. m., & ohland, m. w. (2014, june). a critical review of measures of innovativeness. in 2014 asee annual conference & exposition (pp. 24-41). mıhcı-türker, p., & pala, f. k. (2018). ortaokul öğrencilerinin, öğretmenlerin ve öğrenci velilerinin kodlamaya yönelik görüşleri [opinions of secondary school students, teachers and parents about coding]. elementary education online, 17(4), 2013-2029. https://doi.org/ 10.17051/ilkonline.2019.506939 midilli, m. (2019). özel yetenekli ilkokul öğrencilerinin yaratıcı düşünme becerilerinin bazı değişkenlere göre incelenmesi [invastigation of gifted primary school students' creative thinking skills in terms of some variables]. (unpublished master’s thesis). university of ordu. mikhailova, o. b. (2019). high school students involved and not involved in mmorpg: creativity and innovativeness. international journal of cognitive research in science, engineering and education, 7(2), 29-39. mone (turkey ministry of national education). (2018). science course curriculum (primary and secondary 3, 4, 5, 6, 7, and 8 grades). http://mufredat.meb.gov.tr/programdetay.aspx?pid=325 nguyen, d., pietsch, m., & gümüş, s. (2021). collective teacher innovativeness in 48 countries: effects of teacher autonomy, collaborative culture, and professional learning. teaching and teacher education, 106, 103463 https://doi.org/10.1016/j.tate.2021.103463 öğüt, a., aygen, s., & demirsel, m. t. (2007). personel güçlendirme inovasyonu hızlandırır mı? antalya ili beş yıldızlı konaklama işletmelerine yönelik görgül bir araştırma [will staff empowerment accelerate innovation? an empirical research on five-star accommodation establishments in antalya]. selçuk üniversitesi karaman i̇ktisadi ve i̇dari bilimler fakültesi dergisi, special issue, 163172. özbulak, b. e., aypay, a., & aypay, a. (2011). ortaöğretim öğrencilerinin problem çözme ve atılganlık becerilerinin bazı değişkenlerle ilişkisi [high school students’ problem solving and https://doi.org/10.29129/inujgse.395132 https://doi.org/10.14689/issn.2148-2624.1.8c.1s.15m https://doi.org/10.7827/turkishstudies.13746 https://doi.org/10.2399/yod.18.001 http://dx.doi.org/10.15390/eb.2015.2613 https://doi.org/10.18844/cjes.v15i3.4642 http://mufredat.meb.gov.tr/programdetay.aspx?pid=325 https://doi.org/10.1016/j.tate.2021.103463 journal of science learning article doi: 10.17509/jsl.v5i1.33533 54 j.sci.learn.2022.5(1).42-54 assertiveness skills and their relationships with some variables]. electronic journal of social sciences, 10(36), 77-93. öztürk, e., önder, a. n., & güven-yıldırım, e. (2019). fen bilgisi öğretmen adaylarının girişimcilik özelliklerinin ve eleştirel düşünme eğilimlerinin bazı değişkenler açısından incelenmesi [investigation of prospective science teachers' entrepreneurship characteristics and critical thinking dispositions in terms of some variables]. journal of individual differences in education, 1(2), 89-107. pallant, j. (2020). spss survival manual: a step-by-step guide to data analysis using ibm spss. routledge. parlar, h., polatcan, m., & cansoy, r. (2019). the relationship between social capital and innovativeness climate in schools: the intermediary role of professional learning communities. international journal of educational management. https://doi.org/10.1108/ijem-10-2018-0322 polat, m. (2017). sınıf öğretmenlerinin eleştirel düşünme eğilimleri ile yaratıcılık düzeylerinin bazı değişkenlere göre incelenmesi [examining primary school teachers'critical thinking disposition and creativity levels in terms of difference variable]. (unpublished master’s thesis). university of adıyaman. rıza, e. t. (1999). yaratıcılığı geliştirme teknikleri [techniques for enhancing creativity]. anadolu press. rogers, m. e. (1995). diffusion of innovations (fifth edition). free press. sarı, i̇., & kartal, f. (2018). sosyal bilgiler öğretmen adaylarının teknoloji ile ilgili tutumlarının bireysel yenilikçilik düzeyleri ve bazı değişkenler açısından incelenmesi [investigation of social studies teacher candidates' attitudes towards using technology in terms of individual innovativeness levels and some variables]. kırşehir journal of education, 19(2), 1673-1689. https://doi.org/10.29299/kefad.2018.19.02.017 seechaliao, t. (2017). instructional strategies to support creativity and innovation in education. journal of education and learning, 6(4), 201208 http://doi.org/10.5539/jel.v6n4p201 siew, n. m., & ambo, n. (2018). development and evaluation of an integrated project-based and stem teaching and learning module on enhancing scientific creativity among fifth graders. journal of baltic science education, 17(6), 1017 http://doi.org/10.33225/jbse/18.17.1017 soyuçok, h. (2018). tübi̇tak 4006 bilim fuarları kapsamında hazırlanan fen projeleri hakkında çalışmalara katılan farklı kesimlerin görüşleri [views of different participants in studies about science projects prepared in scope of tubitak 4006 scientific fairs]. (unpublished master’s thesis). university of i̇brahim çeçen. tabachnick, b. g., & fidell, l. s. (2013). using multivariate statistics (6th). pearson education. tunç, y., & taşgın, a. (2018, november). examination of middle school students' perceptions of problem-solving skills [abstract presentation]. iğdır international conference on multidisciplinary study, iğdır, turkey. ulusoy-yılmaz, d., & yıldız, y. (2019). müzik öğretmen adaylarının yaratıcı düşünme becerilerinin çeşitli değişkenler açısından incelenmesi [investigation of creative thinking skills of music teacher candidates in terms of various variables]. cumhuriyet international journal of education, 8(2), 516-530. http://dx.doi.org/10.30703/cije.522714 ünsal, k. (2019). ortaokul ve lise okul yöneticilerinin kodlama eğitimine yönelik görüşlerinin incelenmesi (bağcılar ilçesi örneği) [investigation of secondary and high school pirincipals' opinions on coding training (sample of bagcilar district)]. (unpublished master’s thesis). university of sabahattin zaim. yenice, n., & alpak-tunç, g. (2019). öğretmen adaylarının yaşam boyu öğrenme eğilimleri ile bireysel yenilikçilik düzeylerinin incelenmesi [an investigation of pre-service teachers’ lifelong learning tendencies and their individual innovativeness levels]. kastamonu education journal, 27(2), 753-765. https://doi.org/10.24106/kefdergi.2716 yıldırım, b. (2018). stem uygulamalarına yönelik öğretmen görüşlerinin incelenmesi [research on teacher opinions on stem practices]. journal of education theory and practical research, 4(1), 4253. yılmaz-öztürk, z. (2015). i̇lköğretim okulu öğretmenlerinin bireysel yenilikçilik düzeyleri ve bu düzeylere etki eden etmenlerin incelenmesi [an analysis of primary school teachers' level of individual innovativeness and the factors affecting it]. (unpublished doctoral dissertation). university of gaziantep. yoon, s. (2018). a study on the effect of epl on programing, computing thinking and problem solving ability of programing education. the journal of the convergence on culture technology, 4(4), 287-294. webster, c. a., mindrila, d., moore, c., stewart, g., orendorff, k., & taunton, s. (2020). exploring the role of physical education teachers’ domain-specific innovativeness, educational background, and perceived school support in cspap adoption. journal of teaching in physical education, 39(1), 36-47. weis, s., scharf, c., & gryl, i. (2017). new and even newer fostering innovativeness in primary education. international e-journal of advances in education, 3(7), 209-219. https://doi.org/10.1108/ijem-10-2018-0322 https://doi.org/10.29299/kefad.2018.19.02.017 http://doi.org/10.5539/jel.v6n4p201 http://doi.org/10.33225/jbse/18.17.1017 http://dx.doi.org/10.30703/cije.522714 https://doi.org/10.24106/kefdergi.2716 a © 2022 indonesian society for science educator 115 j.sci.learn.2022.5(1).115-126 received: 24 june 2021 revised: 3 october 2021 published: 1 march 2022 the effect of using web-based interactive learning media for vocational high school students to understanding of looping: qualitative approach dwi fitria al husaeni1*, enrico nabil qois budisantoso1, mushfani ainul urwah1, nissa nur azizah1, putri zukhruf dinata1, shandini apriliany1, herbert siregar1 1department of computer science education, faculty of mathematics and science education, universitas pendidikan indonesia, bandung, indonesia *corresponding author dwifitriaalhusaeni@upi.edu abstract this study aims to evaluate the effect of the use and development of interactive and visual learning media in learning looping by utilizing a web-based digital platform called loopers. the research design used a qualitative descriptive with case study approach in-depth interview technique. the subjects of this research are five students of the state vocational school pekerjaan umum, bandung, indonesia. we use three phases in conducting research through 1) problem and needs analysis, 2) web design, and 3) implementation and testing. loopers were developed using the waterfall method. loopers can demonstrate the implementation of a productive learning process and measure the effectiveness of interactive and visualized learning media. the findings show that loopers can effectively facilitate experimental demonstrations and written modules. respondents think this learning media is effective and can increase their interest, motivation, and interest in learning looping material. the next research is to enrich the loopers feature and conduct trials with larger respondents. keywords looping, basic programming, experimental demonstration, vocational school, website 1. introduction looping is a method of repeating a procedure as many times as predetermined stopping conditions allow (shouthiri & thushika, 2018). since many software applications repeat their work until the desired condition is reached, looping is an essential part of programming (grover, jackiw, & lundh, 2019). with iterations, programmers are not required to write as much program code as they would like (brown & wilson, 2018). looping is one of the material scopes that must be studied in basic programming subjects at the basic competency (known as kd) 3.7 and 4.7 levels of vocational school education, namely applying a loop control structure in programming languages and creating program code for a loop control structure. the material taught is looping with initial, final, user-inputted conditions, continue, and a break statement. however, when it comes to learning the loop concept, vocational school students find the material difficult. this is evidenced by the results of a survey of 23 vocational students conducted by the author using google forms, in which 45.5% responded that there were subjects that were difficult to understand, namely basic programming subjects on looping material. this is consistent with esteves, fonseca, morgado, & martins (2011), who state that a programming language has a broad and complex syntax (attallah, 2017). learning programming languages will undoubtedly be difficult for inexperienced students. programming languages are one of the subjects that require a high level of qualification (çoklar & akçay, 2018). as a result of the high qualifications possessed by this subject, many students are required to repeat and even drop out of school. respondents' reasons for difficult-to-understand material include (1) a lack of creative learning media, (2) unclear explanations of material, (3) a lack of material sources/references, (4) a lack of examples of application provided, and (5) inadequate learning facilities. it is critical to assist students in understanding loops. one method for learning loops more easily is visualizing and creating a learning media. presentation of material is more visual; for example, videos can help students learn more effectively (afriani & agustin, 2019; cetin, 2020). vocational schools continue to use learning media in mailto:dwifitriaalhusaeni@upi.edu journal of science learning article doi: 10.17509/jsl.v5i1.35534 116 j.sci.learn.2022.5(1).115-126 printed books, digital e-books, tutorial articles on websites, and video tutorials for learning programming languages. many studies on looping material have been conducted, including research related to the expression of the term looping in semantic programming (goncharov, ospichev, ponomaryov, & sviridenko, 2019), research conducted by shouthiri and thushika (2018) regarding comparative analysis of the looping structure by comparing the loop structure of the while loop and do-while loop in c++ language (shouthiri & thushika, 2018), making looping a formal concept (manik, 2020), research on combining active learning with inductive logic programming to close the loop in machine learning (bryant, muggleton, page, & sternberg, 1999) and teaching various kinds of loops for beginners with program examples (tam, 1992). however, no research has been conducted on web-based learning media in terms of looping material that is explicitly presented and in detail with a higher and more dominant level of complexity in presenting the material and the interaction between the system and students through the incorporation of descriptive, visual, audio-video, and other media. quizzes on media and other practice questions. as a result, this study was carried out to aid in the efficient facilitation of the learning process. the preliminary research findings to ascertain vocational school students' opinions on learning media development revealed that students agreed with digital learning assistance in learning. increasing student learning time with digital learning, in particular, increases learning achievement relatively (lin & chen, 2017). the digital platform's innovation is to combine various learning media on one platform, making it easier for students to access loop material, such as learning videos, learning modules, and quizzes/games that are free, open-source, and user friendly. this study aims to test the effect of using and developing more interactive and visual learning media on understanding looping material through experimental demonstrations using a web-based digital platform for vocational students. analysis of student learning outcomes and understanding and the effectiveness of learning media is carried out by interviewing students and reviewing the results of the quizzes provided on the created website. this research is expected to help students understand basic programming concepts, particularly looping, so that students can meet kd 3.7 and 4.7. 2. method 2.1. research subject respondents in this study included five students from the vocational school computer and network engineering expertise program, two in grade 10 and three in grade 11. we assigned r1, r2, r3, r4, and r5 to five respondents. respondents comprised 40% male students and 60% female students, with a median age of 16 years and a maximum age of 17 years. 2.2. design analysis the qualitative descriptive method with the case study approach was used in this study. in addition, the researchers used an in-depth interview technique. the waterfall method is used in the creation of learning media. the waterfall is carried out systematically, beginning with the system requirements design stage and progressing to the analysis stage, web design, web testing, and maintenance. figure 1 depicts the research's three phases: problem and needs analysis, web design, and implementation and testing. 2.3. analysis of the problem and needs in the problem and needs analysis step, we identified the inputs and outputs and the media used to develop loopers. an open questionnaire distributed via google forms to 23 vocational school students with expertise in computer and informatics engineering in bandung, figure 1 research stages table 1 question for analysis of the problem and needs number question 1 in your opinion, are there subjects that are difficult to understand? 2 what subjects do you find difficult to understand? 3 from these subjects, what material do you think is the most difficult to understand (can write one or more chapters of material)? 4 in your opinion, what causes the material to be challenging to understand? (explain whether there is a lack of sources or references, whether the teacher does not explain in detail, the learning media lacks, etc.). 5 choose the learning media that you want to make it easier to understand the material (can be more than one choice) journal of science learning article doi: 10.17509/jsl.v5i1.35534 117 j.sci.learn.2022.5(1).115-126 indonesia, was used to analyze problems and needs. we asked respondents five questions in table 1 about difficultto-understand skill subjects and preferred learning media to help students understand the material. figure 2 shows that 45.5% of the 23 students responded to difficult-to-understand subjects, namely basic programming subjects on looping material. the questionnaire answers are used to identify subjects, sub-materials, and media that will be utilized to create loopers web-based learning media. 2.4. web design the analysis results are implemented using a programming language during the design stage. several diagram design media are used in the design process (such as data flow diagram (dfd), flowchart, entity relationship diagram (erd), and sitemap). in addition, we design learning media that will be used on the website loopers creation system based on the problem results and need analysis. learning videos, portable document format (pdf) materials, quizzes, enrichment questions, and a top score system are among the media used. figure 2 subjects that are difficult for students to understand table 2 question of interview process number question 1 have you studied repetition in school? if so, did you have difficulty in learning it? please explain the difficulties experienced if yes. if not, do you have an interest in learning basic programming, especially looping material? 2 previously, have you ever tried using learning media like loopers? if yes, state the website's name and how you feel or the difference between the two websites. if not, what do you think about making learning media like loopers? 3 what do you think about the appearance of web loopers (first impressions)? 4 what do you think when operating this website? are you having trouble accessing some of the features on this website? 5 how far have you learned the material? has everything been done? 6 is the material displayed on the web in accordance with the learning material described in class? 7 has the looper website helped you understand basic programming subjects, especially looping material? state the reason. 8 are there any difficulties in understanding the looping material on this web? if so, what materials are still difficult to understand in this web learning media? 9 there are teaching materials in modules, videos, and quizzes on the web. do you think the teaching materials are sufficient to understand your knowledge of repetition? 10 does visualizing more (video) the material makes you more interested in learning a material? explain why. 11 how is the learning video shown? is it interesting to watch? if so, why? if not, why? 12 have you filled out the quiz? if so, what were the results? and give your opinion on the quiz as a whole. 13 what do you think about providing learning media such as this looper website? 14 what do you think about web loopers? who is interested in learning or not? if so, why? if not, why? 15 can the existence of learning media such as this looper increase your motivation in learning? state the reason. 16 in your opinion, is this web-based learning media effective, efficient, and useful for people who have difficulty understanding repetition? explain. 17 mention what you like on this website. 18 can friends tell us about the advantages and disadvantages after trying the loopers learning web? 19 would you recommend this website to your friends? 20 state your hopes for this website in the future. journal of science learning article doi: 10.17509/jsl.v5i1.35534 118 j.sci.learn.2022.5(1).115-126 the use of video in learning is more appealing to students. learning videos help students understand looping material through audio-visual means, making the material delivered more interactive and visual. meanwhile, the material module in pdf format is presented for students who prefer to read. furthermore, the pdf module includes a function that allows students to review material on the loopers web without playing videos. before taking the school exam, students can easily review the looping material. the quiz function is a collection of practice questions that students can complete after studying the offered information. this quiz allows students to assess themselves after completing the learning process. in addition, the loopers system now includes a scoring system and a feature for determining the highest score to increase students' motivation to learn. furthermore, the scoring tool enables students to see their abilities and progress in studying. 2.5. implementation and testing we began designing the system using a programming language throughout the implementation stage. php: hypertext preprocessor (php) and mysql are utilized to create loopers learning media accessible via a website. php is used to create client-side web pages. mysql is a database management system that is utilized on the serverside. the loopers website will then be tested. nine students from the indonesia university of education participated in the testing stage. six women and three men served as examiners. the test is conducted in three stages: 1) the examiner runs the loopers web, 2) the examiner tries all of the loopers web's features, and 3) the examiner provides criticism, suggestions, and input. figure 3 shows the stages of testing. 2.6. teaching method the teaching method employed is experimental demonstration development media via a web-based digital platform. the research was carried out in stages, the first of which was the distribution of learning media to students. following that, interviews were conducted in which students were asked 20 open-ended questions shown in table 2. before beginning the learning and interview process, information about the importance of basic programming subjects is gathered to determine the students' starting abilities. 3. results and discussion the web loopers are divided into six sections: home, about, class, login, material, and contact. the feature has a hierarchy that can be linked to other features. the home feature, which can access logins, classes, about, materials, and contracts, becomes the center of the hierarchy. class features provide access to the material. login feature for accessing home, sign up, and class. logins, classes, materials, and quizzes can all be accessed via material features. loopers learning media is an open-source learning media because it does not require students to register first. on the loopers web, students can freely access all features except the quiz feature. it is intended that students find it simple to carry out learning activities. students who access the loopers site will be provided a loopers learning module that includes two forms of media, video and pdf. in addition, students can assess their learning by taking quizzes available on the loopers website. the system will respond to students by displaying the quiz scores they have completed. in addition, the loopers web system will provide students with answer keys and corrections for filling out quizzes. the answer key function allows students to discover what mistakes they made when filling out the quiz, allowing them to learn again and impacting student learning outcomes (sa’adah, nurkamto, & suparno, 2018). 3.1. features of web loopers learning media class features figure 4 shows a class page with a list of looping materials for students to follow. students can choose from 14 classes, which include the introduction of basic looping, increment (++) and decrement (--) operators, mathematical expressions in looping, counters, for statements, nested for statements, while statements, nested while statements, do-while statements, infinity statements, break statements, continue statements, and go-to statements. figure 3 stages of testing web loopers figure 4 class page journal of science learning article doi: 10.17509/jsl.v5i1.35534 119 j.sci.learn.2022.5(1).115-126 tutorial video web loopers has several interactive and visual features for presenting content, one of which is the use of video, as shown in figure 5. because many students propose presenting material through video, learning videos are used to present material. moreover, video-based learning can increase students' interest and focus on learning (nadeak & naibaho, 2020). figure 6 describes an example of the video content. loopers web and youtube are used to present learning videos. learning videos are created based on material categories, resulting in 14 learning videos. students can learn looping material by using the video feature on the loopers web. audio explanations of the material accompany the learning videos. using audio-video in learning can encourage students' active participation in their hearing and vision, causing them to become more focused on their studies (thézé et al., 2020). pdf module web loopers includes a material module in pdf format. students who enjoy reading will benefit from written material modules (ling, 2018). some students may not have enough time to watch learning videos, so reading this module is a good alternative. furthermore, the pdf module makes it easier for students to see a summary of the material as study material when facing exams at school. figure 7 represents an example of a pdf module for loop material. the pdf format is designed to look like a study summary, with short, concise, and clear content. the number of pdf pages per material ranges from 3 to 8 pages. the appearance of the pdf is also enhanced by the use of different colors to help students understand the material. quiz the quiz features available on the loopers web are displayed in figure 8. the quiz feature trains students and improves their understanding of each topic. there are 70 figure 5 presentation on the web loopers figure 6 contents of learning videos figure 7 material pdf module journal of science learning article doi: 10.17509/jsl.v5i1.35534 120 j.sci.learn.2022.5(1).115-126 questions in each material, with two easy questions, two difficult questions, and one difficult question. the quiz is also intended to familiarize students with various types of questions. students who want to practice additional sample questions can download the enrichment questions found on the class and material pages. quizzes can help students learn basic terms and concepts in various subjects. furthermore, providing students with random quiz questions (demonstrating how concepts become more practical) can help them understand concepts and apply them to new contexts on subsequent tests (nguyen & mcdaniel, 2015). enrichment instruments the enrichment instrument is a feature available on the loopers web that allows students to explore additional learning materials related to their learning tasks. in order to achieve the best level of development and mastery of the material. every student has direct access to and can download this feature. enrichment services are provided to students who master the material faster by presenting them with more difficult learning challenges to help them achieve their maximum learning capacity (brigandi, weiner, siegle, gubbins, & little, 2018). individual answers have been provided in each instrument to assist students in being more responsive and understanding each question criteria. figure 9 represents the enrichment questions for all looping materials on the loopers website. features of score and top scoreboard figure 10 represents the top score page on the loopers website. the top score represents the acquisition of the highest score to the lowest score of each student who completed the quiz. because their scores and names will be displayed on the top scoreboard, the top score is intended to foster students' competitive spirit and encourage them figure 8 quiz on loopers web figure 9 enrichment instruments journal of science learning article doi: 10.17509/jsl.v5i1.35534 121 j.sci.learn.2022.5(1).115-126 to compete and do their best work. the purpose of making a scoreboard is the same as the specific purpose of assessing learning outcomes which includes four things, that is a) knowing the success of the process education and teaching in schools, b) diagnosing learning difficulties, and c) motivating students' learning by knowing and understanding themselves and stimulating them to make improvement efforts (dobson, 2008). the view score page is visualized in figure 11. the quiz scores completed by students are displayed on the score view page. the data is displayed in the form of quiz scores, name, school origin, major, and the last date of the exam and is sorted from highest to lowest score. students can repeat quizzes that have been completed previously. students who repeat the quizzes can correct the mistakes to get a maximum score. in addition, students can see the answers to each quiz question on the score page. students were given answer keys and error information when filling out quizzes to re-learn material that they felt was difficult when working on quiz questions, which improved student learning outcomes (sa’adah et al., 2018). features with an answer key and error correction figure 12 shows a feature in web loopers that allows students to view the quiz answer keys. we also added a feature that detects errors in web loopers quizzes answered by students. this feature allows web loopers to interact with students more effectively. the system responds to students by displaying the quiz scores they have completed. the system provides answer keys and corrections for student-completed quizzes. students can find out what errors they made when filling out the quiz. following that, students can re-study the material with a less-than-ideal quiz score. this affects student learning outcomes (sa’adah et al., 2018). 3.2 test results of the device through test table 3 displays the results of the students' basic programming subjects. the majority of students have initial basic programming scores of more than 80. this demonstrates that students' initial mastery of basic programming material is suitable (anggraeni et al., 2020). table 4 compares students' pre-test and post-test scores after receiving the loopers learning media. the students' average score increased after using the loopers web, reaching 90 from 77.20 at the start. the standard deviation of the student's post-test is 3.73, which is less than the standard deviation of the student's pre-test of 7.40, indicating that the difference between the sample values and the average is shrinking. the distribution of each figure 10 top scoreboard figure 11 score history page journal of science learning article doi: 10.17509/jsl.v5i1.35534 122 j.sci.learn.2022.5(1).115-126 student's knowledge of the looping material is improving and becoming more evenly distributed (the difference between) (nissen, talbot, thompson, & van dusen, 2018). this demonstrates that the loopers web has proven an effective learning media. 3.3. descriptive qualitative data analysis for learning outcomes this study analyzed the data by processing the information collected through the interview method, which is part of the descriptive qualitative research method. in addition, the results of the interviews were described in terms of predetermined problems. based on yusrizal, safiah, & nurhaidah, (2017) research, we used nine points to determine the interview results by adjusting the questions asked to the respondents. the study mentioned seven factors to consider when determining the feasibility of learning media, some of which were the suitability of using learning media, the accuracy of learning media, and the ease of learning media. the following are the nine points based on the findings of an analysis of vocational school students' use of the loopers web learning media to learn to loop. 1. the applicability of loopers web learning media for learning to loop. based on the results of interviews with the five respondents about the suitability of using loopers web learning media in learning to loop, r1 responded, "the learning media is very suitable, especially for students who are lazy to read. because you can see the provided learning videos and the features are simple to use.". r2 responded, "the learning media is very suitable to use because the features that are easy to understand make me not bored using it.". r3 responded, "it is very suitable for web loopers for students who want to learn looping, because the content of the explanation of the material is not complicated and easy to learn.". r4 responded, "it seems appropriate to me. this looping material necessitates strong logic and numerous application examples to ensure that the concept is understood ". r5 replied, "web loopers are very suitable to be used for learning looping material, it's just that i want additional material not only looping but all about basic programming." based on the results of interviews that have been conducted, it was found that loopers, which is a webbased learning media with an emphasis on the use of figure 12 error info and correct answer correction table 3 student basic programming grades students score r1 80 r2 64 r3 80 r4 80 r5 81 table 4 comparison of student’s pre-test and post-test scores students pre-test post-test r1 80.00 95.00 r2 64.00 90.00 r3 80.00 91.67 r4 80.00 88.33 r5 81.00 85.00 average 77.20 90.00 standard deviation 7.40 3.73 journal of science learning article doi: 10.17509/jsl.v5i1.35534 123 j.sci.learn.2022.5(1).115-126 various learning media, videos, texts, and practice questions, are suitable for use in looping learning. this is because learning using videos and other interactive media can increase students' understanding and interest (anwar, choirudin, ningsih, dewi, & maseleno, 2019). 2. the suitability of the looping material on the loopers learning media for school-based looping learning. based on the findings of five interviews with respondents about the suitability of the looping material in loopers learning media for looping learning at school, r1 responded, "the material presented at school and in loopers is not much different. it's just that the difference in loopers is much easier to understand than the explanation in school.” r2 responded, "the material presented at school and in loopers is not much different, and if at school the teacher conveys at length, but students do not understand, then in loopers the delivery is short but students can easily understand." r3 responded, "learning media on web loopers is appropriate as material in general, made in a simple and easy to understand manner." r4 responded, "web loopers are ideal for looping school lessons. it is just that the material from web loopers is easier to understand than explanations at school, such as reading from printed books or listening to teacher explanations." r5 responded, "the learning media is very easy to use for learning; you don't need to read a lot because the suitability of the material taught at school has been very clearly summarized in loopers." loopers learning media is assessed according to what students learn at school. students think that loopers are very easy to use in learning because the material provided by loopers is in accordance with what is taught at school. in addition, the loopers material is made more concise to be easy to learn and understand. the material presented is also more structured with clear instructions. this agrees with the research conducted by faust and paulson (1998), where instructors can guide students toward a deeper understanding of the material presented. 3. loopers learning media tools in the understanding of looping material. based on the results of interviews with the five respondents, r1 responded, "it is very helpful, when we run the quiz on this loopers there will be an explanation and can be tried repeatedly." r2 answered, "very helpful and easier to understand". r3 responded, "very helpful, because i am too lazy to read, so seeing the learning video directly helps me understand the material. when there is a test or quiz, it is not difficult to find a collection of looping material. " r4 responded, "very helpful, because the looping material is very complete; there are learning modules, videos, and quiz questions that can be studied repeatedly." r5 responded, "it is helpful, easy to understand, and provides students with easy access to the internet." based on the interviews, many students who used loopers were helped and more easily understood the lessons, including videos, modules learning, and quizzes. they are helped to understand looping because the learning videos can be played repeatedly. besides that, the quizzes can help them to increase their skills. this agrees with the research by saeed, yang, & sinnappan (2009) on the effect of using multiple and interesting learning media on web loopers. 4. the efficiency of video as a learning media based on the results of interviews with the five respondents about the effectiveness of using video as a learning medium, r1 responded, "it is very effective, especially for online school conditions during the current pandemic." however, it is preferable if a teacher directly explains the material rather than simply providing learning videos." r2 responded, "effective because students can easily try it themselves right away." r3 responded, "very effective, because i am lazy to read, i can see videos that are easy to understand and can be put into practice right away." r4 responded, "very effective, because the explanation is quite clear and easy to understand." r5 responded, "effective, good because there is a video explanation. i'd rather have an explanation before doing/practicing coding.” besides that, respondents stated that learning through video could be more enjoyable. this demonstrates the effectiveness of video as a learning medium (yusuf, zuhrawardi, & wardani, 2020). in addition, according to research by arthur, sekartaji, arris maulana, & dewi (2019) video media can increase student learning outcomes. 5. availability of quizzes in loopers learning media. based on the results of the loopers quiz interview, r1 responded, "good, the quiz is linked to the discussion module, and the questions are not strange, in my opinion." while r2 responded, "it is great that there is a quiz; you can try it right away; you do not have to wait for assignments anymore, so you understand right away." r3 responded, "it is excellent that the quiz can measure the ability to understand looping material." r4 responded, "surprisingly, quizzes are available in loopers. you can learn from the quiz if there is an exam." finally, r5 responded, "it is good to have a quiz, but there are not many quiz questions because the knowledge is used for relearning." in addition, the availability of score and top score features on the loopers web that shows the result of the quiz can be used to determine the limits of students' knowledge about the looping material and can also be used as material for student evaluation to be even better. as mentioned by r1, "with the availability of the score and journal of science learning article doi: 10.17509/jsl.v5i1.35534 124 j.sci.learn.2022.5(1).115-126 top score features on the loopers web, we can determine the limits of our abilities and evaluate them so that they can perform better in future quizzes or questions." according to the following respondents, the top score feature can increase students' motivation and learning challenges and serve as a reward for themselves. for example, r3 stated, "it is interesting that i want to get a score of 100 continuously." while r4 stated, "it is more of a challenge for self-motivation if i can get a high score by studying," r5 also stated, "the score and top score feature is afraid to get a small score, but that way there is a reward for myself if i can and do not be lazy to learn basic programming." by giving quizzes or practice questions on web loopers, students are helped to understand the material being studied. students also become happy when they find challenges in the quizzes or practice questions provided. it was also approved by the journal making a game out of it: using web-based competitive quizzes for quantitative analysis content review, which stated that quizzes could help students learn and increase students' enjoyment of learning (grinias, 2017). 6. interest in learning looping material after making loopers learning media. based on the results of interviews related to their interest in learning after using loopers to study looping material, r1 answered, "i am happy to be interested in learning. maybe we can add other material". while r2 replied, "i am very interested, very easy to understand, always free". r3 replied, "very interested, because the explanation is short but easy to understand". r4 replied, "interested, you don't need access to a lot of quotas, you can download learning modules". finally, r5 replied, "web loopers are cool, so they are interested in learning looping by presenting material that doesn't bore us". based on the interviews from the five respondents after using the loopers learning media, they were very interested in learning the looping material. because the loopers learning media is a freeware that is easily accessible by anyone. in addition, interactive learning media such as loopers can make it easier for students to understand the material being taught with help, questions, videos, and learning modules. this agrees with ramkissoon, belle, & bhurosy (2020) that students prefer interactive online learning because it is adapted to their needs and interest in technology. 7. web-based learning media increases learning motivation in basic programming subjects based on the results of interviews conducted to increase respondents' learning motivation in basic programming subjects through the use of web-based learning media (vania, setiawan, & wijaya, 2018), r1 replied, "it is quite motivated, but it is normal". in contrast to r2, he responded, "it increases when i am alone, it is just as easy to understand, and it is just as nice". r3 responded, "yes, i am highly motivated, particularly during the pandemic. r4 answered, "motivated to learn basic programming, especially looping". r5 answered, "yes, obviously motivated, online learning at loopers can be done anywhere." loopers motivate students to learn repetition material. loopers can be done anywhere, especially during the pandemic, many students have to study independently at home. the learning media can increase students' motivation to learn a lesson material that according to research by winarto, hardyanto, & sugianto (2019). 8. loopers learning media include videos, content modules in pdfs, and quizzes. based on the results of interviews regarding the availability of videos, material modules in the form of pdf, and quizzes, r1 replied, "that is enough", r2 responded, "enough. r3 responded, "it is cool. there are free videos, learning modules, and quizzes, no need to pay like other learning websites." r4 answered, "good and enough, but you can add more interesting learning video animation. r5 answered, "learning videos can be played repeatedly. i can read the pdfs provided on the web at any time. so, when there is a test, the material provided on the web can be used for re-study." based on the interviews, the availability of videos, material modules in pdf form, and quizzes was measured. many students believe that loopers is quite clear in presenting material through videos, pdfs, and quizzes. it helps students to understand and evaluate themselves. in addition, the presentation of interesting material with delivery style through quizzes supports students to understand the material well (higashinaka, dohsaka, amano, & isozaki, 2007). 9. web-based learning media's effectiveness and efficiency based on the results of interviews related to the effectiveness and efficiency of using web-based learning media, r1 answered, "the website is good for the motivation of today's children who have difficulty learning, this is one of the most effective and efficient learning media, highly recommended because it can be accessed anywhere and anytime". besides that, r2 answered "effective and efficient because the learning media is easily accessible anywhere". r3 answered, "very effective and efficient, through web-based learning, you can learn independently without time restrictions like in school". r4 replied, "effective and efficient, especially now that the pandemic is all online, so you have to get used to online learning and access the learning web". r5 responded, "it is incredibly effective and efficient if you miss material at school, you can visit learning webs like loopers learn anywhere and at any time." based on the interviews with the five respondents, they all agreed that web-based learning media was effective and efficient. loopers web-based learning media can be helpful for students who have learning development journal of science learning article doi: 10.17509/jsl.v5i1.35534 125 j.sci.learn.2022.5(1).115-126 problems. it also provides many conveniences for users, and it is recommended that they learn from loopers. this is in accordance with the findings of daniela, kalniņa, & strods (2017) and astuti, wihardi, & rochintaniawati (2020), who found that web-based learning media were feasible and effective. the researcher concluded several characteristics of each respondent based on the five respondents' interviews. r1 is a student who prefers to learn through video media because he dislikes reading; the availability of videos can help r1 understand the concept of looping in basic programming subjects. furthermore, r1 is a student who enjoys challenges, is pleased with the quiz, and is eager to re-learn the material when he receives unsatisfactory quiz results. finally, r1 is very interested in learning looping through the loopers website. r2 is a student who enjoys practical concepts. r2 believes that the features available on the loopers web are simple to understand. web loopers can be accessed at any time and from any location. when using web loopers, r2 is interested and motivated to learn about looping. r3 is a person who does not enjoy reading and prefers to learn through online videos. loopers assists r3 in comprehending the looping material. r3 can use the practice questions to help him understand the looping material. r3 gains independence in learning looping with web loopers. r4 is a student who requires an application example to understand a learning material concept. many examples from everyday life can be found in the loopers web videos. as a result, r4 is very interested in learning how to use web loopers, making looping material easier to understand. r5 is a student who understands the material after reading a brief description. loopers offers brief material in pdf format. when faced with an exam, r5 can use learning media in a pdf module to review the material. as a result, r5 enjoys using web loopers and is motivated to learn looping through them. according to the findings of the interviews mentioned above, web loopers can assist some students in their learning process. web loopers are popular among students and in high demand. loopers can be used to prepare for school exams due to the compatibility of the looping material provided on the loopers web with what students learn at school. the comprehensiveness of learning media used in web loopers, such as creating experimental demonstration videos, descriptive modules in the form of pdfs, and practice questions (quiz and enrichment instruments), can boost students' learning motivation. using more interactive, visual, and diverse learning media can boost students' motivation and learning performance (saputri, rukayah, & indriayu, 2018). many students offered ideas for developing loopers learning media. as a result, we have been inspired to improve some of the features in web loopers. in the future, we hope to add an animation feature to the material delivery process and some source code looping directly in the loopers web. the addition of animation features was made in response to the requests of several students who participated in this study. the animation feature is thought to be more apparent in describing the material, allowing students to understand better the material presented and master learning outcomes more efficiently and effectively (ismail, othman, amiruddin, & ariffin, 2017). in addition, the inclusion of source code in loopers web allows students to directly practice looping coding by copying and pasting the provided code. the loopers learning media does not discuss the entire material on basic programming subjects. so that there is a need for further research and feature updates on loopers based on student suggestions contained in the results and discussion. loopers are expected to be a learning media that all groups can use. loopers are not only used by students because loopers make looping material easier to understand by using web media to present more interactive and visual material. conclusion based on the study, the making of learning media for looping materials on basic programming subjects using a web-based digital platform called loopers is as follows: (1) loopers is declared effective and suitable for use in productive learning processes involving repetition materials; (2) students' responses to loopers showed a strong desire to use loopers because it increased their interest, motivation, and interest in the loopers material. loopers is learning media that discuss looping material clearly and complexly. therefore, loopers are expected to be a learning media used by all groups, not only students, because loopers makes looping material easier to understand by using web media to present more interactive and visual material. acknowledgment we would like to thank the five respondents from smk negeri pu bandung who took in this research. references afriani, t., & agustin, r. r. (2019). the effect of guided inquiry laboratory activity with video embedded on students’ understanding and motivation in learning light and optics. journal of science learning, 2(4), 79-84. anggraeni, s., nadhira, s. f., azizah, n. n., al husaeni, d. f., ragadhita, r., hofifah, s. n., & nandiyanto, a. b. d. (2020). teaching the effect of variations in types of flour on mechanical characteristics of chicken skin crackers through experimental demonstrations to senior high school students. journal of engineering education transformations, 34(2020), 102-107. anwar, m. s., choirudin, c., ningsih, e. f., dewi, t., & maseleno, a. (2019). developing an interactive mathematics multimedia learning based on ispring presenter in increasing students’ interest in learning mathematics. al-jabar: jurnal pendidikan matematika, 10(1), 135-150. journal of science learning article doi: 10.17509/jsl.v5i1.35534 126 j.sci.learn.2022.5(1).115-126 arthur, r., sekartaji, g. t., arris maulana, l., & dewi, k. (2019). pengembangan media video presentasi pada mata kuliah hidrologi di universitas negeri jakarta [development of video presentation media for hydrology courses at the state university of jakarta]. jurnal teknologi pendidikan, 7(02), 170-183. astuti, l., wihardi, y., & rochintaniawati, d. (2020). the development of web-based learning using interactive media for science learning on levers in human body topic. journal of science learning, 3(2), 8998. attallah, b. (2017). visualizing computer programming in a computerbased simulated environment. international journal of advanced computer science and applications (ijacsa), 8(8), 369-378. brigandi, c. b., weiner, j. m., siegle, d., gubbins, e. j., & little, c. a. (2018). environmental perceptions of gifted secondary school students engaged in an evidence-based enrichment practice. gifted child quarterly, 62(3), 289-305. brown, n. c. c., & wilson, g. (2018). ten quick tips for teaching programming. plos computational biology, 14(4), e1006023. bryant, c. h., muggleton, s. h., page, c. d., & sternberg, m. j. (1999). combining active learning with inductive logic programming to close the loop in machine learning. in proc. aisb symposium on ai and scientific creativity (pp. 221-234). cetin, i. (2020). teaching loops concept through visualization construction. informatics in education-an international journal, 19(4), 589-609. çoklar, a. n., & akçay, a. (2018). evaluating programming self-efficacy in the context of inquiry skills and problem-solving skills: a perspective from tteacher eeducation. world journal on educational technology: current issues, 10(3), 153-164. daniela, l., kalniņa, d., & strods, r. (2017). an overview on effectiveness of technology enhanced learning (tel). international journal of knowledge society research (ijksr), 8(1), 79-91. dobson, j. l. (2008). the use of formative online quizzes to enhance class preparation and scores on summative exams. advances in physiology education, 32(4), 297-302. esteves, m., fonseca, b., morgado, l., & martins, p. (2011). improving teaching and learning of computer programming through the use of the second life virtual world. british journal of educational technology, 42(4), 624-637. faust, j. l., & paulson, d. r. (1998). active learning in the college classroom. journal on excellence in college teaching, 9(2), 3-24. goncharov, s., ospichev, s., ponomaryov, d., & sviridenko, d. (2019). the expressiveness of looping terms in the semantic programming. arxiv preprint arxiv:1912.02731. grinias, j. p. (2017). making a game out of it: using web-based competitive quizzes for quantitative analysis content review. journal of chemical education, 94(9), 1363-1366. grover, s., jackiw, n., & lundh, p. (2019). concepts before coding: non-programming interactives to advance learning of introductory programming concepts in middle school. computer science education, 29(2–3), 106-135. higashinaka, r., dohsaka, k., amano, s., & isozaki, h. (2007). effects of quiz-style information presentation on user understanding. in eighth annual conference of the international speech communication association. ismail, m. e., othman, h., amiruddin, m. h., & ariffin, a. (2017). the use of animation video in teaching to enhance the imagination and visualization of student in engineering drawing. in iop conference series: materials science and engineering (vol. 203, no. 1, p. 012023). iop publishing. lin, m. h., & chen, h. g. (2017). a study of the effects of digital learning on learning motivation and learning outcome. eurasia journal of mathematics, science and technology education, 13(7), 35533564. ling, l. t. y. (2018). meaningful gamification and students' motivation: a strategy for scaffolding reading material. online learning, 22(2), 141-155. manik, e. (2020). visual basic 6.0 untuk media pembelajaran interaktif [visual basic 6.0 for interactive learning media]. lppm uhn press. nadeak, b., & naibaho, l. (2020). video-based learning on improving students learning output. palarch's journal of archaeology of egypt/egyptology, 17(2), 44-54. nguyen, k., & mcdaniel, m. a. (2015). using quizzing to assist student learning in the classroom: the good, the bad, and the ugly. teaching of psychology, 42(1), 87-92. nissen, j. m., talbot, r. m., thompson, a. n., & van dusen, b. (2018). comparison of normalized gain and cohen’s d for analyzing gains on concept inventories. physical review physics education research, 14(1), 010115, 1-12. ramkissoon, p., belle, l. j., & bhurosy, t. (2020). perceptions and experiences of students on the use of interactive online learning technologies in mauritius. international journal of evaluation and research in education, 9(4), 833-839. sa’adah, l., nurkamto, j., & suparno, s. (2018). oral corrective feedback: exploring the relationship between teacher’s strategy and students’ willingness to communicate. studies in english language and education, 5(2), 240-252. saeed, n., yang, y., & sinnappan, s. (2009). emerging web technologies in higher education: a case of incorporating blogs, podcasts and social bookmarks in a web programming course based on student's learning styles and technology preferences. journal of educational technology and & society, 12(4), 98-109. saputri, d. y., rukayah, r., & indriayu, m. (2018). need assessment of interactive multimedia based on game in elementary school: a challenge into learning in 21st century. international journal of educational research review, 3(3), 1-8. shouthiri, p., & thushika, n. (2018). a comparative analysis of looping structures: comparison of ‘while’loop and ‘do-while’loop in the c++ llanguage. asian journal of research in computer science, 2(3), 15. tam, w. c. (1992). teaching loop invariants to beginners by examples. proceedings of the twenty-third sigcse technical symposium on computer science education, 92-96. thézé, r., gadiri, m. a., albert, l., provost, a., giraud, a.-l., & mégevand, p. (2020). animated virtual characters to explore audiovisual speech is controlled and naturalistic environments. scientific reports, 10(1), 1-12. vania, p. f., setiawan, w., & wijaya, a. f. c. (2018). edmodo as webbased learning to improve student’s cognitive and motivation in learning thermal physics. journal of science learning, 1(3), 110-115. winarto, a., hardyanto, w., & sugianto, s. (2019). telegram development in dokeos-based e-learning as a learning media to improve students' motivation in learning physics. physics communication, 3(2), 78-85. yusrizal, y., safiah, i., & nurhaidah, n. (2017). kompetensi guru dalam memanfaatkan media pembelajaran berbasis teknologi informasi dan komunikasi (tik) di sd negeri 16 banda aceh [teacher competence in utilizing information and communication technology (ict)-based learning media in sd negeri 16 banda aceh]. jurnal ilmiah mahasiswa pendidikan guru sekolah dasar, 2(2), 126-134. yusuf, m., zuhrawardi, z., & wardani, e. (2020). the effectiveness of animated video as learning media towards the perception of healthy snacks on elementary school students in indonesia. the international journal of tropical veterinary and biomedical research, 5(2), 1-6. a © 2022 indonesian society for science educator 141 j.sci.learn.2022.5(1).141-153 received: 30 april 2021 revised: 15 august 2021 published: 1 march 2022 parent involvement in pre-school science activities: what do parents think about it? cigdem sahin cakir1, gonca uludag2* 1department of science education, faculty of education, giresun university, turkey 2department of preschool education, faculty of education, giresun university, turkey *corresponding author: drgoncauludag@gmail.com abstract this study aimed to investigate parents' views about science and parent involvement in pre-school science activities. according to cresswell's designs' sequential explanatory mixed-method design, the study was conducted. the participants were composed of parents of 39 children (60-72 months) who studied in two different classrooms, there were 20 children in one of the classrooms, and there were 19 children in the other classroom at a public pre-school in ankara/turkey in 2015-2016 school year. the parents' views about science and preschool science activities scale and semi-structured interviews were used to collect data. in addition, semi-structured interviews were conducted with three volunteer parents, each from the parents who got high, medium, and low average scores from the scale. thus, parents' views about science and parent involvement in pre-school science activities were examined in detail. according to the results of this study, it was determined that the parents who participated in this study had positive views on science and science education in pre-school, realized the importance of science activities in pre-school, were eager to participate in pre-school science activities. therefore, this study suggests that teachers and parents should be informed that parents' involvement is essential for doing science activities in pre-school. keywords parent involvement, science education, science, early childhood education, pre-school 1. introduction "why is the sky blue? how do seeds grow? what makes sound and music?..." young children ask their parents or teachers hundreds of questions like these, so we use science to answer them (paulu & martin, 1992). science is a systematic process for discovering knowledge or uncovering general truths based on observation and experimentation (sawah & clark, 2015). children are born with a natural sense of curiosity and exploration, and daily life experiences offer children numerous opportunities to perceive, know and make sense of the world. these experiences are also an opportunity for children to understand science. according to eliason and jenkins (2003), science is part of our daily life, so science education should be meaningful for young children and associated with everyday life. the primary purpose of science education in early childhood education is to deepen children's views on the world and their experimental studies and increase children's knowledge through new conceptual perceptions. kelly (2015) lists the reasons for teaching science in the early years as follows: • young children are interested in science and want to know about the world. • science is interesting for children and gives young children a better understanding of their world. • children are introduced to scientific methods, techniques, and concepts. • children's vocabulary develops with the scientific terms they learn. • science is strongly associated with other areas of the curriculum. science education in early childhood is essential for supporting children's curiosity towards nature and science and informing their children with scientific knowledge without getting bored of their children's questions about nature and science (lehr, 2005). in the early years, parents play an essential role in helping the child acquire the first knowledge of life and science and gaining experience at home/out-of-school. paulu and martin (1992) stated that parents are crucial for journal of science learning article doi: 10.17509/jsl.v5i1.33912 142 j.sci.learn.2022.5(1).141-153 the child's science learning, and parents' enthusiasm and encouragement can spark the child's interest in science. they also said that scientific knowledge is cumulative; the child should start learning it early, so the more the child's curiosity about science is encouraged by their parents, the better it will be. therefore, we can say that parent involvement studies in early childhood education are essential in science education. 1.1. parent involvement in early childhood education parent involvement (we use the terms parent involvement, family involvement, and parental involvement interchangeably in this study) is an essential component of early childhood education. according to the structure-process model (figure 1), quality in early childhood education consists of four main parts. each component views individually, but together all four components influence the development of children and their families. also, this model assumes that quality is quantifiable/measurable (kluczniok & roßbach, 2014). it is seen that interaction with the parent is also included in the model as a component. in its most traditional definition, parent involvement refers to participating in activities at home and the child's school (martinez, 2015). parent involvement “has been operationally defined as parental aspirations for their children’s academic achievement, parents’ communication with their children about education and school matters, parents’ participation in school activities, parents’ communication with teachers about their children, and parental supervision at home” (fan, 2001). deloatche, bradley klug, ogg, kromrey, and sundman wheat (2015) have described parent involvement as recommended strategy in engaging parents in children's educational experiences. parent involvement is parents' contribution to children's learning at home, at school, and in the community (marin & bocoş, 2017). a wide typology to account for different levels of parent involvement in education was suggested by epstein (fan & chen, 2001). epstein (1987a; 1987b; 1990; 1995; 2011) has included parent involvement in her various works. epstein’s typology is based on the following six types: (1) parenting, (2) communicating, (3) volunteering, (4) student learning at home, (5) decision making, (6) collaborating with the community (epstein, 2011). rodriguez, collinsparks, and garza (2013) proposed a revision to epstein's typology. the proposed model (figure 2) organizes various categories of parent involvement at school into three dynamic dimensions: home environment (parents and students), parents and school/community, and students and school/community. the home environment (parents and students) dimension has been expanded to include figure 1 the conception of educational quality: structure-process model of quality (kluczniok & roßbach, 2014, p.147) figure 2 dynamic dimensions of the parent, school/ community, and student involvement (cf. authors’ illustration based on rodriguez et al., 2013) journal of science learning article doi: 10.17509/jsl.v5i1.33912 143 j.sci.learn.2022.5(1).141-153 peer/sibling interaction and the influence of parents' expectations and monitoring epstein's category, "student learning at home". the parents and school/community dimension involves the school's relationship and relevant community agencies with parents and includes epstein's categories of parent support/training, communication, participation in decisions, and volunteering. students and community/school environment dimensions include the interactions among school officials and teachers with students in their specific community contexts. thus, epstein's category of community collaboration has been integrated. however, three new categories have been added to draw attention to the importance of connecting school curriculum to students' everyday lives, the realities of the community in which they live, and their sense of agency (rodriguez et al., 2013). parent involvement is known to have many benefits. accordingly, parent involvement activities positively affect children's cognitive development, communication skills, literacy development, pre-literacy skills, pre-writing skills, knowledge of print, vocabulary growth, expressive language development, comprehension skills, interaction with peers and adults, and learning (deloatche et al., 2015; fantuzzo, mcwayne, perry, & childs, 2004; harvard family research project, 2006; powell, son, file, & san juan, 2010). parent involvement is also essential for academic and social-emotional development in early childhood and later (cohen & anders, 2019; fasina, 2011; handayani, wirabrata, & magta, 2020). arnold, zeljo, doctoroff, and ortiz (2008) found that pre-literacy developed in children as parent involvement increased in pre-school education. research has shown that parent involvement in early childhood education affects children's academic achievement and personal development (camlıbel cakmak, 2010; celenk, 2003; daniel, 2015; keceli kaysılı, 2008; sahin & kalburan, 2009). parent involvement in early childhood programs helps children succeed in kindergarten and elementary school (carter, 2002). it is believed that it is imperative to know what is necessary for parent involvement and understand its impact in early childhood programs (hilado, kallemeyn, leow, lundy, & israel, 2011). 1.2. parent involvement in science education: why important? children will most likely experience their first formal education on how everyday life and science are connected during early childhood. this first formal encounter with science knowledge could be strongly reinforced at home if parents participated as resources to contribute to their children’s science education. parents are their children's first teachers, and the home environment is essentially a giant science laboratory for pre-school children (aktas arnas, aslan, & günay bilaloglu, 2012). children may answer numerous questions in this laboratory by working with their parents (flannagan & rockenbaugh, 2010). hence, parent involvement in science education will help bridge the home and school environment and children's early development. the national science teachers association (nsta) strongly advocates parent involvement in science education. also, parent involvement in early childhood science activities plays an essential role in children's science literacy (tekerci, 2020). parents encourage the daily use of science concepts and process skills to enhance their child's ability to learn the skills necessary for success (nsta, 1994). parent involvement in science education can be at home, out-of-school learning environments, or school. at home with the child, cooking, doing household chores, repairing a household object, reading science books, watching science-related television programs, examining online or computer-based resources, playing with science materials can be given as examples of involvement in science education (nsta, 2009). out-ofschool learning environments (zoo, science museum, science center, planetarium, aquarium, etc.) are critical in science education in early childhood. uludag and erkan (in press) determined that the use of out-of-school learning environments in science education positively affects the science process skills of young children. similarly, cui, zhang, and leung (2021) reported that visiting science and technology museums, aquariums, botanical gardens, or planetariums with their parents positively affected their science achievement. also, parents involvement in science education provides opportunities to spend time together conducting fun activities, such as visiting the zoo, planetarium, botanical gardens, nature walks, and so on (alisinanoglu, ozbey, & kahveci, 2015; fleer & rillero, 1999; hofstein & rosenfeld, 1996, nsta, 2009). participating in school trips and participating in science activities are examples of parent involvement in science education (nsta, 2009). in school, parents can experiment with children, share information about their profession with children, participate in the drama, help children grow plants, care for an animal in the schoolyard, and participate in science projects. according to wasik, bond, and hindman (2002), having parents more involved with school activities can improve parent and school communication and cooperation. sahin (2019) stated that parents should be involved in the process at school and home for effective science education in early childhood, and they need attention and support for it. the benefits of parent involvement in science education for children have been demonstrated in various research. parent involvement positively affects children's science achievement and attitudes (fleer & rillero, 1999). atci (2003) has mentioned that children’s interest in science starts in the family, and there is a positive relationship between parent involvement and children's success in science. salli, dagal, kucukoglu, niran, and tezcan (2013) have developed a project-based parent involvement journal of science learning article doi: 10.17509/jsl.v5i1.33912 144 j.sci.learn.2022.5(1).141-153 program for 60-72-month-old children, and this project has been designed to provide the permanence of the recycling concept. as a result of the research, it has been found that this program has been effective. sackes (2014) has revealed that early childhood teachers tend to teach science compared to other fields, which is compatible with the science teaching preferences of parents. sahin (2019) has examined the relationship between primary school 7thgrade students' science achievement and their parents' science literacy levels. as a result of the study, it has been determined that the science literacy levels of mother and father have positively influenced students' achievement. also, the mother's science literacy level is more effective than the father's science literacy level on students' achievement. aksu and karaçöp (2015) have examined parents’ involvement in home-based learning activities in 5th, 6th, 7th, and 8th-grade science lessons. consequently, they have determined that parents are aware of their responsibility for their children's homebased learning activities. however, they have also determined that parents' lack of scientific knowledge and self-confidence has negatively affected their active involvement in home-based learning science activities. so, it is imperative to discuss the things to promote parent involvement. starting from early childhood education, parent involvement in science education is necessary at other education levels. therefore, we argue that it is essential to promote a good attitude toward science amongst parents and increase their science knowledge. then, the question is, "how can educators promote parents' active involvement in the science education of their children?" because parents often may have low confidence in their science knowledge, they may not be sure about supporting teachers at pre-school. therefore, it is essential that educators better inform parents and provide strategies to become more involved in their children's education at home and pre-school. in this study, we sought to investigate parents' views on science and how they commonly engaged with their pre-school children in science-related activities outside of school. firstly, determining parents' views about science and parent involvement in pre-school science activities is a need to explore a way to train parents to be more involved in doing science with their children and encourage parents to participate more in science activities. in this context, this study aimed to investigate parents' views about science and parent involvement in pre-school science activities. sub-problems: 1. do parents' views of science and pre-school science activities differ significantly according to their gender? 2. do parents 'views of science and parent involvement in pre-school science activities differ significantly according to their educational stage? 3. what is the average score of the parents' views on science and pre-school science activities according to the scale factors? 4. what are the views of parents who got high, medium, and low mean scores on the scale about parent involvement in pre-school science activities? 2. method the research design, participants, data collection tools, data collection, and data analysis were discussed in this part. 2.1. research design this study was conducted according to cresswell's designs' sequential explanatory (quantitative-qualitative) mixed-method design. qualitative data were collected after collecting and analyzing quantitative data in a sequential explanatory mixed-method design. in this design, quantitative data are dominant and qualitative data supports quantitative data. firstly, the researcher collects and analyzes the quantitative data. secondly, the researcher collects the qualitative data and analyzes it. qualitative data help explain, or elaborate on, the quantitative results obtained in the first phase. quantitative and qualitative data are analyzed separately and combined in the comment and discussion section (ivankova, creswell & stick, 2006). 2.2. participants while determining the sample in sequential mixed method research requires an approach that expands and narrows the field of view. this study's participants include selecting probabilistic and purposeful sampling strategies (quantitative-qualitative) for sequential mixed method research (teddlie & yu, 2007). in the study, first of all (n = 39), the parents of the students studying in a public preschool were determined randomly to collect quantitative data. parents participated in the research voluntarily. the participants of this study were composed of parents of 39 children (60-72 months) who studied in two different classrooms, there were 20 children in one of the classrooms, and there were 19 children in the other classroom at a public pre-school located in etimesgut, ankara/turkey in 2015-2016 school year. the pre-school in the study group was randomly selected among the schools that the researchers had easy access to and the preschool administration allowed. a standard program prepared by the republic of turkey ministry of national education is used in public pre-schools in turkey. science activities are also included in the program, but the teachers plan the frequency of application of these activities. the teachers of the classes in the study group stated that they applied science activities twice a week. demographic characteristics of parents are presented in table 1. quantitative data were analyzed after data were obtained. according to the quantitative data analysis results, semi-structured interviews were conducted with three volunteer parents, each from the parents who got journal of science learning article doi: 10.17509/jsl.v5i1.33912 145 j.sci.learn.2022.5(1).141-153 high, medium, and low average scores from the scale. in addition, parents' views about science and parent involvement in pre-school science activities were examined in detail with a semi-structured interview. when table 1 was examined, it was seen that mothers filled out the scale voluntarily. when parents' educational stage was reviewed, it was seen that their educational stage was primarily high school, associate degree, and bachelor’s degree. also, the educational stage of 16 parents (nmother = 13; nfather = 3) was bachelor's degree, the educational stage of 2 parents was the master's degree, and the educational stage of 1 parent was doctorate. three voluntary parents were selected with high, medium, and low scale scores to collect the qualitative data. moreover, the voluntary participation of parents in the semi-structured interview was adopted. the maximum average score of the scale was five, and the scale's minimum average score was one. the parent with a high-level average scale score was coded as parent 1 (p1). the parent with a medium-level average scale score was coded as parent 2 (p2), and the parent with a low-level average scale score was coded as parent 3 (p3). demographic characteristics of parents are presented in table 2. p1 is 33 years old, and she is a pre-school teacher. she did not study any course related to science education but was touched on some science issues in some courses during the undergraduate education process. p1 said that followed developments related to science and technology through television, the internet, book, etc. p2 is 33 years old, and she is a public official. she said that she studied science education, but it was not enough. p2 said that she did not follow developments related to science and technology but watched health programs. also, her husband is interested in science and technology. p3 is 32 years old, and she is a housewife. she said that she studied science education, but it was not enough. p3 said that she did not follow developments related to science and technology and did not know issues related to science and technology. also, she said that she watched health programs. 2.3. data collection tools a scale and semi-structured interview were used as data collecting tools in this study. data were collected in march 2016. the scale was filled out by all parents (n = 39). semistructured interviews were conducted with three volunteer parents who scored high, medium, and low on the scale. the scale for determining parents' views about science and pre-school science activities was used to gather quantitative data. this scale has been developed by sahin, uludag, gedikli, and karakaya (2018). this was a likert scale which is instructed to select one of these five responses: strongly agree, agree, neutral, disagree, or strongly disagree [strongly disagree: 1 point, disagree: 2 points, neutral: 3 points, agree: 4 points, strongly agree: 5 points]. the scale was composed of two parts: the first part includes demographic information, and the second part contains five factors for assessing parents' views about science and pre-school science activities. the results of the confirmatory factor analysis (cfa) and item analysis have revealed that within the scope of five-factors structure, which are (1) science and preschool science activities, (2) life sciences activities in preschool education, (3) physical sciences activities in preschool education, (4) earth and space sciences activities in preschool education and (5) applied science activities in preschool education, construct validity has been high for target characteristics to be measured. cmin/df (χ²/sd = 2.092), chi-square goodness, cmin (χ² = 2437.116), goodness of fit index, (gfi = .540), adjusted goodness of fit index (agfi = .497), comparative fit index (cfi = .610), root mean square residuals, (rmr = .078) and root mean square error of approximation (rmsea = table 1 participating parents’ educational stage educational stage ps ms hs ad bd md dd mother (n = 31) 2 1 6 6 13 2 1 father (n = 8) 1 3 1 3 ps: primary school, ms: middle school, hs: high school, ad: associate degree, bd: bachelor’s degree, md: master’s degree, dd: doctorate degree table 2 demographic characteristics of parents who were semi-structured interviewed* parent 𝐗 age es ways parents use to follow developments related to science. tv internet book trip-observation uninterested p1 5 33 bd √ √ √ √ p2 3.95 33 hs √ p3 2.79 32 ms √ * p1 = parent had a high-level average scale score, p2 = parent had a medium level average scale score, p3 = parent had a low-level average scale score, bd: bachelor’s degree, hs: high school, ms: middle school, es: educational stage journal of science learning article doi: 10.17509/jsl.v5i1.33912 146 j.sci.learn.2022.5(1).141-153 .105) indices were examined with the cfa. these fit indices indicate the suitability of the scale items. in addition, the cfa data are significant (p < .000). the estimated load values of the items in each scale factor were examined. in addition, the results of the item analysis for the construct validity of the scale can be interpreted as the validity of the items in the scale is high, the items in the factors distinguish the parents in the lower and upper groups, and the items in the factors are items intended to measure the same behavior. the correlation between scale factors has ranged from .711 to .837. cronbach's alpha reliability coefficient of the scale has been calculated to be .935. the cronbach alpha reliability coefficient of the scale factors has varied between .734 and .913 (sahin et al., 2018). these results indicate that the scale has validity and reliability. the semi-structured interviews were conducted to collect qualitative data in the study. there are ten questions in the semi-structured interview form used for interviews. nine questions were prepared in the first draft of the form. for the interview questions, the opinions of 3 experts, one of which is science education and two of them pre-school education, were consulted. one of the pre-school education experts and the science education expert stated that all the questions in the draft form were suitable for the study and could be used as they are. one of the pre-school education experts stated that in addition to the questions in the draft form, whether the child shares the science activities at pre-school with their parents may be essential. the researchers also found this suggestion necessary, and thus, it was added to the interview form. the questions in the form in terms of language and intelligibility were directed to two mothers who were not included in the study. one of the mothers had difficulty answering a question (do you have any information about science activities at your child's pre-school?), so various examples (by sharing the child with bulletins, by getting information from the teacher, by following the education program, etc.) were given to make the question understandable. questions of the semi-structured interview form were presented in appendix1. interviews were conducted in march 2016 and were recorded with the knowledge and consent of the parents. each interview lasted approximately 35-40 minutes. 2.4. data analysis to test the conformity of the data obtained from the scale to normal distribution, mean, median, skewness, kurtosis coefficients, and shapiro-wilk normality test p values were calculated. if the sample group is less than 50 people, the shapiro-wilks test is used to test the compliance of the data to a normal distribution (büyüköztürk, 2004). in addition, quantitative data were analyzed independent-samples t-test, one-way anova test, and descriptive statistical. the data obtained from semi-structured interviews were analyzed contently according to qualitative data analysis. data were coded, and themes were created from codes. researchers created initial codes and themes independently to provide credibility for data analysis. a matrix was then developed to compare and contrast the codes/themes across by researchers. this implementation allowed for the triangulation of scale findings. researchers coded data by re-reading the transcripts, combining, excluding, redefining, identifying emergent codes and themes, and discussing with the other researchers to provide credibility for data analysis. so, researchers decided on standard codes and themes together. data obtained from the scale and semi-structured interviews form were analyzed separately, but these data were discussed by being compared with each other. method triangulation controls the consistency of findings reached by different data collection tools (patton, 2014). also, quotes from three parents' statements were presented to provide data credibility. 3. result in this section, the findings obtained for the research sub-problems were presented. the conformity findings to normality distribution of the data obtained regarding which tests to be used in the spss 22.0 statistical package program used in the analysis of the data were presented in table 3. when table 3 was examined, it was seen that the pvalue was more significant than 0.05 according to the shapiro-wilk test, so the scale data met the normality assumption (p > .05). in addition, when the mean and median values of the scale were examined, the fact that these values were close to each other supports that the scale data meet the normality assumption. the skewness and kurtosis values were between -1 and +1 indicates that the scale data were typically distributed. therefore, parametric tests were used in the analysis of the scale data. 3.1. do parents' views of science, and pre-school science activities differ significantly according to their gender? results from the independent-samples t-test comparison of parents' mean scores of the scale and its factors were presented in table 4. table 3 normality test results of data obtained from the scale test n �̅� median skewness kurtosis shapiro-wilk test p normality test 39 3.9211 3.8988 -.010 -.242 .382 journal of science learning article doi: 10.17509/jsl.v5i1.33912 147 j.sci.learn.2022.5(1).141-153 table 4 shows the findings obtained from the independent-samples t-test comparison of the scale's mean scores and its factors. accordingly, when the average scores of the parents for scale factors are compared, it was seen that the mothers' average scores were higher than the fathers' average scores. however, there was no significant difference between the parents' average scores (p >, 05). for example, in the "science and preschool science activities" factor, when the parents' average scores were compared, the mothers' mean scores were higher than the average scores of the fathers, and there was no significant difference between the average scores. (mother = 3.9496, father = 3.4844, t = 1.854, p > .05, r = .085). 3.2. do parents' views of science and parent involvement in pre-school science activities differ significantly according to their educational stage? findings from the one-way anova comparison of the average scores of parents obtained from the scale according to their educational stage were presented in table 5. when the results of one-way anova are examined in table 5, it was seen that the average scores of the parents obtained from the scale did not show a statistically significant difference according to their education level [f(6) = 2.226, p > .05]. eta squared value (r = .294) also supports this situation. 3.3. do parents' views of science, and pre-school science activities differ significantly according to scale factors? findings from the descriptive statistics are presented in table 6. table 6 shows the parents' mean scores and standard deviation results from the scale and scale factors. parents' scale average score is 3.9211. this case indicates that parents' views and parent involvement in science education in pre-school are in the "agree" category of the five-point likert-type scale, and therefore parents have positive opinions. similarly, it is seen that the average scores of the scale factors, which are science and preschool science activities, life sciences activities, physical sciences activities, earth and space sciences activities, and table 4 results from independent-samples t-test comparison of parents' mean scores of the scale and its factors the scale factors parent n �̅� sd df t p r (η 2) science and pre-school science activities mother 31 3.9496 .61691 37 1.854 .072 .085 father 8 3.4844 .69737 life sciences activities mother 31 4.0115 .50477 37 1.120 .270 .033 father 8 3.7857 .52350 physical sciences activities mother 31 3.8065 .76447 37 .512 .611 .007 father 8 3.6607 .46409 earth and space sciences activities mother 31 3.9677 .75697 37 .891 .379 .793 father 8 3.7083 .62836 applied science activities mother 31 4.1843 .57522 37 1.829 .075 .021 father 8 3.7500 .69042 the scale means mother 31 3.9839 .58208 37 1.348 .186 .047 father 8 3.6778 .52897 table 5 findings from the one-way anova comparison of the average scores of parents obtained from the scale according to their educational stage educational stage n �̅� sd df f p r (η2) significant difference primary school 2 4.2955 .48361 6 2.226 .066 .294 no middle school 2 2.8824 .13090 high school 9 3.7907 .57734 associate degree 7 3.7986 .58115 bachelor’s degree 16 4.0555 .49787 master’s degree 2 4.5375 .55811 doctorate degree 1 3.8988 . total 39 3.9211 .57854 table 6 descriptive statistical findings of the scale and its factors factor n �̅� sd science and pre-school science activities 39 3.8542 .65289 life sciences activities 39 3.9652 .51007 physical sciences activities 39 3.7766 .71036 earth and space sciences activities 39 3.9145 .73237 applied science activities 39 4.0952 .61692 the scale average scores 39 3.9211 .57854 journal of science learning article doi: 10.17509/jsl.v5i1.33912 148 j.sci.learn.2022.5(1).141-153 applied science activities are in the "agree" category. in the light of these results, it can be said that parents' views and parent involvement in science education in pre-school are positive. 3.4. what are the views of parents who got high, medium, and low mean scores on the scale about parent involvement in pre-school science activities? semi-structured interviews were conducted to reveal parents' views on involvement in pre-school science activities. codes and themes were formed from the data obtained from the semi-structured interviews made with p1, p2, and p3. the themes were "the importance of science education in pre-school education," "cases of parents doing science activities with their children," and "parents' awareness on science activities done at pre-school and their involvement in these activities." results obtained from semi-structured interviews are presented in table 7. according to table 7, it was seen that there are seven codes in the theme of "the importance of science education in pre-school education. "should science education take place in pre-school period? why?" question was asked to parents, both p1 and p2 stated their ideas related to "necessity for science education in future and "arouse curiosity" codes on "the importance of science education in pre-school education" themes. quotes of parents are presented below. “i believe that he will study science education in the future years, but if he starts at an early age, it will be so good. for example, "is the air temperature measured?" i asked. he gave very reasonable answers. i saw that science-related subjects aroused curiosity in my child (p1)”. “because this education will contribute in the future times namely the education life after pre-school education. science education is not like a geography course in which a child can learn listening or is not like a mathematic course that puzzles a child's brain. the child learns something by wondering in a science course (p2)”. just p2 stated her views on "it provides sensitivity to the environment," "it provides sensitivity to lives," and "it provides identification of nature" codes. the quote of p2 is presented below. “when we look to our environment, plants, animals, everything is related to science. previously my son and i put the chickpea into the cotton at our home, and we waited for the germination for days. we have a home in the village; my son is spudding up land there with his grandfather, growing tomatoes. he eats and picks up tomatoes from the tomato branch. these both aroused my son's curiosity and encouraged my son more to be interested in plants, environment, and animals (p2)”. p1 stated her views on the "it encourages talking science" code. quote of p1 was presented below. “issues related to science attract the interest of my child, and he can talk to us on these issues. for example, yesterday, he asked me a different question about science. i do not know the answer to this question. i said that if you want you can learn from your teacher. so, he was more curious (p1).” both p1 and p3 expressed their views on the “science is in our life” code. quotes of p1 and p3 parents were presented below. “science is a field that exists in all areas of our lives (p1).” “science is related to everyday life. science is the life itself (p3).” when table 7 was examined, it was seen that the “cases of parents doing science activities with their children” table 7 results obtained from semi-structured interviews themes codes parents p1 p2 p3 the importance of science education in pre-school education it encourages talking about science + it is required for future science learning. + + it arouses curiosity + + it provides sensitivity to the environment. + it provides sensitivity to lives. + it provides identification of nature. + science is in our life. + + cases of parents doing science activities with their children doing kitchen experiments + asking question-related to science + not doing a science activity. + making science activities given as homework + growing plant + parents' awareness of science activities done at school and their involvement in these activities explanation of the science activity done at school by children to their parents + + + worksheets sent home by teachers. + + + no time + + no teacher’s invitation + + no interest in science activities + +: commented on code -: no idea about code. journal of science learning article doi: 10.17509/jsl.v5i1.33912 149 j.sci.learn.2022.5(1).141-153 theme were involved. p1’s expressions on “doing kitchen experiments,” “asking questions related to science,” and “growing plant” codes demonstrated that parents carried out various science activities.” a quote of the p1 was presented below. “i love growing flowers at home. i gave tasks to my child related to it. for example, i asked him to water these flowers together; i asked why the flowers' leaves turned yellow, and they were laid. he does not hurt flowers because he owns the flowers. he likes kitchen activities, and when i was cooking, i allow him to participate in the cooking process. we examine science journals that we subscribe to together. i ask him questions (p1)”. p2 mentioned her views on the "activities given as homework" code related to science activities which p2 does together with her child. the quote of p2 is presented below. “we do homework that our teacher sometimes gives related to science. for example, as i mentioned in the examples, chickpea germination and tomato growing (p2).” p3 did not mention her view related to this issue. because she said that she did not know about doing science activities and could not teach her child something she did not know. according to these findings, it was seen that p1 and p2 did various science activities with their children, and they offered learning opportunities for their children at home/out-of-school. when table 7 was examined, it was seen that "parents' awareness on science activities done at the pre-school and their involvement to these activities" themes were involved. for example, three parents said they were aware of explaining the science activity done at pre-school by children to their parents. quotes of parents were presented below: “my son explains (p1).” “my son explains to us things he does about science at the preschool; he says, "we do it like this (p2).” “my daughter explains about things that are interesting and remembered (p3).” parents' quotes on the “worksheets sent home by teachers” codes were presented below. “our teacher sends weekly worksheets; if there are activities related to learned science issues, we do them (p1).” “the teacher usually provides weekly information programs and gives homework; also, i am trying to do these with my son (p2)”. “the teacher sends us worksheet (p3).” according to these findings, teachers use worksheets for parents' awareness about science issues and activities studied in the pre-school. p1 mentioned her views in the "no time" and "no teacher's invitation" codes, p2 mentioned her views in the "no time" code, p3 parent mentioned her views on "no teacher's invitation" and "disinterested in science activities" codes. in addition, quotes from parents' views on why parents cannot participate in science activities are presented below. “the teacher of my son invited to class for parent involvement activity, but she did not invite for a science activity. this will be the first time i will participate in a science activity in my son's class. i am a teacher. my son and i will be in pre-school afternoon; even if there is such involvement activity it is tough to join, i have no time (p1).” “i have no time because i have a baby. this activity is the first involvement activity for me (p2).” “teacher did not invite to us for parent involvement activity. we did not wonder and go (p3).” p1 could not involve in the education process because she is working, and p2 could not apply in the education process because she looked after her baby. however, p3 could not apply in the education process because the teacher did not invite her, and she had no interest in science. 4. discussion parenting is one of the primary influences on children's development. parents are instrumental in facilitating their children's early experiences because these experiences have a lasting impact on children's ability to learn and succeed, both at school and in life (prieto, 2018). parent involvement refers to the different activities and actions that families can do to support their offspring in education (camarero-figuerola, dueñas, & renta-davids, 2020). it is known and accepted worldwide that parent involvement has positive educational and behavioral outcomes and positively supports the child's development (ahmetoglu, acar, sezer, & akşin yavuz, 2018). parent involvement in early science education is also crucial in many ways because education is also a collaborative process. collaboration for learning and teaching is essential in helping children acquire 21st-century skills. so much so that parents are one of the collaborating stakeholders in oecd the future of education and skills 2030 project (oecd, 2018), which aims to create a shared understanding of the knowledge, skills, attitudes, and values to be gained by children. the results of this study, which aims to investigate parents' views about science and parent involvement in pre-school science activities where parents are an essential stakeholder, are discussed below. when the mean scores of the parents for scale factors were compared, it was seen that the mothers' averages were higher than the fathers’ averages. however, there was no significant difference between the parents’ mean scores (p > .05). similarly, kılıc and unal (2020) and dere and unlu (2020) found that parents' views on science activities in the science and pre-school period did not differ significantly according to gender. this may result from parents' equal interest in the pre-school science education of their children. this situation is promising for parent involvement studies in pre-school science education. furthermore, parents' views do not differ significantly journal of science learning article doi: 10.17509/jsl.v5i1.33912 150 j.sci.learn.2022.5(1).141-153 according to their educational stage. contrary to this study, kılıc and unal (2020) found that parents' views on preschool science and science activities differ significantly according to their educational stage. also, it has been determined that the direction of this difference favors parents who are high school, undergraduate, and graduate graduates. dere and unlu (2020) stated that the higher the educational stage, the better parents could contribute to their children's science education. however, it was not ensured that the number of parents who volunteered to participate in this study was equal according to their education level. perhaps, the fact that there was no significant difference found in this study according to the parents' education levels may be due to this situation. however, regardless of the education level, it is considered very important for parents to support their children's science education in the pre-school period. it is seen that the average scores obtained by the parents from the scale and the scale factors are generally at the agreed level (4 points) (table 6). this situation indicates that the parents' views are generally positive. this finding is promising. because, according to perera (2014), parents' attitudes towards science have a significant positive effect on their children's science achievement. as a result of this study, it was determined that parents did not actively involve in science activities at pre-school. however, they did science experiments with their children at their homes and listened to their children's views on preschool science activities. this result supported parents with higher average scores toward the "applied science activities in preschool education" factor of the scale. according to this result, it can be said that parents' involvement in science education is limited to home, but parents consider it essential to do science activities with their children. parents "science and preschool science activities," "life sciences," "physical sciences," "earth and space sciences," "activities in preschool education", and "applied science activities in preschool education" factors of the average scale score supported this case. parents got an average score between "agree" and "strongly agree". the scale results indicated that parents had positive views about pre-school science and science activities. however, this result did not provide enough information about parents' involvement in science activities in preschool. so, three volunteer parents (p1, p2, and p3) were interviewed who were determined to get high, medium, and low average scores from the scale. the results of the semistructured interviews demonstrated that each of these three parents has positive views on science education in preschool education. however, it was identified that p1 and p2 expressed more positive views than p3 in the "the importance of science education in pre-school education" theme. this result demonstrated that the scale results supported the results of the semi-structured interviews. in other words, the parents who got high and medium average scores expressed more positive views than the parent who got a low average score from the scale. parent involvement is essential in pre-school science education, as in all early childhood education. in a study, kiraz and aytac (2020) developed family education through science activities (fetsa) practices for the parents of five-year-old children receiving pre-school education. they thus aimed to investigate the effect of parents on school involvement and children's academic achievement. parents and their children did 17 science activities at home during their studies. at the end of the study, it was determined that the academic success of children who received pre-school education and whose parents participated in the fetsa practices increased. accordingly, it is possible to say that parents' positive views about pre-school science education and various practices will positively affect children. however, p3 was aware of the importance of science in pre-school education. she said that "science is in our life." also, semi-structured interview results showed that p1 was involved in various science activities with her child. p2 just helped her child's science activities given as homework by the pre-school teacher. p3 stated that she was growing plants with her child as a science activity. according to semi-structured interview results, it can be noted that p1 was aware of science activities such as "doing kitchen experiments" and "asking a question related to science." as a result of the interviews, it is seen that the parents give examples of science activities at home and school. in gross et al.'s (2020) research, pre-k and kindergarten parents gave similar examples as behavioral indicators of parent involvement (helping with homework, talking about school, etc.). however, in this study, examples of involvement at pre-school are limited, and involvement barriers are included in the codes of the "parents' awareness of science activities done at school and their involvement in these activities" theme. parent involvement in early childhood has been linked with stronger pre-literacy skills, acquisition of basic skills in mathematics and science, well-developed social skills, early reading skills, language skills, and positive attitudes toward school (hu, zou, & ren, 2020; jeffries, 2012; powell et al., 2010; marcon, 1999). also, adair (2020) stated that new research suggests that secondary students' science knowledge and achievement can be predicted from their science knowledge as early as pre-school. in addition, he emphasized the importance of considering the roles of the parent and the teacher, and the child in early childhood. in other words, parent involvement in science activities should be considered. however, as a result of the study, it was seen that the parents interviewed did not get involved in science activities at pre-school for various reasons. p1 and p3 said that the pre-school teacher did not invite them to school activities. therefore, it can be deduced that if the teacher invites the parents to the school, they will participate in the school activities. it can be said that the journal of science learning article doi: 10.17509/jsl.v5i1.33912 151 j.sci.learn.2022.5(1).141-153 teacher's approach to parent involvement draws attention here. teachers must encourage parents to plan and practice parent involvement activities. according to the result of baeck's research (2015), although many teachers acknowledge the importance of parent involvement and home-school cooperation, they do not care about parent involvement due to a lack of time and resources. also, p3 said that she did not participate in science activities because she is irrelevant. p1 said she could not be involved because she could not find time because of work. in preston, macphee, and o'keefe's (2018) study, kindergarten teachers stated that many parents have an intense work schedule. therefore, it is difficult for them to participate in school hours. erkan, uludag, and dereli (2016) determined that parents think that the fundamental factors preventing their involvement in school-based activities are the school's management, which does not adequately support parents’ involvement activities, time insufficiency, and intensity working life, parents’ indifference, and reluctance. also, in the results of atakan's (2010) research, it has been determined that parents have expressed that teachers are reluctant to involve parents in classroom activities. unuvar (2010) has stated that the purpose of research on parent involvement has not been reached, the teachers' efforts are either on paper or in files, and the parent does not notice the purpose of parent involvement. in conjunction with in-service education, the teacher should provide opportunities to encourage parent involvement research and ensure that teachers can provide quality guidance for parents in participating in pre-school science activities. similarly, sackes (2014) has demonstrated that parental preferences align well with pre-school teachers' views to teach less science than other content areas. it can be interpreted that teacher is a predictive factor qualifying parent involvement. the parents must be guided by teachers effectively and correctly. parents should be encouraged to observe science in their environment and do science activities with their children (veziroglu, 2011). conclusion in summary, this study determined that the parents who participated in this study had positive views on science and science education in pre-school and realized the importance of science activities in pre-school. according to the results of this study, parents generally had positive views about science and parent involvement in pre-school science activities. results obtained from the scale and semistructured interviews supported each other. it is understood from the statements in the semi-structured interviews that parents have a positive perception about their involvement in science activities. these positive views promise parent involvement in science education in preschool and supporting learning environments with parent involvement. there are some limitations to this study. it is limited to the views of 39 parents and three parents who were semistructured interviewed voluntarily. it was not ensured that the number of parents who volunteered to participate in this study was equal according to their education level. also, the numbers of mothers and fathers participating in this study are not equal. fathers did not participate in the semi-structured interview. these situations stated in future research should be tried to be eliminated. similar studies can be done with larger sample groups. this study shows that pre-school teachers and parents should be informed that parent involvement is essential for science activities. however, mothers volunteered to participate in both the scale in the study and the interviews. therefore, it can be suggested to develop policies and practices that will encourage fathers' active participation in parent involvement activities in science education and pre-school education. both school and out-of-school science activities should be supported with parent involvement because parent involvement is critical for improving a positive attitude to science education in early childhood education. it is suggested that pre-school teachers prepare in-service training to support parent involvement in pre-school science education. parent involvement in science activities can be examined with qualitatively focused research. acknowledgment there is no conflict of interest between the authors in this study. the authors contributed equally to the study. during this study, scientific research and publication ethics were followed by researchers. ethical rules have been followed. we thank the parents who voluntarily participated in this study for their contribution. and a part of this study was presented as an oral presentation at the 7th international conference on new horizons in education which was performed in vienna/austria, 13-15 july 2016. references adair, a. e. (2020). measuring pre-school parents' attitudes towards science: development and psychometric evaluation of a parent version of the dimension of attitudes towards science instrument. (unpublished doctoral dissertation). the graduate center, the city university of new york, usa. ahmetoglu, e., acar, i̇. h., sezer, t., & akşin yavuz, e. (2018). adaptation of family involvement questionnaire into turkish culture. bolu abant izzet baysal university journal of faculty of education, 18(1), 1-20. aksu, f. f., & karaçöp, a. (2015). the parental involvement in students’ home based science learning activities. erzincan university journal of education faculty, 17(2), 456-476. aktas arnas, y., aslan, d., & günay bilaloglu, r. (2012). okul öncesi dönemde fen eğitimi [science education in the preschool years]. vize publishing. alisinanoglu, f., ozbey, s., & kahveci, g. (2015). okul öncesinde fen eğitimi [science education in the preschool]. ankara: pegem academy publishing. arnold, d. h., zeljo, a., doctoroff, g. l., & ortiz, c. (2008). parent involvement in pre-school: predictors and the relation of involvement to pre-literacy development. school psychology review, 37(1), 74-90. https://doi.org/10.1080/02796015.2008.12087910 journal of science learning article doi: 10.17509/jsl.v5i1.33912 152 j.sci.learn.2022.5(1).141-153 atakan, h. (2010). participation of family studies in pre-school education according to ideas of teachers and parents. (unpublished master thesis). çanakkale onsekiz mart university, social sciences institute, çanakkale, turkey. atci, ş. (2003). impact of the family participation on the 4th class primary school students success of the subject environment we live. (unpublished master’s thesis). marmara university, educational sciences institute, i̇stanbul, turkey. baeck, u. k. (2015). beyond the fancy cakes. teachers’ relationship to home-school cooperation in a study from norway. international journal about parents in education, 9(1), 37-46. büyüköztürk, ş. (2004). sosyal bilimler için veri analizi el kitabi [manual of data analysis for social sciences]. ankara: pegem-a yayınları. camlıbel cakmak, ö. (2010). parent involvement in pre-school. abant izzet baysal university journal of social sciences, 20(20), 1-18. https://doi.org/10.11616/abantsbe.256 camarero-figuerola, m., dueñas, j. m., & renta-davids, a. i. (2020). the relationship between family involvement and academic variables: a systematic review. research in social sciences and technology, 5(2), 57-71. doi.org/10.46303/ressat.05.02.4 carter, s. (2002). the impact of parent/family involvement of student outcomes: an annotated bibliography of research from the past decade. eric number: ed476296. washington, dc. celenk, s. (2003). okul başarısının ön koşulu: okul aile dayanışması [the prerequisite for school success: home-school cooperation]. elementary education online, 2(2), 28-34. cohen, f., & anders, y. (2019): family involvement in early childhood education and care and its effects on the social-emotional and language skills of 3-year-old children. school effectiveness and school improvement. https://doi.org/10.1080/09243453.2019.1646293 cui, y., zhang, d., & leung, f. k. s. (2021). the influence of parental educational involvement in early childhood on 4th-grade students’ mathematics achievement. early education and development, 32(1), 113133. https://doi.org/10.1080/10409289.2019.1677131 daniel, g. (2015). patterns of parent involvement: a longitudinal analysis of family-school partnerships in the early years of school in australia. australasian journal of early childhood, 40(1), 119-128. https://doi.org/10.1177/183693911504000115 deloatche, k. j., bradley klug, k. l., ogg, j., kromrey, j. d., & sundman wheat, a. n. (2015). increasing parent involvement among head start families: a randomized control group study. early childhood education journal, 43, 271–279. https://doi.org/10.1007/s10643-014-0660-7 dere, z., & unlu, z. k. (2020). examination of parent participation in preschool science activities. ahi evran üniversitesi kırşehir eğitim fakültesi dergisi, 21(1), 154-182. https://doi.org/10.29299/kefad.2020.21.01.005 eliason, c., & jenkins, l. (2003). a practical guide to early childhood curriculum (7th ed.). upper saddle river, nj: prentice-hall. epstein, j. l. (1987a). parent involvement: what research says to administrators. education urban society, 19(2), 119–136. https://doi.org/10.1177/0013124587019002002 epstein, j. l. (1987b). what principals should know about parent involvement? principal, 66(3), 6-9. epstein, j. l. (1990). school and family connections. marriage & family review, 15:1-2, 99-126. https://doi.org/10.1300/j002v15n01_06 epstein, j. l. (1995). school/family/community/ partnerships: caring for the children we share. phi delta kappan, 76(9), 701-712. https://doi.org/10.1177/003172171009200326 epstein, j. l. (2011). school, family, and community partnerships: preparing educators and improving schools (2nd ed.). boulder, co: westview press. erkan, s., uludag, g., & dereli, f. (2016). an investigation on the perceptions of pre-school teachers, school administrators and parents on parent involvement. ahi evran university journal of kırşehir education faculty, 17(1), 221-240. fan, x. (2001). parental involvement and students' academic achievement: a growth modeling analysis. the journal of experimental education, 70(1), 27-61. https://doi.org/10.1080/00220970109599497 fan, x., & chen, m. (2001). parental involvement and students’ academic achievement: a meta-analysis. educational psychology review, 13(1), 1-22. https://doi.org/10.1023/a:1009048817385 fantuzzo, j., mcwayne, c., perry, m. a., & childs, s. (2004). multiple dimensions of family involvement and their relations to behavioral and learning competencies for urban, low-income children. school psychology review, 33, 467–480. https://doi.org/10.1080/02796015.2004.12086262 fasina, f. f. (2011). the role of parents in early childhood education: a case study of ikeja, lagos state, nigeria. global journal of human social science, 11(2), 42-52. flannagan, j. s., & rockenbaugh, l. (2010). curiosity + kindergarten = future scientists. science and children, 48(4), 28-31. fleer, m., & rillero, p. (1999). family involvement in science education: what are the outcomes for parents and students?. studies in science education, 34(1), 93-114, https://doi.org/10.1080/03057269908560150 gross, d., bettencourt, a. f., taylor, k., francis, l., bower, k., & singleton, d. l. (2020). what is parent engagement in early learning? depends who you ask. journal of child and family studies, 29, 747–760. https://doi.org/10.1007/s10826-019-01680-6 harvard family research project. (2006). family involvement in early childhood education. family involvement makes a difference: evidence that family involvement promotes school success for every child of every age. harvard family research project, 1, 1-8. handayani, d. a. p., wirabrata, d. g. f., & magta, m. (2020). how parents’ academic background can affect parental involvement in preschooler’s education. jurnal pendidikan anak usia dini undiksha, 8(1), 53-60. http://dx.doi.org/10.23887/paud.v8i1.24560 hilado, a., kallemeyn, l., leow, c., lundy, m., & israel, m. (2011). supporting child welfare and parent involvement in pre-school programs. early childhood education journal, 39, 343–353. https://doi.org/10.1007/s10643-011-0471-z hofstein, a., & rosenfeld, s. (1996). bridging the gap between formal and informal science learning. studies in science education, 28(1), 87-112. hu, b. y., zou, x., & ren, l. (2020). parent-child number application activities predict children's math trajectories from pre-school to primary school. journal of educational psychology, 112(8), 1521–1531. https://doi.org/10.1037/edu0000457 jeffries, k. (2012). increasing parental involvement in early childhood education. university of south florida. ivankova, n. v., creswell, j. w., & stick, s. l. (2006). using mixed-methods sequential explanatory design: from theory to practice. field methods, 18(1), 3-20. https://doi.org/10.1177/1525822x05282260 keceli kaysılı, b. (2008). parent involvement to improve academic achievement. ankara university faculty of educational sciences journal of special education, 9(1), 69-86. kelly, l. (2015). is science important in the early years?: exploring good practice. in d. stead, & l. kelly (eds.), inspiring science in the early years (pp. 1-15). new york: open university press. kılıc, r., & unal, m. (2020). investigation of parents’ view’s about science and pre-school science activities (sample of elazığ province). mehmet akif ersoy university journal of the institute of educational sciences, 8(10), 120. kiraz, a., & aytac, p. (2020). practices supporting family involvement in pre-school: family education through science activities. pamukkale university journal of university, 50, 226246. https://doi.org/0.9779/pauefd.561702 kluczniok, k., & roßbach, h. g. (2014). conceptions of educational quality for kindergartens. zeitschrift für erziehungswissenschaft, 17, 145–158. https://doi.org/10.1007/s11618-014-0578-2. lehr, f. (2005). helping your child learn science. washington, d.c.: u.s. department of education, office of communications and outreach. retrieved from https://www2.ed.gov/parents/academic/help/science/science.pdf marcon, r. a. (1999). positive relationships between parent school involvement and public school inner-city pre-schoolers’ development and academic performance. school psychology review, 28, 395–412. https://doi.org/10.1080/02796015.1999.12085973 marin, d. c., & bocoş, m. (2017). factors which influence the involvement of the family in their children’s education at the beginning of the romanian primary education. educatia 21 journal, 15(5), 35–39. https://doi.org/10.24193/ed21.2017.15.05 martinez, a. (2015). parent involvement and its affects on student academic achievement. (doctoral dissertation). national science teachers association [nsta] (1994). parent involvement in science education. nsta board of directors. retrieved from https://doi.org/10.11616/abantsbe.256 https://doi.org/10.1080/09243453.2019.1646293 http://dx.doi.org/10.1007/s10643-014-0660-7 https://link.springer.com/journal/10826 https://doi.org/10.1007/s10643-011-0471-z https://doi.org/10.1007/s10643-011-0471-z https://www2.ed.gov/parents/academic/help/science/science.pdf https://doi.org/10.1080/02796015.1999.12085973 https://doi.org/10.24193/ed21.2017.15.05 journal of science learning article doi: 10.17509/jsl.v5i1.33912 153 j.sci.learn.2022.5(1).141-153 http://wupcenter.mtu.edu/education/ed3510/2006/nstaparent_involvement_in_science_educ.pdf national science teachers association [nsta] (2009). parent ınvolvement in science learning. nsta board of directors. retrieved from https://www.nsta.org/nstas-official-positions/parent-involvementscience-learning oecd. (2018). the future of education and skills education 2030: the future we want. retrieved from https://www.oecd.org/education/2030-project/ patton, m. q. (2014). qualitative research & evaluation methods [nitel araştırma ve değerlendirme yontemleri]. enhancing the quality and credibility of qualitative studies [nitel analizin kalitesinin ve i̇nanırlığının artırılması]. (celik, s. & ö. f. karatas, trans.). in m. bütün & s. b. demir (trans. eds.), (pp. 541-598). pegem academy publishing. paulu, n., & martin, m. (1992). helping your child learn science. usa: oxford university press. perera, l. d. (2014). parents’ attitudes towards science and their children’s science achievement. international journal of science education, 36(18), 3021-3041. https://doi.org/10.1080/09500693.2014.949900 powell, d. r., son, s., file, n., & san juan, r. r. (2010). parent–school relationships and children’s academic and social outcomes in public school pre-kindergarten. journal of school psychology, 48(4), 269–292. https://doi.org/10.1016/j.jsp.2010.03.002 preston, j. p., macphee, m. m., & roach o’keefe, a. (2018). kindergarten teachers’ notions of parent involvement and perceived challenges. mcgill journal of education / revue des sciences de l’éducation de mcgill, 53(3). https://doi.org/10.7202/1058416ar prieto, j. p. (2018). enhancing the quality of early childhood education and care: ecec tutors’ perspectives of family engagement in spain. early child development and care, 188(5), 613-623. https://doi.org/10.1080/03004430.2017.1417272 rodriguez, a. j., collins-parks, t., & garza, j. (2013). interpreting research on parent involvement and connecting it to the science classroom. theory into practice, 52(1), 51-58. https://doi.org/10.1080/07351690.2013.743775 sackes, m. (2014). parents who want their prek children to have science learning experiences are outliers. early childhood research quarterly, 29(2), 132–143. http://dx.doi.org/10.1016/j.ecresq.2013.11.005 sahin, u. (2019). parents' participation types in school education. international journal of educational methodology, 5(3), 315-324. sahin, f. t., & kalburan, f. n. c. (2009). parent education programs and their effectiveness: what type of programs are implemented in the world?. pamukkale university journal of education, 25(25), 1-12. sahin, c., uludag, g., gedikli, e., & karakaya, l. (2018). developing of the scale on determining parents’ views about science and pre-school science activities. kastamonu university kastamonu education journal, 26(1), 101-108. http://dx.doi.org/10.24106/kefdergi.375673 salli, d., dagal, a. b., kucukoglu, e. k., niran, s. s., & tezcan, g. (2013). the concept of recycling in pre-school: example of a project based family involvement programme. journal of research in education and teaching, 2(2), 234-241. sawah, i., & clark, a. (2015). the everything stem handbook: help your child learn and succeed in the fields of science, technology, engineering, and math. usa: f+w media. teddlie, c., & yu, f. (2007). mixed methods sampling: a typology with examples. journal of mixed methods research, 1(1), 77-100. https://doi.org/10.1177/1558689806292430 tekerci, h. (2020). erken çocukluk dönemi fen eğitiminde ebeveynin rolü [role of parent in science education in early childhood]. in g. uludağ (ed.), erken çocukluk döneminde fen eğitimi: çocuğun keşif yolculuğu [science education in early childhood: child's journey of discovery], (pp. 403-432). ankara: nobel academic publishing. uludag, g., & erkan, n. s. (in press). effect of the science education program with the activities in the out-of-school learning environments on the science process skills of the 60-72 months old children. hacettepe university journal of education, x(x). https://doi.org/10.16986/huje.2020064760 unuvar, p. (2010). comparison of parent and teacher opinions about family involvement studies. kastamonu journal of education, 18(3), 719-730. veziroglu, m. (2011). fen eğitiminde ailenin ve toplumun rolü [the role of community and society in science education]. in b. akman, g. uyanık balat & t. güler (eds.), okul öncesi dönemde fen eğitimi [science education in preschool], (2nd ed.) (pp.163-180). ankara: pegem academy publishing. wasik, b. a., bond, m. a., & hindman, a. (2002). contemporary perspectives on early childhood curriculum. in o. n. saracho & b. spodek (eds.), effective early childhood curriculum for children at risk. iap. appendix 1. semi-structured interview questions parent in the interview: mother () father () mother/father’s age: ……… child's age: ……… (in months) education status (for mother): educational status (for father): 1. have you received science education throughout your education life? yes no ( ) 2. do you think that the science education you received is sufficient? yes no ( ) 3. if the answer is yes, what was the training content you received? due to which feature do you think the science education you received is sufficient? 4. do you think science education is necessary? why is that? 5. do you follow the developments/news about science and technology? why? 6. do you think your child should have a science education? why is that? 7. do you do science activities with your child? yes no ( ) i. if the answer is yes; can you give an example? ....................... ii. if the answer is no; why is that? …………………………………… 8. do you have any information about your child's pre-school science activities? yes no ( ) i. if the answer is yes; what kinds of activities are held? can you give an example? ii. if the answer is yes; how do you get information about these events? (by sharing the child with bulletins, getting information from the teacher, following the education program, etc.) 9. does your child share science activities at pre-school with you? how? 10. do you involvement in science activities at pre-school? yes no ( ) i. if the answer is yes; how? ………………… .. (e.g., providing materials, participating in school, etc.) ii. if the answer is no; why don't you involve? http://wupcenter.mtu.edu/education/ed3510/2006/nsta-parent_involvement_in_science_educ.pdf http://wupcenter.mtu.edu/education/ed3510/2006/nsta-parent_involvement_in_science_educ.pdf https://www.nsta.org/nstas-official-positions/parent-involvement-science-learning https://www.nsta.org/nstas-official-positions/parent-involvement-science-learning https://doi.org/10.1080/09500693.2014.949900 https://doi.org/10.1016/j.jsp.2010.03.002 https://doi.org/10.7202/1058416ar https://doi.org/10.1080/07351690.2013.743775 https://doi.org/10.1080/07351690.2013.743775 https://doi.org/10.1177/1558689806292430 a © 2021 indonesian society for science educator 394 j.sci.learn.2021.4(4).394-411 received: 9 february 2021 revised: 22 may 2021 published: 19 september 2021 managing a discursive journey for classroom inquiry: examination of a teacher’s discursive moves yi̇lmaz soysal1* 1department of elementary education, faculty of education, istanbul aydin university, istanbul, turkey *corresponding author: yilmazsoysal8706@gmail.com abstract this study presents an analysis of teacher discursive moves (tdms) that aid students in altering their thinking and talking systems. the participants were a science who handled the immersion inquiry activities. the primary data source was the video recorded in the classroom. this video-based data was analyzed through systematic observation in two phases comprising coding and counting to reveal the mechanics of the discursive journey. three assertions were made for the dynamics of the discursive jou rney. first, the teacher enacted a wide range of tdms incorporating dialogically/monologically oriented, simplified (observe-comparepredict), and rather sophisticated moves (challenging). the challenging moves were the most featured among all analytical tdms. second, once higher-order categories were composed by collapsing subcategories of the displayed tdms, the communicatingframing moves were the most prominent performed moves. lastly, the teacher created an argumentative atmosphere in which the students had the right to evaluate and judge their classmates and teacher's utterances that modified the epistemic and social authority of the discursive journey. finally, educational recommendations are offered in the context of teachers noticing the mechanics and dynamics of the discourse journey. keywords teacher discursive moves, classroom discourse, social languages, science learning, vygostkian perspective 1. introduction in his seminal work, acts of meaning, jerome bruner (1990) signaled the newly emerged discursive psychology signs. this new psychology advocates the fundamental assumptions of the learning and teaching of vygotskian tenets (vygotsky, 1981; vygotsky, 1978; eun, 2019). in a vygotskian sense, learning signifies acquiring an alternative language incorporating specific thinking and talking styles (bakhtin, 1986; kim & roth, 2018). however, before elaborating on this view of learning, certain concepts must be elaborated to demonstrate the need for the current study. in the vygotskian sense, the meaning-making of a phenomenon can be attained in two planes (vygotsky, 1978): interpsychological (social plane) and intrapsychological (cognitive plane). on the interpsychological plane, a teacher and students can rehearse and perform various social languages (bakhtin, 1986) under diverse semiotic mechanisms (symbols, diagrams, graphics, gestures, intonations, and mimicking) as in the forms of speech genres (wertsch 1991). on the intrapsychological plane, following the internalization of the reproduced phenomena among the group members, individual thinking as the appropriation of the previously negotiated concepts for individualized schemes is performed (vygotsky, 1978). for the interpsychological plane, vygotsky (1987) clarified two terms: spontaneous and scientific concepts. the former "is developed through everyday experience and communication and are formed aside from any process aimed specifically at mastering them" (scott, 1997; p. 16). on the other hand, vygotsky (1987) believed that scientific concepts could be formed through formal instruction as "the birth of the scientific concept begins not with an immediate encounter with things but with a mediated relationship to the object" (p. 219). this differentiation between the spontaneous and scientific concepts implies that there can be different thinking and talking approaches to a phenomenon for different groups. while learners consider and apply spontaneous concepts in explicating a phenomenon, scientists tend to operate a more formal array of terminologies or jargon. in this sense, bakhtin (1986) defined social languages as "a discourse peculiar to a mailto:yilmazsoysal8706@gmail.com journal of science learning article doi: 10.17509/jsl.v4i4.32029 395 j.sci.learn.2021.4(4).394-411 specific stratum of society (professional, age group, etc.) within a given system at a given time" (holquist & emerson, 1981; p. 430). a stratum of society refers to the communities of children, scientists, teachers, or any other specific groups of learners or thinkers and talkers. a social language implies that the same phenomenon can be conceived differently by different groups of learners. for instance, a solid-state physicist would consider a glass using the existence of the intermolecular forces and interactions among these forces in terms of resting in the solid state. on the other hand, a glassblower deals with the artistic aspects of the produced glass. for the solid-state physicist and glassblower, the realities of glass within social, cultural, historical, and contextual worlds influence the ways of thinking and talking. the former discerns the glass through scientific experimenting accompanied by particular discourses (e.g., states of matter, intermolecular forces, and atoms). the latter would discuss how glass blowing should be undertaken to design state-ofart creations. it is needed since s/he has artistic design concerns about shaping the glasses aesthetically by applying specific glass-blowing techniques. this example directly reveals the intimate relationship between thought (ways of thinking) and language (ways of talking). leach and scott (2002) and scott (1998) explained the social languages phenomenon for the school science by defining three social languages for science teaching. these are everyday social languages of the learners, social languages of scientists, and the social languages of school science. first, the everyday social languages of learners refer to spontaneous concepts. second, the social languages of scientists mean the specific and formal ways of meaning-making of the same world which the students inhabit. thirdly, the social languages of school science are delimited. students can use expressions such as "flowers feed on the earth." or "i have consumed my energy today." both of which are far from being scientifically appropriate. however, students using this everyday social language can express the occurrences in their environment and do not feel uncomfortable about this. this is because the learner has observed plants in soil and when s/he adds some nutrients such as water to the soil, and then the plant draws up the nutrients through its roots and grows. moreover, when a child becomes tired after playing tag, s/he can think that the activity was energy-consuming. for the first instance, an expert in plant physiology would account for the feeding of plants by photosynthesis through chemical equations using specific jargon. for the second example, an expert in biological energy systems can explain a human becoming tired by considering the energy transformations. 1.1 the main tension of science teaching and significance of the current study when this is the case, there are two alternative languages: everyday social languages learners and social languages of scientists. when this dichotomy is infused into the instruction, there will be an inevitable discursive tension for the teachers in general and science teachers in particular. science teachers' pedagogical decisions and accompanied actions as teacher discursive moves (tdms) are essential to managing the discursive exchanges. in addition, science teachers should consider the social languages that learners bring to the class. for the curricular reality, a teacher has to convey specific content to students, and the curricular contents are inherently closer to the social languages of scientists. at this point, there are numerous questions to be raised: which social languages of the different groups of learners should be prioritized by teachers during classroom discourse? is there an order of importance between social languages of learner groups regarding the meaningful learning of science concepts? suppose a teacher starts by considering the everyday social languages of learners. what are the ways for her to maintain and finalize classroom discourse (discursive journey, a sequence of discourse) to recognize and appropriate an alternative thinking and talking system in the form of scientists' social languages or social languages of school science? only a few scholars have responded to the questions given above. one example is the work of mortimer and scott (2003). they presented insights into each part of the framework through episode analysis by ignoring any quantitative cumulative analysis of tdms during a discursive journey. they paid less attention to teacher interventions that were revealed by a few categories within their framework. there have been numerous contributing studies exploring tdms in-depth, taking different research purposes into account (e.g., carpenter et al., 2020; kawalkar & vijapurkar, 2013; mcmahon, 2012; mcneill & pimentel, 2010; ng et al., 2021; pimentel & mcneill, 2013; oh, 2010; oh & campbell, 2013; van booven, 2015). however, this was not implemented within a continuum of classroom discourse as a discursive journey. therefore, two scholarly purposes are awaiting further research: 1) to make a fine-grained analysis of tdms displayed during a student-led in-class science inquiry, 2) to demonstrate how a teacher handles and manages a discursive journey or sequence by performing particular tdms. some assumptions regarding the fundamentals of the classroom discourse within a vygotskian sense were initially hypothesized in the current study. firstly, the existence of presumable clashes between the two social languages was accepted. the confrontation of the two social languages can create a genuine discursive tension for teachers. if a teacher allows for the two differentiated journal of science learning article doi: 10.17509/jsl.v4i4.32029 396 j.sci.learn.2021.4(4).394-411 speaking styles, s/he should manage the classroom discourse through specific tdms. this kind of classroom discourse flow (from the everyday social languages of learners to the social languages of school science) depicts a compelling discursive journey. as expected, handling and managing this thorny discursive flow is possible through the aid of particular tdms that should be performed in the appropriate contexts and relevant moments during the negotiations. given the assumptions mentioned above, the research questions of the current study were: which tdms were enacted by an experienced science teacher of sixth-graders during a discursive journey through a student-centered teaching activity? how was the teacher able to manage the discursive journey by initially considering and negotiating the everyday social languages of learners and completed by prompting the students to recognize and appropriate the social languages of school science? this study attaches importance in terms of several aspects. first, the tdms were not considered isolated from the flow of the classroom discourse in the current study. this implies that the present study was not aimed to produce a sole description of the what-aspects of the tdms. in addition, the current study tries to exemplify how a science teacher enacts different versions and combinations of the tdms to persuade students that there may be novel ways of seeing and speaking about natural phenomena. the current study accepts science teaching as convincing a group of learners to think and talk in novel ways to capture different or more explicative conceptual profiles developed and used by experts or scientists. however, it has not been exemplified how a science teacher uses his/her talk moves to press students to get a discursive journey from a narrower conceptual profile to a broader one. in addition, the current study was conceptualized around the teacher noticing term that has been centralized for teacher education research in terms of planning and implementing high-quality, pedagogically-oriented professional development programs. in the context of the current study, the teacher noticing refers that science teachers may not have an explicit understanding and pedagogic cognition about the tdms that are used effectively in convincing students to think and talk in new ways. thus, the current study can be thought of as a prototype in depicturing teacher-led talk initiations purposed to maintain science teaching as persuasion that can be thought of as an alternative teaching philosophy established within the sociocultural paradigm for the conventional conceptual change theory. in other words, the outcomes of the present study can be considered by science teacher educators in designing and implementing talk-based professional development programs where discourses of discursive psychology are featured and handled. 2. literature review to manage a discursive journey, a teacher can perform miscellaneous tdms. these tdms can consist of discursive purposes to lead the students to a single or multiple aspects of the phenomenon or reality under discussion. moreover, while some tdms can permit social and verbal interactions, others can inhibit the interactive discursive exchanges (mortimer & scott, 2003). a teacher can provide information to students in a discursive journey by direct lecturing, logical expositions, or verbal cloze (chin, 2007; mau & harkness, 2020; mcmahon, 2012). a teacher can make solid evaluations of student responses based on the canonical knowledge of science using comprehension checks or affirmation-cumdirect instruction (oliveira, 2010; van booven, 2015). if a teacher employs only knowledge providers and evaluator moves, then the classroom discourse can be considered subject-centered, incorporating only school science social language (bansal, 2018; grinath & southerland, 2019; tabach, hershkowitz, azmon, & dreyfus, 2020). in order to expand the scope of the negotiation, a teacher can elicit student-led utterances (kawalkar & vijapurkar, 2013; wei, murphy, & firetto, 2018). a teacher can also direct student attention to a focal aspect of the discussion through, for instance, introducing a scientific story by selecting students to rehearse it (leach & scott, 2002; soysal, 2020a). after collecting several ideas from the students, a teacher can select, summarise, and consolidate these ideas that would be prominent in determining the other streaming of classroom discourse (oh & campbell, 2013, van booven, 2015; soysal, 2020a). a teacher can demonstrate to students how scientists have studied a phenomenon using the modeling and rehearsing aspects of the processes of science (grey & rogan-klyve, 2018; mcmahon, 2012; oh, 2010). a teacher can also clarify the background thinking, reasoning, or intention of a studentled utterance by explicitly asking for clarification or further elucidation of the proposed utterances (pimentel & mcneill, 2013; soysal, 2021). in order to invoke reflective thinking, a teacher can operate reflective toss or toss-back by throwing the responsibility of learning back to the students (magnusson, 2021; van zee & minstrell, 1997b). furthermore, in associating the student-led utterances with each other, a teacher can prompt for and link students' ideas to build on the shared topics (brown & kennedy, 2011; soysal, 2021). another way of associating the student-led ideas with each other is to engage students in legitimizing the evaluation of the other student's conceptual and procedural discourse (van zee & minstrell, 1997a; soysal, 2021). a teacher can act as a challenger, discussant, or negotiator by posing constructive challenges or debating and applying ideas (mcmahon, 2012; simon, erduran, & osborne, 2006). for an argumentative discourse, a teacher can encourage justified and evidence-based reasoning by journal of science learning article doi: 10.17509/jsl.v4i4.32029 397 j.sci.learn.2021.4(4).394-411 checking the student-led evidence or directly prompting students for justified or warranted reasoning (simon et al., 2006). in particular, for in-depth negotiations of meaning, a teacher can encourage students to monitor discursive events in the classroom discourse to keep them cognitively alive for the internally consistent streaming of classroom discourse or asking about a mind-change that led students to monitor their previous and current thinking (van zee & minstrell, 1997a; soysal, 2020b). the discursive exchanges given above can be more visible within an intellectually comfortable classroom atmosphere. a teacher may maintain neutrality and foster a respectful environment to present changes in previously held thinking and talking (van booven, 2015; van zee & minstrell, 1997a). to summarise, the current studies reveal several aspects of the displayed tdms. there is a need for further research into the ways and extent to which a teacher plays out a particular combination of tdms in convincing students to appropriate novels ways of seeing and speaking about natural phenomena. 3. method 3.1 research design the present study was designed and conducted as a naturalistic inquiry (creswell & poth, 2016) where there was no intervention to the teacher's in-class implementations. in other words, the tdms enacted by the teacher was solely observed and analyzed within its naturalistic setting, where the teacher and students socially and verbally interacted without any external intervention. as a qualitative research methodology (creswell & poth, 2016), by a case study, it was purposed to explore a phenomenon within some particular context in the current study. the phenomenon explored in the current study was the discursive journey where the teacher performed specific tdms to convince the students to think and speak in different ways while considering science concepts. the tdms were explored within a specific context in which the teacher behaved as a discussant, challenger, and negotiator to show the students that their existing mental models as the core component of their alternative conceptions may be less explanatory when it comes to identifying a natural phenomenon that was subjected to science lessons. through a case study (creswell & poth, 2016), in the present study, it was undertaken to explore a variety of lenses to extract multiple facets of the phenomenon under examination. diversifying and qualitatively distinctive categories of the enacted tdms were deeply explored as the multifaceted aspects of the discursive journey. 3.2 participants the participants were an experienced science teacher and 26 sixth-grade students (females = 12, males = 14) aged 11-12 years. they attended a private school that had several instructional and leisure time facilities. the teacher (male) was 33-year-old, and he was a ph.d. student in science education. he had four years of in-class teaching experience with middle schoolers in a state school located in a big city in turkey. he worked for an international project that aimed to disseminate student-centered teaching in the turkish context. he was on a journey to become a science teacher educator. he had joined a professional group that was designing, planning, and implementing higher quality professional development programs for elementary and secondary science teachers. the participant teacher was engaged in designing and implementing professional development activities to enhance novice science teachers' knowledge and skills regarding the student-centered approach of argumentbased inquiry (abi; hand & keys, 1999). after managing university-led workshops for prompting the novice teachers to recognize a new teaching approach, the participant teacher provided on-site professional support for the development of teachers in their schools. thus, the teacher had on-site and on-site in-depth experiences to manage the well-structured abi implementation detailed below. 3.3 abi implementation and the science content the teacher had implemented several abi activities in the project schools, one of which was selected for this study. the content of the implementation was the theory of matter in general and the properties of matter in particular, which was embedded in the available elementary science curriculum content. this topic was selected because it contained an outstanding negotiation of meaning (mortimer, 1998). the discursive purpose of the teacher was to create a discursive atmosphere in which the students would present their theoretical models (buty & mortimer, 2008; ziman, 2001) in response to their conceptual, epistemological, and ontological contradictions revealed by the teacher during the initial phases of the implementation. thus, there were three interwoven phases of the implementation. phase 1: initial social negotiations of meanings in this phase, the teacher produced many challenging discursive moments based on the utterances of the students. he initiated the negotiation with an array of questioning-based discursive exchanges that were mindstretching and thought-provoking for the students. the teacher-led questioning served to show the students that they could hold conceptual, epistemological, and ontological conflicts regarding the properties of matter. therefore, the students had to ponder these contradictions to clarify them by establishing and negotiating their theoretical models. the teacher's aim in this stage was to listen actively to the students' utterances, present their thinking fallacies about atoms and matter to the class, and pose scaffolding questions to guide the students towards an alternative way of thinking about matter and the building blocks. at the end of this phase, the students were asked to design their theoretical models regarding positions journal of science learning article doi: 10.17509/jsl.v4i4.32029 398 j.sci.learn.2021.4(4).394-411 of the atoms and molecules in specific matter or within a solution of two items of matter (salt-water), the existence of the intermolecular forces, particular states of the matters, matter combinations as solutions (salt-water, sugar-water, water-oil), and the solid-liquid-gas states of solutions as the mixtures of different matter (e.g., the solidstate imagining of a salt-water solution). phase 2: students’ modelling the students constructed their models and engaged in reasoning about the models to generate evidence in the form of arguments in this phase. the discursive quality of the next phase (whole group negotiations) was firmly based upon the diversity of the models that the students produced. in this phase, the teacher supported the students in rethinking, redefining, and regenerating their initial models by pointing out the non-functional parts of the models they had created (buty, tiberghien, & le maréchal, 2004; mortimer 1998). some of the student groups were working on the same modeling procedures. the teacher, therefore, prompted the students to alternative model aspects of the phenomenon to create a variation of generated models to augment the scope of negotiation after modeling. phase 3: whole group negotiations during the whole group negotiations, each group engaged in discussions about their theoretical models and presented them to the other groups. the teacher drew up a specific order of presentation of the groups. one of the fundamental purposes of the third phase was to increase the breadth of the student-student negotiations by comparing and contrasting models students produced (cavagnetto, 2010; cavagnetto & hand, 2012). thus, different aspects of the same phenomenon (e.g., representations of the intermolecular forces within a watersalt solution) could be modeled by two different student groups. they might generate distinctive models even though they engaged in mental reflections on the same phenomenon. these examples were the most productive discursive moments for the classroom discourse. the teacher deliberately invited other student groups to criticize, evaluate and judge their classmates' thinking. the groups criticized each other in terms of the relevancy and actuality of the model and the derived arguments regarding matter and properties from the generated model. this resulted in more verbal exchanges between the teacher and students and between the students. every group tried to convince the other groups that their models revealed reality in the best way. 3.4 data collection the abi implementation lasted about 190 minutes, and the discursive exchanges were video recorded in the laboratory. the teacher was aided by an assistant who located the cameras in the best points in the laboratory to capture the discursive exchanges. the assistant also walked around the classroom using the camera to record the oneto-one negotiations. the video recording quality was sufficient for the simultaneous verbal initiations of the students to be differentiated. the class participants had been informed about the video recording purposes when they had completed consent forms agreeing to participate in the study. 3.5 data analysis coding and quantifying by systematic observation before the analysis, the video-taped data was verbatim transcribed. verbal and non-verbal interactions were incorporated into the transcript. analysing non-verbal interactions (gestures, intonation, and body language) helped capture the nuances among the presented tdms. the data was analyzed by systematic observation, a branch of discourse analysis (mercer, 2010; sandoval, kawasaki, & clark, 2021). in the context of the current study, the displayed tdms were allocated to a set of collapsed categories. the primary aim of the categorization was to obtain quantitative proportions regarding the relative occurrences of the tdms. the researcher determined the relative frequencies of talk turns, which he abstracted from the types of utterances given by the teacher. a set of categories was generated into which the whole teacher-led talks could be discerned and classified. discerning the given tdms was very important in avoiding divergent tdms labeled with the same codes (mercer, 2010; sandoval et al., 2021). the researcher had trained himself to allocate any piece of teacher-led talk to a category that had been generated for the tdms. the researcher established the following two control mechanisms. first, the tdms labeled in the same way were continuously compared with themselves to achieve internal consistency. second, a labeled tdm was also continuously compared and contrasted with other possible labels to accomplish an external consistency. the internal consistency showed the homogeneity of the tdms coded with the same code from the catalog. the external consistency displayed the heterogeneity of the presented tdms labeled by exclusively mutual codes from the catalog described below. the developed coding catalogue for a systematic observation, researchers can develop their categorizing system or adopt an "off the shelf" system (mercer, 2010, p. 4; sandoval et al., 2021). for the current study, both theory-based and data-driven codes were operated together for coding and counting. the coding catalog (see table 1) incorporated six higher-order categories, 17 subcategories, and more than 150 analytical codes gathered around the subcategories. the labels within the catalog (table 1) were contextually influenced and were applied in this sense. this implies that throughout the negotiations, the context of the discussion was inherently dynamic and changeable. in particular episodes of the negotiations, the students asserted claims. journal of science learning article doi: 10.17509/jsl.v4i4.32029 399 j.sci.learn.2021.4(4).394-411 thus, the content or orientation of the proposed claim determined the context or streaming of the negotiation (furtak, hardy, beinbrech, shavelson, & shemwell, 2010). for the created and continuously changed contexts of the negotiations, the labels for the tdms were also subject to change even though the teacher presented the same discursive move. for instance, in one episode, the teacher asked the students to undertake a simple observation and make an accompanying prediction. within the same episode context, after a few talk turns, the teacher used the student's simple observation and related prediction to challenge the previously proposed student-led idea. in this situation, even though the teacher asked for the students to make a simple observation and an associated prediction, the label for the displayed move was determined as challenging. because the teacher was pointing out contradictions or detecting the flaws in the claims of the student. several tdms were labeled by different codes even though the teacher presented them within the same verbal utterance to change the context of the negotiation. this discernment increased the accuracy of the calibration of the analysis process and the validity of the internal and external consistency of the catalog (mercer, 2010; sandoval et al., 2021). 4. result and discussion the findings are presented according to the assertions and discussed in this section. #assertion-i: the teacher presented a wide range of tdms incorporating both dialogically and monologically oriented ones. moreover, the teacher displayed simplified tdms such as observecompare-predict and more sophisticated moves, for instance, challenging. as seen in table 1, the teacher presented a broad genre of tdms to manage the discursive journey. the 17 subcategories of the displayed tdms were gathered around six higher-order categories. the percentages of the presented tdms are shown in figure 1. by varying proportions, the teacher was able to diversify the range of his discursive moves. the tdms incorporated both monologic (e.g., knowledge providing and evaluating moves) and dialogic orientations (e.g., evaluating-judgingtable 1 detected higher-order categories and characterizing subcategories for the tdms higher-order categories (label) subcategories as tdms label descriptions knowledge providing & evaluating (kpe) direct affirmation da the teacher accepts the correctness of the student-led response. soft rejection sr the teacher acknowledges and welcomes the student-led response. presents logical expositions ple the teacher tries to present an internally consistent idea or argument observecomparepredict (ocp) asks for making observations mo the teacher prompts the students to make an on-moment observation regarding an event during classroom discourse. asks for simple comparisons sc teacher requests from the students to make a comparison between two cases, objects, ideas, etc. asks for making predictions mp the teacher guides the students to make guesses and projections communicatin g & framing (cof) embodies emb the teacher provides concrete analogies, examples, cases, or ideas to increase students' comprehension. probes pro the teacher asks the student to expand on his/her response, either asking for further explanation or clarification of the student's response. requests for clarification rfc the teacher asks for further clarification and explanation of studentled ideas. monitors mon the teacher encourages students to monitor their understanding of a classmate's thinking. evaluatingjudgingcritiquing (ejc) prompt students for evaluating classmates’ ideas eci the teacher asks students to judge and critiquing ideas, arguments, opinions already mentioned in the discourse. prompt students for evaluating teacher-led ideas eti the teacher prompts the students to evaluate and judging a teacherled argument, example, or idea. challenging (cha) challenges (plays devil’s advocate role) cha the teacher points out counter-arguments, contradictions, and flaws in the argument of the students. asks for alternative points of views apv the teacher seeks alternative student-led responses by collecting ideas through low interanimation managing the discourse's flow (mdf)* consults for determining further negotiation flow cnf the teacher invites the student to determine which negotiation topic should be featured. determining further negotiation flow dnf the teacher selects a specific point of view to negotiate and eliminate others. asks for permission to engage in the negotiation pen the teacher tries to get permission to be involved in the negotiations among the students. journal of science learning article doi: 10.17509/jsl.v4i4.32029 400 j.sci.learn.2021.4(4).394-411 critiquing moves). in addition, the teacher presented both simplified (e.g., observe-compare-predict) and more sophisticated tdms (e.g., challenging). during the implementation, the teacher confirmed the student-led responses. she also asked for simple observations, comparisons, and projections. the teacher also performed communicating-tdms to capture the underlying meanings, assumptions, and reasoning behind the student-led utterances. moreover, the teacher allowed for student-led evaluations and judgments when the students were negotiating their models. in addition, he acted as a challenger to show the students their conceptual, epistemological, and ontological conflicts to carry the negotiation one step further. finally, the teacher also presented particular tdms to manage the flow of the discursive exchanges. during the discursive journey, it appeared that the teacher tried to draw the distinctive social languages together. by performing more dialogically oriented tdms, the teacher seemed to present the everyday social languages of the learners. by displaying more monologically oriented tdms, the teacher intended to invite the students to recognize an alternative thinking-talking system invented and used by scientists. the discursive journey seemed to be characterized by the co-existence and co-operation of the dialogical and monological interactions with the aid of the diversified tdms. this is because the discursive journey was designed to be open-ended to welcome the students' responses and slightly structured to convince them that there can be alternative and more robust social languages in addition to their less explanatory everyday social languages (cavagnetto & hand, 2012). during the discursive journey, there was also a gradual sophistication in the revealed tdms. the more monologic tdms are only associated with the teacher's voice, and the student-led contributions were taken as listening and comprehending what the teacher was saying. on the other hand, the dialogic interactions in this journey can be considered the students' open-ended contributions, which could be independent of the presented tdms. however, this was not the reality of the described discursive journey since there should be interdependence between more-less sophisticated or more-less dialogicmonologic tdms. in the first phase of the implementation, the teacher allowed for more open-ended exchanges through more dialogically oriented tdms. at this stage, the teacher made more space for the students' vocabularies rather than the vocabularies of school science. creating a wide range of tdms might provide a gradual decontextualization implying that the students isolated the descriptions of scientific concepts from their contexts (mortimer & scott, 2003; martin, xu, & seah, 2021). in this discursive journey, the teacher had to display a variety of tdms to guide the students to a transformation in which the learned phenomena from the here-and-now of everyday views were relocated to the generalizable statements of science (scott, 1998; tang, tan, & mortimer, 2021). in this implementation stage, the students considered the newly introduced social language as belonging to others (bakhtin, 1934; barreto et al., 2021). thus, the teacher had to perform more dialogically oriented discursive moves to comprehend the ways of the student-led talking-thinking styles. for this purpose, the teacher presented moves such as prompting the students to clarify their ideas (ng et al., 2021; van zee & minstrell, 1997a), make simple observations, comparisons, or predictions (ng et al., 2021; oh, 2010; oh & campbell, 2013), evaluating, judging and critiquing others (crawford, 2000; ng et al., 2021; van zee & minstrell, 1997b). other arguments can explicate the fruitfulness of the tdms. the teacher had inherent accountability in recontextualizing the de-contextualized students' ideas produced around their theoretical models. the teacher frequently acted as a challenger and negotiator. herein the figure 1 comparison of the occurrences of the tdms (%) 0.98 0.49 6.86 1.96 3.92 10.3 7.35 8.33 13.7 8.33 9.8 4.9 15.73 0.98 0.98 3.43 1.96 0 2 4 6 8 10 12 14 16 18 20 f re q u e n c y a s p e rc e n ta g e s types of the occurred dıscursıve moves journal of science learning article doi: 10.17509/jsl.v4i4.32029 401 j.sci.learn.2021.4(4).394-411 instructional purpose was to convince the students that their everyday speaking did not work well to deal with the recognized conflicts made explicit. in the discursive journey, the teacher innately transmitted particular scientific facts about atoms, molecules, intermolecular forces, and solutions through specific tdms such as direct lecturing (aukerman, johnson, & chambers schuldt, 2017; edwards & mercer, 1987), presenting logical expositions (aukerman et al., 2017; lemke, 1990), providing authoritative narratives (scott, 1998) or presenting verbal cloze (chin, 2007). these particular tdms were gathered around the first higher category (table 1). indeed, the domination of these more dialogically oriented tdms is not in keeping with the discursive journey depicted in the current study (cavagnetto & hand, 2012). in this discursive journey, the students had already negotiated the content regarding the properties of matter. during the whole group negotiations, there were plenty of opportunities for the students to move on to the worlds of scientific thinking and talking as an alternative. this recontextualization was more attainable by presenting more monologic tdms but applied after the ample dialogical interactions (mortimer & scott, 2003; van der veen, dobber, & van oers, 2018). in this context, there were no drawbacks to the dominance of the teachercentered reviewing or wrapping-up discursive moves since, for the students, the newly recognized and accepted speaking system had gradually appeared to belong to them (bakhtin, 1934). #assertion-ii: analytically, challenging tdms were the most featured moves among all presented tdms. when higher-order categories were composed by collapsing subcategories of the presented tdms, communicating and framing tdms were the most prominent performed moves by the teacher. the teacher presented 204 analytically coded tdms that had been collapsed to compose higher-order categories. particular tdms were prominent (figure 1), and certain discursive moves that aimed to challenge the students' ideas (e.g., playing the role of devil's advocate) table 2 example excerpt displaying types of the revealed tdms and patterns of interactions t u r n s p e a k e r utterance pi* tdm** 30 t anyway, let us return to the beginning. now, a friend of yours previously mentioned that atoms are gluey. so let us discuss this idea, what do you think about this? i monitors // consults for determining further negotiation topics 31 s4 it was my idea! in matter, the atoms compose the building blocks of a substance by adhering to each other. that is my idea. r 32 t then atoms are within matter? should i understand this from your idea? fq requests for clarification 33 s4 yes, sir, the atoms are within matter. r 34 t all right, how many atoms? only one atom? fq probes 35 s1 sir, considerably plentiful, millions and millions. r 36 t [showing the pen in his hand…] students, are there atoms within that pen? fq embodies 37 s1 there are, of course. r 38 s2 sir, the pen you are holding is solid; the atoms within it are so closely positioned… i mean, they are very close to each other. r 39 t do you mean that there are atoms within that pen? fq requests for clarification 40 s2 yes! no? r 41 s4 but in the form of particles. r 42 t if that is true, i am now touching the atoms of that pen. is that so? fq requests for clarification 43 s4 in my opinion, there are no atoms on the surface of the pen. otherwise, how could anyone touch the atoms! r 44 t are there no atoms on the surface? or are they positioned within matter or on the surfaces of matter? fq probes 45 s4 inside! r 46 s1 no! on the surface! r 47 s3 inside… no… on the surface! r 48 t is it important whether the atoms are inside or on the surface of matter? as you had mentioned, all matter is composed of atoms. however, then, i am touching that pen's atoms! i could alter the places of the atoms by turning the pen's cover from one side to another. so, are the atoms within matter or dispersed on the surface of the matter, or are the atoms dispersed to all parts of matter? fq challenges (plays devil’s advocate role) 49 s4 all points of matter are filled with atoms. neither inside nor on the surface of a matter. everywhere! r 50 t if that is so, then am i able to touch the atoms? is this what you imply? fq requests for clarification * pi: patterns of interaction; **tdm: teacher discursive move journal of science learning article doi: 10.17509/jsl.v4i4.32029 402 j.sci.learn.2021.4(4).394-411 constituted about 16% of the teacher's presented moves and were most frequently displayed. the other prominent moves were requesting clarification move (13.7%), asking the students to make simple predictions (10.3%), and guiding the students to make judgments, legitimization, evaluations, and criticisms for their classmates' thinking (9.8%). table 2 presents an example in which about eightminute discursive interactions can be traced, showing how the teacher performed the tdms. several columns display the different aspects of a sequence of the classroom activity that incorporated several talk turns (table 2; t: teacher; s1: student-1 who provided an answer initially) from turn-30 to turn-71. the utterances of the contributors are classified as "i" for initiate, "r" for response, "f" for follow-up questioning as a triadic dialogue; irf and that triadic dialogue pattern characterizes any discursive talk (mehan, 1979; mercer & barnes, 2020; sinclair & coulthard, 1975). this sequence (from turn-30 to turn-71) was the second episode in which the teacher pooled the student-led ideas but did not interrogate them. this is redefined as low interanimation of ideas (mortimer & scott, 2003). this implies that teachers gather student ideas without examining them. in other words, teachers only want to be informed about the range of the ideas to eliminate irrelevant ones. beyond, teachers select some other ideas table 2 example excerpt displaying types of the revealed tdms and patterns of interactions (continued) t u r n s p e a k e r utterance pi* tdm** 51 s4 [using a nervous voice tone…] they inside matter! r 52 s2 within matter, sir. r 53 s3 namely, they are inside. the atoms are the raw materials of the substances. r 54 t i have no idea… let us discuss this. what do you think [the teacher uttered by turning to face all the class] fq consults for determining further negotiation topics 55 s4 sir, all i know is that the inside of the pen is composed of atoms, and the outside is filled with particles. r 56 s3 sir, in my opinion, the atoms are inside. r 57 s7 sir, for me, atoms are everywhere. everywhere within matter. since these (atoms) are the building blocks, everything has atoms! r 58 s9 indeed, an atom is a substance that is composed of merged structures. i mean, atoms multiply by merging. r 59 s7 sir, atoms are the tiniest building blocks of all matter. r 60 s9 as i agreed, the thing called an atom is very tiny. so, we cannot see it. nevertheless, they adhere together when forming matter. r 61 t how smart are these atoms? they have been neatly arranged. i mean, is there something that keeps the atoms together? do they come side by side through their consciousness? in your opinion, how would it happen? so symmetrical and smooth… as you see, there are handsome students in the classroom. how did the atoms of their faces come to be side by side? [the teacher poured out some water on a front desk and made an explanation: "it is not spreading out so much. as you know, it will spread out a little more; then, it stops. then, how is it possible for the water to rest in an orderly manner?"] fq challenges (plays devil’s advocate role) // embodies 62 s9 sir, once you poured the water on this place… that puddle has a mass and volume. however, sir, the bonds among the atoms of water are decisive in gathering. it will stay gathered unless we vaporize the puddle. r 63 t i am looking so closely, but i do not see any bounds you mentioned. [the teacher leans towards the desktop and takes a closer look at the puddle] fq challenges (plays devil’s advocate role) 64 s9 sir! because the atoms are already invisible. thus, the bounds are smaller than the atoms, and then they are invisible. r 65 s7 sir, solids are more jointed, but gases are separated, and liquids are relatively separated compared to solids. r 66 s9 yes, sir. the bounds are more pressed for the solids, lesser for the liquids… moreover, the gases are independent. r 67 t all right! i am going to reduce the spaces you mentioned! let us bring the water molecules closer… is that ok now? can i reduce the spaces within the atoms you talked about? [the teacher tried to bring the puddle together with his hand] fq challenges (plays devil’s advocate role) 68 s7 no! sir, there is no such thing! the bounds i mentioned are inside the atoms! r 69 t however, you previously mentioned that everywhere on the matter is covered by atoms. that is where i touch the atoms. so then, am i wrong? fq challenges (plays devil’s advocate role) 70 s7 sir! as we mentioned, "…the particles are more compressed for the solids, lesser for the liquids… furthermore, the gases hold independent particles." r 71 t then, i understand that matter is composed of particles rather than atoms. however, what do you mean by particles? fq reformulates // probes * pi: patterns of interaction; **tdm: teacher discursive move journal of science learning article doi: 10.17509/jsl.v4i4.32029 403 j.sci.learn.2021.4(4).394-411 that are important in fitting the streamlining the classroom discourse to a teaching agenda that was more associated with the social languages of school science. in this sense, within turn-30, the teacher recalled a proposed idea of a student (student-4) and reformulated it. this was a discursive monitoring move to encourage the students to monitor their understanding of a classmate's thinking (phillips-galloway & mcclain, 2020; van zee & minstrell, 1997a). the teacher then asked the students to determine the flow or further aspects of the topic under negotiation (turn-30). in a latent sense, the teacher selected student-4's idea since; she proposed that atoms adhere to each other within matter (turn-31). thus, the teacher wished to discuss intermolecular forces (social languages of school science) or something (everyday social languages of the learners) that holds atoms together. as expected, the proposal of student-4 was closer to the social languages of school science as indirectly presented within the available curriculum. after the response from student-4, the teacher requested clarification from the student concerning whether the atoms are in the substances (turn-32). student-4 claimed that the atoms are within the substances (turn-33). to probe the proposed response, the teacher asked how many atoms there were within a substance if they were embedded in the atoms (turn-34). student-1 immediately responded that there were countless atoms within the materials (turn-33). within turn-36, the teacher tried to prepare the background for the further deeper negotiation initiated in turn-48. within turn-36, the teacher proposed an idea of whether there were atoms in the pen she was holding to make this abstract phenomenon concrete for the sake of the students' reasoning. within turn-37 and turn-38, student-1 and student-2 consecutively provided utterances in response to the teacher-led question. even though the teacher requested clarification using turn-39, she proposed a thoughtprovoking idea by asking, "do you think there are atoms within the pen i am holding?" the theoretical aspect of the teacher-led questioning was revealed when student-2 (turn-40) and student-4 (turn-41) responded cautiously with, "isn't it?". then, using the student-led information, the teacher presented reasoning concerned with atoms based on the students' responses and asked for clarification in terms of whether she was touching the atoms of the pen (turn-41). within turn-43, student-4 withdrew his answer and stated that since it is not possible to touch the atoms, the surfaces of the matters could not consist of any atoms. in order to probe this response, the teacher rehearsed her first question to obtain a more in-depth explanation by trying to increase the possible or alternative responses (turn-44). however, within the next three turns, the student-led responses appeared confused as they provided discordant utterances regarding the positioning of the atoms within the substances (turn-45, turn-46, and turn-47). when the teacher noticed that there were competing responses, he presented the first challenge. the teacher first mentioned the previous arguments of the students concerning atoms and matter. the teacher then proposed a challenging example that pointed to the idea that when he replaces a part of a pen, he moves the atoms of the pen from one side to another (turn-48). he completed his utterance by rehearsing whether there are atoms within all parts of a substance. after the quick response of student-4 (turn-49), the teacher repeated the possibility of whether he could touch the atoms of any matter to understand the background reasoning of the student-led response (turn50). the students provided simplistic responses within consecutive talk turns (51-52-53); however, there was a consensus in the student-led responses implying there are atoms within substances. finally, in turn-54, the teacher invited all class members to contribute again to the classroom discourse by stating the importance of the discussion point concerning the positioning of the atoms within matter. there was an array of student-led responses from turn55 to turn-60 based on the invitation from the teacher. from the evaluation of these six student-led responses, it appears that the consensus was around the idea that atoms are dispersed to all parts of a substance. in order to augment the scope of the negotiation for supporting the further theoretical modeling procedures of the students, the teacher presented the second challenge (turn-61). based on the student-led responses, the teacher explicated his understanding by referring to the idea of whether atoms are conscious entities since they gathered together to create perfectly symmetrical substances. next, the teacher presented an individual think-aloud interrogation. this included the idea of whether atoms gathered together to create smooth figures in nature or just in front of the students in the class. then, the teacher poured some water on a desk placed at the front of the lab, asked how the waterhole stayed static and did not spread, and waited for the students' reactions to this more profound challenge. in turn-62, student-9 gave a sophisticated response referring to the intermolecular forces and the relations between these forces and states of matter. by playing the devil's advocate role, the teacher leaned towards the pool of water and exclaimed that he could not see anything that bound the molecules together (turn-63). within the next three talk turns (64-65-66), two students argued that the naked eye could not observe the bonds among the atoms as atoms are not subjected to their basic observation (student-9). student-7 supported the proposal of student9 by adding the idea that the bonds among the atoms of a substance can be moved apart or come closer based on the states of matter (solid, liquid, and gas formations). finally, journal of science learning article doi: 10.17509/jsl.v4i4.32029 404 j.sci.learn.2021.4(4).394-411 within these three student-led talk turns, student-9 reinforced the proposal of student-7 (turn-66). the teacher listened carefully to the students and reacted to their responses with the challenging move (turn-67). student-7 rigorously objected to the last challenge of the teacher and countered it by defending the idea that the bounds of the atoms are embedded in the water (turn-68). immediately, adopting a background discursive move, the teacher reminded the students that they had claimed that atoms were dispersed to all parts of the substances (turn-69). this move was coded as a challenging move rather than a monitoring move, as shown in the catalog through which the teacher enacted a counterargument. student-7 responded by comparing the distances between the particles of an atom in terms of the three states of matter (turn-70). the teacher then presented the term particle to lead and probe the topic under negotiation, reformulated the student-led response, and initiated the next episode. the students were to discuss the particular states of matter (turn-71). to explicate, in constructing a discursive journey from the everyday social languages of the students to the social languages of school science, the teacher presented challenging tdms for particular purposes. the students were expected to convince the teacher who had adopted a primarily neutral position regarding the student-led claims; however, this did not mean that the teacher could be persuaded with simplified premises. as explained above, the teacher behaved as a class member seeking justified arguments that were rational, convincing, acknowledgeable, and believable. in other words, the teacher offered concrete meta-communicative signals that if the students wanted to be accepted by the teacher and the other members of the class. therefore, s/he had to propose well-structured arguments that might at the least be warranted. these findings should be elaborated, first concerning the nature and necessity of the argumentative discourse for a theoretical modeling process. during the implementation, the students were involved in theoretical modeling to create an argument concerning the nature and structure of matter and its properties. when the students were rigorously challenged, they felt that they had to generate stronger arguments to defend their claims and persuade others regarding the validity of their utterances (manz & suárez, 2018; mcmahon, 2012; simon et al., 2006). through the challenging tdms, all the proposed arguments consisted of resolving the problems or issues emanating from the contradictions. according to bachelard (1968), "two people must first contradict each other if they wish to understand each other. truth is the child of argument, not of fond affinity." (p. 114). in the presence of concrete challenging tdms, the end goal of creating a valid theoretical model was essential to the resolution of the differences in the arguments of the students (erduran, simon, & osborne, 2004; firetto et al., 2019; osborne, erduran, & simon, 2004; simon et al., 2006). challenging tdms were considerably instrumental in aiding the students to select one explanation over another. in addition, this group of moves was enacted to eliminate a student-led explanation regarding the topic under negotiation during the discursive journey. another explanation comes from van eemeren and grootendorst (2004), who defined argument construction as a "verbal, social and rational activity aimed at convincing a reasonable critic of the acceptability of a standpoint by putting forward a constellation of propositions justifying or refuting the proposition expressed in the standpoint." (p. 1). there should be justification and refutation procedures in the student-led argument construction as substantially observed during this discursive journey. they mainly were made possible with challenging tdms (firetto et al., 2019; mcmahon, 2012; simon et al., 2006). thus, the teacher made available and public the conceptual, ontological, and epistemological conflicts of the students (mortimer & scott, 2003) mainly through challenging tdms. in support of this idea, the students proposed explanations in response to the teacher-led questioning. the students inherently used their everyday social languages in meaning-making of the phenomenon presented by the teacher. however, the teacher was able to show the conflicts within the students' everyday thinking and talking when discussing matter and properties. in the presence of the concrete challenges, the students were mainly convinced that their reasoning did not incorporate explanatory and exploratory power in analyzing the unidentified facets of the phenomenon. when this was the case, during the discursive journey, the students tried to appropriate other thinking-talking systems as an alternative to their social languages and including more powerful aspects in accounting for the dynamics of matter and properties (bakhtin, 1934, mortimer & scott, 2003; soysal, 2021). thus, detecting the fallacious aspects within the everyday social languages of the students and prompting the students to embrace a more intellectually stronger social language was more feasible under the guidance of the teacher (engle & conant, 2002; lemke, 1990; mortimer & scott, 2003; soysal, 2021). another aspect of assertion-ii is that when the higherorder categories were composed, in a broader sense, the communicating-framing tdms had the highest occurrence proportion (38%) among the other moves (see figure 2). the communicating-framing moves included four different sub-moves in which the teacher; embodied (7.35%), probed (8.33%), requested for clarification (13.7%), and monitored (8.33%). the teacher presented the two-fold communicatingframing tdms to identify, elaborate, reformulate, probing, and frame the student-led utterances. the teacher conducted the communicating-tdms to clarify and discern the background meaning embedded in the studentjournal of science learning article doi: 10.17509/jsl.v4i4.32029 405 j.sci.learn.2021.4(4).394-411 led utterances (edwards and mercer 1987; lemke 1990; kawalkar and vijapurkar 2013; van booven 2015). after the teacher clarified and probed the student-led utterances, the students could externalize their understanding of the topic under negotiation. the externalization of the background meanings of the student-led responses appeared to be considered influential in extending the scope of the negotiations. the teacher first had to capture the meaning underlying the student-led utterances to determine that they possessed conceptual, epistemological, and ontological cognitive contributions. the primary purpose of presenting the communicatingtdms was to make the student-led utterances available for whole-class members in the process of revoicing (chapin, o’connor, & anderson, 2003; o’connor & michaels, 2019). this was a procedure of the familiarisation of the proposed ideas during the negotiations. when the teacher performed the communicating moves, the underlying meanings of the student-led utterances become apparent to all the students. some studies found that these tdms offered the students opportunities to support, argue or refute their classmates' utterances since the contributors had comprehended each other's utterances (martin & hand, 2009; mcneill & pimentel, 2010; simon et al., 2006). as revealed in this study, first, the teacher and students needed to understand the implicit or explicit lexical orientations of each other's utterances. then, it was possible to detect the fallacious utterances embedded in the student-led utterances' semantic structures. thus, if class members wanted to contradict each other, they first had to find whether the ideas of the others were authentic or whether they had an underlying meaning. the teacher's framing/monitoring discursive move encouraged the students to follow their understanding of a classmate's utterances. the teacher stimulated his students to be cognisant of others' proposed ideas (soysal, 2021; van zee & minstrell, 1997a; 1997b; van zee, 2000). the monitoring move of the teacher provided the students with a metacognitive framing tool (berland & hammer, 2012; ha & kim, 2017) in tracing the occurrences of the classroom discourse (e.g., “if you notice… she is drawing the atoms as she had seen them before, did you notice this?” turn-75). the teacher frequently reminded his students to eliminate particular ideas or attach importance to other ideas during the discursive journey. by way of monitoring discursive moves, the students were led to be conscious of the stream of the featured points during the negotiations (e.g., "for your attention, i am now mixing up these two solid substances and these two solid substances are now mixed, aren't they?” turn-200). in this discursive journey, the teacher repeatedly reminded his students to determine where the context of the negotiation was and what would be the context of the negotiation in further phases of the negotiations (e.g., “we are going to discuss whether making models with play dough reflects reality, whether we are close the actuality and not. but a little later, ok?” turn-275). there were teacher-led foreground and background reminders, allowing the students to monitor which aspects of the everyday social languages were not helpful and to which aspects of the social languages of the school science should they pay attention in illustrating the various features of the matters and properties when designing theoretical models (e.g., “a major question has been raised just now concerning how many salt molecules are inserted into the water molecules” turn-308). the monitoring move provided a frame for the students, allowing them to comprehend the initial conceptual, epistemological, and ontological points they had discussed. in this sense, framing through monitoring characterized the student-led discursive expectations, onmoment conceptual or procedural schemas, and negotiation patterns for what was taking place in the classroom discourse at that moment (ha & kim, 2017). therefore, the students comprehended that the negotiation groups and teacher drove forward certain concepts, explanations, illustrations, drawings, or reasoning while excluding or ignoring others. namely, the monitoring move helped the students in individual sense-making or an on-moment cognitive patterning regarding “what is it that’s going on here?” to summarise, during the discursive journey, both challenging and communicating-framing discursive moves were of significant importance. the teacher brought the student-led cognitive contradictions to the fore through the challenging discursive moves, permitting them to shift from their everyday social languages. however, as shown in this study, these appropriating processes of the students were more viable in the frequent and intentional uses of the communicating-framing moves. #assertion-iii: the teacher created an argumentative and evaluative context in which the students had the right to legitimate, criticize, evaluate and judge their classmates and teacher's utterances that modified the epistemic and social authority discursive journey. for the depicted discursive journey, the teacher needed to present particular discursive moves to prompt the students to evaluate their classmates or teacher's figure 2 collapsed categories of the tdms (%) 8.3 16.1 37.7 14.7 16.9 6.3 0 10 20 30 40 50 kpe ocp cof ejc cha mdf f re q u e n c y a s p e rc e n ta g e s types of aggregated tdms journal of science learning article doi: 10.17509/jsl.v4i4.32029 406 j.sci.learn.2021.4(4).394-411 utterances. this group of tdms is labeled as evaluatingjudging-critiquing (ejc; table 1). the ejc moves occupied an important place within the moves presented by the teacher (14.7%, figure 2). the teacher prompted the students to evaluate and judge their utterances and of their classmates through the ejc moves (e.g., “is there anyone who wants to pose a question to a classmate? would it happen the way she explained?” turn-88). ejc moves were also used in the case studies carried out by van zee and minstrell (1997a, 1997b) and van zee (2000) in the same manner as in the current study. the teacher placed the route of the discursive journey under the control and guidance of the student-led evaluations, legitimization, criticisms, and judgments. this was more feasible through ejc moves performed by the teacher as a persuasion process. to explicate, the teacher continuously directed the students to judge and evaluate the teacher-led or student-led utterances (“according to what she says, is there a hole in that wall? what do you want to say to her?” turn-98). in an evaluative discursive context, the teacher tried to persuade the students that they already held cognitive contradictions regarding the topic under negotiation ("did you see the modeling of your friends? is there anyone who wants to critique it? or do you have questions for this modeling?" turn-208). the teacher allowed the students to contrast the utterances of others that substantially differed. however, some had to be selected while eliminating others to reach a consensus (e.g., “then, ok. all right, for your consideration, is this a true model that explicates the reality? or does it demonstrate the actuality? so, what are your evaluations, ideas? please talk about them.” turn-220). thus, this was not a conventional classroom discourse; instead, the teacher adapted reflective discourse through the ejc moves. in the setting of this discursive journey, the teacher implemented reflective discourse (chin, 2006; 2007; kelly & licona, 2018) instead of a binary socratic discourse while he executed the ejc moves. socratic discourse is conducted "in a humiliating manner, in front of company, with plenty of irony and sarcasm"; (santas, 1979, p. 5). however, the ejc moves favored another type of discursive intentionality. the teacher acknowledged the student-led responses as valuable contributions by considering the students as co-legitimators or co-evaluators (van booven, 2015). the teacher tried to convince the students that the thinking-talking style of the school science social languages might be more relevant in theorizing. the teacher was not alone in coping with countless counter or alternative student-led ideas in the classroom. the teacher was supported by members of the class in the process of the discursive journey. based on the teacher-led demands, the students made detailed evaluations and judgments when the utterances of their peers were not plausible or incorporated less explanatory power. moreover, when the teacher presented the ejc moves, responsiveness was obtained (studhalter et al., 2021). this shows that the teacher was able to use the student-led information to manage the discursive journey. during the discursive journey, the teacher performed the ejc moves by deliberately associating the various student-led ideas with each other, which led to the students pondering on what we know and why we believe the utterances of others (berland & hammer, 2012; colley & windschitl, 2021; hutchison & hammer, 2010; scherr & hammer, 2009). thus, in the negotiations, the teacher did not act as the sole person who decided what was correct and why this was so (cazden, 1988; colley & windschitl, 2021; lemke, 1990); instead, the student body was the co-evaluators and cojudgers. the presence of the co-evaluators and co-judgers of the discursive journey signified that there were co-authorities of the journey. the teacher’s ejc moves had the potential to modify the power relations. this implies a novel reconfiguration of the power allocations in shifting the speaking styles of the students. for this discursive journey, there are two authorities: epistemic and social authority (berland & hammer, 2012; maeng, 2021). a teacher can prioritize what counts and what does not count as an intellectual contribution, which implies the epistemic authority of classroom discourse (lefstein, 2008; lin, 2007; maeng, 2021). this provides a discursive control mechanism and places the teacher as an undisputable primary knower. suppose a teacher monopolizes this epistemic authority (berland & hammer, 2012). in that case, s/he also creates the social authority, and therefore determines, for example, the students' talk turns or permits students to say something about the topic under negotiation. in this discursive journey, the teacher allocated his epistemic and social authority through the ejc moves. once the teacher allocated his authority, there was more than one authority to manage the classroom discourse. first, the students had the epistemic authority in that they were guided in legitimizing what (student-led utterances) counted as plausible in the theoretical modeling. in other words, they had opportunities to determine whether a student-led utterance would be acknowledged when the teacher said, "so, what do you want to say, class? do you agree with her comment? i mean, is it possible that it occurs as your classmate mentioned?” (turn-224). for this discursive journey, the social aspect of authority was monitored by taking the allocations of patterns of interactions (talk turns) into account (jin, wei, duan, guo, & wang, 2016) and the basic frame in which to fragment the teacher-student interactions was a triadic dialogue (mercer & dawes, 2014). the dominative triadic dialogues mainly occurred as irf questioning with the teacher initiating the verbal exchanges through, for instance, a question (i); then, the students responded (r), journal of science learning article doi: 10.17509/jsl.v4i4.32029 407 j.sci.learn.2021.4(4).394-411 and lastly, the teacher posed another contingent question (f) based on the students' previous responses (lemke, 1990). during this discursive journey, the interactions between the teacher and students occurred as open-ended chains: irfrfrf. beyond the teacher executing the ejc moves, the streaming of the patterns of interactions was anticipated to be sequenced as tsss…sss (t: teacher; s: student). thus, the social negotiations of meanings were mainly carried out among the students as they were expected to hold the social authority by legitimatizing the thinking of their classmates (maeng, 2021; mcneill & pimentel, 2009). this discursive patterning confirmed the need for a modification in the social authority of the discursive journey in addition to the epistemic authority. a closer look at the patterns of interactions that emerged showed another reality in the current study. teacher-student and student-student exchanges were counted to determine the patterns of interactions, as shown in figure 3. the [t-s] patterning signifies a sole teacherstudent interaction continuing as t-s-t-s-…-t-s with a [t-s-s] pattern revealing a triadic exchange of teacherstudent-student. [t-s-s-s] displays another discursive pattern which is a quadruple of teacher-student-studentstudent. the [s-s] pattern represents a student-led initiation of the dialogue through a question and another student responding to the posed question. in addition, [ss-s] demonstrates a triadic exchange incorporating three different student-led voices. finally, the pattern [s-t] confirms a student-posed question to the teacher, who was required to respond. most verbal exchanges were between the teacher who engaged in follow-up questioning and a student who responded ([t-s]: 62.5%; figure 3). about 15% of all the patterning was detected as [t-s-s], with other types of patterns of interactions being detected at lower frequencies. at first appearance, it seems that the proportions of the patterning of interaction stated above might have a damaging influence on social authority sharing. to explain, when the teacher drew away from the negotiations, the discursive streaming could have been blocked for that type of discursive journey. however, when the teacher noticed that the streaming of the classroom discourse was disrupted, he began to initiate another streaming that was more instrumental in advancing the classroom discourse. in addition, the teacher needed to behave as a moderator and strictly manage purposeful negotiation streaming approximating to the school science social languages. indeed, the teacher wanted to arrive at a position with his students at the end of the negotiations. this seemed more attainable under the purposeful guidance of the teacher who was monitoring and modifying the streaming of the negotiation through a meta-discursive lens. the students provided diversified responses pooled, selected, eliminated, and featured for the discursive journey. the teacher was not the sole person who selected or eliminated the proposed ideas. instead, the teacher recognized and assigned the students as selectors and eliminators mainly through the ejc moves. thus, in the current study, this sharing of the social-epistemic authority authorized the students to evaluate and judge their classmates' ideas, but under the scaffolding guidance of the teacher. this process might be directly reflected in the patterns of the interactions. three seminal works are given below support this finding in the current study concerning the relationship between the ejc moves and epistemic-social authority sharing. firstly, hogan, nastasi, and pressley (1999) undertook a fine-grained analysis of discourse patterns by comparing peer-guided and teacher-guided discussions. in their study, even though more talk took place in small student groups, the students' arguments were under-articulated and piecemeal. the students could not recognize what they figure 3 percentages of patterns of interaction 62.5 14.9 7.74 1.79 1.19 5.95 1.19 0.6 0.6 0.6 2.98 0 10 20 30 40 50 60 70 80 f re q u e n c y a s p e rc e n ta g e s types of patterns of ınteractıon journal of science learning article doi: 10.17509/jsl.v4i4.32029 408 j.sci.learn.2021.4(4).394-411 needed to clarify and present to the group members without guiding tdms. in this sense, the teachers crystallized the need and essence of discussion streaming more precisely for purposeful meaning-making (hogan et al., 1999). thus, ts-dominated discursive patterning cannot confirm a damaging influence on social authority sharing, as seen in the current study. once the teacher prompted the students to evaluate and judge their classmates' reasoning, he had to abide by the end purpose of the implementation-defined as transiting the students from a social language to another. secondly, the teacher performed particular tdms that were intrinsic to the current study. the teacher enacted certain tdms in modifying the streaming of the negotiations. the teacher performed managing moves. he consulted the students to determine further negotiation streaming or asked for their permission to engage in the negotiations (table 1). even though the relative percentages of managing moves remained at the lower boundary (6.3%; figure 3), it can be considered that these moves were worthwhile in restructuring and sharing social authority in addition to the ejc moves. this contributed to the ethos of mutual respect in which neither the teacher nor the students were the primary motivators of the initiation of their new topics (christoph & nystrand, 2001; hartl, 2021). instead, during the negotiations, the students were engaged in listening and responding to the utterances of others through the managing (mdf moves, table 1) and ejc moves. in conclusion, the ts-dominated patterning of interactions did not have a damaging influence; instead, it promoted scaffolding and aided in continuing to travel during the discursive journey (christoph & nystrand, 2001; hartl, 2021). thirdly, engle and conant (2002) defined productive disciplinary engagement (pde) as incorporating four principles: problematizing, authority, accountability, and resources. during a discursive journey, the students need to be prompted to undertake intellectual problems, as was the case in the current study in making the student-led cognitive contradictions public (problematizing). after this, the students need to be given authority in addressing the conceptual, epistemological, and ontological contradictions (authority). however, there must be a contract for accountability that implies that "students' intellectual work is made accountable to others and disciplinary norms" (engle & conant, 2002; p. 401). during the discursive journey, the students had to provide plausible arguments to convince their classmates and the teacher. they favored the social languages of the school science in handling the discursive journey. finally, the students must be provided with sufficient resources to understand the first three principles. in the current study, during the modeling processes, the students were provided with adequate materials to establish their theoretical modeling. in terms of sharing the authority through the ejc moves, the second and third guiding principles of pde, in particular, are explanatory in terms of the findings of this study. first, the teacher delivered authority to the students by allowing them to evaluate, judge, criticize and legitimatize the thinking and talking of their classmates. in this way, the teacher assigned the students to be the cojudgers and co-evaluators of the proposed ideas. this (epistemic-social) authority sharing through the ejc moves was undertaken to make the students accountable to others and the disciplinary norms (school science social languages). the teacher had to guide the students to make evaluations and judgments of the proposed ideas. thus, intentionally, as the discussant chair, he directed the students to discuss with each other, creating a discursive environment in which authority sharing and accountability fluctuated during the journey from one social language to another. while the students were held responsible for the evaluations and judgments of the other's thinking and made themselves accountable to the others' thinking and the social languages of school science, the teacher positioned himself as a moderator frequently involved in the negotiations. the teacher, therefore, maintains an internally consistent discursive streaming through the tsdominated exchanges. 5. conclusion and recommendation this study shows several dynamics of a discursive journey managed through particular tdms. a teacher can benefit from diverse tdms, which can be dialogical/monologically oriented and simplistic/sophisticated in the presence of alternative thinking-talking styles. furthermore, shifting an everyday social language into a dissimilar one can be better accomplished in making the student-led conceptual, epistemological, or ontological conflictions public, convincing the students that their social languages tend to be less illustrative in shedding light on phenomena. however, this persuasion process can be more easily attained when the arguers comprehend each other's thinking and talking background intentions and meanings. moreover, a teacher can find proponents (students as coevaluators) that bring to light that the everyday social languages of students cannot be sufficiently instrumental in explicating a phenomenon compared to alternative social languages when s/he presents the ejc moves. finally, in the discursive journey investigated in-depth in this study, the primary issue is that in order to achieve an intellectual consensus, it is necessary to share the epistemic-social authority and configure an ethos of mutual respect rather than creating an environment in which the students only contradict or try to outdo each other. one of the most paramount educational recommendations of the present study is teacher noticing about the dynamics of the discourse journey. this study infers that science teaching is journal of science learning article doi: 10.17509/jsl.v4i4.32029 409 j.sci.learn.2021.4(4).394-411 a multifaceted enterprise in the presence of mutually exclusive thinking and talking systems. therefore, science teachers should be engaged in professional development programs to capture their talk moves' discourse functions within an in-class discourse journey by acting as reflective practitioners. 6. limitation of the study and future direction even though the current study presents an in-depth and fine-grained qualitative analysis of the tdms in the science teaching context, some limitations should be acknowledged and further detailed for future directions. first and foremost, there was only one teacher whose talks moves were investigated in-depth. it would be more illuminating to examine more than a science teacher's tdms. as mentioned, the participatory science teacher was an experienced one. it would be a limitation of the current study not to compare and contrast the diversity of the tdms that might be enacted differently by more experienced and beginner science teachers. in addition, limited verbal data corpus was deeply systematically analyzed in the present study. a more longitudinal data collection, analysis, and interpretation may permit the detecting of additional typologies of the tdms extracted in the present study. a chemistry-based content was used in the implementation to trigger and maintain the verbal interactions. to detect additional aspects of the tdms, biology-related, physics-related, astrophysics-related, or the topics in the earth sciences should be used to see saturated typologies of the tdms based on the concepts under consideration. references aukerman, m., johnson, e. m., & chambers schuldt, l. (2017). reciprocity of student and teacher discourse practices in monologically and dialogically organized text discussion. journal of language and literacy education, 13(2), 1-52. bachelard, g. (1968). the philosophy of no. paris: paris university press. bansal, g. (2018). teacher discursive moves: conceptualising a schema of dialogic discourse in science classrooms. international journal of science education, 40(15), 1891-1912. bakhtin, m. m. (1934). discourse in the novel. the dialogic imagination: four essays (m. holquist & c. emerson, trans.). austin: university of texas. bakhtin, m. m. (1986). speech genres & other late essays (c. emerson & m. holquist, ed. and v. w. mcgee, trans.). austin: university of texas press. barreto, l. p., rodrigues, a. a. d., de oliveira, g. c. b., de almeida, l. t. g., felix, m. a. c., de souza silva, p., ... & mortimer, e. f. (2021). the use of different translation devices to analyze knowledge-building in a university chemistry classroom. research in science education, 51(1), 135-152. berland, l. k., & hammer, d. (2012). framing for scientific argumentation. journal of research in science teaching, 49(1), 68-94. brown, k., & kennedy, h. (2011). learning through conversation: exploring and extending teacher and children’s involvement in classroom talk. social psychology international, 32(4), 377-396. bruner, j. (1990). acts of meaning. cambridge, mass.: harvard university press. buty, c., tiberghien, a., & le maréchal, j-f. (2004). learning hypotheses and an associated tool to design and to analyse teaching-learning sequences. international journal of science education, 26, 579-604. buty, c. & mortimer, e. f. (2008). dialogic/authoritative discourse and modelling in a high school teaching sequence on optics. international journal of science education, 30(12), 1635-1660. cavagnetto, a. r. (2010). argument to foster scientific literacy: a review of argument interventions in k-12 science contexts. review of educational research, 80(3), 336-371. cavagnetto, a., & hand, b. m., (2012). the importance of embedding argument within science classrooms. in m.s. khine (ed.), perspectives on scientific argumentation (pp. 39-53). springer science+business media bv 2012. carpenter, s. l., kim, j., nilsen, k., irish, t., bianchini, j. a., & berkowitz, a. r. (2020). secondary science teachers’ use of discourse moves to work with student ideas in classroom discussions. international journal of science education, 42(15), 25132533. cazden, c. (1988). classroom discourse: the language of teaching and learning. portsmouth, nh: heinemann. chapin, s.h., o’connor, c., & anderson, n.c. (2003). classroom discussions: using math talk to help students learn. sausalito, ca: math solutions publications. chi, m. t. h. (2008). three types of conceptual change: belief revision, mental model transformation, and categorical shift. in s. vosniadou (ed.), international handbook of research on conceptual change (pp. 61-82). new york: routledge. chin, c. (2006). classroom interaction in science: teacher questioning and feedback to students’ responses. international journal of science education, 28, 1315-1346. chin, c. (2007). teacher questioning in science classrooms: approaches that stimulate productive thinking. journal of research in science teaching, 44(6), 815-843. colley, c., & windschitl, m. (2021). a tool for visualizing and inquiring into whole-class sense-making discussions. research in science education, 51(1), 51-70. crawford, b.a. (2000). embracing the essence of inquiry: new roles for science teachers. journal of research in science teaching, 37, 916-937. christoph, j., & nystrand, m. (2001). taking risks, negotiating relationships: one teacher’s transition toward a dialogic classroom. research in the teaching of english, 36, 249–286. creswell, j. w., & poth, c. n. (2016). qualitative inquiry and research design: choosing among five approaches. sage publications. edwards, d., & mercer, n. (1987). common knowledge: the development of understanding in the classroom. london: methuen. eemeren, f.h. van, & groote ndorst, r. (2004 ). a systematic theory of argumentation: the pragma-dialectical approach. cambridge: cambridge university press. engle, r. a., & conant, f. r. (2002). guiding principles for fostering productive disciplinary engagement: explaining an emergent argument in a community of learners classroom. cognition and instruction, 20, 399-484. erduran, s., simon, s., & osborne, j. (2004). tapping into argumentation: developments in the application of toulmin’s argument pattern for studying science discourse. science education, 88, 915-933. eun, b. (2019). adopting a stance: bandura and vygotsky on professional development. research in education, 105(1), 74-88. firetto, c. m., murphy, p. k., greene, j. a., li, m., wei, l., montalbano, c., ... & croninger, r. m. (2019). bolstering students’ written argumentation by refining an effective discourse intervention: negotiating the fine line between flexibility and fidelity. instructional science, 47(2), 181-214. furtak, e. m., hardy, i., beinbrech, c., shavelson, r. j., & shemwell, j. t. (2010). a framework for analyzing evidence-based reasoning in science classroom discourse. educational assessment, (15), 3-4, 175-196. journal of science learning article doi: 10.17509/jsl.v4i4.32029 410 j.sci.learn.2021.4(4).394-411 gray, r., & rogan-klyve, a. (2018). talking modelling: examining secondary science teachers’ modelling-related talk during a modelbased inquiry unit. international journal of science education, 40(11), 1345-1366. grinath a.s., & southerland, s.a. (2019). applying the ambitious science teaching framework in undergraduate biology: responsive talk moves that support explanatory rigor. science education, 103, 92122. ha, h., & kim, h. b. (2017). exploring responsive teaching's effect on students' epistemological framing in small group argumentation. journal of the korean association for science education, 37(1), 63-75. hand, b., & keys, c. (1999). inquiry investigation. the science teacher, 66(4), 27-29. hartl, p. (2021). the ethos of science and central planning: merton and michael polanyi on the autonomy of science. in science, freedom, democracy (pp. 39-67). routledge. hogan, k., nastasi, b. k., & pressley, m. (2000). discourse patterns and collaborative scientific reasoning in peer and teacher-guided discussions. cognition and instruction, 17(4), 379-432. holquist, m., & emerson,c. (1981) glossary for the dialogic imagination: four essays by m. m. bakhtin (m. holquist, ed., m. holquist and c. emerson, trans.). austin: university of texas press. hutchison, p., & hammer, d. (2010). attending to student epistemological framing in a science classroom. science education, 94(3), 506-524. jin, h., wei, x., duan, p., guo, y., & wang, w. (2016). promoting cognitive and social aspects of inquiry through classroom discourse. international journal of science education, 38(2), 319-343. kawalkar, a., & vijapurkar, j. (2013). scaffolding science talk: the role of teachers' questions in the inquiry classroom. international journal of science education, 35(12), 2004-2027. kelly, g. j., & licona, p. (2018). epistemic practices and science education. in history, philosophy and science teaching (pp. 139-165). springer, cham. kim, m., & roth, w. m. (2018). dialogical argumentation in elementary science classrooms. cultural studies of science education, 13(4), 10611085. leach, j. t., & scott, p. h. (2002). designing and evaluating science teaching sequences: an approach drawing upon the concept of learning demand and a social constructivist perspective on learning. studies in science education, 38, 115-142. lefstein, a. (2008). changing classroom practice through the english national literacy strategy: a micro-interactional perspective. american educational research journal, 45, 701-737. lemke, j. l. (1990). talking science: language, learning and values. norwoord, nj: ablex. lin, a. m. y. (2007). what’s the use of “triadic dialogue”? activity theory, conversation analysis and analysis of pedagogical practices. pedagogies, 2(2), 77-94. maeng, s. (2021). explicating epistemic process in elementary students’ language use by practical epistemology and discourse register analyses. research in science education, 51(1), 153-170. magnusson, c. g. (2021). reading literacy practices in norwegian lower-secondary classrooms: examining the patterns of teacher questions. scandinavian journal of educational research, 1-15. manz, e., & suárez, e. (2018). supporting teachers to negotiate uncertainty for science, students, and teaching. science education, 102(4), 771-795. martin, j., xu, l., & seah, l. h. (2021). discourse analysis and multimodal meaning-making in a science classroom: metamethodological insights from three theoretical perspectives. research in science education, 51(1), 187-207. martin, a. m., & hand, b. (2009). factors affecting the implementation of argument in the elementary science classroom. a longitudinal case study. research in science education, 39, 17-38. mau, s. t., & harkness, s. s. (2020). the role of teacher educators and university supervisors to help student teachers reflect: from monological reflection toward dialogical conversation. reflective practice, 21(2), 171-182. mcmahon, k. (2012). case studies of interactive whole-class teaching in primary science: communicative approach and pedagogic purposes. international journal of science education, 34(11), 1687-1708. mcneill, k. l., & pimentel, d. s. (2010). scientific discourse in three urban classrooms: the role of the teacher in engaging high school students in argumentation. science education, 94, 203-229. mehan, h. (1979). learning lessons. cambridge, ma: harvard university press. mercer, n. (2010). the analysis of classroom talk: methods and methodologies. british journal of educational psychology, 80, 1-14. mercer, n., & dawes, l. (2014). the study of talk between teachers and students, from the 1970s until the 2010s. oxford review of education, 40(4), 430-445. mercer, n., & barnes, d. (2020). english as a classroom language. in learning english (pp. 117-149). routledge. mortimer, e. f. (1998) multivoicedness and univocality in classroom discourse: an example from theory of matter. international journal of science education, 20(1), 67-82. mortimer, e., & scott, p. (2003). meaning-making in secondary science classrooms. maidenhead, england: open university press. o’connor, c., & michaels, s. (2019). supporting teachers in taking up productive talk moves: the long road to professional learning at scale. international journal of educational research, 97, 166-175. oh, p. s. (2010). how can teachers help students formulate scientific hypotheses? some strategies found in abductive inquiry activities of earth science. international journal of science education, 32(4), 541560. oh, p.s., & campbell, t. (2013). understanding of science classrooms in different countries through the analysis of discourse modes for building ‘classroom science knowledge’ (csk). journal of korean association for science education, 33(3), 597-625. oliveira, a. w., (2010). improving teacher questioning in science inquiry discussions through professional development. journal of research in science teaching, 47(4), 422-453. osborne, j., erduran, s., & simon, s. (2004). enhancing the quality of argumentation in school science. journal of research in science teaching, 41(10), 994-1020. phillips galloway, e., & mcclain, j. b. (2020). metatalk moves: examining tools for collective academic discourse learning. the reading teacher, 74(3), 305-314. pimentel, d. s., & mcneill, k. l. (2013). conducting talk in science classrooms: investigating instructional moves and teachers’ beliefs. science education, 97(3), 367-394. sandoval, w. a., kawasaki, j., & clark, h. f. (2021). characterizing science classroom discourse across scales. research in science education, 51(1), 35-49. santas, g.x. (1979). socrates. london: routledge. scherr, r., & hammer, d. (2009). student behavior and epistemological framing: examples from collaborative active-learning activities in physics. cognition and instruction, 27(2), 147-174. scott, p. h. (1997). teaching and learning science concepts in classroom: talking a path from spontaneous to scientific knowledge. in linguagem, cultura e cognicao reflexoes para o ensino de ciencias [language, culture and cognition reflections for science teaching]. belo horizonte, brazil: faculdade de educacao da ufmg. scott, p. h. (1998). teacher talk and meaning-making in science classrooms: a vygotskian analysis and review. studies in science education, 32, 45–80. simon, s., erduran, s., & osborne, j. (2006). learning to teach argumentation: research and development in the science classroom. international journal of science education, 27, 137-162. sinclair, j. mch., & coulthard, r.m. (1975). towards an analysis of discourse: the english used by teachers and pupils. london: oxford university press. journal of science learning article doi: 10.17509/jsl.v4i4.32029 411 j.sci.learn.2021.4(4).394-411 soysal, y. (2020a). exploring elementary and middle school science teachers’ metadiscourse moves: a vygotskian analysis and interpretation. learning: research and practice. doi: 10.1080/23735082.2020.1761432. soysal, y. (2020b). investigating discursive functions and potential cognitive demands of teacher questioning in the science classroom. learning: research and practice, 6(2), 167-194. soysal, y. (2021). an exploration of the determinants of middle school students’ argument quality by classroom discourse analysis. research in science & technological education, 1-29. studhalter, u. t., leuchter, m., tettenborn, a., elmer, a., edelsbrunner, p. a., & saalbach, h. (2021). early science learning: the effects of teacher talk. learning and instruction, 71. tabach, m., hershkowitz, r., azmon, s., & dreyfus, t. (2019). following the traces of teachers’ talk-moves in their students’ verbal and written responses. international journal of science and mathematics education, 18, 509-528 tang, k. s., tan, a. l., & mortimer, e. f. (2021). the multi-timescale, multi-modal and multi-perspectival aspects of classroom discourse analysis in science education. research in science education, 51(1), 111. van booven, d. (2015). revisiting the authoritative–dialogic tension in inquiry-based elementary science teacher questioning. international journal of science education, 37(8), 1182-1201. van zee, e. h., & minstrell, j. (1997a). reflective discourse: developing shared understandings in a physics classroom. international journal of science education, 19, 209-228. van zee, e. h., & minstrell, j. (1997b). using questioning to guide student thinking. the journal of the learning sciences, 6, 229-271. van zee, e. h. (2000). analysis of a student-generated inquiry discussion. international journal of science education, 22(2), 115-142. van der veen, c., dobber, m., & van oers, b. (2018). engaging children in dialogic classroom talk: does it contribute to a dialogical self? in the dialogical self theory in education (pp. 49-63). springer, cham. wei, l., murphy, p. k., & firetto, c. m. (2018). how can teachers facilitate productive small-group talk? an integrated taxonomy of teacher discourse moves. the elementary school journal, 118(4), 578609. wertsch, j. v. (1991). a social approach to socially shared cognition. in l. b. resnick, j. m. levine & s. d. teasley (ed.), perspectives on socially shared cognition (pp. 85－100). washington, dc: american psychological association. vygotsky, l. s. (1978). mind in society. cambridge, ma: harvard university press. vygotsky, l. s. (1981). the genesis of higher mental functions. in j. w. wertsch (ed.), the concept of activity in soviet psychology (pp. 144-188). armonk, ny: sharpe. vygotsky, l. s. (1987). thinking and speech (n. minick, trans.). in r. w. rieber & a. s.carton (ed.), the collected works of l. s. vygotsky: vol. 1. problems of general psychology (pp. 39285). new york: plenum press. (original work published 1934). ziman j. (2001). real science: what it is, and what it means. cambridge: cambridge university press. a © 2022 indonesian society for science educator 127 j.sci.learn.2022.5(1).127-140 received: 21 june 2021 revised: 2 october 2021 published: 1 march 2022 science curriculum objectives’ intellectual demands: a thematic analysis yi̇lmaz soysal1* 1istanbul aydin university, higher education studies application and research center, turkey *corresponding author. yilmazsoysal8706@gmail.com abstract science curriculums and curricular materials are essential guidelines in materializing effective science teaching. the primary goal of the current study aims to present a thematic analysis of the last three elementary and middle school science curriculums objectives released in 2013, 2017, and 2018 to determine whether they provide a base for science teachers to design intellectually demanding instructional tasks. this study conducted an in-depth document analysis to describe the curricular themes and objectives' intellectual demands beyond a mere description. moreover, a critical document-based thematic analysis achieved a call for an in-depth interrogation of the intended science curricula. the current study reveals that the explored science curriculums mainly include physics-related and biology-related topics and chemistry-related topics. there is less place for the issues related to astrophysics and earth sciences. although three curricular changes (2013, 2017, and 2018) were actualized to enrich the science curriculums' scope, intellectual capacity, and thematic variation, the conceptual emphasis seemed to be strictly copied over the years. the curriculums under examination appeared to let the teachers design high intellectually demanding tasks to teach science knowledge and epistemic practices, however, to a certain extent. it is concluded that the sharp decreases in the number of objectives observed in the abstraction zone may hinder teachers from generating teaching environments where students can transfer acquired knowledge and practices to external contexts. educational recommendations are offered in the sense of curriculum development and teacher education. keywords science curriculum, curricular objective, intellectual demand, instructional task, science teacher 1. introduction 1.1. background of the study developing countries like turkey have invested heavily in schooling systems since it is considerably related to the economic development and quality of social life. however, despite the efforts around education on both national and international scales, lewin (1993) indicates some severe issues in instructional quality and students' intellectual acquirements in general. brown-acquaye (2001) directly states that these general and particular problems are of either an appropriateness or implementation issue of curricula in general and science curricula in particular. educational policymakers see the value of science education in economic growth and technological modernization to support the problems (koh, tan, & cheah, 2008). unfortunately, most of the developing nations try to use a version of the westernregulated curricula system by, most of the time, simply adapting it into their educational context (cross et al., 2020). on a national and international scale, the curriculum development process has been considered a problematic duty for educators and educational policymakers (cross et al., 2020). for instance, curriculum reforms should incorporate clear research-based evidence or school system outcomes. however, çalik and ayas (2008) reported that over a relatively short time, four major revisions and 11 different versions of the turkish science curriculum were released from 1924 to 2005, with six since 1968. thus, çalik and ayas (2008) concluded that turkish science teachers have never had a chance to implement a particular curriculum entirely before replacing it with a new version. similarly, as turkish science teacher educators, we have never actually managed to conduct careful research to fully delve into a being implemented curriculum before it was changed with a novel version exemplified in the present study (e.g., 2013, 2017, and 2018 curriculum). there has been an ongoing call for educators and educational policymakers to rehabilitate the schooling systems by fostering the curricular objectives and contents. firstly, curriculum development must be considered a mailto:yilmazsoysal8706@gmail.com journal of science learning article doi: 10.17509/jsl.v5i1.35439 128 j.sci.learn.2022.5(1).127-140 needs-based process. secondly, local experience is essential in shaping a science curriculum's frame. thirdly, curricular objectives, contents, and teaching strategies imposed by curriculums should be compact with an appropriate and supporting assessment and evaluation regime. fourthly, teacher professional development should accompany an effort to enrich a curricular paucity. finally, curriculum development, implementation, and assessment take time since it is a longitudinal process (cross et al., 2020). 1.2. justification for the study and theoretical framework in the context of turkish education, radical changes in science curriculums were actualized for elementary and middle school (grades 3-8) two decades ago. these newly designed science curriculums favored co-constructivist pedagogies to teach science concepts and practices (e.g., çalik, alipaşa, & coll, 2007; 2008). thus, turkish science teachers could practice intellectually effective instruction at the classroom level to foster students' cognitive and affective outcomes in light of the reform-based curriculums. one of the prominent parts of an intellectually demanding classroom setting is the task (as an instruction unit; tekkumru-kisa, stein, & schunn, 2015; tekkumrukisa, schunn, stein, & reynolds, 2019). thus, there would be a concrete interaction between intellectually demanding tasks and students' acquisition of science concepts and epistemic practices. the present study conducted a thematic or content-based analysis of the objectives of the elementary and middle school science curriculum. it is needed to determine whether the curriculums provide pedagogic opportunities for turkish science teachers to design and conduct cognitively demanding in-class implementations. morphology of in-class tasks implies a sophisticated relationship between science teachers' pedagogy and curricular materials (dela fuente, 2021; elizabeth, fred, janssen, & van driel, 2016). understanding this relationship attaches importance since an intended curriculum regulates how teaching and learning occur in the classroom (kim, 2019). curriculums are pedagogically guiding collections or intended planning tools (schmidt et al., 2001). this implies that curriculums are a springboard in determining students' opportunities to learn (milner, 2011). a curriculum is "a plan for the experiences that learners will encounter, as well as the actual experiences they do encounter" (remillard & heck, 2014). thus, science teachers should design and implement intellectually demanding tasks for more skill-based instruction based on the objectives in a curriculum. the intellectual capacity of a curriculum is explicitly related to its objectives' demands. a curriculum's objectives may have fluctuations regarding academic demands embedded in them implicitly. for example, recalling an event's date differs from paraphrasing a concept or analyzing an argument regarding cognitive effort an individual displays. examining a curriculum’s intellectual capacity links the teaching processes with a curriculum’s objectives (lee, kim, & yoon, 2015; lee, kim, jin, yoon, & matsubara, 2017). a curriculum may provide diverse educational opportunities to science teachers to sustain a rhythm (dela fuente, 2021; dela fuente & biñas, 2020; mortimer & scott, 2003) when designing and implementing lower-demanding or higher-demanding tasks for optimal student learning. the cognitive load of a curriculum is hidden in its objectives and should sustain a balance regarding intellectual correspondence on the side of students (lee et al., 2015; 2017). it would facilitate working with a homogenous science curriculum that harmonizes its objectives with lower, moderate, and higher cognitive demands. a heterogeneous curriculum incorporates an irregular dispersion of high, medium, or low intellectually demanding objectives. it is unlikely to implement a curriculum with low cognitively demanding objectives effectively. on the other hand, it is still problematic to implement a curriculum incorporating high cognitively demanding objectives pervasively. consequently, presumptive intellectual demand embedded in the objectives of a curriculum should disperse homogeneously to support the highest student acquisition of science concepts and practices. therefore, this study explored whether the 3-8 grade science curriculums implemented in the 2013, 2017, and 2018 years in turkey had an internal balance regarding the cognitive demands embedded in their objectives, prompting teachers to plan and conduct instructional tasks to teach elementary, middle school science. 1.3. curricular objective, instructional task, and intellectual demand the central term of the present study is intellectual demand that is the level or kind of thinking required of students to successfully engage with a science task (tekkumru-kisa et al., 2015). intellectual demand is more about what learners must know and do and productive disciplinary engagement through in-class assignments formatted by the curricular objectives (tekkumru-kisa et al., 2015). increasing intellectual demand presses students to operate several psychological or cognitive abilities, functions, or variables (psychometric variables). the task’s sophistication predicts a task’s intellectual demand. in the present study, three aspects of science teaching (curricular objective, instructional task, intellectual demand) are linked. by reinterpreting curricular objectives, science teachers define the boundaries of in-class implementations that may create diverse cognitive demands on the side of students. examples make the point mentioned above clearer. objective-1: “a student can describe the basic functions of the sense organs.” journal of science learning article doi: 10.17509/jsl.v5i1.35439 129 j.sci.learn.2022.5(1).127-140 objective-2: “a student can do experiments to compare the response rate of different sense organs while reacting to an event requiring rapid reflexes.” in the first objective, students should listen to the teacher, comprehend the functions of the sense organs, and provide relevant examples of the functions based on their everyday observations. however, the second objective would demand different levels and kinds of student thinking (tekkumru-kisa et al., 2015). in the second objective, science teachers should initially encourage students to negotiate which sense organ may be faster than reflexing to a rapidly occurring event compared to the others. moreover, science teachers should guide students to construct hypothetical claims and researchable questions before data collection, analysis, and interpretation. then, teachers should guide students to design and conduct reliable and valid experiments to compare and contrast different sense organs' reflex rates by collecting data from different individuals. in addition, students have to communicate their experimental findings with their classmates. there will be counter-arguments to rebut a group's data-based interpretations, and the group has to convince other groups that their argument is valid and explanatory. in the second objective, students have to handle more cognitive work/effort to understand how the sense organs function in the presence of the different reallife events (e.g., catching a free-falling mass quickly by seeing or hearing). in the second objective, there will be more argumentative discourse within a possibly challenging learning environment where different student groups might draw contradictory conclusions even though they address similar research questions. in the second example, students listen to the teacher and capture what is being uttered. in addition, they have to listen to their classmates, generate evidence-based claims, protect their arguments from others' refutations, criticize others' claims, etc. these cognitive operations will therefore demand higher or highest intellectual activity from students. in conclusion, a curricular objective’s underlying pedagogical aim profoundly influences teachers’ execution of an instructional task. this is explicitly related to what a science teacher may do with an objective that incorporates low or high cognitive, intellectual correspondence when transformed into a teaching process. the outcomes of the present study will be helpful for teachers to realize in what ways and to what extent science curriculums' objectives permit them to produce high and low cognitively demanding instructional settings. this study is also instrumental for curriculum developers to develop a holistic portrayal of the intellectual demands embedded in the examined curricular objectives. in the national context, most studies dedicated to a documentbased analysis of the curriculums tended to analyze only a curriculum in an isolated manner from others (e.g., aktan, 2019; aydin, ayyildiz, & nakiboğlu, 2019; cangüven, öz, binzet, & avcı, 2017; zorluoğlu, kizilaslan, & sozbilir, 2020). the current study takes a longitudinal style. the last three curriculums' objectives were investigated and constantly compared across the years (e.g., 2013, 2017, and 2018) regarding the potential intellectual demand. apart from the previous research, in the present study, the thematic/conceptual background of the objectives was considered to clarify how their intellectual demands vary based on different science content areas (e.g., physicsrelated objectives, chemistry-related objectives; biologyrelated objectives, the objectives found for earth sciences and astrophysics content areas). wan and lee (2020) proposed that a curriculum can be examined thematically regarding different aspects such as contents' coverage, focus, sequencing, and emphasis. the current study considers especially coverage, focus, or emphasis on science content areas' curricular objectives. coverage implies selecting many specific science concepts in a curriculum (wang & mcdougall, 2019). more significant numbers of topics indicate soft focus and vice versa. thus, in the present study, apart from the previous studies (e.g., aktan, 2019; aydın et al., 2019; cangüven et al., 2017; zorluoğlu et al., 2020), it was also possible to detect whether the objectives’ cognitive demands fluctuate across the different content areas or science themes/concepts. there are a few methodologically sound studies on intellectual demands of curricular objectives created in the sense of different science contents areas (e.g., wan & lee, 2020). 2. method 2.1. research design the current study was a document analysis (karppinen & moe, 2019). the researcher’s purpose was not to merely describe the curricular themes and objectives’ intellectual demands. instead, a critical exploration of them was the purpose. in the present study, the document analysis approach was used as a thinking tool to detect the written data trends systematically (e.g., intended curriculums released in 2013, 2017, and 2018). by a theoryladen document analysis, the curriculums’ internal potential in creating spaces for science teachers to plan and implement both low, moderate, and high intellectually demanding tasks for meaningful science teaching was explored in the present study. the analysis of the written documents was theory-laden since specific pedagogical lenses, e.g., the interaction between curricular objective, instructional task, and intellectual demand interpreted above, were treated to describe the curriculums differently to make them more readable to the external users. 2.2. data collection, analysis, and trustworthiness the data corpus was the intended elementary and middle school science curriculums released in 2013, 2017, and 2018, gathered through official correspondence with the national education ministry. the data analysis included six steps detailed below. journal of science learning article doi: 10.17509/jsl.v5i1.35439 130 j.sci.learn.2022.5(1).127-140 extraction: first, the curriculums' objectives were organized, and the unit of analysis was decided as an objective-based examination. the curriculums included objectives classified in cognitive, affective, and psychomotor domains in line with the revised bloom taxonomy (anderson et al., 2001) detailed below. a substantial amount of the objectives, more than 90%examined in the present study, were placed in the cognitive domain category. this step was executed as a selection process to pick out the objectives in the cognitive domain category. finally, being selected objectives were controlled and organized for an in-depth examination. two researchers performed the extraction process, and the inter-coder reliability was 91%. thematic analysis: in the present study, to detect the discipline-based differences, first, the objectives were categorized regarding the science content areas. the curricular units' titles (e.g., "let us know the force") guided this initial and broader classification. then, for a more finegrained description, individual objectives were examined to refine the key concepts or keywords embedded in them (see table 4 for detailed examples). next, for each curricular unit and sub-units, several keywords were reproduced based on the contents of the objectives. finally, the extracted keywords were recategorized around the different science content areas. selection of the tool for analyzing the objectives' intellectual demand: the revised bloom taxonomy (anderson et al., 2001) was selected to use as a thinking tool in mapping out the possible intellectual demands embedded in the objectives. there are, of course, other versions of categories (e.g., marzano and kendall’s (2006) taxonomy, the smith’s taxonomy for mathematics by taking the rationality of bloom’s taxonomy into account (smith et al., 1996), the structure of the observed learning outcome (solo) taxonomy (biggs, 1995)) to examine a curriculum’s objectives’ intellectual demands. the rbt was selected among other assessment tools for some reasons. first, the rbt has been more familiar to researchers to use as an assessment tool (e.g., elmas et al., 2020; toledo & dubas, 2015). several teacher educators have also comprehended it more precisely to present a panoramic picture of an intended curriculum's intellectual capacity. second, the rbt permits researchers to examine a curriculum both analytically and holistically. analytically, individual objectives' intellectual capacity can be more knowable with the rbt. by the holistic manner, e.g., coding and quantifying intellectual capacity, the trends in the objectives' structural and pedagogic characteristics can be traceable or knowable for their users. in addition, the rbt has been central for different research purposes. for example, apart from the curricular studies, the author (2020) used the rbt to analyze a science teacher's in-class questions' intellectual demands that might be instrumental in triggering higher-order student reasoning and intellectual contributions to classroom talks. the rbt consists of two dimensions as knowledge and cognitive processes (anderson et al., 2001). in the present study, the cognitive process dimension was used for more analytical and fine-grained analysis of the objectives. the cognitive process dimension includes six categories: remember, understand, apply, analyze, evaluate, create, and 19 sub-categories characterized by specific action verbs. there is a hierarchy between the categories regarding intellectual demand (anderson et al., 2001). a low intellectually demanding category (understand) occurs before achieving a high cognitively demanding category intellectual demand cognitive process definition perception remember (recognizing, recalling) retrieve knowledge from long-term memory understand (interpreting, exemplifying, classifying, summarising, inferring, comparing, explaining) construct meaning from instructional messages, including oral, written, and graphic communication conception apply (executing, implementing) applying a procedure to a familiar task analyse (differentiating, organizing, attributing) break material into its constituent parts and determine how the parts relate to one another and an overall structure or purpose abstraction evaluate (checking, critiquing) make judgments based on criteria and standards create (generating, planning, hypothesizing) put elements together to form a coherent or functional whole; reorganizing elements into a new pattern or structure figure 1 the revised bloom taxonomy (anderson et al., 2001) journal of science learning article doi: 10.17509/jsl.v5i1.35439 131 j.sci.learn.2022.5(1).127-140 (apply), representing a more sophisticated cognitive processing (analyze) of an individual. in the present study, the six categories of the cognitive dimension were divided into three higher-order categories; perception, conception, and abstraction (figure 1), observed in the previous studies (e.g., author, 2021). the perception level includes the intellectual demands at remembering and understanding levels where students perceive and comprehend what aspects of a scientific phenomenon through an in-class implementation developed based on an objective. at the perception level, an in-class implementation demands retrieving knowledge from longterm memory or constructing meaning from teachers' instructional messages from students. the conception level includes the intellectual demands at apply and analysis levels. students must operate a procedure to a familiar task or distinguish relevant from irrelevant parts or necessary from unimportant parts of presented science contents in these levels. an implementation based on an objective at the conception level may demand students to decide how elements fit or function within a structure. the abstraction level incorporates the highest intellectual demands observed at evaluating and creating levels where in-class implementations allow students to transfer their understanding of science concepts and practices to unfamiliar contexts or different problems. an in-class task designed based on an objective at the abstraction level demands students to make judgments based on criteria and standards or put elements together to form a coherent or functional whole and reorganize elements into a new pattern or structure. sample analysis by translating theory into practice: in the previous studies (e.g., elmas et al., 2020; yaz & kurnaz, 2020), the researchers adapted a methodology where an objective is scanned and matched with an action verb stated in the rbt to predict the objective's intellectual demand. this approach is theoretically valid and expectedly common; however, it is practically incomplete. the objectives guide a curriculum's principles as they are translated into practice through in-class implementations by teachers. therefore, a mere action verb matching method will be oversimplified for a fine-grained systematic observation. therefore, it was imperative in the present study to take a hypothetical stance to analyze the presumably created intellectual demands on the side of students through in-class implementations developed based on the curricular objectives. for this purpose, four researchers (two experts in the field of elementary/middle school science teaching and teacher education and two supporting investigators having expertise in the educational sciences) conducted the intellectual demand analysis by taking the possible usability of an objective in the sense of designing an in-class implementation into account. first, all coders scanned an objective, then thought and imagined like a science teacher by frequently asking the question, “if i work with this objective which type(s) of in-class implementation(s) i design and share with my students?" to themselves. this was functional for exploring the curricular objectives' intellectual demands in-depth. next, four coders tried to ponder an objective's possible in-class usage(s) to make a more solid and practice-based estimation of its intellectual demand. finally, a specific thinking-deciding diagram was developed and exemplified in table 1 for the multifaceted intellectual demand analysis. three steps helped define an objective’s intellectual demand using the thinking-deciding diagram: carefully scan an objective, estimate an intellectual demand the objective may possess, justify the estimation by proposing hypothetical instructional designs (table 1). nine hundred twenty-three objectives were analyzed by this method. randomly selected 53 objectives from different curriculums were analyzed with the help of the thinking-deciding diagram by the full participation of the four coders. technical (e.g., pedagogical, conceptual, etc.) issues were resolved through the constant comparisons and negotiations of different understandings on the triadic (scan-estimate-justify): considering an objective > instructional translation: identifying a presumptive in-class implementation by taking the objective's structural, semantic, and pedagogical content into account > determining an estimated intellectual demand of the objective. the remaining 870 objectives were analyzed independently by the researchers. finally, 100 randomly selected thinking-deciding diagrams, including personally justified representations of the analyzed objectives, were reconsidered for predicting intercoder reliability. at the outset, inter-coder reliability was 69%. significantly, the coders had trouble assigning codes to the objectives pitched at the evaluate and create levels. some contradictions were resolved by negotiating divergent meanings on the same objective (reliability coefficient: 81%). considering expert science teachers’ reinterpretations: to foster the credibility of the analysis, 13 expert science teachers voluntarily checked the analyzed data by acting as external audits. each science teacher controlled randomly selected 25 sample analyses completed by the primary coders. first, the teachers had information regarding the analysis sequence as the primary coders' thinking mechanism (thinking-deciding diagram) delivered to the teachers as diagrammatical representations (table 1). they were then requested to determine whether the main coders' decisions are pedagogically plausible and pedagogically appropriate for authentic classroom settings. second, the science teachers performed peer reviews by checking the data analysis and interpretations. the teachers played the devil's advocate role (morse, 2015) and tried to make the primary coders honest. the science teachers jotted down explanations and rigorous questions on the analysis documents to press the primary coders to revise their initial interpretations. therefore, the science teachers were able journal of science learning article doi: 10.17509/jsl.v5i1.35439 132 j.sci.learn.2022.5(1).127-140 to peruse both the process and the product of the account by constantly assessing their accuracy (morse, 2015). quantifying and interpreting: after taking revision suggestions from the external audits, final decisions regarding the pedagogically-oriented intellectual demands of the curricular objectives were made. finally, the systematic observations were quantified to have a more holistic portrayal of the distributions of the intellectual demands across the years and science contents. this was needed to draw a generic picture of the distributions of the low, moderate, and high intellectually demanding curricular objectives. 3. results as seen in table 2, 132 curricular units' objectives were analyzed. on average, the curriculums seem to be table 1 the thinking-deciding diagram as a chain: scan-estimate-justify objective and estimated intellectual demand hypothetical instructional scene staging (justification for the estimation) potential cognitive pathways a student can classify light sources as natural and artificial. u n de rs ta n d if i work with this objective, first, i request students to produce a list of light sources they have experienced. then, through a brainstorming activity, i gather and pool the possible student-led examples of the light sources. then, i invite students to categorize them. the students, of course, may propose different labels for their classifications. for example, suppose the intended classification as "natural vs. artificial" will not be proposed by students. in that case, i will propose this version of categorization, and then i request them to re-classify the light sources based on my proposal. • categorizing • subsuming • contrasting • mapping • matching a student can associate exercise, breathing, and heart rate. a pp ly i believe that this is an objective that may be coded at the application level. to justify, if one wants to make linkages between exercise, breathtaking, and heart rate, s/he has to make direct observations. first, we have to go out of the classroom. imagine that we are on the 1st floor. then, i choose two students to carry out the directions i give them. the direction is that one of them runs to the 2nd floor and returns with the same speed. others may go to the 2nd floor by walking. other students are responsible for recording their observations. alternatively, other students have to collect data from their friends whose different exercises influence their frequency of breath-taking and heart rate. after observations, students will be able to understand how these three different phenomena work together. • comparing • contrasting • illustrating • categorizing • concluding • predicting • executing • implementing • carrying out • using a student can determine the selection criteria for the thermal insulation materials used in buildings. e va lu at e this objective demands many things from students. first, students must know the primary difference between the two terms: heat, temperature. the students also have a knowledge base regarding heat transmission and the conditions in which heat transmission occurs. second, a cumulative talk may propose relevant or irrelevant materials, such as a thermal insulator. then, i demand students to differentiate the examples of the thermal insulator materials from the most effective one to the fewer ones. for this differentiation, they have to warrant their ideas. therefore, they have to design experimental settings to test their initial hypothesis regarding the cost-effectiveness of thermal insulators. then, they have to compare and contrast their experimental results for understanding, analyzing, criticizing, evaluating, and legitimating each other's interpretations. then, collective thinking suggests the selection criteria for the thermal insulation materials used in buildings. • identifying • categorizing • concluding • predicting • comparing • contrasting • executing • implementing • carrying out • using • discriminating • distinguishing • selecting • testing • judging table 2 distribution of the units across three curriculums curriculum physics f(%) chemistry f(%) biology f(%) earth sciences f(%) astrophysics f(%) total f(%) 2013 18(40.9) 8(18.18) 12(27.27) 2(4.54) 4(8.68) 44(100) 2017 22(47.82) 6(13.04) 12(26.08) 2(4.34) 4(8.72) 46(100) 2018 19(45.23) 6(14.28) 12(28.57) 2(4.76) 3(7.16) 42(100) total 59(44.69) 20(15.15) 36(27.27) 6(4.54) 11(8.35) 132(100) journal of science learning article doi: 10.17509/jsl.v5i1.35439 133 j.sci.learn.2022.5(1).127-140 dominated by the physics-based units in addition to the biologybased ones. on average, more than 40% of the units were in the curriculum’s physics concepts. there are fewer chemistry-based units (15.15%) in the curriculum compared to the physics-based or biologybased units (27.27%). on average, more than one-fourth of the units across the curriculums were for the biology concepts. on the other hand, there was less place for the earth sciences (4.54%) and astrophysics (8.35%) units. even though three changes were actualized to improve the science curriculums, the distribution of science contents or conceptual structure of the curriculums were strictly protected over the years. similarly, the physics-based objectives seem to be included in the curriculums more pervasively than other content areas. more than one-third of the 923 objectives were explicitly related to the physics concepts. secondly, the biology-based objectives seem to dominate the curricular objectives’ distribution. even though fewer units (table 2, 15.15%) were for the chemistry-based concepts on average, more chemistry-based objectives were in the curriculums (table 3, 21.66%). the objectives devoted to the earth sciences concepts were under 5% on average across the curriculums. only one-tenth of the objectives in the curriculums were for the astrophysics concepts (table 3). five content areas were characterized by different concepts across the curriculums presented in table 4. the three curriculums showed a similar thematic sequence regarding the extracted five content areas. some specific concepts were strictly rehearsed across the curriculums. for example, three higher-order themes were for the physics content area: force, electricity, light, sound, and energy. in the physics content area, especially force and electricity concepts and sub-concepts, nature’s mechanics (table 4) dominated the objectives’ thematic structure. three themes were for the biology content area: microbiological perspective, macro biological perspective, and ecology. most biology-related concepts in the curriculums appeared, prompting students to comprehend part-whole relations regarding biological systems. there is a similar conceptual sequence for the chemistry-based concepts around three segments: micro world, macro world, and chemistry in daily life. microworld includes an explanatory dimension where students should think and talk about invisible segments of substances. the macro world defines a descriptivefunctional dimension, and the concepts are only experienced or observed at the visible and tangible levels. the macro dimension requires in-depth articulations, table 3 distribution of the objectives’ thematic contents across the curriculums curriculum physics f(%) chemistry f(%) biology f(%) earth sciences f(%) astrophysics f(%) total 2013 109 75 93 26 17 320 2017 122 62 87 9 30 310 2018 114 63 86 8 22 293 total 345(34.06) 200(21.66) 266(28.8) 43(4.65) 69(10.82) 923 table 4 the science concepts characterising the five content areas in the curriculums science content area characterizing topics as key concepts or keywords 2013 curriculum physics force and motion; effects of forces; measurement of force; force and energy; simple machines; electric vehicles; simple electrical circuits; transmission of electricity; electrical energy; electricity in our lives sound and light; lighting and sound technologies; spread of light and sound; reflection and refraction of light biology microscopic creatures five senses; the journey to the world of the living; systems in our body; reproduction, growth, and development in living things human and environment relations, living things, and their energy relationships chemistry particulate state of matter; substances and energy (heat, energy, etc.) substances and their general features; state change of substances; states of matter and heat earth sciences the earth’s crust and the movements of our earth; earthquakes the motion of the earth around its axis; weather events astrophysics planet earth; interactions between sun, earth, and moon solar system and beyond journal of science learning article doi: 10.17509/jsl.v5i1.35439 134 j.sci.learn.2022.5(1).127-140 explications, or causations of concepts at the micro-level. earth sciences content area incorporated two thematic parts: a formation mainly related to nature’s materialistic history and earth’s cycles as laws of nature. lastly, the astrophysics content area also included two thematic flows as terrestrial and extra-terrestrial systems. 3.1. horizontal interpretations the 2013 curriculum: table 5 shows the distributions of the objectives’ intellectual demands placed in different science content areas and curriculums. for the 2013 curriculum, about 30% of the physics and biology contents objectives stayed at the perception level (remember, understand). more objectives at the level of perception were for the chemistry topics (37.3%) compared to the physics-based or biology-based contents. on the other hand, the objectives’ intellectual demands were dramatically pitched at the perception level for the earth sciences and astrophysics contents. most of the objectives were evaluated at the conception level (apply, analyze). for instance, more than six out of 10 physics objectives were at the conception level. however, there was a sharp decrease in the objectives evaluated at the abstraction level (evaluate, create) in the 2013 curriculum. less than one-tenth of the objectives of the physics, chemistry, and astrophysics content areas were at the abstraction level. however, especially for the biology-based contents, more objectives table 4 the science concepts characterising the five content areas in the curriculums (continued) science content area characterizing topics as key concepts or keywords 2017 curriculum physics characteristics of forces; force and motion; force and energy; effects of forces; measurement of force and friction; pressure; simple machines electric charges and electric energy; transmission of electricity; electric vehicles; simple electrical circuits lights and sounds around us; lighting and sound technologies; spread of light; sound and its features; interaction of light with matter biology cell division; dna and genetic codes five senses; the journey to the world of the living; foods and biological systems; systems in our body; systems in our body and their health; reproduction, growth, and development in living things human and environment; energy transformations and environmental science chemistry substances and energy (heat, energy, etc.); pure substances and mixtures substances and their general features; state change of substances substances and industry earth sciences the earth’s crust and the movements of our earth seasons and climate astrophysics planet earth; sun, earth, and moon solar system and eclipses; solar system and its beyond 2018 curriculum physics force and motion; force and energy; effects of forces; measurement of force and friction; pressure; simple machines electricity, electric vehicles, simple electrical circuits, electrical circuit components, the transmission of electricity; electric charges, and electric energy lighting and sound technologies; spread of light; sound and its features; interaction of light with matter biology dna and genetic codes cell and its division; foods and biological systems, systems in our body, five senses, the health of systems in our body; reproduction, growth, and development in living things human and environment relations, energy transformations and environmental science chemistry substances and energy (heat, energy, etc.), pure substances and mixtures substances and their general features; state change of substances substances and industry earth sciences the earth’s crust and the movements of our earth seasons and climate astrophysics planet earth; interactions between sun, earth, and moon solar system and eclipses; solar system and its beyond journal of science learning article doi: 10.17509/jsl.v5i1.35439 135 j.sci.learn.2022.5(1).127-140 were found at the abstraction level (22.59%) like the earth sciences objectives (19.27%). the 2017 curriculum: more homogeneous distributions of increasing and decreasing intellectual demands across the content areas were valid in the 2017 curriculum. for the physics, chemistry, and biology content areas, more than 30% of all the objectives were in the perception zone (table 5). however, a substantial amount of the objectives in the earth sciences and astrophysics contents were at the perception zone. in terms of the conception zone, the chemistry and biology contents had a very similar tendency of the objectives' intellectual demand; however, nearly one out of two objectives in the physics contents were at the conception level. the objectives of the astrophysics contents were more frequently observed than the earth sciences contents' objectives in terms of conception zone. regarding the abstraction zone, the objectives in the physics, chemistry, and biology contents were around 20%. this was slightly frequent than the objectives placed in the earth sciences contents and significantly frequent than the astrophysics contents. the 2018 curriculum: on average, 40% of the objectives stayed at the perception level in the physics, biology, and chemistry contents. for the earth sciences and astrophysics contents, the objectives stayed at the perception level dominated the curriculum (table 5). there was a decremental tendency of the objectives pitched at the conception level from the physics (50.87%) to chemistry (44.44%) to biology (37.2%) contents. there were sharp decreases in the earth sciences (12.5%) and astrophysics (27.27%) contents from the objectives stayed at the perception level to the objectives observed at the conception level. the objectives observed in the chemistry and biology contents were significantly higher than the physics-related objectives observed at the abstraction level (table 5). 3.2. vertical interpretations physics content areas: across the curriculums, there was a consistent increase from the former (2013) to the latter curriculums (2017, 2018) regarding the objectives observed at the perception level for this content area (table 5). there was a dramatic decrease from the 2013 curriculum to the 2017 curriculum regarding the conception level. the 2018 curriculum incorporated similar objectives in the physics content areas compared to the 2017 curriculum. regarding the abstraction level, there was an increase from the 2013 curriculum to the 2017 curriculum; however, in the 2018 curriculum, there was a decrease in this level of intellectual demand for the physics contents. chemistry content areas: a consistency (on average, 40%) in the objectives stayed at the perception level across the curriculums for the chemistry-related topics (table 5). there was an apparent decrease from the 2013 curriculum to the 2017 and the 2018 curriculum regarding the conception level. regarding the objectives observed at the abstraction level, there was a significant increase from the 2013 curriculum to the 2017 curriculum. a slight decrease was also observed from the 2017 curriculum to the 2018 curriculum for the objectives of the chemistry topics. biology content areas: regarding the perception level, there was a constant increase from 2013 to the 2017 curriculum, but the increase from 2013 to the 2018 curriculum was more significant than the difference between the 2013 and 2017 curriculums (table 5). there was a consistent decrease from the former to the latter curriculums regarding the conception level. however, regarding the abstraction level, there was a consistency in the amounts of the objectives of the biology topics. table 5 vertical and horizontal frequencies of the objectives regarding year, content area, and intellectual demand curriculum cognitive demand physics f(%) chemistry f(%) biology f(%) earth sciences f(%) astrophysics f(%) 2013 perception 30(28.3) 28(37.3) 29(31.18) 13(50) 8(47.05) conception 69(63.3) 40(53.3) 43(46.23) 8(30.76) 8(47.05) abstraction 10(8.4) 7(9.4) 21(22.59) 5(19.27) 1(5.9) total 109(100) 75(100) 93(100) 26(100) 17(100) cognitive demand physics chemistry biology earth sciences astrophysics 2017 perception 41(33.6) 23(37.09) 33(37.93) 7(77.78) 20(66.66) conception 58(47.54) 25(40.32) 34(39.08) 1(11.11) 9(30) abstraction 23(18.86) 14(22.59) 20(22.99) 1(11.11) 1(3.34) total 122(100) 62(100) 87(100) 9(100) 30(100) cognitive demand physics chemistry biology earth sciences astrophysics 2018 perception 44(38.59) 24(38.09) 37(43.02) 7(87.5) 15(68.18) conception 58(50.87) 28(44.44) 32(37.2) 1(12.5) 6(27.27) abstraction 12(10.54) 11(17.51) 17(19.78) 0(0) 1(4.55) total 114(100) 63(100) 86(100) 8(100) 22(100) journal of science learning article doi: 10.17509/jsl.v5i1.35439 136 j.sci.learn.2022.5(1).127-140 therefore, the objectives observed at the perception level were replaced with the objectives observed at the conception level from 2013 to the 2018 curriculum in the sense of biology topics. earth sciences content areas: there was a sharp increase in the objectives found at the perception level in this content area from 2013 to the 2018 curriculum (table 5). however, there was a more dramatic decrease from the 2013 curriculum to the 2018 curriculum regarding the conception level. this implies that the objectives observed at the perception and conception level were replaced, favoring the perception level for the earth sciences topics. regarding the abstraction level, even though there was a slight decremental tendency from 2013 to the 2017 curriculum, a very sharp decrease was also observed from the 2013 to 2018 curriculum about the objectives observed at the abstraction level (table 5). astrophysics content areas: from the 2013 curriculum to the 2017 curriculum, there was a tremendous increase in the objectives coded at the perception level for this content area. however, there was no substantial change between the 2017 and 2018 curriculums regarding the objectives observed at the perception level (table 5). similarly, regarding the conception level, there was a dramatic decrease from 2013 to the 2017 curriculum; however, the 2017 and 2018 curriculums included relative amounts of the objectives evaluated at the conception level. finally, there was a dramatic change for the objectives at the abstraction level from 2013 to the 2018 curriculum (table 5). the examined objectives in the three curriculums were weighted regarding their presumptive intellectual capacity represented as figure 2 for generating a holistic portrayal. as inferred from the cumulated averages, there was a heterogeneous distribution of the objectives regarding their intellectual demands across the curriculums. in addition, the curricular content areas were dominated mainly by the objectives observed at the perception or conception zones (figure 2, more than 80% of all objectives). as a result, fewer curricular objectives were observed in the abstraction zone in the curriculums. in the 2013 curriculum, more than 85% of all the objectives were at either perception or conception zone. the 2013 curriculum seemed to be dominated by the applying or analyzing level (conception zone). however, the objectives observed at the abstraction zone were significantly rare compared to the perception or conception zone for the 2013 curriculum. the 2017 and 2018 curriculums, on the other hand, appeared to have a higher intellectual capacity regarding the objectives observed at the abstraction zone compared to the 2013 curriculum. thus, the 2018 and 2017 curriculums seemed to have a very similar pattern of intellectual capacity. however, even though there was an increase in the abstraction zone in the latest curriculums compared to the 2013 curriculum, there was also a dramatic incremental tendency in the perception zone in the 2017 and 2018 curriculum compared to the 2013 curriculum. in conclusion, the overall intellectual capacity of the curriculums seemed to be stable, particularly from the 2017 curriculum to the 2018 curriculum that was seemed to be created by simply cloning the 2017 curriculum. 4. discussion in the current student, a specific question is addressed, such as describing the embedded intellectual capacity of the science curricula. it was found that the science curriculums examined in the present study mainly were incorporated physics-related (main themes: force, electricity, and light, sound, and energy) and biology-related topics (main themes: microbiological perspective, macro biological perspective, and ecology) in addition to the chemistry-related topics (main themes: micro world, macro world, and chemistry in daily life). there is less place for the topics related to astrophysics and earth sciences. this infers that although three modifications were actualized to enrich the intended science curriculums, the distribution of the conceptual coverage or emphasis seemed to be strictly protected over the years. the static nature of the curriculums may signal that the curriculum developers did not intend to boost the easiness, meaningfulness, and quality of learning in school subjects that are possible through the wise and emergent modifications of coverage or focus of a curriculum (fortus & krajcik, 2012). it has known that students may have misconceptions, especially regarding the physics-related topics (e.g., i have consumed my energy today) covered by the investigated curriculums (duit & treagust, 2012; vosniadou, 2012). there are conceptual differences between students' existing mental schemes and physics topics teachers try to introduce (vosniadou, 2012). studies showed that, especially for the context of physics-related topics, conceptual, epistemological, and ontological distances figure 2 the general picture of the objectives’ intellectual demands across the curriculums 43.35 40 33.75 42.66 40.96 52.5 13.99 19.04 13.75 0 10 20 30 40 50 60 2018 2017 2013 perception conception abstraction journal of science learning article doi: 10.17509/jsl.v5i1.35439 137 j.sci.learn.2022.5(1).127-140 between the social language of school science and students' every day or primarily incomplete/invalid social languages might emerge. when this distance is the case, a version of learning demand (mortimer & scott, 2003) that is considerably related to the intellectual demand occurs. when there is higher learning demand on the side of students, science teachers have to create high intellectually demanding in-class tasks to invite students to adopt new ways of seeing and understanding science phenomena. once a teaching sequence displays a higher learning demand, it would guide science teachers to take an internally influential role in pressing students for an intentional conceptual change (duit & treagust, 2012; vosniadou, 2012). however, science curriculums must permit teachers to generate high cognitively demanding tasks (luke, 2010; tekkumru-kisa et al., 2019; wan & lee, 2020) to persuade students to think and talk in new ways as social languages of school science. as seen in the present study, physics-related objectives were mainly pitched at the low intellectual demand levels across the curriculums. this was also valid for the biologyrelated and chemistry-related objectives that dominated the curriculums. these results follow the recently reported nation-based studies (aktan, 2019; aydın et al., 2019; cangüven et al., 2017; zorluoğlu et al., 2020). this implies that the curricular objectives had less potentiality in activating a version of higher-order learning demand that is possibly created on the side of students by teacher-led inclass tasks structured based on the objectives. as presented, a more heterogeneous distribution of the curricular objectives regarding the intellectual demand was detected. regardless of the year and science content area variables (table 5), the intellectual demands embedded in the curricular objectives primarily accumulated in the perception (remember, understand) and conception zone (apply, analyze) (figure 2). in the abstraction zone (evaluate, create), there was a sharp decrease across the curriculums (figure 2) and content areas (table 5) compared to the less intellectually demanding objectives observed at the perception or conception zone. the curriculums under examination seemed to let the teachers design and implement high intellectually demanding tasks to teach science knowledge and epistemic practices, however, to a certain extent. the sharp decreases in the number of objectives observed in the abstraction zone may hinder teachers from generating in-class teaching environments where students can transfer acquired knowledge and practices to diverse external contexts. when the thematic emphases of the curricular objectives are considered, especially in the biology-related and chemistry-related topics, a zigzag or back-and-forth sequence was valid. these science content areas seemed to be structured based on the whole-part relationships as the curricular objectives appeared, prompting science teachers to design instructional settings where students connect the micro-worlds and the macro worlds of, for instance, livings things or substances. johnstone (1991; 1993; 2000) proposed two dimensions of teaching and learning chemistry-related concepts: macro and micro, observed in the examined curricular objectives. the macro dimension implies that concepts (five senses, substances, and their general features) are placed in students’ existing mental schemes as workable models. the macro dimension is inherently visible and tangible in students’ experiential imaginations. the micro dimension includes more analytically-oriented worlds of science phenomena, including articulations, explications, or causations of concepts under consideration (gilbert & treagust, 2009). the changes in the micro dimension determine the changes in a related macro dimension as a part-whole relationship sequence. students must establish reality as an interactional mixture of micro and macro dimensions. for doing science, especially in the chemistry-related concepts (e.g., solutions, atoms, molecules), students may hold cognitive difficulties in describing and materializing them (gilbert & treagust 2009). some chemistry-related concepts (e.g., atoms, molecules, intermolecular forces) characterizing the micro dimension may also substantially challenge teachers to teach them. however, these concepts may present more significant teaching opportunities for science teachers by designing higher intellectually demanding tasks (wan & lee, 2020) and attaching micro and macro dimensions. students feel more comfortable comprehending a macro dimension of a science concept that is more concrete or simpler than the micro dimension of the concept. this means that more intellectual demand will occur when science teachers design a teaching task based on the objective indicating a micro dimension of a concept compared to the macro dimension of the concept. students may have difficulties elucidating the macro dimension changes by referring to the micro dimension (gilbert & treagust, 2009). drinking a cup of sugared tea and knowing that its taste is due to sugar is simply about students' personal experiences and expressed through a low intellectually demanding objective structured in the macro dimension observed in the present study. however, understanding the interactions in the micro dimension will be complicated for students to imagine how water and sugar molecules behave differently when they are mixed. a high intellectually demanding objective expresses this. herein, it should be in question whether science teachers can use the mentioned instructional opportunity. to achieve this, first, science teachers should have a conscious awareness (e.g., the curricular objectives stated in micro and macro dimensions may incorporate lower and higher degrees of intellectual demand) regarding micro and macro dimensions observed in the curricular topics. therefore, the present study's outcomes can guide teachers in reading and knowing the journal of science learning article doi: 10.17509/jsl.v5i1.35439 138 j.sci.learn.2022.5(1).127-140 curricular objectives in a new way by attributing the interaction between the micro and macro dimensions. secondly, as detected, the distribution of the curricular objectives was dramatically heterogeneous. low (perception: remember, understand) and moderate (conception: apply, analyze) intellectually demanding curricular objectives dominated the curriculums across the science content areas compared to the highest intellectually demanding ones (abstraction: evaluate, create). thus, it is still a problem since even though the content coverage has flowed as a way in which students may perform inductive/deductive thinking by working with micro/macro dimensions together, the heterogeneous distribution of the intellectual demand may inhibit this highly recommended instructional sequence as a journey between the visible and invisible parts of natural phenomena. the tendencies observed in the curricular objectives’ intellectual capacity also show that intellectual rigor is not trivial (wan & lee, 2020). the intended curriculums explored in the present study signal the apparent lack of intellectually rich and discrepant science content areas. therefore, the present study is an invitation to reconsider the content standards or tendencies and the quality of those standards (schmidt, wang, & mcknight, 2005). the turkish curriculums may contain extensive coverage of science topics. however, another significant matter is whether an instructional opportunity embedded in a curricular objective can guide science teachers to design and conduct an intellectually demanding task. this issue is also visible in the other countries' science curriculums; for instance, it was doubtful whether australian science curriculums would up the intellectual capacity in primary classrooms (luke, 2010). schmidt et al. (2001) reported that the american science curriculums included shallow intellectual capacity in the presence of pervasive coverage of topics, resulting in poor academic performance. this situation is not different in the canadian science curriculums in increasing students' scientific literacy with the low intellectually demanding curricular objectives (fitzpatrick & schulz, 2015). east-asian countries have had the same problem (wei & ou, 2019) to materialize a more demanding science curriculum by inviting science teachers to foster students' cognitive capacity via in-class tasks where intellectual rigor is centralized. in the turkish science curriculums, yaz and kurnaz (2020) concluded that a sharp decrease was visible regarding their intellectualinstructional potentiality and capacity. yaz and kurnaz (2020) drawn attention to the point that one of the significant indicators of poor student performance in the international assessments (e.g., pisa (programme for international student assessment) and timss (trends in international mathematics and science study)) is the intended science curriculums including less knowledge building capacity showed evidently in the present study. conclusion this study concluded that turkey’s last three science curricula implemented might not give teachers some crucial instructional opportunities to handle high cognitively demanding in-class implementations by taking different science-related concepts into account. based on the present study results, for the turkish curriculum development context, it may be time to quit the top-down pressures, adapted for a very long time, by deliberately welcoming in-service science teachers as curriculum designers. teachers critically interact with curriculum resources such as curricular objectives (beyer & davis, 2012a; 2012b). carlgren (1999) indicated that teachers “actively construct, invents, develops, and designs the practice of schooling” (p. 50). therefore, teachers are not mere doers of the curriculum; they are the curriculum designers to generate authentic learning opportunities for students (remillard, 1999). therefore, it will be a strategic tactic for turkish curriculum developers to acknowledge science teachers as translators of curricular contents into teachable entities expected to be intellectually demanding to form the future’s minds. for instance, in the present study, the science teachers were seen as external audits or peer reviewers identifying the intellectual demands embedded in the curricular objectives. in the informal conversations, the supporting teachers declared that they had never read the science curriculums in this way (e.g., analyzing intellectual demand of an objective) introduced to them via the present study. one of the limitations of the current study is that science teachers' in-class implementations are not systematically observed. this would be vital to determine whether a science curriculum would be used in an intended manner or precisely as defined by science teachers. to be clear, it is more about a science teacher's pedagogical capacity as s/he may use a low cognitively demanding objective to design and implement a high demanding instructional scene staging. on the other hand, a science teacher may underestimate a curricular objective's intellectual capacity and plan and conduct a low intellectually demanding instructional sequence based on the objective even though it pitches at the analyze, evaluate, or create levels. therefore, further research must consider the reciprocal interactions between the presumable intellectual capacity of a science curriculum and instructional sequence building capacity and capability of a science teacher since science teachers are well accepted as the translators of the curricular content. moreover, in the current study, only the cognitive dimension of the rbt was used to re-read the intellectual capacity of the science curricula. therefore, a further analysis based on the knowledge dimension of the rbt would also be elucidatory in terms of giving some specific answers to the research questions addressed in the present study. journal of science learning article doi: 10.17509/jsl.v5i1.35439 139 j.sci.learn.2022.5(1).127-140 references aktan, o. (2019). investigation of primary school mathematics curriculum lesson acquisitions according to renewed bloom taxonomy. pau journal of education, 48(1), 15-36. anderson, l. w., krathwohl, d. r., airasian, p. w., cruikshank, k. a., mayer, r. e., pintrich, p. r., ... & wittrock, m. c. (2001). a taxonomy for learning, teaching, and assessing: a revision of bloom’s taxonomy of educational objectives, abridged edition. white plains, ny: longman, 5(1). aydin, a., ayyildiz, y., & nakiboğlu, c. (2019) investigation of the gains of the 2018 science high school chemistry curriculum according to the revised bloom’s taxonomy and comparison with 2018 chemistry curriculum. necatibey faculty of education electronic journal of science and mathematics education, 13(2), 1186-1215. biggs j. (1995). assessing for learning: some dimensions underlying new approaches to educational assessment. alberta j. educ. res., 41(1), 1-17. beyer, c., & davis, e. a. (2012a). developing preservice elementary teachers’ pedagogical design capacity for reform‐based curriculum design. curriculum inquiry, 42, 386-413. beyer, c., & davis, e. a. (2012b). learning to critique and adapt science curriculum materials: examining the development of preservice elementary teachers’ pedagogical content knowledge. science education, 96, 130-157. brown-acquaye, h. a. (2001). each is necessary and none is redundant: the need for science education in developing countries. science education, 85, 68–70. çalik, m., alipaşa a., & coll, r. k. (2007). investigating the effectiveness of a constructivist-based teaching model on student understanding of the dissolution of gases in liquids. journal of science education and technology, 16, 257-270. çalik, m., & alipaşa a. (2008). a critical review of the development of the turkish science curriculum. in r. k. coll & n. taylor (eds.), science education in context: an international examination of the influence of context on science curricula development and implementation (pp. 161-174). rotterdam: sense publishers. çalik, m., alipaşa a., & coll, r. k. (2009). investigating the effectiveness of an analogy activity in improving students’ conceptual change for solution chemistry concepts. international journal of science and mathematics education, 7, 651-676. cangüven, h. d., öz, o., binzet, g., & avcı, g. (2017). examination of ministry of national education 2017 draft science program according to revised bloom taxonomy. international journal of eurasian education and culture, 2, 62-80. carlgren, i. (1999). professionalism and teachers as designers. journal of curriculum studies, 31(1), 43-56. cross, r., bone, e., ampt, p., bell, t., quinnell, r., & gongora, j. (2020). embedding cultural competence in science curricula. in cultural competence and the higher education sector (pp. 255-275). singapore: springer. dela fuente, j. a. (2021). implementing inclusive education in the philippines: college teacher experiences with deaf students. issues in educational research, 31(1), 94-110. dela fuente, j. a. & biñas, l. c. (2020). teachers’ competence in information and communications technology (ict) as an educational tool in teaching: an empirical analysis for program intervention. journal of research in education, science and technology, 5(2), 61-76. duit, r., & treagust, d. f. (2012). how can conceptual change contribute to theory and practice in science education? in second international handbook of science education (pp. 107-118). springer, dordrecht. elizabeth, a. d., fred, j. j., janssen, m., & van driel, j. h. (2016) teachers and science curriculum materials: where we are and where we need to go. studies in science education, 52(2), 127-160. elmas, r., rusek, m., lindell, a., nieminen, p., kasapoğlu, k., & bílek, m. (2020). the intellectual demands of the intended chemistry curriculum in czechia, finland, and turkey: a comparative analysis based on the revised bloom’s taxonomy. chemistry education research and practice, 21(3), 839-851. fitzpatrick, b., & schulz, h. (2015). do curriculum outcomes and assessment activities in science encourage higher order thinking?. canadian journal of science, mathematics, and technology education, 15(2), 136-154. fortus, d., & krajcik, j. (2012). curriculum coherence and learning progressions. in b. fraser, k. tobin, & c. mcrobbie (eds.), second international handbook of science education (pp. 783–798). dordrecht: springer. gilbert, j. k., & treagust, d. (2009). introduction: macro, submicro and symbolic representations and the relationship between them: key models in chemical education. in j. k. gilbert & d. treagust (eds.), multiple representations in chemical education (pp. 1–8). the netherlands: springer. johnstone, a. h. (1991). why is science difficult to learn? things are seldom like they seem. journal of computer assisted learning, 7(2), 7583. johnstone, a. h. (1993). the development of chemistry teaching: a changing response to changing demand. journal of chemical education, 70(9), 701-705. johnstone, a. h. (2000). teaching of chemistry: logical or psychological?. chemistry education: research and practice in europe, 1(1), 9-15. karppinen, k., & moe, h. (2019). texts as data: document analysis. in the palgrave handbook of methods for media policy research (pp. 249-262). palgrave macmillan, cham. kim, y. (2019). global convergence or national variation? examining national patterns of classroom instructional practices. globalisation, societies and education, 17(3), 353-377. koh, t. s., tan, k. c. d., & cheah, h. m. (2008). science education in singapore: meeting the challenges ahead. in r. k. coll & n. taylor (eds.), science education in context: an international examination of the influence of context on science curricula development and implementation (pp. 283–290). rotterdam: sense publishers. lee, y.-j., kim, m., & yoon, h.-g. (2015). the intellectual demands of the intended primary science curriculum in korea and singapore: an analysis based on revised bloom’s taxonomy. international journal of science education, 37(13), 2193-2213. lee, y.-j., kim, m., jin, q., yoon, h.-g., & matsubara, k. (2017). eastasian primary science curricula: an overview using revised bloom’s taxonomy. dordrecht: springer. lewin, k. (1993). planning policy on science education in developing countries. international journal of science education, 15, 1-15. luke, a. (2010). will the australian curriculum up the intellectual ante in primary classrooms?. curriculum perspectives, 30(3), 59-64. marzano r. j., & kendall j. s. (2006). the new taxonomy of educational objectives (2nd ed.). thousand oaks, ca: corwin press. milner, h. r. (2011). culturally relevant pedagogy in a diverse urban classroom. the urban review, 43(1), 66-89. morse, j. m. (2015). critical analysis of strategies for determining rigor in qualitative inquiry. qualitative health research, 25(9), 1212-1222. mortimer, e., & scott, p. (2003). meaning making in secondary science classrooms. mcgraw-hill education (uk). remillard, j. t. (1999). curriculum materials in mathematics education reform: a framework for examining teachers’ curriculum development. curriculum inquiry, 29(3), 315–342. remillard, j. t., & heck, d. j. (2014). conceptualising the curriculum enactment process in mathematics education. zdm mathematics education, 46(5), 705-718. schmidt, w. h., mcknight, c. c., houang, r. t., wang, h., wiley, d. e., cogan, l. s., & wolfe, r. g. (2001). why schools matter: a crossnational comparison of curriculum and learning. san francisco, ca: jossey-bass, a wiley. schmidt, w. h., wang, h. c., & mcknight, c. c. (2005). curriculum coherence: an examination of us mathematics and science content journal of science learning article doi: 10.17509/jsl.v5i1.35439 140 j.sci.learn.2022.5(1).127-140 standards from an international perspective. journal of curriculum studies, 37(5), 525-559. smith g., wood l., coupland m., stephenson b., crawford k. & ball g. (1996). constructing mathematical examinations to assess a range of knowledge and skills. int. j. math. educ. sci. technol., 27(1), 65-77. soysal, y. (2020). investigating discursive functions and potential cognitive demands of teacher questioning in the science classroom. learning: research and practice, 6(2), 167-194. soysal, y. (2021). an exploration of the determinants of middle school students’ argument quality by classroom discourse analysis. research in science & technological education, 1-29. doi: 10.1080/02635143.2021.1908981. tekkumru-kisa, m., stein, m. k., & schunn, c. (2015). a framework for analysing cognitive demand and content-practices integration: task analysis guide in science. journal of research in science teaching, 52(5), 659-685. tekkumru-kisa, m., schunn, c., stein, m. k., & reynolds, b. (2019). change in thinking demands for students across the phases of a science task: an exploratory study. research in science education, 49(3), 859-883. toledo s., & dubas j. m. (2015). encouraging higher-order thinking in general chemistry by scaffolding student learning using marzano’s taxonomy. j. chem. educ., 93(1), 64-69. vosniadou, s. (2012). reframing the classical approach to conceptual change: preconceptions, misconceptions and synthetic models. in second international handbook of science education (pp. 119-130). springer, dordrecht. wan, d., & lee, y. j. (2020). the intellectual demands and coherency of topics of reformed primary science curricula from three east-asian regions. international journal of science and mathematics education, 1-20. wang, z., & mcdougall, d. (2019). curriculum matters: what we teach and what students gain. international journal of science and mathematics education, 17(6), 1129-1149. wei, b., & ou, y. (2019). a comparative analysis of junior high school science curriculum standards in mainland china, taiwan, hong kong, and macao: based on revised bloom’s taxonomy. international journal of science and mathematics education, 17(8), 14591474. yaz, ö. v., & kurnaz, m. a. (2020). comparative analysis of the science teaching curricula in turkey. sage open, 10(1), 2158244019899432. zorluoğlu, s. l., kizilaslan, a., & sozbilir, m. (2016). analysis and evaluation of learning outcomes in high school chemistry curriculum according to revised bloom taxonomy. necatibey faculty of education electronic journal of science and mathematics education, 10(1), 260-279. a © 2022 indonesian society for science educator 176 j.sci.learn.2022.5(1).176-192 received: 1 june 2021 revised: 5 october 2021 published: 1 march 2022 analysis of prospective primary school teachers’ knowledge regarding chemical representations on crystallization experiment melis arzu uyulgan1*, nalan akkuzu güven1 1chemistry education, mathematics and science education department, faculty of education, dokuz eylul university, turkey *corresponding author: melisarzucekci@gmail.com abstract the study aims to determine the subject matter knowledge of prospective primary school teachers (ppsts) and analyze their chemical representation levels on crystallization. the study was carried out with descriptive research with a qualitative approach based on this purpose. the study participants were eighty freshman students studying at the department of primary education in a state university in the aegean region of turkey. the data were collected using a worksheet containing seven openended questions regarding the crystallization experiment. the questions were about the solubility of salt in water, the formation of the salt water and its solubility-temperature graph, the formation of the saturated salt water, and the appearance of particles formed during crystallization. a worksheet was prepared to determine the chemical representation levels of the ppsts, and the data were subjected to document analysis. the researchers conducted a demonstration experiment and an animated video on the extraction of table salt by crystallization as an activity during the study process. the results indicated that ppsts' responses related to crystallization were mainly at the macroscopic level. at the same time, they had great difficulty explaining at the levels of sub-microscopic and symbolic representations. moreover, they could not explain the concept of dissolution with scientific expressions and distinguish the mixtures from each other. additionally, most prospective teachers could not draw the correct solubility-temperature graph, so they had difficulty in symbolic representations. the study results imply that to raise the quality of science education in turkey, ppsts must attend a quality teaching of science, so primary school students acquire scientifically accurate knowledge of the basic science subjects and concepts such as dissolution, solubility, and crystallization. keywords chemical representations, prospective primary school teachers, crystallization, science education 1. introduction the primary education period is the first step where basic education starts, and students acquire various skills, behaviors, and attitudes. among the main goals of this step is to deliver quality science education. science is one of the most important fields that uses observation and experimentation to describe and explain natural phenomena. as can be understood, science has a structure that gives more information about the state of things in macroscopic structures. on the other hand, science learning involves observing various natural phenomena and understanding how natural phenomena exist or occur (merino & sanmarti, 2008). thus science courses also challenge teaching students about phenomena that humans often cannot directly interact with. in this case, it becomes difficult to understand due to its complex structure. therefore, it is extremely crucial to employ representations to realize meaningful learning (johnstone, 2010; lindawati, wardani, & sumarti, 2019). when looked at from the point of science, representations have played an essential role in constructing science knowledge because representations convey knowledge many times about objects which humans cannot directly interact with or quickly summarize (polifka, 2021). conveying information helps support science processes, including learning (wu & puntambekar, 2012) and communication (kozma, 2003). students can develop better scientific understanding by engaging in the construction of representation. representations are essential, especially for students to efficiently learn the basic abstract concepts in scientific fields, differentiate observable and invisible phenomena, and experience correct and meaningful learning in their minds. suppose we recognize the acts of thinking and interrogating each as a building block of meaningfulness (aubrey, ghent, & mailto:melisarzucekci@gmail.com journal of science learning article doi: 10.17509/jsl.v5i1.34772 177 j.sci.learn.2022.5(1).176-192 kanira, 2012). quality science education can offer students various mental processes to acquire higher cognitive skills and representations. at this point, it is highly crucial to use dynamic analogies (hermita et al., 2020), activities such as experiments, animations, educational videos (akkuzu güven & uyulgan, 2021), and online activities (polifka, 2021) to reveal representations and to provide meaningful learning. according to recent studies, these activities help students observe, investigate, and understand the physical world through direct experiences with the phenomena or manipulating objects and materials (akkuzu güven & uyulgan, 2021; banawi, sopandi, kadarohman, & solehuddin, 2019). moreover, primary school teachers are the most important actors who guide science education to ensure that students possess all these acquirements by performing the activities and are raised by the intended goals. the system's success is directly related to teachers' achievement (knowles, holton, & swanson, 2005). therefore, it is critically important that primary school teachers, and indeed, prospective primary school teachers (ppsts) have advanced competency in science teaching to ensure a quality process of science learning. many studies emphasize that this level of education plays a crucial role in raising individuals interested in and exhibiting positive attitudes towards science (duschl, schweingruber, & shouse, 2007; pine, messer, & st. john, 2001; uludüz, 2017). because the theory of particles is a foundation of the mindset in science education that starts from primary school (treagust et al., 2010), understanding the microscopic world and establishing its relationship with other representations is essential for meaningful learning to occur. from this point forth, the present study focuses on ppsts' subject matter knowledge in terms of macroscopic, sub-microscopic, and symbolic levels of chemical representations using demonstration experiments and animated video. specifically, educators need to identify ppsts' basic chemistry knowledge through chemical representations. when they start their professional life, they are the only experts who will teach students with in-depth knowledge of the subjects, not superficial, by using various chemical representations. they have a big part in making science courses more real. in their future professional lives, prospective teachers cannot be expected to teach their students the knowledge and skills that they do not readily have. it is also necessary that they transfer them through practical and meaningful ways. it is therefore crucial that prospective teachers do not have misconceptions and inadequate or inaccurate knowledge of their subject matter (härmälä-braskén, hemmi, & kurtén, 2020). primary school teachers administer basic science education starting from the 3rd and 4th years of primary school. furthermore, it requires ppsts to be trained in a way that equips them with the necessary subject matter knowledge related to basic science concepts (knowles et al., 2005). it is reported that prospective teachers have inadequate or incorrect knowledge of science concepts in the studies covering the northern part of turkey (demircioğlu, demircioğlu, & ayas, 2004) and the south-eastern part of the usa (schulte, 2001). an established finding is that these prospective teachers graduate in an incompetent state of misconceptions and incorrect knowledge of science concepts, resulting in their students carrying similar misconceptions in science (lemma, 2013). a teacher who is competently knowledgeable in the subject matter will undoubtedly tend to be more self-confident, devoted, and conscious about what and how to teach. therefore, among the essential characteristics expected of teachers is to possess substantial subject matter knowledge (aydeniz, bilican, & kirbulut, 2017). the initially-acquired knowledge and affective behaviors of science could be had at the early years' stage (dusch et al., 2007). in this respect, primary school teachers have an essential responsibility. studies reported that science was one of the least taught subjects at the primary school level. primary prospective school teachers had mainly negative beliefs about science teaching and had less efficacy for teaching science than for other subjects in the primary curriculum (buss, 2010; petersen & treagust, 2014). considering this situation, primary school teachers must be competent in science concepts and ensure that their students acquire this knowledge in a laboratory environment (kurt & birinci konur, 2017). herga, čagran, and dinevski (2016) suggested that laboratory work is the most effective primary method when acquiring science knowledge. practical work provides students with cognitive processes such as concretizing abstract concepts, problemsolving, analyzing, critical thinking, synthesizing, evaluating, decision making, and creativity (kurt & birinci konur, 2017). according to the current four-year undergraduate curriculum of universities in turkey, ppsts are trained about the main concepts of science and science teaching during the courses basic science in elementary school and teaching science. moreover, through the course laboratory applications in science, they become competent at various science experiments to acquire the skill to design experiments suitable for all grades of primary education (council of higher education [cohe], 2018). however, since they do not have the option to pick among various subject matters for practicing during practice in teaching, they graduate with less experience and poor comprehension in teaching science compared to other subject matters of teaching (mansfield & woodsmcconney, 2012). the current study includes ppsts who conducted an experiment on crystallization, a physical purification technique in science and is intended to ensure that the participants gain experience for correct and meaningful learning. it is believed that the study might also journal of science learning article doi: 10.17509/jsl.v5i1.34772 178 j.sci.learn.2022.5(1).176-192 help identify the aspects where the prospective teachers are incompetent in crystallization. it is known that chemistry is the field where students experience comprehension problems the most. it is mainly because most concepts are abstract, and an unfamiliar language is used in chemistry (barak & dori, 2004; eilks & hofstein, 2015). the concern among science educators about when is the most appropriate time to introduce the concept of particles involving atoms and molecules in the science curriculum is the primary school level (johnstone, 2010; tsaparlis, kolioulis, & pappa, 2010). tsaparlis et al. (2010) have suggested delaying the introduction of the concepts of atoms and molecules until such time when students are ready to assimilate these ideas into their cognitive structures. when primary school students have misconstruction of the basic chemistry concepts such as atom and molecule, they subsequently experience difficulties in comprehending advanced concepts and associating them with the knowledge they were taught previously (canpolat & pınarbaşı, 2012). additionally, ppsts have difficulty understanding chemistry because it requires them to transfer between macro (observable, concrete), sub-microscopic (invisible, abstract), and symbolic levels of reaction (derman & ebenezer, 2020; johnstone, 2000). the macro is what can be explained by sense organs; the sub-microscopic is the cognitive model of the matter which is used to explain or predict the properties of substances; and the symbolic is the representation of the macro and sub-microscopic area with symbols (ye, lu, & bi, 2019). head, yoder, genton, and sumperl (2017) emphasized that in the absence of a chemical representation, pre-service teachers experience difficulty constructing an accurate mental model or internal representation based solely on a text description of abstract chemical concepts. johnstone (2000) uses a chemistry triangle to symbolize these representations that facilitate comprehension of chemistry knowledge. johnstone also noted that students' correct comprehension of chemistry concepts occurs within the triangle, not at just one apex of the triangle or along one edge of the triangle. underlining that expert teachers achieve it as they move fluently across this triad from one representational domain to another, johnstone recommends that teachers plan to teach students chemical representations by bridging and seeing the connections between the different domains of chemical representations. juriševič, glažar, razdevšek pučko, and devetak (2008) reported that a reasonable understanding could be established when all three levels of the concept describe each other in a specific way in students' working memory. it is, therefore, necessary that teachers are aware of the connections between each domain. this cognition will enable the students to construct an appropriate internal representation and connect among these three levels, helping them learn chemical concepts easily. recent studies show that primary teachers and teacher candidates have considerable difficulties in understanding the basic science and chemistry concepts, such concepts as chemical and physical changes, dissolution, heterogeneous mixture, and homogenous mixture (akkuzu güven & uyulgan, 2021; aydeniz, bilican, & kirbulut, 2017; derman & ebenezer, 2020; håland, 2010; härmäläbraskén et al., 2020; hermita et al., 2020; lemma, 2013; nandiyanto et al., 2018; sopandi, kadarohman, rosbiono, latip, & sukardi, 2018; yerrick & simons, 2017). therefore, more studies are needed to explain their chemical knowledge in great detail. this study seeks to determine the ppsts' subject matter knowledge related to crystallization and analyze their chemical representation levels (macroscopic, sub-microscopic, and symbolic levels) on crystallization. 2. method 2.1. research design this research is a descriptive study with a qualitative approach. descriptive research describes the existing situation using scientific procedures to answer real problems. descriptive data are produced in written or oral words of observed behavior, activities, features, changes, relationships, similarities, and differences between one phenomenon and another. in this type of research, the understanding and meaning of individuals and groups are tried to be described (creswell, 2012). as this study is descriptive research, it aims to determine and describe the macroscopic, sub-microscopic, and symbolic representation levels of the ppsts related to the science subject of crystallization, which is one of the methods of separating mixtures. this study is an investigation designed to provide baseline information that will aid and facilitate the development of other complex study strategies. 2.2. participants the current study participants consisted of first-year students studying at the department of primary education in a state university located in the aegean region of turkey. participants were selected based on the purposive sampling method that could be very useful when the researcher needs to reach a targeted sample quickly (creswell, 2012). due to the limitations in terms of time and labor, this sample is chosen from easily accessible (yıldırım & şimşek, 2013). in addition to this choice, the ppsts have quite similar backgrounds, such as socioeconomic status and grade level. although they took general chemistry lessons in the first year of high school, they did not take any chemistry or science lessons in the following terms. therefore, their background concerning science subjects is approximately the same. a total of 80 ppsts have voluntarily participated in the study. 46 of them were women (57.5%), whereas 34 were men (42.5%). data were collected within a 2-hour practice in the general chemistry course taught in the 2018 spring semester. in journal of science learning article doi: 10.17509/jsl.v5i1.34772 179 j.sci.learn.2022.5(1).176-192 order to have the study conform to ethical considerations, all of the prospective teachers were informed about the purpose and process of the research. 2.3. data collection the data in the study were obtained through the use of worksheets developed by the researchers. a worksheet on the crystallization experiment was prepared to determine the macroscopic, sub-microscopic, and symbolic representation levels of the ppsts. worksheets are written documents containing the process of the activities participants will do in a particular order. these documents ensure the participation of the whole class in the given activity simultaneously. the worksheet consisted of 7 open-ended questions regarding the experiment of crystallization. the questions were about the solubility of salt in water, the formation of the salt water and its solubility-temperature graph, the formation of the saturated salt water, and the appearance of particles formed during crystallization. ppsts were asked questions about each representation in open-ended questions, and ppsts were also asked to draw. two experts from the mathematics and science education department evaluated the questions for validity and reliability. the final versions of the questions examined within the scope of the research are included in the results section. 2.4. validity and reliability of data collection tool in order to provide the validity of the data collection tool, the worksheet questions were evaluated by two experts in terms of content, language, and comprehensibility. the experts scored the questions according to the specified degrees adequate (3), adequate but correction required (2), and not enough (1) (see table 1). in order to establish the reliability of the study, the worksheets were analyzed and scored by two researchers independently at different times. the reliability formula of miles and huberman (reliability = agreement/[agreement+disagreement] × 100) was applied to each question of the worksheet form. the agreement percentage representing interrater reliability among researchers was determined as 96%, indicating that the coding was reliable and the coherence excellent (miles & huberman, 1994). 2.5. procedure during the data collection process, worksheets related to the crystallization experiment of impure salt were distributed to the ppsts. the researchers conducted a demonstration experiment on the extraction of table salt by crystallization as an activity for the ppsts in the classroom environment. immediately after the experiment, an animated experiment video on the preparation of table salt was shown (see table 2). the experiment video was chosen according to the level of the ppsts. in the meantime, they were asked to answer the macroscopic, sub-microscopic, and symbolic questions on the worksheet according to the flow of the experiment. table 1 evaluation criteria for worksheet’s validity expert 1 expert 2 total q u e st io n n u m b e r c o n te n t l an g u ag e c o m p re h e n si b il it y c o n te n t l an g u ag e c o m p re h e n si b il it y 1 3 3 3 3 3 2 17 2 3 2 3 3 3 3 17 3 3 3 3 3 3 3 18 4 3 3 3 3 3 3 18 5 3 3 2 3 3 3 17 6 3 3 3 3 3 3 18 7 3 3 3 3 3 3 18 table 2 the stages of demonstration experiment and animated video experiment stages demonstration experiment animated video experiment 1 15 g of impure table salt is weighed and placed in a beaker containing 50 ml of distilled water. some water is taken in a container and heated. 2 the mixture in the beaker is heated up to the boiling temperature. when water (solvent) starts boiling, some table salt is added slowly. 3 the saturated hot solution is filtered into an erlenmeyer flask using filter paper and glass funnel and thus separated from water-insoluble impurities. the mixture is constantly stirred, and salt is added until more salt dissolves in water. 4 the saturated sodium chloride (nacl) solution is allowed to cool down at room temperature. then, the solution is left to crystallize. the saturated solution is filtered into an erlenmeyer flask using filter paper and a glass funnel. then, it is allowed to cool down. 5 at the end of crystallization, the crystals formed are washed with pure water. a stick is taken, and the string is tied to the stick. finally, the string is placed inside the solution. 6 the crystals formed are filtered and dried in the air. after a while, large salt crystals suspended are observed to form on the string. journal of science learning article doi: 10.17509/jsl.v5i1.34772 180 j.sci.learn.2022.5(1).176-192 2.6. data analysis documents enable uncovering meaning, developing understanding, and discovering insights relevant to the research problem (merriam, 1988). since the worksheet was used as a document in this study, the data were subjected to document analysis. according to this approach, the content of written documents is analyzed systematically (wach & ward, 2013). for this purpose, the answers of the ppsts were firstly examined under chemical representation levels. in order to determine these levels of the ppsts, the categories were identified in line with their direct statements. for some questions (1, 2, 3, 6a), written responses and drawings were coded according to four categories that were correct, partially correct, incorrect and no answer (n/a) in line with specific criteria (coştu, ayas, çalık, ünal, & karataş, 2005). meanwhile, the answers to the graphic drawing in the 4th, 5th, and b part of the 6th questions were categorized as correct, partially correct, and incorrect drawing. "correct" responses correspond to the ppsts' answers that match the scientific knowledge, "partially correct" responses include ppsts' answers that overlap with scientific knowledge but contain missing information about the question. "incorrect" responses include answers that do not correspond to scientific knowledge, and "n/a" means that are not answered or whose responses are meaningless. from the written statements of the ppsts, not only was it possible to find how they formed associations for the chemical representations, it was possible to discover where they had difficulties related to chemical representations on crystallization. responses in terms of accuracy categories were analyzed descriptively by frequencies and percentages regarding the number of ppsts. also, the analyses concerning the chemical representations were done according to frequencies within the ppsts' responses. the frequency numbers were the number of repetitions of the statements of ppsts. therefore, the total of the frequencies was not equal to the total number of participants (n = 80). more or fewer frequencies than the number of participants would be seen in the presented findings tables. figure 1 shows a step-by-step summary of how to analyze the worksheet. 3. results and discussion in the light of research questions, the ppsts' responses were first categorized as correct, partially correct, incorrect, and n/a; then, their analyses results were assessed by figure 1 step-by-step worksheet analysis figure 2 ppsts’ responses to the first question presented with their frequency and percentage distribution by categories f = 65 (81%) f = 14 (18%) f = 1 (1%) correct response partially correct response incorrect response journal of science learning article doi: 10.17509/jsl.v5i1.34772 181 j.sci.learn.2022.5(1).176-192 levels of chemical representations. finally, the results of their frequency and percentage distribution values were presented descriptively in the following graphs. the ppsts were asked the first question, i.e., "how is salt obtained?" and instructed to make necessary explanations. as seen in figure 2, most of them (81%) responded with correct answers; the rests were 18% partially correct and 1% incorrect. from the analysis results in table 3, according to the levels of chemical representation, we found that the ppsts had mostly macroscopic-level statements (f = 70). table 3 shows the chemical representation levels of the ppsts' responses to this question, along with the relevant sample statements. among the correct answers given by the ppsts were evaporating seawater and crystallization, while some of their partially-correct statements also included simple distillation. the first statement that came to the mind of the ppsts in their responses to how salt was obtained was the method of evaporation of seawater. crystallization is ranked second among their responses. this result indicates that the prospective teachers only have a casual knowledge of the subject, and they were inadequate to think of practical procedures in terms of chemistry teaching. laboratory practices are the applications that enable the concretization of abstract concepts. they also allow prospective teachers to discover the associations between the three levels of chemical representations, thereby facilitating better learning. this result is consistent with the study of kelly, akaygün, hansen, and villalta-cerdas (2017), who reported that laboratory experiments could support students' thinking skills. through these experiments, the students can connect the macro and micro levels of an abstract science concept. sanchez (2021) also pointed out that implementing strategies such as an experiment-oriented approach can lead a better understanding of the topics as the students visualize the given phenomena. as well as macroscopic level statements, there were some noteworthy answers given at sub-microscopic (f = 11) and symbolic (f = 2) levels (see table 2). the sub-microscopic statements explained that salt is obtained through ionic bonding and with the sodium and chlorine atoms compounding. at the same time, their answers of symbolic level referred to the reaction equation of sodium and chloride ions. the answers of these levels revealed that most ppsts gave the compound of nacl as an example to refer to salt, which suggests that they perceive salt to be a compound of sodium and chlorine ions. the second question was intended to investigate how the ppsts identify crystallization used to purify solids and is one of the production and purification techniques used, especially for saturated salt solutions. as shown in figure 3, 36% of the ppsts responded with scientifically correct answers, 40% partially-correct answers, and 16% incorrect answers, and 8% of them had n/a. to consider the analysis results in terms of representation levels, we found that all of the relevant statements were at the macroscopic level (see table 4). the analysis results in table 4 revealed table 3 the results of the analysis of the ppsts’ responses to the first question representation levels statements frequency values of the statements macroscopic obtained by evaporating and purifying seawater. 49 obtained by crystallizing. 12 obtained through simple distillation. 7 obtained by grinding rock salt. 2 total 70 sub-microscopic obtained through ionic bonding. obtained with the sodium and chlorine atoms compounding. 11 total 11 symbolic obtained using the equation na+ + cl-→nacl 2 total 2 figure 3 ppsts’ responses to the second question presented with their frequency and percentage distribution by categories f = 29 (36%) f = 32 (40%) f = 13(16%) f = 6 (8%) correct response partially correct response incorrect response n/a journal of science learning article doi: 10.17509/jsl.v5i1.34772 182 j.sci.learn.2022.5(1).176-192 that most ppsts described crystallization as the technique where water is evaporated (f = 18). this finding might be due to the ppsts’ thoughts that water in salt-water mixture vanishes through evaporation. similar to this finding, some ppsts (f = 10) described crystallization as the method used to separate liquids from solids through evaporation. these findings indicate that the ppsts confuse crystallization with evaporation. in response to the second question where we expected them to explain crystallization, the ppsts answered most, stating that it is water evaporation. this result also shows that the prospective teachers had inadequate knowledge about crystallization, which is one of the separation methods and mostly confused it with the evaporation method. in parallel to this result, coştu, ayas, açıkkar, and çalık (2007) reported that in response to an open-ended question asking students how they would separate a solid matter dissolved in a solvent from the solvent, majority of the students referred to evaporation as the separation method they would use. we also found out that some ppsts described crystallization as a technique used to separate solid-liquid mixtures, which concluded that the ppsts had inadequate knowledge about the technique (f = 16). only a few of the ppsts’ responses contained correct information about crystallization (f = 9), while there were also many incorrect answers describing it as the phase transition from gas to solid (f = 4) and the phase transition of a dissolved liquid to solid through crystallizing (f = 4). this finding points out that the ppsts cannot associate crystallization and solubility. the purpose of the crystallization experiment is to explain how the solubility of a solid matter changes depending on the temperature while it is being purified. therefore, the prospective teachers are required to be knowledgeable about solubility, the phenomenon of dissolving, and the factors affecting solubility. our results, however, found out that they had incorrect knowledge about the concept of dissolution. following this result, taylor and coll (2002) revealed that pre-service primary teachers could not explain the concept of dissolution with scientific expressions, and they reported that only one of them could explain dissolving in terms of attraction and bonding between molecules. akkuzu güven and uyulgan (2021) stated that the lack of understanding of the prospective primary school teachers in the concepts of mixtures and dissolution might also be due to their insufficient laboratory experience. we also determined that the prospective teachers could only give mostly macroscopic answers to the questions. likewise, derman and ebenezer (2020) reported that the prospective primary teachers knew physical and chemical changes mainly composed of macroscopic concepts. moreover, they also revealed the need for the submicroscopic and symbolic knowledge representations table 4 the results of the analysis of the ppsts’ responses to the second question representation levels statements frequency values of the statements macroscopic it is the evaporation of water. 18 it is a technique used to distill solid-liquid mixtures. 16 it is a technique of separating the liquid from the solids through evaporation. 10 it is to purify solid matters. 9 it is the extraction of substances like salt and sugar. 6 it is the transformation of a dissolved liquid into a solid by crystallizing. 5 it is to freeze and reheat a mixture. 4 it is the transition phase from gas to solid. 4 it is a separation technique by using the difference in solubility. 2 total 74 figure 4 ppsts' responses to question 3a presented with their frequency and percentage distribution by categories f = 25 (31%) f = 15 (19%) f = 38 (47%) f = 2 (3%) correct response partially correct response incorrect response n/a journal of science learning article doi: 10.17509/jsl.v5i1.34772 183 j.sci.learn.2022.5(1).176-192 connected with the missing steps of the instructional design. the third question on the worksheet is composed of three parts of 3a, 3b, and 3c. in the macroscopic question 3a, the ppsts were asked what they would observe when adding salt into the water in a beaker. unfortunately, 47% of them had incorrect answers, while only 31% had correct answers; additionally, 19% had partially correct answers; and 3% had n/a (see figure 4). this is because the ppsts answered this macroscopic question at sub-microscopic and symbolic levels and were unable to differentiate homogeneous and heterogeneous mixtures. the results presented in table 5, considering the ppsts' chemical representation levels, show that they had responses about other representations while expecting macroscopic answers. this might be because they could not comprehend the differences between representation levels soundly. among the incorrect answers, some ppsts stated that salt sinks to the beaker's bottom (f = 39), they form a heterogeneous mixture (f = 7), and salt particles stay visible in the beaker (f = 3). these findings are linked with the inadequate knowledge that the ppsts have about the term “dissolution”. our results also demonstrated that the prospective teachers frequently used macroscopic statements such as disappearing and becoming particles invisible to the naked eye while addressing the salt dissolving in water. studies' results are consistent with this result report that both students and prospective teachers defined the concept of dissolution as the disappearance of the dissolving substance (demirbaş, tanrıverdi, altınışık, & şahintürk, 2011; kabapınar, leach, & scott, 2004; tarkın çelikkıran & gökçe, 2019). ye et al. (2019) maintain that due to the abstract idea of the charge of the ions, students cannot ''see'' at the microscopic level. the answers given by the ppsts to question 3b, "what happens when you heat salt-water mixture?" are shown in figure 5. the majority of the ppsts (41%) had incorrect answers. table 6 shows that the explanations of the ppsts were at macroscopic and sub-microscopic levels. these statements at the macroscopic level included that salt dissolves in water and becomes invisible (f = 24). the previously heterogeneous mixture becomes a homogeneous mixture (f = 4). although their statements are observed to pertain to the macroscopic level, it is also table 5 the results of the analysis of the ppsts' responses to question 3a representation levels statements frequency values of the statements macroscopic salt sinks to the bottom. 39 they form a homogeneous mixture. 21 they form a heterogeneous mixture. 7 salt dissolves in the water and becomes invisible. 7 salt becomes visible, and water blurs. 3 total 77 sub-microscopic salt particles stick to each other and float in the water. 6 particles of water and salt are equally dispersed as the solution is homogeneous. 3 compared to particles of water, salt particles stay closer to each other. 2 salt appears to adhere to water molecules as it is a homogeneous mixture. 3 sodium gives 1 electron to chlorine to form an ionic bond. 2 total 16 symbolic salt → nacl water → h2o 2 total 2 figure 5 ppsts' responses to question 3b presented with their frequency and percentage distribution by categories f = 27 (34%) f = 20 (25%) f = 33 (41%) correct response partially correct response incorrect response journal of science learning article doi: 10.17509/jsl.v5i1.34772 184 j.sci.learn.2022.5(1).176-192 understood that they have misunderstandings in terms of scientific accuracy. because they thought that salt dissolves only when heated. some ppsts also expressed that water evaporates and salt sinks to the bottom after the heating (f = 19). as the answers given by the ppsts at the submicroscopic level were examined, the statements pertained to the particulate level. although the overall number of ppsts with correct answers at this level was high, others still had incorrect answers stating that salt particles move closer to each other to have an ordered pattern only when heated as salt is solid (f = 5). the ppsts were asked question 3c, "what is the purpose of heating the salt-water mixture?" as part of the crystallization experiment. only a few of them (15%) had scientifically correct answers to the question, while most (54%) of them had partially-correct answers, 26% had incorrect answers, and the rest (5%) had n/a (see figure 6). the ppsts answered this question at macroscopic and sub-microscopic levels according to chemical representations. although their statements matched the representation levels, there were also scientifically incorrect answers. as in the second question, the ppsts confused evaporation with crystallization in this question as well (f = 22) (see table 7). moreover, the ppsts who stated heating is necessary due to the difference in solubility (f = 12) failed to comprehend that multiple solid substances dissolve in a solvent. many ppsts confused melting with dissolving (f = 15). there were, however, some other scientifically correct answers stating that heating is necessary to accelerate the dissolving process (f = 6) and to increase the amount of salt dissolving in water (f = 10). there were also sub-microscopic answers stating that heat is needed to allow particles of salt and water to make a reaction (f = 4). this situation reveals that the ppsts consider dissolving as a chemical phenomenon. in response to the question on the salt-water mixture, some of the prospective teachers described the mixture as a heterogeneous mixture, and they did not consider the ionic dissolving of salt in their sub-microscopic drawing. the prospective teachers could also not demonstrate the salt ions being surrounded by water molecules while dissolving in their statements and drawings. this result is compatible with the study by tarkın çelikkıran and gökçe (2019), they determined that although prospective chemistry teachers were able to draw a salt consisting of an anionic assembly of cations, anions and water molecules, they could not illustrate the ions being surrounded by water molecules. a similar result is also encountered in the study where uluçınar sağır, tekin and karamustafaoğlu (2012) included ppsts. the study concluded that the prospective table 6 the results of the analysis of the ppsts’ responses to the question 3b representation levels statements frequency values of the statements macroscopic salt begins to dissolve in the water and becomes invisible. 24 water evaporates, and salt sinks to the bottom. 19 the previously heterogeneous mixture converts into a homogeneous mixture. 4 salt melts in the water. 4 salt becomes invisible but keeps its taste. 3 total 54 sub-microscopic the distance between particles is increased. 15 water particles slowly drift apart while salt particles get closer because the solids convert into a regular pattern's solid state of matter. 5 kinetic energy and, therefore, the movement of particles increase when the temperature rises. 2 salt particles ionize. 2 total 24 figure 6 ppsts' responses to question 3c presented with their frequency and percentage distribution by categories f = 12 (15%) f = 43 (54%) f = 21 (26%) f = 4 (5%) correct response partially correct response incorrect response n/a journal of science learning article doi: 10.17509/jsl.v5i1.34772 185 j.sci.learn.2022.5(1).176-192 teachers had difficulty drawing the phenomenon of salt dissolving in water at the particulate level and determining which of the various mixtures were solutions. another remarkable result is encountered in the statements of the ppsts referring that salt and water particles make a reaction when the salt-water solution is heated. the reason why they comprehend the phenomenon of dissolution as a chemical fact might be that they could not observe it at the sub-microscopic level (okumuş, öztürk, doymuş, & alyar, 2014). in the fourth question, the ppsts were asked to draw a solubility-temperature graph to represent the solubility of nacl in water and to explain the reasoning of their drawings. this question was mainly intended to understand how correctly they could concretize the phenomenon of the salt dissolving in water, which is one of the crystallizing processes, using a graph at the symbolic level. most ppsts (75%) had incorrect drawings, while 20% had partially correct and 5% had correct drawings (see figure 7). some of the ppsts with incorrect drawings (f = 35) assume that the salt-water solution is pure. therefore, they confused the solubility-temperature graph with the temperature-time graph. another reason ppsts had great difficulty drawing the graph was that they could not think that the solubilitytemperature curve continued to increase (f = 10). this indicates that they confuse evaporation with boiling. finally, some ppsts (f = 12) explained the saturated saltwater solution by rote based on the temperature-time graph. despite being correct in the case of the temperaturetime graph, these explanations were scientifically incorrect for the question asked about the solubility-temperature graph. the ppsts with the partially correct drawing could establish that nacl had a solubility that increased as the temperature increased. however, they took 0 oc as the initial point where solubility presents (f = 12). their statements also revealed that they had miscomprehension about the solubility of salt being directly proportional to temperature (f = 4). moreover, some ppsts (f = 8) had the misunderstanding that salts with solubility, which increases as temperature rises, dissolve in water only to an extent (see table 8). therefore, most prospective teachers could not draw the correct solubility-temperature graph for the saltwater solution. this also indicated that the prospective teachers had difficulty in symbolic representations. this is because the prospective teachers were unable to distinguish between pure substances and solutions and had inadequate knowledge about the solubility-temperature graph of the solids whose solubility increases with an increase in temperature. however, graphs are indispensable components of scientific statements (glazer, 2011). therefore, symbolic representations such as graphs, tables, table 7 the results of the analysis of the ppsts’ responses to the question 3b representation levels statements frequency values of the statements macroscopic the solution is heated to separate salt from water until the water evaporates. 22 salt melts in the water. 15 we can separate them as they have different solubility values. 12 heat facilitates the dissolution of salt in water. 8 heat accelerates dissolution. 6 the amount of salt dissolving in water increases as the solution is heated. 10 total 73 sub-microscopic heat allows particles of salt and water to make a reaction. 4 it is needed for ionization so that salt dissolves in water. 3 total 7 incorrect drawing partially correct drawing correct drawing figure 7 examples of the graphs drawn by the ppsts in response to the fourth question journal of science learning article doi: 10.17509/jsl.v5i1.34772 186 j.sci.learn.2022.5(1).176-192 formulas, equations, etc., are crucial for understanding the relations between many science concepts and phenomena (tepe & akkuzu güven, 2020; taber, 2009). similar to our results, gheith and aljaberi (2015) found that the preservice classroom teachers had poor graph skills, which is one of the symbolic representations. therefore, we can conclude that the symbolic level representations provide clues that indicate how accurately a science subject has been learned. in the fifth question, the ppsts were asked to explain what changes happen in a mixture when the amount of salt is increased at a specific temperature and to draw an appearance of the particles of the mixture. most of them (67.5%) had incorrect drawings in response to this question (see figure 8). there were only a small number of ppsts with correct drawing (6%), in addition to some others with no drawing at all (7.5%). for example, some of them with incorrect drawings referred to nacl using its molecular formula at the symbolic level. in contrast, those with partially-correct drawings drew at the sub-microscopic level but could not demonstrate the ions of salt and water compounds to be dispersed (see figure 9). this finding reveals that the ppsts cannot consider particle size and cannot comprehend the dissolution phenomenon. from table 8 the results of the analysis of the ppsts' drawings of solubility-temperature graphs about the solubility of nacl in water levels f and % values statements frequency values of the statements correct drawing 4 (5%) solubility increases as temperature rises. 3 solubility changes as the saline solution is a homogeneous mixture, but not much. 1 partially correct drawing 16 (20%) there is a maximum amount of substance that water can dissolve. after a while, no matter how much we increase the temperature, the solubility terminates at some point. 8 the solubility of the salt in water changes directly proportional to temperature. 8 incorrect drawing 60 (75%) water boils at 100 oc and then stays constant. 35 it remains constant after the saturated solution has occurred. 12 solubility remains constant at the temperature where evaporation begins. 10 solubility is a distinctive characteristic and therefore remains unchanged. 3 total 80 (100%) 80 figure 8 distribution of the ppsts’ drawings in response to the fifth question by categories incorrect drawing partially correct drawing correct drawing figure 9 examples of the ppsts’ drawings in response to the fifth question f = 5 (6%) f = 15 (19%) f = 54 (67.5%) f = 6 (7.5%) correct drawing partially correct drawing incorrect drawing n/a journal of science learning article doi: 10.17509/jsl.v5i1.34772 187 j.sci.learn.2022.5(1).176-192 the result of analysis of the ppsts' statements to the fifth question as to the chemical representation levels, we found that these statements were at macroscopic and submicroscopic levels (see table 9). although the ppsts had mostly incorrect drawings, their statements corresponded to the macroscopic and sub-microscopic levels. the macroscopic statements include scientifically correct responses expressing that saturated solution occurs (f = 19) and a more concentrated solution is formed than the previous salt-water solution (f = 18). some ppsts (f = 17) were also observed to refer to a visible phenomenon where water blurs as salt dissolves in the water. this finding indicates that the ppsts had statements matching with the macroscopic level. to assess the statements in terms of scientific accuracy, despite being sub-microscopic responses, these statements show that the ppsts misunderstood that the distance between salt particles shrinks to get closer to each other. among the study results is also the inability of most of the prospective teachers to draw the saturated solution at the sub-microscopic level. the great difficulty is caused by their inability to consider how the ions in the solution would move. many studies have shown that prospective teachers find it challenging to understand the representation of particles existing in solution (adadan, 2014; demirbaş et al., 2011; valanides, 2000). however, the macroscopic statements of the table 9 the results of the analysis of the ppsts’ responses to the fifth question representation levels statements frequency values of the statements macroscopic the saturated solution occurs. 19 it becomes more concentrated. the amount of salt in the mixture increases. 18 a blurry appearance occurs as salt dissolves in it. it becomes oversaturated. 17 salt gets more intense than water. 8 salt does not dissolve. it sinks to the bottom. 5 the ratio of salt to water increases in the mixture. dissolution time prolongs. 2 total 69 submicroscopic the distance between particles shrinks if the amount of salt in the mixture increases. 7 salt particles move closer to each other. 4 total 11 table 10 the results of the analysis of the ppsts' responses to question 6a representation levels statements frequency values of the statements macroscopic salt does not filter through the paper filter. instead, it is separated through filtering. 31 the total amount of the solution filters through the paper filter. the salt has already dissolved in the water. 10 some amount of the salt does not filter through the paper filter. instead, the dissolved part is filtered into the erlenmeyer flask. 14 the solution thoroughly filters through the paper filter, except for the substances that have not dissolved in the water. 22 total 77 submicroscopic the separated substances stay on the filter: sodium and chloride. 3 total 3 figure 10 ppsts' responses to question 6a presented with their frequency and percentage distribution by categories f = 22 (27%) f = 10 (13%)f = 48 (60%) correct response partially correct response incorrect response journal of science learning article doi: 10.17509/jsl.v5i1.34772 188 j.sci.learn.2022.5(1).176-192 prospective teachers concerning the saturated salt-water solution coincide with scientific accuracy. so, this shows that they do not have problems with the visible dimensions of phenomenon or phenomena but with the invisible particle behaviors as seen in their statements and drawing. so mental modeling and a greater degree of interpretation are required to overcome the obstacles of this submicroscopic level and understand the information conveyed in the representation (head et al., 2017). the next question (6a) that the ppsts were asked: "what would you expect to happen if you pour the hot saltwater solution into a filter paper?". figure 10 shows that more than half (60%) of them had incorrect answers, while those with correct answers accounted only for 27%, and partially-correct answers accounted for 13%. as question 6a has a sensate basis of observation, it is a question regarding the macroscopic level. the representation levels indicate that most ppsts had macroscopic statements (see table 10). from the macroscopic statements of the ppsts, we concluded that they had many miscomprehensions. for example, they stated that salt separates from water through filtering (f = 26), and the quantity of non-dissolved salt does not filter through the filter paper (f = 14). this finding shows that the ppsts could not explain filtering, which is one of the mixture separation methods. despite the experiment and video demonstrated to facilitate comprehension of a chemistry subject, the prospective teachers could not distinguish the mixtures from each other. wang, chi, luo, yanga, & huanga (2017) stated that to understand chemical processes such as filtering and crystallization, symbolic representations should be included in the learning process of learners and macroscopic representations. in another question (6b) asked the ppsts, they were instructed to draw how the particles formed in the flask and explain the reasons for their drawings. in table 11, the results of the analysis of the ppsts' drawings revealed that 62.5% of them had correct drawings, while 32.5% of them had incorrect drawings. those with incorrect drawings illustrated the salt particles attached (f = 9), solidified (f = 10), or freeze (f = 7) (see figure 11). these findings demonstrate that they confuse crystallization with freezing, and they could be unable to think that the crystals forming in crystallization are separated from the solvent. the prospective teachers described the crystallization of salt particles as freezing and solidification, demonstrating that they confused crystallization with freezing, although they are two different phenomena. this situation reveals that the prospective teachers completed the science courses of primary and secondary schools where basic concepts are learned and the chemistry course of the grade-1 high school with inadequate and inaccurate knowledge. nevertheless, many studies are emphasizing that science subjects should be taught effectively, especially in the learning process table 11 the results of the analysis of the ppsts’ responses to the question 6b levels f and % values statements frequency values of the statements correct drawing 50 (62.5%) it is crystallized. because salt particles are dissolving in water, move closer. 28 the particles left on the string are salt crystals. 12 the only salt remains as the water evaporates. 10 incorrect drawing 26 (32.5%) it is a solidified form. 10 salt particles are attached to each other. 9 it is frozen. 7 n/a 4 (5%) 4 total 80 (100%) 80 incorrect drawing correct drawing figure 11 examples of the graphs drawn by the ppsts in response to question 6b journal of science learning article doi: 10.17509/jsl.v5i1.34772 189 j.sci.learn.2022.5(1).176-192 extending from primary education to university, and therefore by providing transitions at macro, symbolic and sub-microscopic levels (aydeniz et al., 2017; derman & ebenezer, 2020; kelly et al., 2017; tsaparlis et al., 2010). the final seventh question investigated the other mixtures that the ppsts might know can be separated through crystallization. again, 80% of them responded, referring to sugar-water solution (see figure 12). when all the findings were considered, it was revealed that the ppsts could not think in a triple structure (macroscopic, sub-microscopic, and symbolic) and could not combine them in related content of the crystallization experiment. as emphasized in the literature (banawi et al., 2019; derman & ebenezer, 2020; hermita et al., 2020) and found in this study, the understanding of prospective primary school teachers require to include macroscopic, sub-microscopic, and symbolic levels. conclusion the subject "science" at the primary education contains specific and basic chemistry concepts such as homogeneous and heterogeneous mixtures, water and water solutions, dissolving, and crystallization (januik & mazur, 2010). therefore, prospective teachers who are well trained in these subjects are needed in teaching to ensure quality science education. our results herein revealed that the prospective teachers had inadequate knowledge about the relevant chemistry concepts. moreover, the responses of the prospective teachers discovered the existing problems that they have in terms of chemical representations. the results also demonstrated that their responses were mainly at the macroscopic level, while they had great difficulty explaining at the sub-microscopic and symbolic representations levels. this result shows that the students of the prospective teachers may also have weak knowledge in their professional lives. it will lead to the fact that there are students who cannot respond to the invisible state of matter in daily life. they cannot perform meaningful learning because they cannot establish a relationship between representations. therefore, it is expected that prospective teachers who can understand the microscopic world and teach it by simplifying it will be trained (nandiyanto et al., 2018). additionally, the results of this study intensify the data of prospective primary school teachers' understanding of the crystallization experiment. hence they become the inspiration to ensure quality science education and further research. many studies show that primary school students at the concrete operational stage have difficulty comprehending such concepts as chemical change, physical change, dissolution, heterogeneous mixture, and homogenous mixture unless they associate them with daily-life phenomena (donovan & bransford, 2005; taşdemir & demirbaş, 2010). for this reason, ppsts should teach these concepts by using macroscopic statements and employing sub-microscopic drawings and representations that might appeal to students' concrete learning. papageorgiou, amariotakis, and spiliotopoulou (2017) pointed out that chemical representations serve as a basis for the constructed internal representation and help students consolidate their understanding of an abstract concept. it is also expressed that employing various instructional technologies such as animations for these representations helps reinforce concrete learning of these concepts (kelly & jones, 2008; pekdağ, 2010). in our study, the prospective teachers have displayed an animated video along with the demonstration experiment, which also served as a model in teaching these concepts. the participants of the study have a remarkable impact on our results. many studies in the literature report that prospective teachers have negative emotions rather than positive emotions when teaching topics related to physics or chemistry since the participants are prospective primary school teachers (brígido, bermejo, conde, & mellado, 2010). when the negative attitudes of prospective teachers are reflected upon concepts of physics or chemistry, their students might have difficulties in learning. therefore, prospective teachers should get acquainted with physics figure 12 ppsts' responses to question 7 presented with their frequency and percentage distribution no response f = 8 (10%) sugar-water mixture f = 64 (80%) lime water f = 5 (6%) naphthalene -water mixture f = 2 (3%) snowflakes f = 1 (1%) journal of science learning article doi: 10.17509/jsl.v5i1.34772 190 j.sci.learn.2022.5(1).176-192 and chemistry concepts and learn the conceptual structures properly to develop a positive attitude towards teaching these concepts. implementing science activities in primary education also poses an issue concerning science concepts. for this reason, more effective science training programs should be arranged to use science activities frequently and effectively (papageorgiou, kogianni, & makris, 2007). the present study allowed prospective teachers to explain chemical phenomena using their conceptual structures at different chemical representation levels, thereby allowing them to get curious about and interested in exploring chemistry. there have been some recommendations developed based on the results and the relevant literature: • hands-on science laboratory practices that incorporate different teaching techniques supporting primary school teachers and teaching basic chemistry concepts can be implemented in demonstration and as teamwork. prospective teachers might therefore be allowed to get more experience on science concepts. • difficulties experienced at any level of chemical representation might affect comprehension of other representation levels; thus, determining their state of comprehension at all three levels is critically important to ensure efficient teaching of basic science concepts in chemistry. therefore, teaching plans should be designed in a way attaching importance to these representation levels in teaching basic science concepts such as dissolution, solubility, and crystallization. • ppsts must attend a quality science teaching so primary school students, who will encounter the subjects and basic concepts of science for the first time, acquire accurate knowledge of the basic science subjects and concepts. to raise the quality of science teaching, the education curricula of prospective primary teachers must be revised and improved as needed. references aubrey, c., ghent, k., & kanira, e. (2012). enhancing thinking skills in early childhood. international journal of early years education, 20(4), 332-348. adadan, e. (2014). investigating the influence of pre-service chemistry teachers' understanding of the particle nature of matter on their conceptual understandings of solution chemistry. chemistry education research and practice, 15(2), 219-238. https://dx.doi.org/10.1039/c4rp00002a akkuzu güven, n., & uyulgan, m. a. (2021). linking the representation levels to a physical separation and purification method in chemistry: understanding of distillation experiment. journal of pedagogical research, 5 (3), 80-104. https://doi.org/10.33902/jpr.2021370703 aydeniz, m., bilican, k., & kirbulut, z. d. (2017). exploring pre-service elementary science teachers’ conceptual understanding of particulate nature of matter through three-tier diagnostic test. international journal of education in mathematics, science and technology, 5(3), 221-221. banawi, a., sopandi, w., kadarohman, a., & solehuddin, m. (2019). prospective primary school teachers’ conception change on states of matter and their changes through predict-observe-explain strategy. international journal of instruction, 12(3), 359-374. https://doi.org/10.29333/iji.2019.12322a barak, m., & dori, y. j. (2004). enhancing undergraduate students’ chemistry understanding through project-based learning in an it environment. science education, 8(1), 117-139. brígido, m., bermejo, m. l., conde, c., & mellado, v. (2010). the emotions in teaching and learning nature sciences and physics/chemistry in pre-service primary teachers. us-china education review, 7(12), 25-32. buss, r. r. (2010). efficacy for teaching primary science and mathematics compared to other content. school science and mathematics, 110(6), 290-297. http://dx.doi.org/10.1111/j.19498594.2010.00037.x canpolat, n., & pınarbaşı, t. (2012). kimya öğretmen adaylarının kaynama olayı ile ilgili anlayışları: bir olgubilim çalışması [prospective chemistry teachers’ understanding of boiling: a phenomenological study]. erzincan university journal of education faculty, 14(1), 81-96. coştu, b., ayas, a., açıkkar, e., & çalık, m. (2007). çözünürlük konusuyla ilgili kavramlar ne düzeyde anlaşılıyor? [at which level are concepts about solubility topic understood?]. boğaziçi university journal of education, 24(2), 13-28. coştu, b., ayas, a., çalık, m., ünal, s., & karataş, f. ö. (2015). fen öğretmen adaylarının çözelti hazırlama ve laboratuvar malzemelerini kullanma yeterliliklerinin belirlenmesi [determining preservice science teachers’ competences in preparing solutions and in use of laboratory tools]. hacettepe university journal of education, 28(28), 65-72. council of higher education [cohe]. (2018). sınıf öğretmenliği lisans programı, yeni öğretmen yetiştirme lisans programları [primary education undergraduate program, new teacher training undergraduate programs]. creswell, j. w. (2012). educational research: planning, conducting, and evaluating quantitative and qualitative research. educational research vol. 4 (pp. 232-260). harlow, united kingdom: pearson education limited. demirbaş, m., tanrıverdi, g., altınışık d., & şahintürk, y. (2011). fen bilgisi öğretmen adaylarının çözeltiler konusundaki kavram yanılgılarının giderilmesinde kavramsal değişim metinlerinin etkisi [the impact of conceptual change texts on the elimination of misconceptions of science teacher candidates about the subject of solution]. sakarya university journal of education, 1(2), 52-69. demircioğlu, h., demircioğlu, g., & ayas, a. (2004). sınıf öğretmeni adaylarının bazı temel kimya kavramlarını anlama düzeyleri ve karşılaşılan yanılgılar [prospective science teachers' levels of understanding and misconceptions about some basic chemical concepts]. hayef: journal of education, 1, 29-49. derman, a., & ebenezer, j. (2020). the effect of multiple representations of physical and chemical changes on the development of primary pre-service teachers cognitive structures. research in science education, 50, 1575–1601. https://doi.org/10.1007/s11165-018-9744-5 donovan, m. s., & bransford, j. d. (eds.). (2005). how students learn: history, mathematics, and science in the classroom. washington, dc: national academies press. duschl, r. a., schweingruber, h. a., & shouse, a. w. (eds.). (2007). taking science to school: learning and teaching science in grades k-8. washington, dc: national academies press. eilks, i., & hofstein, a. (eds.). (2015). relevant chemistry education: from theory to practice. rotterdam: sense publishers. gheith, e. m., & aljaberi, n. m. (2015). pre-service classroom teachers’ attitudes toward graphs and their ability to read and interpret them. international journal of humanities and social science, 5(7), 113-124. glazer, n. (2011). challenges with graph interpretation: a review of the literature. studies in science education, 47(2), 183-210. håland, b. (2010). student-teacher conceptions of matter and substances – evaporation and dew formation. nordina, 6(2), 109– 124. https://dx.doi.org/10.1039/c4rp00002a https://doi.org/10.29333/iji.2019.12322a http://dx.doi.org/10.1111/j.1949-8594.2010.00037.x http://dx.doi.org/10.1111/j.1949-8594.2010.00037.x https://doi.org/10.1007/s11165-018-9744-5 journal of science learning article doi: 10.17509/jsl.v5i1.34772 191 j.sci.learn.2022.5(1).176-192 härmälä-braskén, a.-s., hemmi, k., & kurtén, b. (2020). misconceptions in chemistry among finnish prospective primary school teachers-a long-term study. international journal of science education, 42(9), 1447-1464. head, m. l., yoder, k., genton, e., & sumperl, j. (2017). a quantitative method to determine pre-service chemistry teachers' perceptions of chemical representations. chemistry education research and practice, 18(4), 825-840. herga, n. r., čagran, b., & dinevski, d. (2016). virtual laboratory in the role of dynamic visualisation for better understanding of chemistry in primary school. eurasia journal of mathematics, science & technology education, 12(3), 593-608. hermita, n., alpusari, m., noviana, e., putra, z. h., islami, n., basori, h., suhandi, a., & samsudin, a. (2020). improving prospective primary school teachers' mental models through implementation of cdoi supported by multimode visualization. universal journal of educational research, 8(2), 460-467. https://doi.org/10.13189/ujer.2020.080217 januik, r. m., & mazur, p. (2010). poland. in b. risch (ed.), teaching chemistry around the world (pp. 291-309). germany: hubert &co. johnstone, a. h. (2000). teaching of chemistry-logical or psychological? chemistry education research and practice, 1(1), 9–15. johnstone, a. h. (2010). you can’t get there from here. journal of chemical education, 87(1), 22–29. https://dx.doi.org/10.1021/ed800026d juriševič, m., glažar, s. a., razdevšek pučko, c., & devetak, i. (2008). intrinsic motivation of pre-service primary school teachers for learning chemistry in relation to their academic achievement. international journal of science education, 30(1), 87–107. https://dx.doi.org/10.1080/09500690601148517 kabapinar, f., leach, j., & scott, p. (2004). the design and evaluation of a teaching-learning sequence addressing the solubility concept with turkish secondary school students. international journal of science education, 26(5), 635—65. kelly, r. m., & jones, l. l. (2008). investigating students' ability to transfer ideas learned from molecular animations of the dissolution process. journal of chemical education, 85(2), 303. kelly, r. m., akaygün, s., hansen, s. j. r., & villalta-cerdas, a. (2017). the effect that comparing molecular animations of varying accuracy has on students' submicroscopic explanations. chemistry education research and practice, 18(4), 582–600. knowles, m., holton, e. f., & swanson, r. a. (2005). the adult learner: the definitive classic in adult education and human resource development (6th ed.). amsterdam: elsevier. kozma, r. (2003). the material features of multiple representations and their cognitive and social affordances for science understanding. learning and instruction, 13(2), 205-226. kurt, s., & birinci konur, k. (2017). pre-service primary teachers’ impressions towards chemistry experiments based on constructivist laboratory approach. inonu university journal of the faculty of education, 18(3), 145-161. http://dx.doi.org/10.17679/inuefd.296545 lemma, a. (2013). a diagnostic assessment of eight grade students’ and their teachers’ misconceptions about basic chemical concepts. african journal of chemical education, 3(1), 39–59. lindawati, l., wardani, s., & sumarti, s. s. (2019). development of inquiry materials based on chemical representation to improve students’ critical thinking ability. journal of innovative science education, 8(3), 332-343. https://doi.org/10.15294/jise.v8i1.31082 mansfield, c. f., & woods-mcconney, a. (2012). “i didn’t always perceive myself as a science person”: examining efficacy for primary science teaching. australian journal of teacher education, 37(10),37-52. merino, c., & sanmarti, n. (2008). how young children model chemical change. chemistry education research and practice, 9(3), 196–207. merriam, s. b. (1988). case study research in education: a qualitative approach. san francisco: jossey-bass. miles, m. b., & huberman, a. m. (1994). qualitative data analysis. united states of america printed. nandiyanto, a. b. d., asyahidda, f. n., danuwijaya, a. a., abdullah, a. g., amelia, n., hudha, m. n., & aziz, m. (2018). teaching “nanotechnology” for elementary students with deaf and hard of hearing. journal of engineering science and technology, 13(5), 1352-1363. okumuş, s., öztürk, b., doymuş, k., & alyar, m. (2014). maddenin tanecikli yapısının mikro ve makro boyutta anlaşılmasının sağlanması [aiding comprehension of the particulate of matter at the micro and macro levels]. journal of educational sciences research, 4(1), 349-368. papageorgiou, g., amariotakis, v., & spiliotopoulou, v. (2017). visual representations of microcosm in textbooks of chemistry: constructing a systemic network for their main conceptual framework. chemistry education research and practice, 18(4), 559-571. http://dx.doi.org/10.1039/c6rp00253f. papageorgiou, g., kogianni, e., & makris, n. (2007). primary teachers’ views and descriptions regarding some science activities. chemistry education research and practice, 8(1), 52–60. https://dx.doi.org/10.1039/b6rp90019d pekdağ, b. (2010). kimya öğreniminde alternatif yollar: animasyon, simülasyon, video ve multimedya ile öğrenme [alternative methods in learning chemistry: learning with animation, simulation, video and multimedia]. journal of turkish science education volume 7(2), 79110. pine, k., messer, d., & st. john, k. (2001). children’s misconceptions in primary science: a survey of teachers’ views. research in science and technological education, 19(1), 79–96. https://dx.doi.org/ 10.1080/02635140120046240. petersen, j. e., & treagust, d. f. (2014). school and university partnerships: the role of teacher education institutions and primary schools in the development of pre-service teachers' science teaching efficacy. australian journal of teacher education, 39(9), 153167. http://dx.doi.org/10.14221/ajte.2014v39n9.2. polifka, j. (2021). investigating introductory science students’ knowledge of multiple representations using technology. (doctoral dissertation). retrieved from https://www.proquest.com/pqdtglobal/docview/2572566476/pr eviewpdf/db243a053c284c6bpq/1?accountid=10527. sanchez, j. m. p. (2021). understanding of kinetic molecular theory of gases in three modes of representation among tenth-grade students in chemistry. international journal of learning, teaching and educational research, 20(1), 48-63. https://doi.org/10.26803/ijlter.20.1.3 schulte, p. l. (2001). pre-service elementary teachers' alternative conceptions in science and attitudes toward teaching science. (doctoral dissertation). retrieved from https://www.proquest.com/docview/304703069 sopandi, w., kadarohman, a., rosbiono, m., latip, a., & sukardi, r. r. (2018). the courseware of discontinuous nature of matter in teaching the states of matter and their changes. international journal of instruction, 11(1), 61–76. https://dx.doi.org/10.12973/iji.2018.1115a taber, k. s. (2009). learning at the symbolic level. in j. k. gilbert & d. treagust (eds.), multiple representations in chemical education (pp. 75– 105). springer, dordrecht. tarkın çelikkıran, a., & gökçe, c. (2019). kimya öğretmen adaylarının çözünürlük konusuna ilişkin submikroskobik seviyedeki anlama düzeylerinin çizimlerle belirlenmesi [determination of preservice chemistry teachers’ understanding of solubility concept at submicroscopic level by drawings]. pamukkale university journal of education, 46, 57-87. http://dx.doi.org/10.9779/pauefd.457845 taşdemir, a., & demirbaş, m. (2010). i̇lköğretim öğrencilerinin fen ve teknoloji dersinde gördükleri konulardaki kavramları günlük yaşamla ilişkilendirebilme düzeyleri [the level of correlation of concepts that primary students seen topics in science and technology class with daily life]. international journal of human sciences, 7(1), 124-148. taylor, n., & coll, r. k. (2002). pre-service primary teachers' models of kinetic theory: an examination of three different cultural groups. chemistry education research and practice, 3(3), 293-315. https://doi.org/10.13189/ujer.2020.080217 http://dx.doi.org/10.17679/inuefd.296545 https://doi.org/10.26803/ijlter.20.1.3 https://www.proquest.com/docview/304703069 journal of science learning article doi: 10.17509/jsl.v5i1.34772 192 j.sci.learn.2022.5(1).176-192 tepe, o., & akkuzu güven, n. (2020). genel kimya konularına ilişkin grafik okuma, yorumlama ve çizim becerileri testi geliştirme süreci: geçerlilik ve güvenilirlik analizleri [the development process of graph reading, interpreting and drawing skills test concerning general chemistry subjects: validity and reliability analyses]. western anatolia journal of educational sciences, 11(1), 23-43. treagust, d. f., chandrasegaran, a. l., crowley, j., yung, b. h. w., cheong, i. p. a, & othman, j. (2010). evaluating students’ understanding of kinetic particle theory concepts relating to the states of matter, changes of state and diffusion: a cross-national study. international journal of science and mathematics education, 8, 141164. tsaparlis, g., kolioulis, d., & pappa, e. (2010). lower-secondary introductory chemistry course: a novel approach based on scienceeducation theories, with emphasis on the macroscopic approach, and the delayed meaningful teaching of the concepts of molecule and atom. chemistry education research and practice, 11(2), 107–117. uluçınar sağır, ş., tekin, s., & karamustafaoğlu, s. (2012). sınıf öğretmeni adaylarının bazı kimya kavramlarını anlama düzeyleri [the levels of prospective elementary school teachers’ understanding of some chemistry concepts]. journal of dicle university ziya gökalp faculty of education, 19, 112-135. uludüz, ş. m. (2017). sınıf öğretmeni adaylarının fen okuryazarlık düzeyleri ile fen öğretimi öz yeterlik inançlarının karşılaştırılması [the comparison of science literacy levels of primary teacher candidates and science teaching self efficacy beliefs]. (master thesis). retrieved from https://tez.yok.gov.tr/ulusaltezmerkezi/ valanides, n. (2000). primary student teachers' understanding of the particulate nature of matter and its transformations during dissolving. chemistry education: research and practice in europe, 1(2), 249262. wach, e., & ward, r. (2013). learning about qualitative document analysis. retrieved from https://opendocs.ids.ac.uk/opendocs/handle/20.500.12413/298 9 wang, z., chi, s., luo, m., yanga, y., & huanga, m. (2017). development of an instrument to evaluate high school students’ chemical symbol representation abilities. chemical education research and practice, 18(4), 875-892. https://doi.org/10.1039/c7rp00079k wu, h. k., & puntambekar, s. (2012). pedagogical affordances of multiple external representations in scientific processes. journal of science education and technology, 21(6), 754-767. ye, j., lu, s., & bi, h. (2019). the effects of microcomputer-based laboratories on students macro, micro, and symbolic representations when learning about net ionic reactions. chemistry education research and practice, 20(1), 288-301. http://dx.doi.org/10.1039/c8rp00165k yerrick, r. k., & simons, t. (2017). the affordances of fiction for teaching chemistry. science education international, 28(3), 232-243. yıldırım, a., & şimşek, h. (2013). sosyal bilimlerde nitel araştırma yöntemleri [qualitative research methods in the social sciences]. ankara: seçkin kitabevi. notes a part of this study is presented as an oral presentation at the 2nd international congress on seeking new perspectives in education. abbreviations ppsts, prospective primary school teachers; n/a, no answer. https://opendocs.ids.ac.uk/opendocs/handle/20.500.12413/2989 https://opendocs.ids.ac.uk/opendocs/handle/20.500.12413/2989 http://dx.doi.org/10.1039/c8rp00165k