Microsoft Word - Naufal 20377-63853-1-ED.doc Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 13 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context Naufal Ishartono1, Rafiza binti Abdul Razak2, Siti Hajar binti Halili3, Yoga Dwi Windy Kusuma Ningtyas4, Wilda Syam Tonra5, Muhammad Noor Kholid6, Mohamad Waluyo7, Soraya Djamilah8 1-3Faculty of Education, University of Malaya, Malaysia 1,6Faculty of Teacher Training and Education, Universitas Muhammadiyah Surakarta, Indonesia 4Faculty of Teacher Training and Education, Univeritas Muhammadiyah Jember, Indonesia 5Faculty of Teacher Training and Education, Universitas Khairun, Indonesia 7Faculty of Education, University of Szeged, Hungary 8Faculty of Teacher Training and Education, Universitas Muhammadiyah Banjarmasin, Indo- nesia DOI: 10.23917/ijolae.v5i1.20377 Received: November 29th, 2022. Revised: December 5th, 2022. Accepted: December 9th, 2022 Available Online: December 13th, 2022. Published Regularly: January 1st, 2023 Abstract Several previous researchers have tried to review the results of previous research related to the development of learning tools using a systematic literature review (SLR) model. However, SLR research efforts that ex- amine the development of learning tools carried out by Indonesian in-service and pre-service mathematics teachers are still limited. Therefore, this study seeks to answer two questions, namely (1) what is the math- ematics learning sets developed by the Indonesian ISTs and PSTs, and (2) What instructional design model is used by the Indonesian ISTs and PSTs in developing mathematics learning sets. As many as 55 articles were selected using the PRISMA protocol by retrieving articles from the ERIC and Dimensions databases. The findings of the two questions are that the ADDIE model is the most widely used model by Indonesian PSTs and ISTs in developing learning sets. In addition, mathematics learning media and mathematics stu- dent worksheets are the most developed tools by authors. This study also examines several instructional designs and types of mathematics learning sets developed by researchers, which can undoubtedly be a refer- ence for subsequent researchers in developing mathematics learning sets. Keywords: instructional design models, in-service mathematics teachers improvement, learning sets skills improvement, mathematics learning sets, pre-service mathematics teachers improvement, PRISMA Corresponding Author: Naufal Ishartono, Faculty of Education, University of Malaya, Malaysia Email: s2102309@siswa.um.edu.my 1. Introduction Along with the rapid development of technology, the condition of the educational world is also required to be increasingly de- veloped (Firdausy et al., 2019). The teacher's skills in delivering material must be imple- mented effectively and efficiently, but still do not leave the meaningfulness of the learn- ing process. One of the efforts that can be made is to develop suitable mathematics learning sets that are valid and reliable (Agustina et al., 2021). Valid mathematics learning sets are right on target both from the Indonesian Journal on Learning and Advanced Education http://journals.ums.ac.id/index.php/ijolae Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 14 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context material aspect and the conformity aspect to the student. Next, the mathematics learning sets that are declared reliable remain valid even though they are used repeatedly (Ishartono, Nurcahyo, et al., 2022). With decent and up-to-date mathematics learning sets, the material delivery will be more readi- ly accepted by students. Apart from the aspect of the device, the development of mathematics learning sets must also look at the psychological devel- opment and character of students, who are increasingly different in each generation (Rejekiningsih, 2019). Some experts say that students who grew up and were born in the 2000s are called the technological native generation, where they are easier to adapt to the use of digital technology than in previous generations (Kai et al., 2021; Luik & Suviste, 2018; Saltan & Arslan, 2017; Wang et al., 2018). On the other hand, material aspects must also be considered in develop- ing mathematics learning sets (E. Saputra & Fahrizal, 2019). The material aspect deter- mines what kind of learning follows the character of the material being taught, for example, in geometry material in mathemat- ics learning. GeoGebra, an integrated media based on mathematics learning sets, is better than MATLAB, which tends towards algebra and calculus. In the development of mathematics learning sets, basic knowledge, as has been conveyed, must be possessed by a teacher in developing devices (Ishartono, Setyono, et al., 2022). At least four aspects must be con- sidered in developing the mathematics learn- ing sets: student aspects, curriculum aspects, media/device aspects, and aspects of learning approaches/models (Sun et al., 2018). Each of these aspects is interrelated, so failure to understand one aspect affects the other, sig- nificantly affecting the quality of the device that has been developed. The theories that cover these four aspects and the stages of developing mathematics learning sets are summarized in the concept of instructional design. This concept is considered essential to master by in-service teachers (ISTs) and pre-service teachers (PSTs). a. Instructional Design and Indonesian ISTs and PSTs One of the indicators of successful learn- ing is an increased understanding of students on the material taught. Efforts to increase understanding of the material certainly can- not be separated from the meaningfulness of the learning process. Next, fluency is also influenced by models, approaches, media, and learning strategies represented in a mathematics learning set consisting of lesson plans, learning media and resources, and tests. Therefore, good mathematics learning sets must be based on good instructional de- sign. Instructional design is a set of systematic procedures for developing instructional ma- terials (Merrill, 1994). Meanwhile, Chen (2016) argues that the discipline of designing educational experiences that make infor- mation and skill development more efficient, effective, and appealing. Several previous studies have actualized the definition into several instructional design models. The models are used by educational developers, practitioners, and researchers to support the creation of instructional design methods that elicit suitable cognitive processes (Khalil & Elkhider, 2016). Some of the instructional design models developed by previous re- searchers include ADDIE, ASSURE, Dick and Carey, 4D, SAM, and Merrill's Princi- ples (see Table 1 for the details). Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 15 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context Table 1. Types of Instructional Design Models ID Model Yeara Steps MRDb nc ADDIE 1975 Analysis, Design, Development, Implementation, and Analysis (Branch, 2009) 17,000 ASSURE 1999 Analyze Learner, State Standards and Objectives, Select Strategies, Utilize Technology, Require Learner Participa- tion, and Evaluate (Karakis et al., 2016) 2,610 Dick and Carey 1978 Instructional Goals, Instructional Analysis, Entry Behav- iors, Performance Objectives, Criterion, Instructional Strategy, Instructional Materials, Formative Evaluation (Dick, 1996) 2,000 Four-D 1974 Define, Design, Development, and Disseminate (Thiagarajan, 1974) 758 Merrill’s Principles 2002 Activation, Demonstration, Application, and Integration (Merrill, 2002) 44 SAM 2012 Gather Materials, Meet with Team, Create a Prototype, Ask for Feedback, and Make Changes (Allen & Sites, 2012) 486 aYear: the year when the models were invented bMRD: most referred document cn: number of the referred article based on the ERIC and Dimensions database during 2018-2022 Table 1 shows some instructional design models used by previous researchers. The table shows that the Four-D model by Thia- garajan (1974) was the first instructional design developed. This design consists of four stages: Define, Design, Development, and Disseminate. The Define stage deter- mines the product's theme, needs, and phi- losophy—in this context, a mathematics learning set—to be developed. This model initially aims to develop instructional materi- als for training teachers of exceptional chil- dren (Thiagarajan, 1974). However, in its development, this model is used in the field of education but in a broader scope, such as in the development of mathematics learning sets. On the other hand, SAM (Successive Approximation Model) is a newer instruc- tional model developed in 2012 by Allen & Sites (2012). SAM is an ID that results from the ADDIE model's simplification (Jung et al., 2019). This model has five steps: Gather Materials, Meet with Team, Create a Proto- type, Ask for Feedback, and Make Changes (Allen & Sites, 2012). Next, Table 1 also shows that the AD- DIE model is the most referenced model by researchers, with 17,000 articles and books referencing the model. Interestingly, despite having almost the same year the invention of the Four-D model developed by Thiagarajan, the difference in the number of articles refer- encing the Four-D model is far below AD- DIE. Some researchers argue that the AD- DIE model is the ID that has the most un- complicated syntax (Cheung, 2016). At the same time, the least referenced model ID is Merrill's Principles, developed in 2002. In Table 1, there are only 44 references that refer to the model as an aspect of the discus- sion in the article. This could be because Merrill's Principles were initially developed for management training, as stated by Jghamou et al. (2019). Therefore, it is under- stood that there are not many educational studies that examine the model. The interrelationship between instruc- tional design and Indonesian researchers is quite strong in the Indonesian context. This can be seen from the Google Scholar data- base in Indonesian, where there are more Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 16 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context than 6,200 research articles on the develop- ment of mathematics learning sets carried out by Indonesian researchers until 2021 using the keywords "Development" and "Learn- ing". b. Purpose and Significant of The Study Some of the previous studies have sys- tematically examined literature reviews re- lated to instructional design, as done by Challco et al. (2016), which reviewed 18 research articles on computer-based systems for automating the instructional design of collaborative learning scenarios. Next, an SLR was conducted by Pástor et al. (2018), which examines the use of Semantic Web Technologies in the context of instructional design. In the SLR, they reviewed 21 articles and produced recommendations related to virtual learning settings, and the systems should include semantic web technology features. Lastly, Chan et al. (2021) conduct- ed an SLR, which studies the implementation of instructional design pada virtual chemical laboratories. The SLR reviewed 76 research articles and concluded that virtual laborato- ries could be used as an effective comple- mentary tool or tempered alternative to natu- ral, hands-on laboratories; however, future research should focus on investigating skill- based learning outcomes using immersive VR and NUI technologies, as well as consid- ering instructional design in virtual chemical laboratories. However, from some of these studies, SLR related to instructional design in the development of teacher skills in the Indonesian context is limited. This research is necessary because the rapid development of technology requires teachers to know trends related to mathematics learning sets developed by ISTs and PSTs. Therefore, this study aims to describe the role of ID in as- sisting Indonesian PSTs and PSTs in devel- oping mathematics learning sets. The re- search questions are as follows (a) what are the mathematics learning sets developed by the Indonesian ISTs and PSTs; and (b) what ID model is used by the Indonesian ISTs and PSTs in developing mathematics learning sets. 2. Method This study is a systematic literature re- view (SLR) that employs the PRISMA pro- tocol to select the qualified articles to be studied. Nightingale (2009) suggests that the first stage of conducting SLR is by develop- ing a protocol that clearly defines: (1) the aims and objectives of the review, (2) the inclusion and exclusion criteria for studies, (3) how the study will be identified, and (4) the plan of analysis. Among those four defi- nitions, the second is the most critical point determining whether the SLR is well con- ducted. Nightingale (2009) uses six inclusion criteria that are (1) type of study, (2) type of participants, (3) type of intervention, (4) comparison, (5) outcome measures, and (6) other aspect related to the characteristic of the study. To ensure that the protocol is well conducted, then Moher et al. (2009) suggest the concept of PRISMA (Preferred Reporting Items for Systematic Reviews and Meta- Analyze), which consists of four stages of review, namely identification, screening, eligibility, and inclusion (see Figure 1 for the PRISMA steps in this study). Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 17 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context Figure 1. PRISMA Steps a. Search Identification This stage is done by determining the keywords used to find relevant articles. In addition, this study also determined the range of research years, databases, languages, PI- CO principles, and article types. The articles searched for are those in Indonesian, so the database that covers the article is ERIC and Dimensions. Next, the range of years used is the last five years or from 2018-2022, and the type of article used is a research article. Examples of keywords used are "Instruction- al Design", "Development", and "Mathemat- ics learning sets”. Lastly, the PICO (Partici- pant, Intervention, Comparison, and Out- come) principles are used to determine the keywords used in the article search process (Cooke et al., 2012). Examples of PICO- based keywords can be seen in Table 2. This stage resulted in as many as 521 articles that will proceed to the next stage. Table 2. Keywords Base on PICO Principles PICO Aspects Keywords Participants “Pre-Service Teachers”, “Teachers”, and “In-Service Teachers” Intervention “Instructional Design” and “Instructional Design Model” Comparison “Development” Outcomes “Learning Media” b. Article Screening This stage involves issuing research ar- ticles, not the desired publication type. Therefore, some articles of the proceedings, review articles and book chapters are deleted from the list. Proceeding-type articles are excluded since this type has a relatively lim- ited scientific impact, their relative im- Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 18 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context portance is shrinking, and they become obso- lete faster than the scientific literature (Usée et al., 2008). Next, review articles are also excluded since these articles do not convey the results of research carried out empirically (Short, 2009). Besides the article type aspect, exclusion criteria are also based on the lan- guage used. At this stage, this research se- lects only articles written in English. The last criterion is duplicated articles. Because this study uses two international databases, there will likely be duplication of articles between the two databases. This stage issued 374 arti- cles and left 147 articles to proceed to the next stage. c. Article Eligibility and Inclusion This stage is carried out by selecting el- igibility articles from articles that pass the screening stage. Some of the articles exclud- ed from the eligibility stage were because they were not well-structured, their methods were not robust, and they did not use any of the ID models, and a total of 92 articles were excluded from the list. Based on the results of the selection, there were as many as 55 articles that entered the inclusion stage for review. The data analysis process uses VoS Viewer and NVIVO 12. 3. Result and Discussion This section will be described the results of the analysis of articles that have been se- lected. Next, the article was analyzed accord- ing to research questions, namely (1) Math- ematics Learning Sets Developed by Indone- sian PSTs and ISTs, and (2) ID Models Used by the Indonesian PSTs and ISTs to Develop the Mathematics Learning Sets. In general, the results of the analysis of the article can be seen in Figure 2. (a) Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 19 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context (b) Figure 2. The Proportion of (a) Learning Sets, and (b) ID Models a. Mathematics Learning Sets Developed by Indonesian PSTs and ISTs One aspect of a teacher's success is how well the teacher develops the learning sets. Therefore, this section explains the learning sets developed in the 55 research articles (see Table 3 for details). Table 3 shows the types of developed mathematics learning sets In- donesian ISTs and PSTs have developed in the last five years. In general, student work- sheets are the most developed type of learn- ing set by Indonesian ISTs and PSTs, which can be seen for the circle size that is the big- gest of the others. For instance, a study by Basuki & Wijaya (2018) developed ethno- mathematics-based student worksheets. Fol- lowing research results from Wijayanti & Abadi (2021), which also develops STEM and Problem-Based Learning (PBL)-based student worksheets, as well as many more previous studies that develop student work- sheets by the Indonesian PSTs and ISTs (Bilad & Ekawati, 2022; Oktaviyanthi & Dahlan, 2018; Sagita et al., 2018; Sisi Pitriyana, 2019). Table 3. Learning Sets Developed by Previous Studies Type of Learning Sets n Authors Student Worksheet 15 (Basuki & Wijaya, 2018; Bilad & Ekawati, 2022; Effendi et al., 2019; Lestari & Saragih, 2018; Muchsin et al., 2018; Murtiyasa et al., 2020; Naqiyah & Rosana, 2019; Oktaviyanthi & Dahlan, 2018; Purwitaningrum & Prahmana, 2021; Rahmayani et al., 2018; Setiana et al., 2019; Sisi Pitriyana, 2019; Sunendar & Mahmudi, 2019; Ulandari et al., 2019; Wijayanti & Abadi, 2021) Lesson Plan 6 (Muchsin et al., 2018; Naqiyah & Rosana, 2019; Setiana et al., 2019; Sunendar & Mahmudi, 2019; Ulandari et al., 2019; Wijayanti & Abadi, 2021) Module 5 (Feriyanto & Putri, 2020; Hikayat et al., 2020; Patri & Heswari, 2021; Rahman, 2022; Setiana et al., 2019) Book 8 (Alim et al., 2021; Feriyanto & Putri, 2020; Kusumadewi et al., 2021; Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 20 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context Type of Learning Sets n Authors Purwitaningrum & Prahmana, 2021; Alim et al., 2021; Murtiyasa et al., 2020; Pramasdyahsari et al., 2022; Setyawan et al., 2020) Test 1 (Hidayah et al., 2020) Learning Media 15 (Astuti et al., 2019; Az-Zahroh et al., 2019; Hanifah et al., 2019; Hidayati & Irmawati, 2019; H. Kurniawan & Susanti, 2020; Kusumadewi et al., 2021; Murtikusuma et al., 2019; Murtiyasa et al., 2020; Pardimin et al., 2019; Rachmadina & Pratiwi, 2021; W. B. Saputra et al., 2021; Setyaningrum & Waryanto, 2018; Suddin & Deda, 2020; Syarfina et al., 2022; Verawati et al., 2022) Games 6 (Amir et al., 2019; Az-Zahroh et al., 2019; Setyaningrum & Waryanto, 2018; Suddin & Deda, 2020; Umbara et al., 2021; Yansen et al., 2019) Android Apps 5 (Setyaningrum & Waryanto, 2018; Suddin & Deda, 2020; Verawati et al., 2022; Arifin et al., 2021; Kurniawan & Susanti, 2020) Learning Materials 8 (Hanifah et al., 2019; Hikayat et al., 2020; Kusumadewi et al., 2021; Lestari & Saragih, 2018; Naqiyah & Rosana, 2019; Purwitaningrum & Prahmana, 2021; Rusli et al., 2021; Ulandari et al., 2019) Table 3 shows the types of developed mathematics learning sets Indonesian ISTs and PSTs have developed in the last five years. In general, student worksheets are the most developed type of learning set by Indo- nesian ISTs and PSTs, which can be seen for the circle size that is the biggest of the oth- ers. For instance, a study by Basuki & Wija- ya (2018) developed ethnomathematics- based student worksheets. Following re- search results from Wijayanti & Abadi (2021), which also develops STEM and Problem-Based Learning (PBL)-based stu- dent worksheets, as well as many more pre- vious studies that develop student work- sheets by the Indonesian PSTs and ISTs (Bilad & Ekawati, 2022; Oktaviyanthi & Dahlan, 2018; Sagita et al., 2018; Sisi Pitriyana, 2019). Next is learning media, defined as any- thing that may be utilized to channel signals from the sender to the recipient to stimulate learners' ideas, feelings, interests, and readi- ness to learn to attain the aim of learning effectively (Ediyani et al., 2020). From many selected articles, as many as 15 examine the development of learning media as carried out by Rachmadina & Pratiwi (2021), where they develop learning media in the form of videos integrated with the Powtoon applica- tion. In addition, some research results from Syarfina et al. (2022) examine the develop- ment of learning media suitable for teaching numbers in early childhood. After the student worksheet and learning media were developed in 15 studies, there were learning materials developed in 8 stud- ies. The use of learning materials provides good benefits in learning to help teachers explain concepts so that student's under- standing of the concepts being taught be- comes better (Hasibuan et al., 2019; Tuimur & Chemwei, 2015). Good learning materials must maximize learning potential by encour- aging intellectual, aesthetic, and emotional involvement that stimulates right and left brain activity to maximize memory (Harsono, 2015). Learning materials can be loaded in books or digital media (Abadi et al., 2018). Another learning set is a lesson plan, where several previous studies have devel- oped lesson plans as part of the learning sets developed, such as those conducted by Setiana et al. (2019), who develop lesson sets where the lesson plan is part of the lesson Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 21 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context sets. Next is the research of Muchsin et al. (2018), the same as Setiana et al. (2019), where the development of lesson plans is part of the development of learning sets. Next, many android applications are cur- rently being developed for various purposes, including to support mathematics learning. The android application is an example of using technology in learning, where learning will be more varied and attract students' at- tention (Baihaki et al., 2022). Moreover, the use of android applications in learning math- ematics can also help improve problem- solving abilities, such as those conducted by Arifin et al. (2021) and Verawati et al. (2022). In addition to the three types of learning sets described, there are also other types, such as games, tests, books, android apps, and learning media. However, with so many researchers developing student worksheets, it shows that the learning set is the most need- ed by teachers. This is inseparable from the characteristics and benefits of student work- sheets, which are effective for "guiding" stu- dents in learning activities and measuring student understanding (Bakri et al., 2020). b. ID Models Used by the Indonesian PSTs and ISTs to Develop the Math- ematics Learning Sets In a Research and Development study, the ID models used are very decisive and related to the feasibility level of the product being developed. As for the literature study results from 90 articles analyzed, the results were obtained as stated in Table 4. Table 4 shows that the ADDIE model is the most referenced model by the Indonesian PSTs and ISTs (N = 15) as the basis for the theory of developing the learning sets, they devel- oped. This is inseparable from the ADDIE character, which has a simple and easy-to- understand syntax compared to other models. This is in line with the opinion of Peterson (2003) and Ishartono et al. (2016), who argue that the ADDIE model is a practical, simple framework for instructional design. Table 4. The ID Model Used for Previous Studies ID Models n Articles ADDIE model 14 (Astuti et al., 2019; Basuki & Wijaya, 2018; Bilad & Ekawati, 2022; Hanifah et al., 2019; Hikayat et al., 2020; Patri & Heswari, 2021; Rachmadina & Pratiwi, 2021; Rahman, 2022; Rahmayani et al., 2018; W. B. Saputra et al., 2021; Satiti et al., 2021; Setyaningrum & Waryanto, 2018; Suddin & Deda, 2020; Umbara et al., 2021) Plomp model 5 (Alim et al., 2021; Amanda et al., 2022; Oktaviyanthi & Dahlan, 2018; Setyaningsih et al., 2019; Wijayanti & Abadi, 2021) Nieveen model 2 (Amanda et al., 2022; Amir et al., 2019) Formative Re- search 1 (Angriani, 2018) Agile 1 (Arifin et al., 2021) Design Research 11 (Dasaprawira & Susanti, 2019; Effendi et al., 2019; Jannah et al., 2019; Nizar et al., 2018; Nusantara et al., 2020, 2021; Purwitaningrum & Prahmana, 2021; Putri & Zulkardi, 2020; Richardo & Martyanti, 2019; Sumandya, 2020; Yansen et al., 2019) Lee Owen 1 (Az-Zahroh et al., 2019) Tessmer Develop- ment Model 3 (Feriyanto & Putri, 2020; H. Kurniawan & Susanti, 2020; Nasution et al., 2019) Watson-Glaser Critical Thinking Test 1 (Hidayah et al., 2020) Four-D 13 (Hidayati & Irmawati, 2019; Khasanah & Astuti, 2018; Lestari & Saragih, 2018; Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 22 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context ID Models n Articles Muchsin et al., 2018; Murtikusuma et al., 2019; Murtiyasa et al., 2020; Naqiyah & Rosana, 2019; Setyawan et al., 2020; Sisi Pitriyana, 2019; Sunendar & Mahmudi, 2019; Syarfina et al., 2022; Ulandari et al., 2019; Verawati et al., 2022) Borg & Gall 4 (Kusumadewi et al., 2021; Pardimin et al., 2019; Pramasdyahsari et al., 2022; Setiana et al., 2019) Dick & Carey 2 (Hasibuan et al., 2019; Setiana et al., 2019) Software Devel- opment Life Cycle (SDLC) 1 (Takwin et al., 2018) DeVellis 1 (Takwin et al., 2018) The next model was the Four-D devel- oped by Thiagarajan (1974), which recorded 13 studies referencing this model. In addition to ADDIE being considered simple, some previous researchers have also argued that the Four-D model is also categorized as sim- ple and easy to understand (Erawati et al., 2020; Kurniaman & Noviana, 2020). Of the many articles that use the ADDIE model, one study does not fully use ADDIE, namely the one conducted by Murtikusuma et al. (2019) where the syntax only stops at the development stage. According to him—who is a PSTs—this is due to limited research costs and time. The same is true in other In- donesian learning sets development stud- ies—beyond the scope of the articles selected in this study—that use the Four-D model but only stop at the development stage for rela- tively the same reason (Palloan & Swandi, 2019). The next model that becomes a priority is design research, where this model was developed by (Gravemeijer & Cobb, 1999). Design Research is designing systematic educational interventions consisting of de- sign, development, and evaluation activities to improve the quality of educational activi- ties or programs (Putrawangsa, 2018). De- sign Research has two objectives that are interrelated with one another, namely (1) developing educational interventions to solve learning problems and (2) formulating theo- retical arguments that underlie the effective- ness of these interventions (intervention the- ory). Many mathematics education research- ers often use this model to develop Local Instructional Trajectory (LIT), which is often combined with the Realistic Mathematics Education approach first developed by Freudenthal in 1997 (Heuvel-Panhuizen, 1993). In addition to these three models, there are also many other models used by re- searchers from selected articles, such as Plomp Model, Nieveen Model, Formative Research Model, Agile Model, Lee Owen Model, Tessmer Development Model, Wat- son-Glaser Critical Thinking Test Model, Borg & Gall, Dick & Carey, Software De- velopment Life Cycle (SDLC), dan DeVellis Model. c. Significant and Limitations of The Study The findings of this study were that in the first question, it was found that the most studied types of learning sets were mathe- matics student worksheets and mathematics learning media. While the least developed is the mathematical test. In the development of learning sets, tests cannot be separated from student worksheets and learning media (D. A. Kurniawan et al., 2022; Sulistyanto et al., 2022). This is because the test measures how effective the two learning sets are in improv- Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 23 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context ing students' understanding of the mathemat- ics material taught (Hermita et al., 2022; Pramita et al., 2021). Therefore, this finding can provide an overview to subsequent re- searchers to improve the development of the mathematics test as a measuring tool for mathematical understanding. In addition, the findings obtained from the results of the second question analysis found that the ADDIE model was the most widely used in mathematics learning sets development research. This further strength- ens many researchers' opinions that the AD- DIE model is the most straightforward model for researchers to understand and follow in developing learning products. Indeed, this research still has much room that can be studied further, such as in the material aspect, which raises the question of what material is the most studied in the learning sets development research conduct- ed by Indonesian ISTs and PSTs, as well as the breadth of year and database coverage, which in this study only includes research from ERIC and Dimensions in the last five years. 4. Conclusion This SLR research aims to answer two research questions, namely (1) mathematics learning sets developed by Indonesian PSTs and ISTs and (2) ID Models used by the Indonesian PSTs and ISTs to develop the mathematics learning sets. As for the first question, it was found that the development of student worksheets and learning media dominated the learning sets developed by the Indonesian PSTs and ISTs. Meanwhile, the second question found that the ADDIE caters model was the most significant proportion of the research numbers, followed by the Four- D model, which was not far from ADDIE. Subsequent research can be done by developing research questions related to research approaches and mathematical materials studied. 5. References Abadi, M. K., Asih, E. C. M., & Jupri, A. (2018). The Development of Interactive Mathematics Learning Material Based on Local Wisdom with .swf Format. Journal of Physics: Conference Series, 1013(1). https://doi.org/10.1088/1742- 6596/1013/1/012131 Agustina, E. N. S., Widadah, S., & Nisa, P. A. (2021). Developing Realistic Mathematics Problems Based on Sidoarjo Local Wisdom. Mathematics Teaching Research Journal, 13(4), 181–201. https://files.eric.ed.gov/fulltext/EJ13323 36.pdf Alim, J. A., Hermita, N., Alim, M. L., Wijaya, T. T., & Pereira, J. (2021). Developing a Math Textbook using realistic Mathematics Education Approach to increase elementary students’ learning motivation. Jurnal Prima Edukasia, 9(2), 193–201. https://doi.org/10.21831/jpe.v9i2.39393 Introduction Allen, M., & Sites, R. (2012). Leaving ADDIE for SAM: An Agile Model for Developing the Best Learning Experiences. https://www.td.org/books/leaving- addie-for-sam-an-agile-model-for- developing-the-best-learning- experiences Amanda, F. F., Sumitro, S. B., Lestari, S. R., & Ibrohim, I. (2022). Developing Complexity Science-Problem Based Learning Model to Enhance Conceptual Mastery. Journal of Education and Learning (EduLearn), 16(1), 65–75. https://eric.ed.gov/?id=EJ1339625 Amir, M. F., Mufarikhah, I. A., Wahyuni, A., Nasrun, N., & Rudyanto, H. E. (2019). Developing ‘Fort Defending’Game as a Learning Design for Mathematical Literacy Integrated to Primary School Curriculum in Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 24 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context Indonesia. Elementary Education Online, 18(3). Angriani, A. D. (2018). Developing Test Instruments for Measurement of Students’ High-Order Thinking Skill on Mathematics in Junior High School in Makassar. Journal of Physics: Conference Series, 1028(1), 12169. https://iopscience.iop.org/article/10.108 8/1742-6596/1028/1/012169/meta Arifin, M., Sholeh, M., Hafiz, A., Agustin, R., & Wardana, M. (2021). Developing Interactive Mobile Mathematics Inquiry to Enhance Students’ Problem-solving Skill. International Association of Online Engineering. https://www.learntechlib.org/p/218685/ Astuti, D. P., Bhakti, Y. B., & Astuti, I. A. D. (2019). Developing Adobe Flash- based Mathematics Learning Media for 7th-grade Students of Junior High School. Journal of Physics: Conference Series, 1188(1), 12098. https://iopscience.iop.org/article/10.108 8/1742-6596/1188/1/012098/meta Az-Zahroh, S. F., Thaariq, Z. Z. A., Surahman, E., Widyasari, C. M., Qolbi, M. S., & Diana, R. C. (2019). Developing Ethic Game (Ethnomathematics Game): The Instructional Media of Culture Mathematics With Tringo By Ki Hadjar Dewantara. JPP (Jurnal Pendidikan Dan Pembelajaran), 26(2), 43–50. http://journal2.um.ac.id/index.php/jpp/a rticle/view/9762 Baihaki, Djamilah, S., & Lazwardi, A. (2022). Developing Interactive Learning Media Based on Adobe Animate Applications for Geometry Transformation. Kalamatika: Jurnal Pendidikan Matematika, 7(2), 191–206. https://doi.org/10.29103/ijevs.v1i8.2260 Bakri, F., Permana, H., Wulandari, S., & Muliyanti, D. (2020). Journal of Technology and Science Education Student Worksheet with AR Videos : Physics Learning Media. Journal of Technology and Science Education, 10(2), 231–240. Basuki, W. A., & Wijaya, A. (2018). The development of student worksheet based on realistic mathematics education. Journal of Physics: Conference Series, 1097(1), 12112. https://iopscience.iop.org/article/10.108 8/1742-6596/1097/1/012112/meta Bilad, D. I., & Ekawati, R. (2022). Development of Student Worksheet on Probability With Realistic Mathematics Education Approachusing Pandemic Context. Jurnal Ilmiah Pendidikan Matematika Volume, 11(2). Branch, R. M. (2009). Instructional Design: The ADDIE Approach. Springer. https://doi.org/10.1007/978-0-387- 09506-6 Challco, G. C., Bittencourt, I. I., & Isotani, S. (2016). Computer-based systems for automating instructional design of collaborative learning scenarios: A systematic literature review. International Journal of Knowledge and Learning, 11(4), 273–297. https://doi.org/10.1504/IJKL.2016.0847 45 Chen, L.-L. (2016). A Model for Effective Online Instructional Design. Literacy Information and Computer Education Journal, 7(2), 2303–2308. https://doi.org/10.20533/licej.2040.2589 .2016.0304 Cheung, L. (2016). Using the ADDIE Model of Instructional Design to Teach Chest Radiograph Interpretation. Journal of Biomedical Education, 2016, 1–6. https://doi.org/10.1155/2016/9502572 Cooke, A., Smith, D., & Booth, A. (2012). Beyond PICO: The SPIDER tool for qualitative evidence synthesis. Qualitative Health Research, 22(10), 1435–1443. https://doi.org/10.1177/1049732312452 938 Dasaprawira, M. N., & Susanti, E. (2019). Developing Mathematics Questions of PISA Type Using Bangka Context. Journal on Mathematics Education, 10(2), 303–314. https://eric.ed.gov/?id=EJ1218133 Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 25 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context Dick, W. (1996). The Dick and Carey model: Will it Survive the Decade? Educational Technology Research and Development, 44(3), 55–63. https://doi.org/10.1007/BF02300425 Ediyani, M., Hayati, U., Salwa, S., Samsul, S., Nursiah, N., & Fauzi, M. B. (2020). Study on Development of Learning Media. Budapest International Research and Critics Institute (BIRCI- Journal): Humanities and Social Sciences, 3(2), 1336–1342. https://doi.org/10.33258/birci.v3i2.989 Effendi, K. N. S., Putri, R. I. I., & Yanawati, P. (2019). Developing Mathematics Worksheet Using Futsal Context for School Literacy Movement. Journal on Mathematics Education, 10(2), 203– 214. Erawati, L., Waskito, W., & Krismadinata, K. (2020). Development of Weblog Based Learning Media in Information and Communication Technology Guidelines. International Journal of Educational Dynamics, 2(1), 84–90. https://doi.org/10.24036/ijeds.v2i1.236 Feriyanto, F., & Putri, R. O. E. (2020). Developing mathematics module based on literacy and higher order thinking skills (HOTS) questions to train critical thinking ability of high school students in mojokerto. Journal of Physics: Conference Series, 1594(1), 12014. Firdausy, A. R., Setyaningsih, N., Ishabu, L. S., & Waluyo, M. (2019). The Contribution of Student Activity and Learning Facilities to Learning Independency and it’s Impact on Mathematics Learning Outcomes in Junior High School. Indonesian Journal on Learning and Advanced Education (IJOLAE), 1(2), 29–37. https://doi.org/10.23917/ijolae.v1i2.810 4 Gravemeijer, K., & Cobb, P. (1999). Design research from a learning design perspective. In Educational Design Research (pp. 1–206). http://www.eric.ed.gov/ERICWebPortal /recordDetail?accno=EJ815766 Hanifah, N. H., Arifuddin, A., Walid, M., Padil, M., Bashith, A., & Busro, B. (2019). Developing autoplay media based mathematics teaching materials for elementary school. Journal of Physics: Conference Series, 1175(1), 12265. Harsono, Y. M. (2015). Developing Learning Materials for Specific Purposes. TEFLIN Journal - A Publication on the Teaching and Learning of English, 18(2), 169–179. https://doi.org/10.15639/teflinjournal.v1 8i2/169-179 Hasibuan, A. M., Saragih, S., & Amry, Z. (2019). Development of Learning Materials Based on Realistic Mathematics Education to Improve Problem Solving Ability and Student Learning Independence. International Electronic Journal of Mathematics Education, 14(2), 243–252. https://doi.org/10.29333/iejme/5729 Hermita, N., Putra, Z. H., Alim, J. A., Wijaya, T. T., Anggoro, S., & Diniya. (2022). Elementary Teachers’ Perceptions on Genially Learning Media Using Item Response Theory (IRT). Indonesian Journal on Learning and Advanced Education (IJOLAE), 4(1), 1–20. https://doi.org/10.23917/ijolae.v4i2.147 57 Heuvel-Panhuizen, M. Van Den. (1993). The Didactical Use of Models in Realistic Mathematics Education: An Example From A Longitudinal Trajectory and Percentage. Journal of Immunology (Baltimore, Md. : 1950), 25(1), 89–108. https://doi.org/10.1007/BF01274104 Hidayah, N., Ramli, M., Mappiare, A., Hanafi, H., Yuliana, A. T., Kurniawan, N. A., & Eva, N. (2020). Developing Critical Thinking Skills Test In Indonesia. PalArch’s Journal of Archaeology of Egypt/Egyptology, 17(3), 815–826. Hidayati, N., & Irmawati, F. (2019). Developing digital multimedia of human anatomy and physiology Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 26 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context material based on STEM education. JPBI (Jurnal Pendidikan Biologi Indonesia), 5(3), 497–510. Hikayat, C., Suparman, Hairun, Y., & Suharna, H. (2020). Design of Realistic Mathematics Education Approach to Improve Critical Thinking Skills. Universal Journal of Educational Research, 8(6), 2232–2244. https://doi.org/10.13189/ujer.2020.0806 06 Ishartono, N., Juniati, D., & Lukito, A. (2016). Developing Mathematics Teaching Devices in the Topic of Trigonometry Based on Guided Discovery Teaching Method. 1(2), 154– 171. https://doi.org/doi.org/10.23917/jramath edu.v1i2 Ishartono, N., Nurcahyo, A., Waluyo, M., Prayitno, H. J., & Hanifah, M. (2022). Integrating Geogebra Into the Flipped Learning Approach To Improve Students ’ Self-Regulated Learning During the Covid-19 Pandemic : an. Journal on Mathematics Education, 13(1), 69–85. https://doi.org/10.22342/jme.v13i1.pp6 9-86 Ishartono, N., Setyono, I. D., Maharani, A. R., & Firdaus, S. (2022). The Quality of Mathematics Teaching Aids Developed by Mathematics Pre-Service Teachers in Indonesia. Jurnal Varidika, 34(1), 14– 27. https://doi.org/10.23917/varidika.v1i1.1 8034 Jannah, R. D., Putri, R. I. I., & Zulkardi. (2019). Soft Tennis and Volleyball Contexts in Asian Games for Pisa-Like Mathematics Problems. Journal on Mathematics Education, 10(1), 157– 169. https://doi.org/10.22342/jme.10.1.5248. 157-170 Jghamou, A., Maziri, A., Mallil, E. H., & Echaabi, J. (2019). Comparison of training methods with ELECTRE I and Merrill’s principles. European Journal of Training and Development, 43(5–6), 592–618. https://doi.org/10.1108/EJTD- 09-2018-0093 Jung, H., Kim, Y. R., Lee, H., & Shin, Y. (2019). Advanced instructional design for successive E-learning: Based on the successive approximation model (SAM). International Journal on E- Learning: Corporate, Government, Healthcare, and Higher Education, 18(2), 191–204. https://www.learntechlib.org/p/187327/ Kai, S., Chu, W., Reynolds, R. B., Tavares, N. J., & Notari, M. (2021). 21st Century Learning. Springer. https://doi.org/10.4018/978-1-7998- 4102-9.ch002 Karakis, H., Karamete, A., & Okçu, A. (2016). The Effects of a Computer- Assisted Teaching Material, Designed According to the ASSURE Instructional Design and The ARCS Model of Motivation, on Students’ Achievement Levels in a Mathematics Lesson and Their Resulting Attitudes. European Journal of Contemporary Education, 15(1), 105–113. https://doi.org/10.13187/ejced.2016.15. 105 Khalil, M. K., & Elkhider, I. A. (2016). Applying Learning Theories and Instructional Design Models For Effective Instruction. Advances in Physiology Education, 40(2), 147–156. https://doi.org/10.1152/advan.00138.20 15 Khasanah, U., & Astuti, D. (2018). Developing Mathematics Learning Model of Thinking Empowerment by Question (TEQ) with TAI Setting to Improve Students Metacognition Ability. International Journal of Active Learning, 3(2), 80–85. Kurniaman, O., & Noviana, E. (2020). Child Comic Development with The Themes of Prevention Covid-19: Validation Analysis. Metodik Didaktik, 16(2), 88– 94. Kurniawan, D. A., Astalini, A., Darmaji, D., Tanti, T., & Maryani, S. (2022). Innovative Learning: Gender Perception Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 27 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context of e-Module Linear Equations in Mathematics and Physics. Indonesian Journal on Learning and Advanced Education (IJOLAE), 4(2), 92–106. https://doi.org/10.23917/ijolae.v4i2.166 10 Kurniawan, H., & Susanti, E. (2020). Development of M-Learning Media With Indonesian Realistic Mathematics Education’s Approach. SEMANTIK Conference of Mathematics Education (SEMANTIK 2019), 142–145. Kusumadewi, R. F., Neolaka, A., & Yasin, M. (2021). Developing Electronic Teaching Materials Through Comic Mathematics Media to Increase Student Learning Independence During the Covid-19 Pandemic in Indonesia. Ilkogretim Online, 20(1), 708–716. https://www.ilkogretim- online.org/fulltext/218-1609990375.pdf Lestari, S. A. B., & Saragih, S. (2018). Developing Learning Materials Based on Realistic Mathematics Education with Malay Culture Context to Improve Mathematical Communication Ability and Self-Efficacy of Students in SMPN 2 Talawi. American Journal of Educational Research, 6(11), 1473– 1480. Luik, P., & Suviste, R. (2018). Perceptions of Technological, Pedagogical And Content Knowledge (TPACK) among pre-service teachers in Estonia. Educ Inf Technol, 23, 741–755. https://doi.org/10.1007/s10639-017- 9633-y Merrill, M. D. (1994). Designing and Instructional Design. In Educational technology research and development (Issue 42). https://books.google.com/books?hl=en &lr=&id=kPB- _L4JcOoC&oi=fnd&pg=PR7&dq=%22 instructional+design+is%22&ots=kGVo 5nWzVa&sig=CyxwSMpVBOsIFwckO PPmnoRIMZU Merrill, M. D. (2002). First principles of instruction. In Educational Technology Research and Development (Vol. 50, Issue September). Springer. https://doi.org/10.1007/BF02505024 Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Group, P. (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Annals of Internal Medicine, 151(4), 246–269. www.annals.org Muchsin, S. B., Kamaruddin, R., & Rosida, V. (2018). Developing Learning Instruments of Geometry Based on Van Hiele Theory to Improving Students’ Character. Journal of Physics: Conference Series, 1028(1), 12137. Murtikusuma, R. P., Fatahillah, A., Oktavianingtyas, E., Hussen, S., & Lailiya, N. (2019). The Development of Interactive Mathematics Learning Media Based on Schoology and Visual Basic Through Industrial Revolution 4.0. IOP Conference Series: Earth and Environmental Science, 243(1), 12137. Murtiyasa, B., Jannah, I. M., & Rejeki, S. (2020). Designing Mathematics Learning Media Based on Mobile Learning for Ten Graders of Vocational High School. Universal Journal of Educational Research, 8(11), 5637– 5647. https://doi.org/10.13189/ujer.2020.0811 68 Naqiyah, M., & Rosana, D. (2019). Developing Physics Learning Tools Based on Local Wisdom in the Form of Musical Instrument of Gandrang Bulo Dance as Learning Source in Sound Wave. Journal for the Education of Gifted Young Scientists, 7(3), 609–626. Nasution, R. S., Fauzi, K. M. A., & Syahputra, E. (2019). Developing Mathematics Problem Based on PISA Level of Space And Shape Content to Measure Student’s Mathematics Problem Solving Ability. American Journal of Educational Research, 7(10), 660–669. Nightingale, A. (2009). A Guide To Systematic Literature Reviews. Surgery, 27(9), 381–384. Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 28 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context https://doi.org/10.1016/j.mpsur.2009.07 .005 Nizar, H., Putri, R. I. I., & Zulkardi. (2018). Developing Pisa-Like Mathematics Problem Using the 2018 Asian Games Football and Table Tennis Context. Journal on Mathematics Education, 9(2), 183–194. https://doi.org/10.22342/jme.9.2.5246.1 83-194 Nusantara, D. S., Zulkardi, & Putri, R. I. I. (2021). Designing Pisa-like Mathematics Problem Using a COVID- 19 transmission map context. AIP Conference Proceedings, 2438(1), 20005. Nusantara, D. S., Zulkardi, Z., & Putri, R. I. I. (2020). Designing PISA-like mathematics problem relating change and relationship using physical distancing context. Journal of Physics: Conference Series, 1663(1), 12004. Oktaviyanthi, R., & Dahlan, J. A. (2018). Developing guided worksheet for cognitive apprenticeship approach in teaching formal definition of the limit of a function. IOP Conference Series: Materials Science and Engineering, 335(1), 12120. Palloan, P., & Swandi, A. (2019). Development of learning instrument of active learning strategy integrated with computer simulation in physics teaching and learning on makassar state university. Journal of Physics: Conference Series, 1157(3). https://doi.org/10.1088/1742- 6596/1157/3/032016 Pardimin, Arcana, N., & Supriadi, D. (2019). Developing media based on the information and communications technology to improve the effectiveness of the direct instruction method in mathematics learning. Journal for the Education of Gifted Young Scientists, 7(4), 1311–1323. Pástor, D., Jiménez, J., Gómez, O. S., & Isotani, S. (2018). New Perspectives in Instructional Design using Semantic Web Technologies: A Systematic Literature Review. Ingeniería y Desarrollo, 36(1), 215–239. https://doi.org/10.14482/inde.36.1.1094 7 Patri, S. F. D., & Heswari, S. (2021). Development of Ethnomathematic- Based on Mathematics E-Module To Improve Students’ Logical Thinking Skills. AIP Conference Proceedings, 2330(1), 40005. Peterson, C. (2003). Bringing ADDIE to life: instructional design at its best - learning & technology library (LearnTechLib). Journal of Educatioanal Multimedia and Hypermedia, 12(3), 227–241. http://www.learntechlib.org/p/2074/ Pramasdyahsari, A. S., Setyawati, R. D., & Rubowo, M. R. (2022). Developing Group Theory Textbook which Connected to The School Mathematics’ Contents. AIP Conference Proceedings, 2577(1), 20049. Pramita, M., Sukmawati, R. A., Purba, H. S., Wiranda, N., Kusnendar, J., & Sajat, M. S. (2021). Student Acceptance of E- learning to Improve Learning Independence in the Department of Computer Education. Indonesian Journal on Learning and Advanced Education (IJOLAE), 4(1), 34–44. https://doi.org/10.23917/ijolae.v4i1.926 5 Purwitaningrum, R., & Prahmana, R. C. I. (2021). Developing instructional materials on mathematics logical thinking through the Indonesian realistic mathematics education approach. International Journal of Education and Learning, 3(1), 13–19. http://download.garuda.kemdikbud.go.i d/article.php?article=2633816&val=245 31&title=Developing instructional materials on mathematics logical thinking through the Indonesian realistic mathematics education approach Putrawangsa, S. (2018). DESIGN RESEARCH Sebagai Framework Desain Pembelajaran (1st ed., Issue January 2019). Penerbit Sanabil. Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 29 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context Putri, R. I. I., & Zulkardi. (2020). Designing PISA-like mathematics task using Asian games context. Journal on Mathematics Education, 11(1), 135–144. https://doi.org/10.22342/jme.11.1.9786. 135-144 Rachmadina, D., & Pratiwi, D. D. (2021). Developing mathematics video assisted by powtoon application in contextual learning approach. Journal of Physics: Conference Series, 1796(1), 12027. Rahman, S. A. (2022). The Development of Mathematics E-Modules by Using Flip PDF Professional Software on Algebraic Forms of Material. Annual International COnference on Islamic Education for Students, 1(1). Rahmayani, E., Irwandi, I., & Rajibussalim, R. (2018). Developing worksheets through ISLE-based STEM approach and implementing them on senior high school students. Journal of Physics: Conference Series, 1088(1), 12091. Rejekiningsih, T. (2019). Infographic Media in Shaping the Youth ’ s Critical Thinking Ability to Cope with Social Conflict in the Age of Digital Technology. Icoet 2019, 372(Proceedings of the International Conference on Education Technology (ICoET 2019)), 307–312. Richardo, R., & Martyanti, A. (2019). Developing ethnomathematical tasks in the context of yogyakarta to measure critical thinking ability. Journal of Physics: Conference Series, 1188(1), 12063. Rusli, R., Rahman, A., Ahmar, A. S., Musa, H., & Lince, R. (2021). Development of teaching materials for digital higher education in the industrial revolution 4.0 era. Linguistics and Culture Review, 5(1), 361–366. Sagita, L., Widagsa, R., & Dwipa, N. M. S. (2018). Developing Bilingual Scientific-Worksheet for Indefinite Integral. Journal on Mathematics Education, 9(2), 249–258. Saltan, F., & Arslan, K. (2017). A comparison of in-service and pre- service teachers’ technological pedagogical content knowledge self- confidence. Cogent Education, 4(1). https://doi.org/10.1080/2331186X.2017. 1311501 Saputra, E., & Fahrizal, E. (2019). The Development of Mathematics Teaching Materials through Geogebra Software to Improve Learning Independence. Malikussaleh Journal of Mathematics Learning (MJML), 2(2), 39–44. https://doi.org/10.29103/mjml.v2i2.186 0 Saputra, W. B., Utami, N. W., & Kusuma, I. A. (2021). Developing Culture-Based Mathematics Learning Media with Adobe Flash for JHS Students. Journal of Physics: Conference Series, 1823(1), 12114. Satiti, W. S., Alfatah, D. A., & Umardiyah, F. (2021). Development of PISA-like mathematics problems within personal- context for junior high school students. APPLICATION: Applied Science in Learning Research, 1(2), 99–105. Setiana, D. S., Santosa, R. H., Izzaty, R. E., & Herawan, T. (2019). Developing mathematics learning tools to stimulate critical thinking. International Journal of Advanced Science and Technology, 128, 81–90. Setyaningrum, W., & Waryanto, N. H. (2018). Developing mathematics edutainment media for Android based on students’ understanding and interest: A teachers’ review. Journal of Physics: Conference Series, 983(1), 12093. Setyaningsih, N., Rejeki, S., & Ishartono, N. (2019). Developing Realistic and Child- friendly Learning Model for Teaching Mathematics. JRAMathEdu (Journal of Research and Advances in Mathematics Education), 4(2), 79–88. https://doi.org/10.23917/jramathedu.v4i 2.8112 Setyawan, F., Prasetyo, P. W., & Nurnugroho, B. A. (2020). Developing Complex Analysis Textbook to Enhance Students’ Critical Thinking. Journal of Research and Advances in Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 30 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context Mathematics Education, 5(1), 26–37. Short, J. (2009). The art of writing a review article. In Journal of Management (Vol. 35, Issue 6, pp. 1312–1317). https://doi.org/10.1177/0149206309337 489 Sisi Pitriyana, S. (2019). Design of Student Worksheet Problem based Learning to Improve Critical Thinking Skills Students of Class VIII Junior High School in Indonesia. Journal of Advanced Research in Social and Behavioural Sciences, 17(1), 1–8. https://www.akademiabaru.com/doc/AR SBSV17_N1_P1_8.pdf Suddin, S., & Deda, Y. N. (2020). Education game based on Timor local wisdom as an android-based mathematics learning media. Al-Jabar: Jurnal Pendidikan Matematika, 11(2), 227–246. Sulistyanto, H., Anif, S., Narimo, S., Sutopo, A., Izzul Haq, M., & Abdul Nasir Zakaria, G. (2022). Education Application Testing Perspective to Empower Students’ Higher Order Thinking Skills Related to The Concept of Adaptive Learning Media. Indonesian Journal on Learning and Advanced Education, 4(3), 257–271. https://doi.org/10.23917/ijolae.v4i3.194 32 Sumandya, W. (2020). Developing Assessment of Vocation-Based HOTS on Mathematics Subjects for X Class of Vocational School. International Journal of Scientific & Technology Research, 9(2), 2900–2903. http://repo.mahadewa.ac.id/id/eprint/70 3/1/14. Hots%2C IJSTR 2020.pdf Sun, Z., Xie, K., & Anderman, L. H. (2018). The role of self-regulated learning in students’ success in flipped undergraduate math courses. Internet and Higher Education, 36(May 2016), 41–53. https://doi.org/10.1016/j.iheduc.2017.09 .003 Sunendar, A., & Mahmudi, A. (2019). Developing Learning Kit of Geometry for Vocational School Grade X Based on Multiple Intelligence Theory. Southeast Asian Mathematics Education Journal, 9(1), 65–76. Syarfina, Basri, Hasibuan, R. H., Abidin, Z., Ulya, K., Tursina, A., Amri, K., Veryawan, & Fadli, M. (2022). Developing mathematics learning media to introduce the concept of numbers to early childhood. AIP Conference Proceedings, 2524(1), 20005. Takwin, M., Pansri, O., Parnichparinchai, T., & Vibulrangson, S. (2018). Developing a Self-Assessment Instrument for Analysis of the Social and Personal Competencies of Teachers in Senior High Schools in Indonesia. Journal of Physics: Conference Series, 1028(1), 12088. Thiagarajan, S. (1974). Instructional development for training teachers of exceptional children. In A sourcebook (Issue Mc). https://eric.ed.gov/?id=ED090725 Tuimur, H. N., & Chemwei, B. (2015). Availability and Use of Instructional Materials in the Teaching of Conflict and Conflict Resolution in Primary Schools in Nandi North District, Kenya. International Journal of Education and Practice, 3(6), 224–234. https://doi.org/10.18488/journal.61/201 5.3.6./61.6.224.234 Ulandari, L., Amry, Z., & Saragih, S. (2019). Development of Learning Materials Based on Realistic Mathematics Education Approach to Improve Students’ Mathematical Problem Solving Ability and Self-Efficacy. International Electronic Journal of Mathematics Education, 14(2), 375– 383. Umbara, U., Susilana, R., & Puadi, E. F. W. (2021). Algebra Dominoes Game: Re- Designing Mathematics Learning during the COVID-19 Pandemic. International Journal of Instruction, 14(4), 483–502. Usée, C., Larivière, V., & Archambault, É. (2008). Conference proceedings as a source of scientific information: A Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 5 (1) (2023) 13-31 31 The Role of Instructional Design in Improving Pre-Service and In-Service Teacher’s Mathematics Learning Sets Skills: A Systematic Literature Review in Indonesian Context bibliometric analysis. Journal of the American Society for Information Science and Technology, 59(11), 1776– 1784. https://doi.org/10.1002/asi.20888 Verawati, A., Agustito, D., Pusporini, W., Utami, W. B., & Widodo, S. A. (2022). Designing Android learning media to improve problem-solving skills of ratio. Advances in Mobile Learning Educational Research, 2(1), 216–224. Wang, W., Schmidt-Crawford, D., & Jin, Y. (2018). Preservice Teachers’ TPACK Development: A Review of Literature. Journal of Digital Learning in Teacher Education, 34(4), 234–258. https://doi.org/10.1080/21532974.2018. 1498039 Wijayanti, I. K., & Abadi, A. M. (2021). Developing learning set with stem-pbl approach to mathematics connection ability and student’s learning motivation. 7th International Conference on Research, Implementation, and Education of Mathematics and Sciences (ICRIEMS 2020), 352–361. Yansen, D., Putri, R. I. I., Zulkardi, & Fatimah, S. (2019). Developing pisa- like mathematics problems on uncertainty and data using asian games football context. Journal on Mathematics Education, 10(1), 37–46. https://doi.org/10.22342/jme.10.1.5249. 37-46