Article Title (Size 12; Bold; Times New Roman) www.jsser.org Journal of Social Studies Education Research Sosyal Bilgiler Eğitimi Araştırmaları Dergisi 2022:13(2),243-264 Revisiting the EFL curriculum in the outcome-based education framework and freedom to learn program Imroatus Solikhah1 Abstract In this study, three dimensions were evaluated: the process of developing the curriculum, course distribution to serve quality assurance, and development of courses for Merdeka Belajar (Freedom to Learn) program. Curriculum documents and papers presented in the focus group discussion were used as data sources. Content analysis design and qualitative data analysis were used for analyzing the data. A total of 42 participants from Indonesian public and Islamic universities were included in this study. The data were thematically analyzed. The study resulted in three key findings. First, initial curriculum development focused on competency-based curriculum and outcome-based education. Not all learning outcomes and course mapping match the curriculum based on backward design. Second, the National Qualification Framework should be built on the skill and knowledge cluster to adequately assess course learning outcomes. Also, the 144-credit-course distribution should allow the undergraduate program to be completed in seven semesters. Third, curriculum developers can divide 40 semester credits for Merdeka Belajar courses and 104 credits for regular courses. Conversion of credits can be achieved by assigning equal courses in the same semester, combining undergraduate research with fieldwork, or publication in a journal. Keywords: learning outcomes, outcome-based education, competence-based curriculum, freedom to learn, backward design approach. Introduction The purpose of this study is to revisit the English curriculum in the context of Indonesian national qualification framework (Kerangka Kualifikasi Nasional Indonesia; henceforth, KKNI) in Indonesian higher education, with a particular emphasis on four erroneous perceptions about the curriculum. First, KKNI has been positioned as a new curriculum model that is aligned with outcome-based education (OBE) (Hejazi, 2011); however, its implementation falls short of the OBE concept (Spady, 1994). Second, KKNI was developed using a backward design curriculum model (Mendikbud, 2020b; Richards, 2013) and has since lost its roots as a result of adaptation to an unestablished curriculum development theory (Kaya, 2021; Solikhah & Budiharso, 2020). 1 Dr. UIN Raden Mas Said Surakarra, Indonesia; Email: iimqueenoslo@gmail.com mailto:iimqueenoslo@gmail.com Journal of Social Studies Education Research 2022: 13(2), 243-264 Third, KKNI, as a curriculum model, has placed a greater emphasis on outcome formulation (Depdiknas, 2010), referred to as learning outcomes, resulting in an inadequate and incorrect handling of its curriculum dimension (Bonk & Graham, 2006; Budiharso & Tarman, 2020; Driscoll, 2002). Fourth, while KKNI policy, as a curriculum, has entered the realm of public policy (Depdiknas, 2003; Dye, 2017), which is inextricably linked to policy analysis (Dunn, 2018), its implementation is more closely tied to political policy (OECD, 2006; Satoshi et al., 2021; Solikhah, 2015a). Analysis on KKNI-curriculum content and organization conducted by the Indonesian Study Program through the group discussion indicates features in table 1. It shows the more on the policy theories and law application of KKNI’s functions in terms of quality control, course distribution, and cooperation patterns (Depdiknas, 2010). Table 1 Redesign of Course Distribution for Merdeka Belajar No Type of Courses Semester Credit Unit 1 University obligatory course 31 2 University optional course Choose 2 out of 6 3 Faculty obligatory course 4 4 Faculty optional course Choose 2 out of 6 5 Study program obligatory course 44 6 Study program optional course Choose 4 or 6 of 10 7 Merdeka Belajar (in campus) 20 8 Merdeka Belajar (via e-learning) 9 9 Merdeka Belajar (out of campus) 23 10 Merdeka Belajar (out of campus) Choose 4 of 16 145 Depdiknas (2008) asserted that reconstructing the curriculum of the courses distribution amounts to 145 semester credit units for undergraduate (S1) program. However, the proposal shows that KKNI study is limited to theories and norms. Backward design, OBE, and curriculum dimension are not adequately discussed. There is a regional perspective in the terms “university,” “faculty,” and “optional.” KKNI as a curriculum should have global reach and standards. Courses for Merdeka Belajar off-campus with 23 semester credit units should also be highlighted. The Merdeka Belajar guide suggests taking 20–40 semester credit units out of campus (Mendikbud, 2020a, 2020b). The ontological basis of 23 semester credit units should be re-examined. Most importantly, 30.4% of the study program credits for 44 semester credit units are science courses, Solikhah 245 i.e., only 30% of graduate science competencies. The above-mentioned studies show that the proposed curriculum engineering is not feasible. Administrative aspects have been emphasized when teaching KKNI in an applied context. In the view of Richards (2013), Carvalho, et. al., (2022). and Spady (1994), the attainment of the curriculum dimension is not adequately addressed. Inconsistencies in curriculum development have been found by Depdiknas (2010). For developing an adequate curriculum for higher education, KKNI is used as a model, and the curriculum is referred to as the higher education curriculum (Depdiknas, 2010). Learning outcomes are used to develop the curriculum in higher education, and the course group and teaching materials studied are set in a matrix used to determine the curriculum (Depdiknas, 2003). Backward design (Richards, 2013) refers to this pattern, but the government does not indicate that KKNI, as a curriculum, is actually developed on the basis of this model (William et al., 2021; Richards, 2013; Solikhah, 2015b). Furthermore, a number of countries around the world have incorporated OBE into their National Qualifications Framework. By including OBE as a basis for curriculum development, Mendikbud (2020a) issued an updated curriculum guideline. According to the government’s condition, KKNI as a curriculum refers to OBE and curriculum based on backward design. The KKNI curriculum has lost its theoretical roots in the development of the curriculum (Carvalho, et. al., 2022). KKNI is not a curriculum model (Solikhah, 2020). It is rather a competency qualification framework that can juxtapose, equalize, and integrate education and work training and experience to recognize work competency in various sectors (Depdiknas, 2010; Solikhah, 2015a; 2015b Spady, 1994). It is governed by Presidential Regulation No. 8 Year 2021 and Ministerial Regulation No. 49 Year 2014 Regarding National Standards for Higher Education. Mendikbud (2020a) stated that KKNI is a reference for curriculum development, not a model. The KKNI covers formal or informal learning outcomes with internationally recognized standards. The KKNI curriculum has been examined in terms of content and implementation in different courses. Four research findings are reviewed herein to highlight research gaps and novelties. Idris et al. (2020) conducted a study at Education College (LPTK) UIN Sunan Kalijaga Yogyakarta. They described the KKNI curriculum and its application and emphasized investigating the curriculum document. Their findings are based on a review of KKNI curriculum theories, not on a critical analysis of curriculum content. None of the KKNI curriculum’s applications to OBE were critically evaluated by the researchers. Over 5,000 Indonesian students Journal of Social Studies Education Research 2022: 13(2), 243-264 participated in a study conducted by Muhammad & Ariani (2020), which took place at five state Islamic universities. The study results highlighted KKNI aspects in terms of implementation, learning outcome formulation, and program suitability with the graduate learning outcomes (GLO). GLO and course distribution were not examined critically in the development of the KKNI curriculum. Furthermore, KKNI, competency-based curriculum (CBC), and OBE are not defined in this study. The findings of Neliwati et al.’s (2020) study, conducted in Medan, describe KKNI in curriculum theories. The authors, however, did not critically analyze KKNI’s curriculum and learning theories. In another study, Solikhah & Budiharso (2019) investigated the learning outcomes of an INQF-based English language teaching curriculum in Indonesia. They essentially examined the KKNI curriculum on the basis of CBC theories and studied the course distribution in the KKNI curriculum. This study has discussed the CBC KKNI, but it has not explored the KKNI curriculum viewed from the OBE perspective (Tachie & Kariyana, 2022). The lack of a theoretical basis for curriculum formulation was highlighted by the four research groups that examined the KKNI curriculum. To address the shortcomings of previous studies, the existing research gaps are considered the prime focus of this study (Kaya, 2021). As a result, the theories developed in this study, namely, backward design curriculum development and OBE- based curriculum development, are novel. With regard to methodology, this study is novel as it uses focus group discussion (FGD) and content analysis to collect data (Tachie & Kariyana, 2022). A novel approach for Merdeka Belajar curriculum development is proposed herein. There has been some deviation and misinterpretation of the KKNI curriculum as it relates to backward design curriculum theory and the OBE education model. This has been exacerbated by the Merdeka Belajar curriculum, whose reference is also OBE, and which draws more attention to the KKNI-based curriculum and the Merdeka Belajar curriculum. This study shows a theoretical gap between backward design curriculum, OBE, and KKNI. In terms of policy, researchers and educators have placed more emphasis on normative aspects based on regulations than on mastery- of-field-based practices. Research Questions On the basis of the previous research studies and identification of existing research gaps, three research questions were sought to be answered. Solikhah 247 1) What are the main issues encountered by the teacher study programs in Indonesia while incorporating CBC KKNI into the OBE KKNI ? 2) How does the course distribution in the KKNI guarantee the graduates’ graduation period and science quality? 3) How are the curriculum materials for Merdeka Belajar incorporated into the OBE KKNI curriculum by the teacher study programs in Indonesia? Conceptual Framework KKNI A qualification framework is a part of a country’s greatest education and training system. In a qualification system, all structures and activities leading to qualification are included (Cedefop, 2013). A qualification framework encompasses a set of learning outcomes agreed upon by countries in a region. A national qualification framework compares a country’s qualification to other countries (Cedefop, 2017; The European Center for the Development of Vocational Training [CEDEFOP]). Tuck (2007) defined a qualification framework as a tool for developing and classifying qualifications based on learning levels. It is either implicit in qualification descriptors or explicit in level descriptors. In a country, or internationally, a qualification framework is meant to improve transparency, quality, accessibility, relationship, and public recognition or labor market (Carvalho, et. al., (2022). The qualification framework is developed globally. In Indonesia, it is known as Kerangka Kualifikasi Nasional Indonesia, i.e., KKNI. It is a concept of mapping the competency qualification of Indonesian workers based on Presidential Regulation No. 8 of 2012. This concept creates equal labor ability in Indonesia. The ability can be acquired through education, self- education, industry, or profession (Depdiknas, 2010). KKNI is a general reference to how one’s qualifications are recognized in their workplace. Depdiknas (2010) stated that Indonesia urgently needs KKNI because global challenges and competitions in the national or international labor markets are more open. Protective regulations can no longer stop labor migration to and from Indonesia. To juxtapose, equalize, and integrate the fields of education and job training in accordance with job structures in various sectors Presidential Regulation No. 8 Year 2012 Article 1 Verse (1) enumerates nine levels of KKNI, namely: Journal of Social Studies Education Research 2022: 13(2), 243-264 • Levels 1–3, which include operators who graduated from elementary school, junior high school, and senior high schools; • Levels 4–6, which include technicians or analysts who graduated from D1, D2, D3, D4, and S1 programs; • Level 7, which includes graduate experts with a professional education; • Level 8, which includes experts with master- or specialist-1-level education; • Level 9, which includes experts with doctoral- or specialist-2-level education. Conceptually, KKNI divides each level of qualification into four categories: (1) work skills; (2) scope of science (knowledge); (3) methods and levels of application of science; and (4) managerial ability (Depdiknas, 2003). Learning outcomes refer to the internalization and accumulation of the four parameters that should be achieved through a structured education process or through employment (Depdiknas, 2010). Brief Overview of the Development of Education College Curriculum Solikhah (2015) classified national curriculum in Indonesia into six broad categories, which include the pre-1970s materials covering (1) nationalism, (2) pedagogy, (3) general psychology, (4) didactic-methodical method, (5) fields of study taught, and (6) teaching practices. The LPTK curriculum adopted an integrated system between academic education and the teaching profession during the 1970s and 1990s. Each course was assigned a number, (e.g., 1, 2, 3, or 4) according to its level of difficulty. There was also a course on the teaching–learning process. Furthermore, the CBC era spanned from 1994 to 2000. Courses were divided into two categories: the main expertise course and the minor expertise course (post-secondary subject matter). General basic courses, specific basic courses, expertise courses I and II, and expertise courses III and IV made up the course distribution. Between 2000 and 2005, CBC was improved. CBC was developed in 2000 following the Ministry of Education’s Decision Letters 232/U/2000 and 045/U/2002. These include main competencies, supporting competencies, and other competencies. The courses were divided into five categories: personal development, science and skills, work behavior, work expertise, and social life. The KKNI-based curriculum was adopted in 2013, and this era is also known as the “OBE era,” wherein curriculum development followed a backward design approach. Thematic curriculum with a Solikhah 249 scientific approach was used in the elementary, junior high, and high schools. The higher education curriculum refers to work competencies accepted by national and international markets. A curriculum referring to the KKNI is one that refers to this qualification. The Ministry of Education and Culture’s KKNI-based curriculum shows major differences between the 2019 and 2020 versions. The KKNI curriculum prior to 2019 used CBC, while the 2020 version used OBE. For simplicity, this research refers to the 2019 curriculum as CBC KKNI and the 2020 curriculum as OBE KKNI or the new KKNI curriculum. Outcome-Based Education Spady (1984), an academician, educational psychologist, sociologist, educational planner, and the father of OBE, introduced OBE in 1984. As of 2017, OBE has been signed by a number of countries, as listed in Table 2 (Washington Accord, 2012; Gleason, 2018). Table 2 Countries that have Implemented OBE No Year Country 1 1984 The first issue of OBE by Prof. William G. Spady 2 1989 Australia, Canada, Ireland, New Zealand, England, USA 3 1995 China 4 1999 South Africa 5 2005 Japan 6 2006 Singapore 7 2007 South Korea 8 2009 Malaysia 9 2011 Turkey 10 2012 Russia 11 2021 Indonesia OBE is a teaching method that focuses on what students can do after they complete their education (Spady, 1994). It differs from traditional education with regard to educational theories, educational structure, and instructional approaches (Killen, 2007). The terms “competency,” “standard,” and “benchmark” are interchangeable in this model (Uys et al., 2005; Bonk & Graham, 2006). OBE focuses on life skills, basic skills, professional and vocational skills, intellectual skills, and personal skills (Mendikbud, 2020a). The outcomes of OBE should be evident from students' learning outcomes that reflect: (1) what students know, (2) what they can do with their knowledge, and (3) their confidence and motivation in demonstrating their knowledge (Guskey, 1994). Mendikbud (2020a) explained that OBE-based curriculum is developed in three steps: outcome- based curriculum (OBC), OBLT, and outcome-based assessment and evaluation (OBAE). OBE is Journal of Social Studies Education Research 2022: 13(2), 243-264 based on GLO and profiles. To develop learning materials, assessment and evaluation tools based on GLO are derived (Mendikbud, 2020a; Birney & McNamara, 2021). OBTL involves the interaction of lecturers, students, and learning sources. One of the key principles of OBLT is that students should choose learning forms and methods that align with the GLO. OBAE refers to the method of assessing and evaluating GLO to enhance learning quality. Learning process and the GLO results are assessed. OBAE is also used in curriculum evaluation to achieve GLO of study programs (Mendikbud, 2020b). Merdeka Belajar (Freedom to Learn) To modify the educational literacy paradigm in this digital age, Indonesian Minister of Education and Culture, Nadiem Makarim, has implemented Merdeka Belajar (Freedom to Learn) (Mendikbud, 2019). Mendikbud (2020b) defined digital age literacy as the mastery of (1) data literacy, or the ability to read, analyze, and use data (big data), (2) technology literacy, understanding coding, AI, and engineering principles, and (3) strengthening humanity, communication, and design. Students and teachers can conduct various literacy activities (Gleason, 2018). Merdeka Belajar focuses on developing students’ cognitive abilities. Students are challenged to think critically and analyze well to solve problems (Yamin & Syahrir, 2020). According to Mendikbud (2019), educational institutions require not only traditional literacy skills like reading, writing, and counting, but also digital literacy. This viewpoint is in line with autonomous learning and blended learning. According to Gleason (2018), digital literacy should be encouraged in the era of Industrial Revolution 4.0. Teachers, principals, and educational institutions direct, lead, and dig out students’ potential and critical power in the era of Industrial Revolution 4.0 (Siobhan, 2021). An educational ecosystem is created to foster students’ reasoning, character, creativity, independence, comfort, and expertise (Bonk & Graham, 2006; Driscoll, 2002). Merdeka Belajar moves from elementary to high school (Yamin & Syahrir, 2020). Mendikbud (2020a) emphasized that Merdeka Belajar in higher education promotes students to master various scientific skills useful in the workplace. Students can opt for courses from eight different types of programs: (1) industrial practices or internships, (2) village projects, (3) student exchange, (4) research, (5) entrepreneurship, (6) humanitarian, (7) projects, and (8) school teaching. Solikhah 251 Merdeka Belajar is in line with the Minister of Education and Culture of the Republic of Indonesia's Regulation No. 3 Year 2020 Regarding National Education Standards, Verse 18. Students have “three semesters to study off-campus” rights. For three semesters, undergraduate (S1) students can participate in non-course learning activities. The term “learning hour” is replaced by “activity hour.” All activities in Kampus Merdeka must be lectured. Activities outside higher education (like apprenticeships or village projects) can be taken for two semesters or 40 semester credit units. For two semesters, students can take semester credit units elsewhere (equivalent to 40 semester credit units). In the same higher education, they may take semesters credit units in other study programs (equivalent to 20 semester credit units). Methods Design This study used a content analysis design (Zhang & Wildemuth, 2009; Hsieh & Shannon, 2005) and a qualitative approach (Creswell, 2014). The research areas investigated were as follows: (1) the process of designing GLO, learning outcomes, and course distribution in the curriculum, (2) differences between CBC KKNI and OBE KKNI in terms of content and implementation of curriculum formulation, and (3) designing the Merdeka Belajar curriculum and problems in preparing proper courses. With a content analysis design, this study focused on documents in the form of FGD results containing discussion of problems in the KKNI-based curriculum and the KKNI curriculum document. Because the data analyzed were information, argumentation, facts, and narration on the content of the KKNI curriculum, a qualitative approach was used (Creswell, 2014; Yin, 2014). This study was conducted using Zoom, which was hosted by the University of Bandar Lampung, designated as the Merdeka Belajar’s implementing university. Data and Data Source In this study, the most important data came from curriculum documents obtained from an FGD about the Merdeka Belajar curriculum that took place in the fall. The FGD was held with people from all over the country who were studying teacher training. FGD results were used to show how people view the KKNI-based curriculum, how to write GLO, distribute courses in study programs, write LO, develop courses matrix, implement Merdeka Belajar on and off-campus, and write KKNI in Merdeka Belajar. A report on the results of the FGD, a recording, and field notes all Journal of Social Studies Education Research 2022: 13(2), 243-264 talked about how the FGD data came from. Documents in the form of articles about the Merdeka Belajar curriculum written by people who were knowledgeable about the subject were used in this study. They also included the course distribution for Merdeka Belajar, the transcripts of the FGD, the guide to making a curriculum for higher education, and the course distribution in English language education and Indonesian-language curriculum. Participants The FGD participants included 42 lecturers from all over Indonesia. There were two resource persons (4.8%) and 40 participants. The participants consisted of 14 men (33.3%) and 26 women (61.9%). The participants were chosen using a purposive sampling technique based on their availability during the virtual FGD. All participants were lecturers from the following universities: (1) Islamic State Institute Surakarta, (2) University of Lampung, (3) University of Mulawarman Samarinda, (4) Pontianak Islamic State Institute, (5) Palembang Islamic State Institute, (6) Parepare Islamic State Institute, (7) Cirebon Islamic State Institute, (8) Bandarlampung University, (9) University of Lambung Mangkurat, (10) University of Jendral Soedirman, (11) University of Sanata Dharma, (12) University of Jambi, (13) Islamic University Raden Intan, (15) Islamic University NU Jepara, and (16) University of Veteran Bangun Nusantara Sukoharjo. The main speaker came from the Islamic State Institute Surakarta, and the University of Sanata Dharma Yogyakarta. Research Instrument The instrument to collect data in this study was a list of questions developed by the researchers in the form of eight items of the open-ended questions. These eight items were: (1) understanding the CBC KKNI and OBE KKNI, (2) the process and obstacles in formulating GLO and learning outcomes, (3) the course distribution in the curriculum document, (4) grouping the fields of science in the courses, (5) the stages in developing the curriculum using the OBE KKNI, (6) the problems in preparing the curriculum for Merdeka Belajar, (7) the problems encountered while preparing the courses for Merdeka Belajar out of campus, and (8) problems encountered while converting courses when students join lectures out of campus. Before the questions were prepared, the researchers first discussed them with the manager of the Merdeka Belajar Program in the University of Bandar Lampung, the director of the Merdeka Belajar Program, and the lecturers in Solikhah 253 General Higher Education and Islamic Religious Higher Education. The list of questions was then limitedly tried out through interviews with three lecturers at Islamic State Institute Surakarta. The results of the trial were used to improve the items in three aspects. The questions dealing with CBC KKNI and OBE KKNI were sharpened, the scope of the question was developed further, and the problems identified were made more specific. Moreover, the researchers also wrote the transcripts of interviews and analyzed the results of the interviews using a content analysis technique in accordance with the theories proposed by Zhang & Wildemuth (2009) and Hsieh & Shannon (2005). The researchers also adapted the qualitative research theory presented by Creswell (2014) and Yin (2014). Data Collection Technique The FGD and an in-depth interview technique were used to gather the primary data. There were 40 participants in total, divided into eight groups of five participants each. The FGD committee and virtual IT helped the researchers ask open-ended questions to each group. Two minutes were allotted to each group to discuss the answers to the questions they were given. Other groups were able to see the answers and respond openly. The committee team took notes on every discussion and response. Each group took 15–20 minutes to adapt to this new method. Verbatim transcriptions were made at the end of the session of all recordings containing verbal dialogs between one group and the others (Zhang & Wildemuth, 2009; Hsieh & Shannon, 2005), and the transcription results were recapitulated and treated as documents to be analyzed (Creswell, 2014). The secondary data from curriculum texts, the Higher Education Curriculum Arrangement, and KKNI curriculum documents were analyzed in three stages. First, the researchers scanned each document for KKNI-related themes. Second, the researchers identified themes and units of analysis in each document that aligned with the prime focus of this study. Third, the researchers inventoried the themes and units of analysis in each document and summarized them to present them in accordance with the research questions. Data Analysis Technique For this study, a content analysis approach was used, as was a qualitative method of data analysis. Based on Zhang & Wildemuth’s (2009) content analysis theory, the researchers used transcripts of FGD results to identify themes and units of analysis. Following Creswell (2014), the qualitative Journal of Social Studies Education Research 2022: 13(2), 243-264 data analysis went through five stages: (1) converting numerical and textual information into narrative data; (2) creating an analysis-specific coding system guide; (3) implementing the coding system; (4) verifying its accuracy and correctness, and (5) selecting the final data (Zhang & Wildemuth, 2009; Hsieh & Shannon, 2005; Creswell, 2014; Yin, 2014). There were three categories of themes that emerged from the analysis, namely: (1) the themes and units of analysis on the processes of preparing GLO and the learning outcomes, and the distribution of courses in the curriculum; (2) the differences between the OBE and KKNI in terms of curriculum formulation; and (3) preparation for Merdeka Belaja, which was based on the findings of the analysis of research problems. Furthermore, the first research question determined the process of preparing the GLO, LO, and course distribution in terms of themes and the units of analysis from which the answers were obtained. The second theme group was used to answer the second research question, which focused on the differences between the KKNI and the OBE. The third research question, which concerned the Merdeka Belajar curriculum, was addressed using the third theme group (Creswell, 2014; Yin, 2014). Results and Discussion Curriculum Arrangement Process The first finding of this study was the process of OBE curriculum arrangement. The answer to the first research question involves the following nine themes of KKNI curriculum arrangement: 1) The KKNI-based curriculum had diametral changes between the CBC KKNI and the 2020 OBE KKNI owing to the orientation of the educational theory. 2) As the change in the 2013 curriculum occurred, the designation of KKNI also changed into CBC KKNI and 2020 OBE KKNI. 3) The CBC KKNI refers to the faculty learning outcomes, but special learning outcome courses were formulated. 4) The general and special graduate learning outcomes in the CBC KKNI were incoherent as the course distribution was diametrically different. 5) The course learning outcomes and the CBC KKNI were difficult to be formulated in an integrated way as the matrix of the type and the goal of the courses were categorically different. Solikhah 255 6) The 2020 OBE KKNI determines the GLO and the learning outcomes on the basis of the stages of the curriculum development (OBC), the success in the outcomes (OBLT), and the assessment results (OBAEI). 7) The 2020 OBE KKNI specifies the stages of the curriculum development (OBC) on the basis of the learning outcomes and the GLO. 8) The 2020 OBE KKNI made some curriculum improvement (OBLT) based on the learning outcomes and the GLO that had been reached. 9) The 2020 OBE KKNI determines the quality assurance (OBAEI) through assessments and evaluation of the learning outcomes and the GLO. The above findings indicate that participants were unaware of the changes in the orientation and scientific foundation of the old KKNI and the 2020 KKNI versions. The distinctions were discovered in the theoretical foundations upon which the curriculum was developed. The previous KKNI curriculum was based on the CBC model, and the 2020 KKNI curriculum is based on the OBE model. As a result, participants proposed that the KKNI curriculum be classified as CBC KKNI for the old KKNI and 2020 OBE KKNI for the new KKNI curriculum version. The year 2020 refers to the designation of the 2013 Curriculum as the year of curriculum change. According to the Head of the Mathematics Education Study Program. (Data 1): (1) “We are not aware of the change in the orientation from the CBC into the OBE. We just see the curriculum format and its development. What we emphasize is the preparation of the Graduate Learning Outcomes and the Learning Outcomes” (DD-26-R-20-T-1). The change had no impact on the study program’s formulation of learning outcomes. Learning outcomes according to the CBC and OBE KKNI changed fundamentally. The CBC KKNI learning outcomes focused on institutional learning outcomes. The learning outcomes should be based on knowledge distribution and should refer to the course learning outcomes. A participant in the FGD confirmed a head of Islamic Religion study program (2). (2) “What I know is that there are three learning outcomes: institutional learning outcomes to assert the university goal; faculty learning outcomes to show the outcomes that would be reached by the faculty; and course learning outcomes to show the scientific target. The problem is that the guide for the CBC KKNI merely leads to the faculty learning outcomes” (D-13- R-34-T-1). Journal of Social Studies Education Research 2022: 13(2), 243-264 The problem in developing the KKNI curriculum, according to the participants, was threefold: (1) the learning outcomes developed were directed toward the faculty learning outcomes, (2) the discourse learning outcomes were not developed, and (3) the approach to preparing matrix-based courses made the GLO incoherent with the faculty learning outcomes. The emergence of the OBE KKNI curriculum paradigm results in different categories that, in turn, result in document reconstruction. One of the FGD participants expressed his opinion in data (3). (3) “Clearly, the curriculum structure should be completely changed. In the OBE KKNI, there is an obligation to prepare a curriculum based on graduate learning outcomes, graduate learning outcomes, and the evaluation results of the graduate learning outcomes. Indeed, the OB KKNI is rather flexible in preparing the course distribution, but we have seen how it was practiced. Many higher education institutes have applied this curriculum for tens of years, and it is successful, for example, State University of Malang (UM). Although there is a new curriculum, it is impossible for us to change the course structure into new courses. What we might do is to merely reorganize it by adding or reducing its contents” (D-12-R- 4-T1) Until now, the KKNI curriculum has been prepared only when a study program has been accredited. “Well-planned changes based on curriculum evaluation are not implemented” (D-4-R- 2-T1). Moreover, the curriculum revision process is not accompanied by a theoretical workshop. “We were asked to review the curriculum. In the field, we fill out a check list and present our findings” (D-31-R-5-T-1). The theoretical basis for the CBC KKNI and the OBE KKNI has been oriented as described above. Neither program directors nor lecturers in higher education institutes were aware of the change. The change included the GLO, learning outcomes, and courses distribution based on the difficult matrix. Theme 2: Science Courses as Quality Insurance The second finding of the substance of the CBC KKNI and OBE KKNI is used to answer the second research question. The themes found in the FGD included: 1) The CBC KKNI and the 2020 OBE KKNI did not determine the course matrix based on the study program knowledge cluster. 2) The formulation of the study of teaching materials to become courses among the same study programs substantially and categorically is different. Solikhah 257 3) The number of semesters for 144 semester credit units has not been explicitly formulated so that it can be completed in seven semesters or eight semesters on an average. 4) A grouping of university, faculty, and study program occurs diametrically due to understanding of concepts and authorities. 5) The course cluster does not refer to the clear fields of science and responsibility, so that the weight of study contents is difficult to meet quality assurance. 6) The course cluster proposed for the English education study program is as follows: nationalism course, basic English skills, linguistics, teaching, curriculum, and scientific development. 7) The guarantee of success for the English knowledge cluster of at least 60% is poured into basic English skills, language knowledge, and English teaching. With this in mind, it can be concluded that KKNI curriculum content development has been hindered by the lack of knowledge clusters in the courses. As an example, the English language education program has a knowledge cluster that includes: (1) nationalism development courses such as Indonesian, citizenship, and religion; (2) basic English skills; (3) language teaching; (4) curriculum development and research; and (5) linguistics, among others. Knowledge cluster development is expected to make it easier to formulate course learning outcomes and thus GLOs according to participants of the FGD. In accordance with the previous findings, 144 semester credit units must be the minimum number of credits required for the undergraduate program. Students must complete their undergraduate degree in at least seven semesters, if the curriculum documents legally permit it. Consequently, a clear hierarchy and set of rules should be established for distributing courses and organizing knowledge clusters. Participants in the focus group, including the director of the English Language Education program, all agreed: (4) “The Regulation of the Minister of the National Education of the Republic of Indonesia allows smart students to complete their undergraduate program in seven semesters. However, the course distribution in our curriculum still requires eight semesters to complete the final task. Thus, no legal foundation exists that could allow students to complete their undergraduate program in seven semesters. If there are students who graduate in seven semesters, the graduation is considered incidental” (D-19-R-16-T-2). According to this study, the distribution of courses that support the final task, such as research method and data analysis, proposal writing, and science courses like curriculum and teaching Journal of Social Studies Education Research 2022: 13(2), 243-264 material development should be set early in the semester. “A research course usually hinders the writing of a skripsi. These courses should be scheduled earlier, like in the fourth semester. Early fifth semester data on student skripsi candidate allowed the study program to determine advisors” (D-32-R-23-T-2). The findings in this theme confirmed the importance of science courses as quality assurance. The KKNI curriculum credit weight is 33–40 semester credit units or 30.30%. This does not guarantee that the graduates will be competent in their chosen science field. “The weight of science courses is a real issue. It usually reaches 33%. How can we be sure of their knowledge if we just give them so much weight?” (D-2-R-10-T-2). “I totally agree,” said one supporter. The issue is that the KKNI curriculum does not specify the weight of each field of expertise. This was stated in the old curriculum. Science courses range from 40%–80%” (D-13-R- 27-T-2). Theme 3: Material Development for Merdeka Belajar in the OBE KKNI The answer to the third research question regarding the process of the Merdeka Belajar material development in the OBE KKNI consists of seven themes, namely: 1) In general, the number of credits in the OBE KKNI was divided into two, 40 semester credit units for Merdeka Belajar courses and 104 credits for regular courses. 2) The Merdeka Belajar materials among study programs in a university were developed together with the study programs in a faculty or other faculties so that the contents of the materials may be matched. 3) The Merdeka Belajar course materials outside a university are converted according to the contents, fields of science, and the number of credits. 4) The Merdeka Belajar course materials studied out of campus with the weight of 20–40 semester credit units were converted into obligatory courses at that time. 5) A final task in the form of skripsi may be done together with the Merdeka Belajar out of campus with the weight of 20 or 40 semester credit units. 6) The final task (skripsi, undergraduate thesis) is written with a theme in accordance with the field of expertise of the study program with the number of credit equivalent to skripsi. 7) The skripsi may be equalized with an article published in Sinta 2 journals or indexed international journals. Solikhah 259 This third finding revealed issues with the number of semester credit units, equivalence of off- campus courses, writing of skripsi, and equivalence of skripsi and scientific articles published in journals. The first stage divided the 144 semester credit units into two categories: 40 credits for Merdeka Belajar and 104 credits for the regular campus courses. The issue is that the 40 semester credit units should match the course type, credit amount, and scientific field specified in the curriculum. The FGD participants said it was difficult to convert regular courses and apprenticeships or teaching assistance. The issue is how the guidance process is done if students taking Merdeka Belajar courses intend to finish their skripsies off-campus. If they are in higher education, they can ask the lecturers on campus to help them if they require guidance. Then another thought is required. Students write reports, which are then turned into articles for publication in Sinta 2 scientific journals or non-Scopus-indexed international journals. The reports should be on topics related to the study program’s science field. This study found that the KKNI curriculum development process is constrained by educational theory, curriculum development theory, and course distribution that is not based on knowledge cluster. It is thus difficult to develop the GLO and learning outcome formulas. Another issue is that the CBC KKNI has become the OBE KKNI. Adding 20–40 semester credit units to Merdeka Belajar courses also limits their conversion to regular courses. The issue sparked debate on three sides. The first finding showed that changing the scientific orientation from CBC to OBE causes problems in curriculum formulation. Consequently, the curriculum document’s GLO, learning outcomes, and course distribution are limited. This finding supports Spady (1984) and Uys et al. (2005). By bridging the gaps between the CBC theory and the OBE theory, this study addressed the shortcomings of previous studies by explaining the OBE and its integration process in curriculum development (Satoshi et al., 2021; Spady, 1984). The backward design curriculum theory can be included as a problem-solving approach based on the curriculum development theory (Richards, 2012; Birney & McNamara, 2021). As a result, the OBE KKNI should be developed using both the backward design curriculum development theory and the OBE educational theory (Budiharso & Tarman, 2020; Killen, 2007; Meyer et al., 2008; Siobhan, 2021; Solikhah & Budiharso, 2020). The second finding revealed that the scientific field of study is given a proportion of 60%–80%, and the knowledge cluster is set in line with the study program’s field of expertise, in agreement Journal of Social Studies Education Research 2022: 13(2), 243-264 with the findings of Solikhah & Budiharso (2020) on quality standards in KKNI, Budiharso & Tarman (2020) on curriculum implementation, and Dye (2017) and Dunn (2018) on accountable, transparent, and quality-oriented public policy implementation. From the policy aspect, this finding corroborates the studies on (1) public policy strategies (Dunn, 2018; Satoshi et al., 2021), (2) quality assurance (Tuck, 2007), and (3) curriculum evaluation (Richards, 2013). Convincingly, the OBE KKNI public policy courses distribution (Dunn, 2018; Dye, 2017) and the demand to meet a legal foundation in terms of policy formulation are novelties in this part (Mendikbud, 2020a; Tuck, 2007; OECD, 2006; Satoshi & Takuya, 2021; Spady, 1984). There were 40 semester credit units of Merdeka Belajar coursework and 104 semester credit units of Merdeka Belajar coursework in the OBE KKNI, according to the third discovery. Merdeka Belajar course development is hampered by course conversion, semester credit units, and scientific field. However, courses for the Merdeka Belajar program off-campus are equivalent to 20–40 semester credit units, and thus, it is not mutually exclusive. OBE theory (Spady, 1984), blended learning (Bonk & Graham, 2006; Driscoll & Niekerk, 2008), and performance qualifications framework (Meyer et al., 2008) are all supported by this finding (Cedefop, 2013; 2017; Gleason, 2018). As theories of the OBE, blended learning, and self-directed learning had not been previously explored, the inclusion of OBE and its analysis in the KKNI curriculum under study is a novel aspect of this study. Conclusion and Suggestions Restating this study’s novelty, the findings highlight how the new KKNI curriculum incorporates OBE theory and show how OBE changed the KKNI curriculum and how Merdeka Belajar should be developed. The backward design curriculum, the OBE, and the public policy-based curriculum reformation theories are all used in this study in a novel way. The OBE KKNI uses blended learning theory in the construction of its courses and analyzes the distribution of courses based on knowledge clusters. To sum up, the change of CBC KKNI curriculum into OBE KKNI curriculum resulted in confusion. The OBE KKNI preparation has its constraints by the understanding of GLO, learning outcomes, and course distribution. OBE KKNI has not properly set the course distribution of 144 semester credit units for the guidelines; determining GLO and learning outcome formulation; Solikhah 261 determining course distribution in curriculum evaluation materials. To create the Merdeka Belajar courses, the OBE KKNI can simply divide the 144 semester credit units into 40 Merdeka Belajar and 104 regular Courses. However, the equivalence of Merdeka Belajar courses (20–40 semester credit units) with regular courses remain a big issue to solve. It is recognized that this research has some weaknesses in the form of data collection method through virtual FGD that we could not control. It is suggested that future researchers change the data collection technique via direct observations and interviews. The pandemic era has certainly introduced numerous constraints for implementing face-to-face studies, and thus, a COVID-19- appropriate protocol should be carefully programmed. References Birney, Lauren & McNamara, M. Denise. (2021). Tackling problem-solving through the curriculum and community enterprise for environmental restoration project, Journal of Curriculum and Teaching, 10(3), 1-10. URL: https://doi.org/10.5430/jct.v10n3p1. Bonk, J. C. & Graham, R. C. (2006). The handbook of blended learning. Pfeiffer. Budiharso, T. & Tarman, B. (2020). Improving quality education through better working conditions of academic institutes, Journal of Ethnic and Cultural Studies, 7(1), 99-115. http://dx.doi.org/10.29333/ejecs/306 Carvalho, M, Cabral, I, Verdasca, J, Alves, M. J. (2022). Strategic Action Plans for School Improvement: An Exploratory Study about Quality Indicators for School Improvement Plan Evaluation. Journal of Social Studies Education Research, 13(1), 143-163. https://jsser.org/index.php/jsser/article/view/3910/554 Cedefop. (2013). Analysis and overview of NQF level descriptors in European countries. Working paper. https://www.cedefop.europa.eu/files/6119_en.pdf Cedefop. (2017). ‘Defining, writing and applying learning outcomes – a European handbook’. https://www.cedefop.europa.eu/sl/news-and-press/news/new-european-handbook-learning- outcomes# Creswell, J.W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th Ed.). SAGE Publications. Depdiknas. (2010). Buku Pedoman Kerangka Kualifikasi Nasional Indonesia. Jakarta: Direktorat Jendral Pendidikan Tinggi. https://doi.org/10.5430/jct.v10n3p1 http://dx.doi.org/10.29333/ejecs/306 https://jsser.org/index.php/jsser/article/view/3910/554 https://www.cedefop.europa.eu/files/6119_en.pdf https://www.cedefop.europa.eu/sl/news-and-press/news/new-european-handbook-learning-outcomes https://www.cedefop.europa.eu/sl/news-and-press/news/new-european-handbook-learning-outcomes Journal of Social Studies Education Research 2022: 13(2), 243-264 Depdiknas. (2008). Pengembangan Kurikulum pada Pendidikan Kesetaraan. Jakarta: Balai Pengembangan Pendidikan Non-Formal dan Informal. Depdiknas, (2003). Undang-Undang Nomor 20 Tahun 2003 tentang Sistem Pendidikan Nasional. Jakarta: Direktorat Jendral Pendidikan Tinggi. Dunn, N. William. (2018). Public Policy Analysis: An Integrated Approach. New York and London: Routledge. Dye, R, Thomas. (2017). Understanding public policy. Fifteenth Edition. Pearson Education, Inc. Driscoll, M. (2002). Blended learning: Let’s get beyond the hype. IBM Global Services. Gleason, W. Nancy. (2018). Higher education in the era of the fourth industrial revolution. Palgrave Macmillan. Guskey, T. (1994). Defining the differences between outcome -based education and mastery learning. The School Administrator, 51, 34-37. Hejasi, Mousavi, Bahar. (2011). Outcome-based education (OBE): A transformational perspective on quality and mobility in higher education. Porland University, Community College Leadership Program. Hsieh, H.-F., & Shannon, S.E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277-1288. Idris, S.; Tabrani; Sulaiman, Fikri & Murziqin, R. (2020). The role of kkni curriculum in supporting the development of education at the LPTK UIN Sunan Kalijaga Yogyakarta, International Journal of Advanced Science and Technology, 29(6), 4011-4024. Kaya, M. M. (2021). State, Citizens and Education in the Ottoman Empire: Civic Education Curriculum in the Early 20th Century. Research in Educational Policy and Management (REPAM), 3(2), 42-72. https://doi.org/10.46303/repam.2021.6 Killen, Roy (2007). Teaching strategies for outcomes-based education, Second Edition. Juta and Company Ltd. p. 48. ISBN 978-0-7021-7680-7. Mendikbud, (2019). Pedoman Pengembangan Kurikulum Perguruan Tinggi Berbasis KKNI. Jakarta: Direktorat Jendral Pendidikan Tinggi. https://doi.org/10.46303/repam.2021.6 https://en.wikipedia.org/wiki/ISBN_(identifier) https://en.wikipedia.org/wiki/Special:BookSources/978-0-7021-7680-7 Solikhah 263 Mendikbud. (2020a). Panduan Penyusunan Kurikulum Pendidikan Tinggi di Era Industri 4.0 untuk Mendukung Merdeka Belajar-Kampus Merdeka. Jakarta: Direktorat Jendral Pendidikan Tinggi. Mendikbud. (2020b). Panduan Merdeka Belajar-Kampus Merdeka. Jakarta: Direktorat Jendral Pendidikan Tinggi. Muhammad, I., & Ariani, S. (2020). The development of KKNI-Based curriculum at the Arabic language education programs in Indonesian Higher Education. Jurnal Ilmiah Peuradeun, 8(3), 451-474. doi:10.26811/peuradeun.v8i3.543 Neliwati; Marbun, Derliana; Mahariah. (2019). The implementation of curriculum based Indonesian National Qualification Framework (KKNI) in Islamic education study program at the Islamic Higher Education in Medan. IJLRES: International Journal on Language, Research and Education Studies, 3(1), 71-84. DOI: 10.30575/2017/IJLRES-2019010406. OECD (Organisation for Economic Cooperation and Development). (2006). Qualifications systems: Bridges to lifelong learning. OECD. http://www.oecd.org/education/innovation- education/qualificationssystemsbridgestolifelonglearning.htm Richards, C. Jack. (2013). Curriculum approach in language teaching: Forward, central and backward design. RELC Journal, 44(1), 5-33. DOI: 10.1177/0033688212473293. Satoshi K., Ismael C. N. & Takuya B. (2021). Analysis of the intended mathematics curriculum revision process in Mozambique from the perspective of relevance, Journal of Curriculum Studies, 53:5, 642-658, DOI: 10.1080/00220272.2020.1855256 Siobhan D. (2021). Intentional, tacit, contingent: knowledge recontextualization in the official History curriculum - a critical discourse analysis, Journal of Curriculum Studies, 53:5, 692- 710, DOI: 10.1080/00220272.2021.1904007 Solikhah, I. & Budiharso, T. (2019). Investigating the learning outcomes of an INQF-based English language teaching curriculum in Indonesia, Journal of Social Studies Education Research, 10(4), 153-175. https://jsser.org/index.php/jsser/article/view/1090/412 Solikhah, I. & Budiharso, T. (2020a). Standardizing BIPA as an international program of a language policy. Asian ESP Journal, 16(5.2), 181-205. https://www.elejournals.com/asian- esp-journal/volume-16-issue-5-2-october-2020/ Solikhah, I. & Budiharso, T. (2020b). Exploring cultural inclusion in the curriculum and practices for teaching Bahasa Indonesia to speakers of other languages, Journal of Social Studies Education Research, 11(3), 177-197. https://jsser.org/index.php/jsser/article/view/2658/465 http://www.oecd.org/education/innovation-education/qualificationssystemsbridgestolifelonglearning.htm http://www.oecd.org/education/innovation-education/qualificationssystemsbridgestolifelonglearning.htm https://doi.org/10.1080/00220272.2020.1855256 https://doi.org/10.1080/00220272.2021.1904007 https://jsser.org/index.php/jsser/article/view/1090/412 https://www.elejournals.com/asian-esp-journal/volume-16-issue-5-2-october-2020/ https://www.elejournals.com/asian-esp-journal/volume-16-issue-5-2-october-2020/ https://jsser.org/index.php/jsser/article/view/2658/465 Journal of Social Studies Education Research 2022: 13(2), 243-264 Solikhah, I. (2015a). Designing academic literacy framework of EAP program for Indonesian learners, Journal of Language and Literature, 6(2), 34-38. Solikhah, I. (2015b). Learning outcomes Berbasis KKNI dalam Kurikulum Pendidikan Bahasa Inggris. Surabaya: Jenggala Pustaka Utama. Spady, W. (1984). Outcome-Based Education: Critical Issues and Answers (PDF). American Association of School Administrators. ISBN 0876521839. Retrieved 31 October 2014. Tachie, S. A. & Kariyana, I. (2022). Time Divested or Time Invested? Freshmen’s Perspectives and Reflective Experiences on Interactive School Learning. Research in Social Sciences and Technology (RESSAT), 7(1), 1-19. https://doi.org/10.46303/ressat.2022.2 Tuck, R. (2007). An introductory guide to national qualifications frameworks. Conceptual and practical issues for policy makers. ILO. https://www.ilo.org/wcmsp5/groups/public/---ed_emp/--- ifp_skills/documents/instructionalmaterial/wcms_103623.pdf Uys, L. & Gwele, N. (2005). Curriculum development in nursing: process and innovation. Routledge. p. 194. ISBN 0415346290. Washington Accord. (2012, August 20). International engineering alliance. Archived from the original on 26 January 2012. William H. Schmidt, Siwen Guo & Richard T. Houang (2021). The role of opportunity to learn in ethnic inequality in mathematics, Journal of Curriculum Studies, 53:5, 579-600, DOI: 10.1080/00220272.2020.1863475 Yamin, Muhammad & Syahrir. (2020). Pembangunan Pendidikan Merdeka Belajar: Telaah Metode Pembalajaran. Jurnal Ilmiah Mandala Education, 6(1), 124-136. Yin, Robert K. (2014). Case Study Research Design and Methods. 5th edition. SAGE Publication Inc Zhang, Y & Wildemuth, B. M. (2009). Qualitative analysis of content, In: B. M. Wildemuth, Ed., Applications of Social Research Methods to Questions in Information and Library Science, Libraries Unlimited, 2009. pp. 1-12. http://files.eric.ed.gov/fulltext/ED380910.pdf https://en.wikipedia.org/wiki/ISBN_(identifier) https://en.wikipedia.org/wiki/Special:BookSources/0876521839 https://www.ilo.org/wcmsp5/groups/public/---ed_emp/---ifp_skills/documents/instructionalmaterial/wcms_103623.pdf https://www.ilo.org/wcmsp5/groups/public/---ed_emp/---ifp_skills/documents/instructionalmaterial/wcms_103623.pdf https://en.wikipedia.org/wiki/ISBN_(identifier) https://en.wikipedia.org/wiki/Special:BookSources/0415346290 http://www.washingtonaccord.org/Washington-Accord/signatories.cfm http://www.washingtonaccord.org/Washington-Accord/signatories.cfm https://doi.org/10.1080/00220272.2020.1863475