Sebuah Kajian Pustaka: English Language Teaching Educational Journal (ELTEJ) Vol. 2, No. 1, 2019, pp. 22-31 E-ISSN: 2621-6485 The CIPP Model-Based Evaluation on Integrated English Learning (IEL) Program at Language Center 1Noor Qomaria Agustina, 2Fanni Mukhtaruddin 1Kent State University 2Institut Teknologi Nasional Yogyakarta, e-mail: nagustin@kent.edu, fanni@itny.ac.id Abstract The challenging situation in the workforce leads the education institution to make suitable programs which later help the learners to be more competitive in the world of work especially in mastering the English. Language Training Centre of a university provides service on teaching language skill especially English. The research purpose is to make an evaluation on the program conducted by the Centre namely Integrated English Learning (IEL) using the CIPP Model developed by Stufflebeam. The methodology was the qualitative design with case study at a language centre at a private university in Yogyakarta Indonesia. The participants were the students from departments that had implemented the IEL in six semesters. The data collection was from the interview and the data analysis used the qualitative data analysis software. The steps in analysing the data used the methods in the grounded research. This study described the IEL program seen from its context, input, process and product (CIPP Evaluation model) from the students’ experience and perspective. The context was described from the effectivity of the program, while the input took the point of view from the quality. The process was to reveal the program implementation and the product described the achievement of the goals. Keywords: CIPP Model, Evaluation, ELT, language center How to Cite: Agustina, N.Q., & Mukhtaruddin, F. (2019). The CIPP Model-Based evaluation on Integrated English Learning (IEL) program at language center. English Language Teaching Educational Journal, 2 (1), 22-31 INTRODUCTION Achieving a set of goals and objectives is one of the main reasons why an educational program is designed in the first place. To ensure that the desired goals are met, an evaluation to a program is an extremely important step in the entire process of conducting a program so that the people in authority has the bases whether to continue or bring any necessary revisions in the program (Bazargan, 2007a; Birjandi, 2009; Piccardo, North & Maldina, 2019). Staub (2017) stated that “…there is the critical groundwork that must be laid for a healthy, productive environment that sees evaluation and assurance not as perfunctory activities, rather opportunities for the organization, or unit, to learn and grow (p. 75)”, so the evaluation has deeper funtion. The aim of an evaluation program is to focus on the goals accomplishment for the next alternative in decision making (Bazargan, 1997; Kusumawati, Darmawan and Latifah, 2018; Stufflebeam, 1971; Sundoyo in Umam and Saripah, 2018). An educational program that is really interesting to observe is an English language program that has been established by Language Training Center (LTC) of Muhammadiyah University of Yogyakarta (UMY) called Integrated English Learning. The program provides four complete skills in English language learning i.e., listening, speaking, reading, and writing. The program is divided into 12 levels that has its own descriptor. Since the first time the program was established, there has not been any official analysis or evaluation ELTEJ ISSN: 2621-6485 ◼ Agustina, Mukhtaruddin 23 regarding the IEL program although the improvement of the program has already been made. There are four reasons to conduct the evaluation on IEL program. First, the program is unique and interesting. It is different from other general language programs in that it has distinctive features such as the conduct of pre-test and post-test to determine the students’ level, special method of teaching using mixed teaching methods, and a detailed scoring system utilizing several components. Second, there has not been any research-based evaluation on the program. Third, there is curiosity to find out the weaknesses and strengths of the program. Fourth, a will to describe the implementation of the program in a detailed form from the project pilot until the last revision of the program. The success of classroom teaching depends on many factors. As shown by Dunkin and Biddle in Chaudron there are four classes of variables which show the complexity and interconnectedness of college teaching (Birjandi, 2009; Kuo in Umam and Saripah, 2018, Oscarson, 2019). This research was guided by two major research questions which are listed as the following: (1) How do the students and the teachers perceive the IEL program at the language center? (2) How is the IEL program implemented? IEL Program Integrated English Learning (IEL) program of UMY is an English learning program that involves all skills (listening, speaking, reading, and writing) in every single meeting and uses levelling categorization-based class (Wisniewski, 2017; Piccardo, North & Maldina, 2019). Before the program is conducted, all the students following the program join the TOEIC-based English proficiency test consisting of listening, speaking, reading, and writing skill. The test is conducted by at least 50 English instructors covering approximately 350 classes in a semester. After the test is graded, the students are categorized in groups based on students’ department, year, and English proficiency level. The main purpose of this categorization is to give a good English program based on their needs and abilities. There are 12 levels of proficiency in the program. Summative and Formative Evaluation Scriven (1991) introduced two different kinds of evaluation namely Formative and Summative. Formative evaluation needs collecting and sharing information to improve the program. The formative evaluators provide the program planners and staff with information, make adustment to the setting and improve it (Dressler, 1997; Bazargan, 2007b; Dixson & Worrel, 2019; Mahshanian, Shoghi & Bahrami, 2109; English & English, 2019). Formative evaluation takes place usually during the development or improvement of a program, product or person and it is held more than once (Scriven, 1991). The purpose is to validate or ensure that the goals of the instruction whether they are being achieved. Else, it is to improve the instruction giving identification and remediation to the problematic aspects (Boon, 1987; Cheng, 1996; Brence & Rivza, 2012). On the other hand, the formative evaluation provides data to made necessary changes during the program (Xiao & Yang, 2019). Learning activities is possible to be changed in the range and depth of instructional activities, and curriculum can be revised in ongoing process (Tylor-Powell, Steele & Douglan, 1996; Dressel, 1997). The changes are possible to both course improvement and students’ growth (Doll, 1992; Sowell, 2004; Darusslam, 2010). In sum, the formative evaluation is operated during the program to provide direct input for the program directors. Summative evaluation is implemented after a program ends to provide the strength and challenging factors after the curriculum is completely designed (Houston & Thompson, 2017). A summative evaluation might be conducted to determine how effective the 24 ◼ E-ISSN: 2621-6485 curriculum is (Boon, 1987; Tyler, 1990; Doll, 1992; Sowell, 2004). The summative evaluator’s function is to obtain the data and write report to describe the program and its achievements. It figures out the information about the product’s efficacy, such as the learners’ satisfaction and what kind of learning that learners have achieved. Summative evaluation usually uses numeric scores or letter grades to assess learner achievement CIPP Evaluation Model There are various evaluation approaches and models, but the researcher should choose the most appropriate one according to the objectives and conditions. Stufflebeam’s CIPP model (Stufflebeam, 1971; Tunc, 2010; Ulumi, 2016; Halimah and Hadjar; 2018) can be utilized to evaluate the effectiveness of the program in general and the quality of input, process and product. CIPP, the abbreviation of Context, Input, Process, and Product, is a program evaluation model useful for evaluating educational programs. As the name suggests, it is the evaluation of context, input, process, and product of an educational program. Context evaluation comprises examining and describing the context of the program, performing a needs and goals assessment, deciding the program objectives, and verifying whether the proposed objectives are sufficiently responsive to the identified needs. Stufflebeam (2000) suggests that it helps in program-planning decisions making, as he said that “Context evaluations assess needs, problems, and opportunities as bases for defining goals and priorities, and judging the significance of outcomes”. In the input evaluation, the activities are the program inputs and resources description, the comparison of the program performance compared to other program, a prospective benefit/cost assessment, an evaluation of the proposed design of the program, and an examination of the recommendations for the alternative strategies and procedures that should be considered as recommended. According to Stufflebeam (2000), “input evaluations assess alternative approaches to meeting needs as a means of planning programs all allocating resources”. Process evaluation involves examining the implementation of the program, monitoring how the program performance, auditing the program to be in line with the required legal and ethical guidelines, and identifying defects in the design or implementation. Evaluators require feedback to program personnel because it can be helpful in making decisions for formative evaluation, as Stufflebeam said “Process evaluations assess the implementation of plans to guide activities and later to help explain outcomes (ibid)”. Product evaluation includes determining and examining the general and specific outcomes of the program, measuring anticipated outcomes, attempting to identify unanticipated outcomes, assessing the merit of the program, conducting a retrospective benefit/cost assessment, and conducting a cost effectiveness assessment (to determine if the program is cost effective compared to other similar programs). Product evaluation is very helpful in making summative evaluation decisions; “Process evaluations identify intended and unintended outcomes both to help keep the process on track and determine effectiveness (ibid)”. Related Research The CIPP evaluation model invented by Stufflebeam can be used to evaluate the program from the Context, Input, Process and Product. The evaluation model gives the comprehensive way to view in the holistic manner. There are many areas that use its system in evaluation, and some researches have been conducted in the topic. Aziz, Mahmood and Rehman (2018) conducted a research to in quality evaluation in the school level. The ELTEJ ISSN: 2621-6485 ◼ Agustina, Mukhtaruddin 25 purpose of this research is to evaluate the educational quality at school level. The participants are the principles, division heads, and teachers. There are two different perspectives on the success of education, from the stakeholders and teachers. The stakeholders’ point of view was more on how to build different strategies to facilitate the success of learning. On the other hand, teachers saw from the theoretical point of view and fall to apply the root learning, that turned out to burden the students and put the learning in more on the uncomfortable situation. Tootian used the CIPP Model to evaluate succession planning training program for human capital management. The aims of the research are to know the whether the training meet the objective and it is being successful and effective. The result of the research also provides with the feedback and guide for the decision making. Ulum used the model of CIPP to evaluate an English language program at state schools in Turkey from students’ perspective to see the effectiveness of the program. There are four aspects of the program that should be highlighted. The first was the textbook that was too hard, the second was the tests that were hard, the third was the topic that needed to be improved and the last was the time that was too much for the students. Another research that used the CIPP Model is the evaluation of the Arabic language learning program at a higher education to seek the quality assurance of the learning system. The evaluation is very comprehensive from the very beginning of the program until it is completed from the context that comprises the purpose and background of the learning process, the input that examine the implementation process of the curriculum, students, lecturers and infrastructure, and the evaluation process that viewed the curriculum, teaching materials, and media in the classroom activities. The product viewed the outputs and outcomes of the language learning. RESEARCH METHOD The objective of this study was to gain the experience of IEL program from the perspectives of the students, instructors, staffs and stake holders. The CIPP (context, input, process, and product) evaluation model developed by Stufflebeam (1971) is utilized in the study. This research used the qualitative research methodology. Qualitative data were gathered through interviews with the UMY students who joined in the IEL program. Written documents were used of to obtain more detailed information. The target participants were the third-year students of Agriculture and International Relation Department, 15 English instructors who has been teaching the programs since the first time, and the head of the two departments of Agriculture and International Relation of the department. The researcher invited them to participate in the current research by interview. FINDINGS AND DISCUSSION The program evaluated is the Integrated English Learning; it is an English levelled program based on the CEFR levelling using integrated learning which combine four skills of English namely listening, reading, speaking, and writing. There are two departments involved for the pilot projects which is conducted in six semesters, and the length of learning per semester is 48.5 hours or 291 hours altogether. The intuition plan to prepare 12 levels, but so far there are only 9 levels are used. The institution develops the program’s curriculum and syllabus on their own empowering the English Instructors to collaborate in establishing the programs. The participants of this research the sixth semester werr the students of batch 2012 from 2 departments that are selected to be used as the pilot project. The data were gathered from the in-depth interview and focus group discussion (FGD) from 25 students consisted 26 ◼ E-ISSN: 2621-6485 of 9 females and 15 males. For the data triangulation, the interview was also conducted to the LTC’s management and instructors of 7 participants consist of 5 females and 2 males. Context Evaluations It is important to define the needs of the learners in designing a program. Stufflebeam (2000) stated that “Context evaluations assess needs, problems, and opportunities as bases for defining goals and priorities, and judging the significance of outcomes”. In this section, there is exploration on the need based on the participant point of view and supported by the questionnaire result. Integrated English Learning (IEL) is designed to teach the general English using integrated learning that integrates four skills namely listening, reading, writing, and speaking and the students are levelled based on the CEFR levelling system (Winiewski, 2107; Oscarson, 2019). To some extends, general levelled English is still acceptable, however, there are also needs to have very specific English to increase specific skills like grammar, vocabulary, public speaking, English proficiency and so forth. “I think it is vocabulary and grammar. Why grammar? Because when we want to speak English…some people do not understand what we said, one wrong word can change the meaning (Jodi).” Another interesting thing in this need analysis is about the levelling system, whether it is appropriate or not. There are also arguments on the appropriateness of the levelling system at this center. There are some problems arise regarding this system. “In my opinion, appropriateness and suitability depend on what is their level, if they are fresh graduate from Senior High School, for one semester or two they will be happy…but if they are in the higher semester, they will need something else like academic English…(Sekar)” Some agree with the levelling system, but there is a need to improve the system so that the mechanism of the levelling system is clear. There is also problem of the students who stay at the same level for some semesters that make them feel bored and get nothing from the teaching learning process. Input Evaluation To accomplish the goals, an institution needs an input evaluation where there should be description of program as well as planning and allocating the resources to accomplish the programs. There is also a need to find comparison to find out the effectiveness of the programs. In Stufflebeam’s (2000) point of view, “input evaluations assess alternative approaches to meeting needs as a means of planning programs all allocating resources”. There are two aspects that will be discussed in this section, they are program development and teacher quality control. Program Development. Integrated English Learning is the general levelled English program run to serve the students in learning English. Because it is levelled, this program needs placement test to place the students according to their level. The test used to put the students into different level using one of the English Proficiency Test. “The Aims of Integrated English Learning is to make students grouping based on their English competency level…the second intention is to make the class more homogeny; for example, the students’ ability in one class is almost the same, therefore, the teaching learning process is easier (Agus).” In comparison to non IEL classes, there is no placement test, so the students’ competency level will vary from one student to another to make the class more heterogenic. ELTEJ ISSN: 2621-6485 ◼ Agustina, Mukhtaruddin 27 The teachers usually have difficulties in doing the teaching learning process since they should carefully prepare the material which is suitable for all. “…in my opinion, IEL program tries to place the students based on their competency, and this will make the instructor easier to handle the class if there are not many differences among the students in a class…there will not be any big gap among them (Ari)” Another aspect that also needs attention is the program development. There are four steps to develop the program, they are defining the standard, determine the descriptors, make syllabus and material development and for the testing, design the Term of Specification (TOS) and develop the test. The first step is determining the standards. The standard used is the Common European Framework of Reference for Languages (CEFR) to measure language mastery (www.cambridgeenglish.org). “…They know well from the very beginning, from the descriptors, then they make the syllabus…after that process they will get feedback…and after the feedback they usually revised either syllabus, module or term of specification, TOS is to make the test…(Bambang)” From the CEFR that is modified into descriptors, the centre plans to provide 12 levels. However, not all levels are made and used. There are 9 levels available, but only 8 levels are used. “Based on the project in general, there are 12 levels, but for this semester there are only 9 levels. The highest level is the eighth level (Dewi).” The process to make the descriptors until the module and test is a project that involves many instructors and management that also function as the professional development since the teachers will learn a lot about developing a program that hopefully this understanding will lead to better teaching. “…IEL, if I am not mistaken, has 12 levels but only 8 levels are used, level 9 is already made but it has not been used yet. So, each level has its own coordinator… Each coordinator will be responsible to the Academic manager as the project leader. After the modules and syllabuses are done, they submit them to me…Each level has coordinator, and these coordinators are led by the academic manager (Wafa).” The explanation above is the process of program making that is a collaboration process that involved many teachers. The function of the management here is the coordinator of the project. For the evaluation and quality control, there is feedback session that is done either per level group or for all levels that is called grand review. Teacher development is a process to make evaluation and quality control process. There are two kinds of evaluation for the teachers. The first is the end of the semester evaluation and monitoring during the semester. “We already have the SOP. So, the first is the one who should handle the problems is the PIC of the sharing group…The PIC of the sharing group should handle the problem first. If he or she cannot handle the problems, the Academic manager will handle it, and at last if the problems are very serious, then it must be handled by the centre’s head (Alia).” To ensure the teacher performance, at least there are three kinds of evaluation, they are end-session feedback from the students in the form of questionnaire, peer sharing or sharing group as the media to share the problems and ideas, and the at last, the students can contact personally to the centre management by phone or face to face. Process Evaluation The evaluation on the implementation of a program is important for the institution to monitor how the program is carried out to see whether the program has already been conducted in line with its standard operating procedures. In this section, from the data http://www.cambridgeenglish.org/ 28 ◼ E-ISSN: 2621-6485 gathered, the implementation of the teaching learning process in the classroom can be seen from three aspects, they are material used, teacher and teaching method and the assessment. In relation to the material, the centre provides the syllabus and module for all levels. Both the syllabus and the module are reviewed every year. Teachers have different opinion about the module. Some of them follow the module in giving the material, but some make modification on the module. “Mmm we get the module from the centre…last semester was wonderful, because the book was new, and the students are happy… (Jodi)” “…the lecturer has done her best…the lecturer always takes the material from the book (Dewi).” However, the module is not always appropriate for the teaching learning process. Therefore, the lecturers then make adjustment and find additional material. “Mmm because I am not module base, what I mean by module base is that I don’t ask the students to do the exercise exactly like what it is in the book, so I have to modify…(Antok)” Except modifying the module, the teachers also find other resources to teach. The resources can be in the form of books, novel, games, interesting activities, games blog, as well as employ the other persons like friends, teachers, and colleagues. “The teacher can adjust the material so that the material can be more effective (Dedi).” “…my class is full of games…(Mawar).” “…for the assignment I gave homework for the students to read the book…(Bambang).” It can be concluded that the centre provides the materials, but it is often that the module need adjustment to make it suitable to be used for the classroom activities. To make the class more engaging, the teachers also use additional materials such as games, books, blog and interesting activities. About the teacher and teaching method, the teachers at the centre possess various ways to teach in the classroom to provide good learning atmosphere. The teaching method is not only to convey the materials, but also to build relationship with the students. Including in the teaching method is the teacher talk during the class, and the language use in the classroom. Another important consideration is to engage the students to the activities and ask them to think on the material. Assessment is one of the important aspects in the teaching learning process. There are different kinds of assessment used in the centre. The centre has already provided official assessment that is the same for all classes and the assessments are stated in the syllabus. There are five categories to do the assessment, they are attendance, in class assessment, home assignment, and progress test. Besides the official assignment, the teachers also use different kinds of assessment. They use the exercise in the book to look the student’s mastery on the material. The assessment can also be in the form of review before the mid and final tests. Book review and reading activities become alternatives of the assessment. Product Evaluation Product evaluation is related to the examining the outcomes of the program. The outcomes can be traced back to the aims of the program to determine whether the aims are already achieved. There were four aspects highlighted for the evaluation. They are IEL program, the levelling system, module, and the teacher. Integrated English Learning (IEL) as its name suggests is the English language program which integrates the teaching learning of the four skills to be taught together in a ELTEJ ISSN: 2621-6485 ◼ Agustina, Mukhtaruddin 29 level. This program is also general English in the sense that the material is not intended for the specific purposes that are in line with department. Levelled classes are used to place the students based on their proficiency level. Because IEL is levelling class, it is very important that there is levelling system. One of the weaknesses of this program is the test design to place the students that are suitable for their level. There are many cases that the students are put in the wrong level they stay at the same level for several semesters. Another important aspect is the module. There has not been yet the research study whether the modules used are effective or not. However, some attempts have already been taken to review the module. The centre already created 9 modules for 9 different levels along with the syllabuses, TOS, and testing. The centre also continuously provided feedback for the development of the module in the form of module review and feedback. The feedback was gathered from the group discussion, teacher meeting and grand review module that was conducted special to review the module. The last feedback is on the teacher. There are differences in the teacher quality in conducting the teaching learning process. There was feedback that there should be benchmarking and careful system in recruiting the teachers so that the teacher quality can be guaranteed. The teacher should also engage and have good communication with the students; that is the one who will promote learning in the classroom. CONCLUSION Educational program is intended to conduct teaching learning process to create quality human resource. It is a kind of difficult task, so to make sure that the program can run well, there must be a kind of goals and objectives as the guidance for the implementation of the program. There must be a way to make sure the objectives and goals are met. Therefore, there is a need to make evaluation and assessment. To do the evaluation and assessment needs tools. CIPP model promoted by Stufflebeam is one of the models that can be used. Integrated English Learning is one of the language programs conducted by a centre of a well-known university in Yogyakarta. The research design was the qualitative method. The participants of this research are the students from two different departments who take the English at the centre and they have already joined the program for six semesters. The result was divided into four research questions asking about the context, input, process, and product evaluation of the IEL program. The explanation is about the description of the program which picture the implementation of the program at the centre. The research gave useful input for the betterment of the program to know how the possible solution if there is any problem. The teacher who engaged in this program and for the teacher in general, it is to portray on how a language program should be conducted, then the information given in this research may inspire the teachers to do better and more in the teaching learning process. REFERENCES Aziz, S., Mahmood, M., & Rehman, Z. (2018). Implementation of CIPP model for quality evaluation at school level: A case study. Journal of Education and Educational Development, 5 (1), June 2018, pp. 189-205 Bazargan, A. (1997). Quality and its evaluation in higher education: A look at national and international experiences. Quarterly Journal of Rahyaft, 4,155-163. Bazargan, A. (2007a). Educational evaluation. Tehran: SAMT publisher press. 30 ◼ E-ISSN: 2621-6485 Bazargan, A. (2007b). Higher Education in Iran. In James J. I. Forest and P.G. Altbach (Eds.); International Handbook of Higher Education; Dordercht, the Netherlands: Springer. Birjandi, P., &Nosratinia, M. (2009). The qualitative program evaluation of the postgraduate English Translation major in Iran. The Journal of Modern Thoughts in Education, 4, 37-5 Boon, K.B. (1987). Effectiveness of the diploma in education program: perceptions of former Bahasa Malaysia methods student. Kuala Lumpur: University of Malaya. Brence, I. & Rivza, B. (2012). Quality evaluation of higher education programs: process and challenges in Latvia. Proceeding of International Conference of Management, knowledge and learning international, Celje, Slovenia (pp. 787-79). Cheng, V.C. (1996). School effectiveness and school-based management: A mechanism for development. London: The Fulnmer Press. Dixson, D.D., & Worrell, F.C. (2016). Formative and summative assessment in the classroom. Theory Into Practice, 55:153–159, 2016. DOI: 10.1080/00405841.2016.1148989 Doll, R. C. (1992). Curriculum improvement: Decision making and process. Boston: Allyn and Bacon. English, J., & English, T. (2019). Combining summative and formative evaluation using automated assessment. Issues in Informing Science and Information Technology, 16, 143-151. https://doi.org/10.28945/4293 Halimah, U. S. & Hadjar, I. (2018). Arabic language learning evaluation in higher education with context input process product (CIPP) model. Alsinatuna- Journal of Arabic Linguistics and Education, 4 (1), December 2018, pp. 33- 47. P-ISSN: 2477-5371; E-ISSN: 2503-2690. Houston, D., & Thompson, J. N., (2017). Blending formative and summative assessment in a capstone subject: ‘It’s not your tools, it’s how you use them’, Journal of University Teaching & Learning Practice, 14(3), 2017. Available at:http://ro.uow.edu.au/jutlp/vol14/iss3/2. Mahshanian, A., Shoghi, R., & Bahrami, M. (2019). Investigating the differential effects of formative and summative assessment on EFL learners’ end-of-term achievement. Journal of Language Teaching and Research, Vol. 10, No. 5, pp. 1055-1066, September 2019. DOI: http://dx.doi.org/10.17507/jltr.1005.19. Oscarson, M. (2019). Framing modern language education – A european approach. Pertanika J. Soc. Sci. & Hum. 27 (2): 773 – 786 (2019). Piccardo, E., North, B., & Maldina, E. (2019). Innovation and reform in course planning, teaching, and assessment: The CEFR in Canada and Switzerland, a comparative study. Canadian Journal of Applied Linguistics, Special Issue: 22, 1 (2019): 103- 128. Sowell, E. J. (2004). Curriculum development: An integrative approach. Upper Saddle River, NJ: Prentice Hall. https://doi.org/10.28945/4293 ELTEJ ISSN: 2621-6485 ◼ Agustina, Mukhtaruddin 31 Staub, D. F. (2017). Quality assurance and foreign language programme evaluation in S. Hidri and C. Coombe (eds.), Evaluation in Foreign Language Education in the Middle East and North Africa. Second Language Learning and Teaching, DOI 10.1007/978-3-319-43234-2_16 © Springer International Publishing Switzerland 2017 Stufflebeam, D. L. (1971). The relevance of the CIPP evaluation model for educational accountability. Journal of Research and Development in Education, 5, 14-28. Stufflebeam, D. L. (2002). CIPP Evaluation Model Checklist. Retrieved on October 19, 2006. From http://www.mich.edu/evalctr/checklists/cippchecklist Tootian, S. (2019). Evaluation of training courses applied in succession planning in organizations using the CIPP model. International Journal of Human Capital in Urban Management (IJHCUM), 4(2):111-118, Spring 2019. Homepage: http://www.ijhcum.net/ Tunc, F. (2010). Evaluation of an English language Teaching program at a public university using CIPP model. Unpublished master thesis, Middle East Technical University, Turkey. Retrieved fromhttp://www.etd.libmetu.edu.tr/upload/12611570/index Tyler, R. (1990). Reporting evaluation of learning outcomes. Oxford: Pergamum. Ulumi, G. O. (2016). Evaluation of english as a foreign Language program - using CIPP (context, input, process and product) model. European Journal of English Language Teaching, 1 (2), 2016, pp. 114-137. www.oapub.org/edu105281/zenodo.203077. Umam, K. A. I. & Saripah, I. (2018). Using the Context, Input, Process and Product (CIPP) Model in the Evaluation of Training Programs. International Journal of Pedagogy and Teacher Education (IJPTE), 2, July 2018), pp.183-194. DOI: 10.20961/ijpte.v%vi%i.26086. Wisniewski, K. (2017). Empirical learner language and the levels of the common European framework of reference. Language Learning, 67(S1), 232-253. Xiao, Y., & Yang, M. (2019). Formative assessment and self-regulated learning: How formative assessment supports students' self-regulation in English language learning. System, 81, 39-49. http://www.mich.edu/evalctr/checklists/cippchecklist