This is an open access article under the CC-BY-SA license. REiD (Research and Evaluation in Education), 5(2), 2019, 130-143 Available online at: http://journal.uny.ac.id/index.php/reid Evaluation of the implementation of Batik-skills training program *1Hendro Prasetyono; 2Dedeh Kurniasari; 3Laila Desnaranti 1,3Department of Economics Education, Universitas Indraprasta PGRI Jl. Nangka No 58C, Tanjung Barat, Jagakarsa, Jakarta Selatan 12530, Indonesia 2Lembaga Pelatihan dan Keterampilan Cosmos Jl. Kasuari E2/106, Raya Patriot-Jakasampurna, Bekasi, Jawa Barat 17145, Indonesia *Corresponding Author. E-mail: hen.dro23@yahoo.com Submitted: 8 March 2019 | Revised: 3 October 2019 | Accepted: 29 October 2020 Abstract The purpose of this study is to evaluate the implementation of batik skills training program as a recommendation for program improvement. The method used in this research is a qualitative approach using Context, Input, Process, and Product evaluation model. Samples were taken from the Institute of Skills and Training in the areas of Jakarta, Bogor, Depok, Bekasi, and Tangerang. The results of the evaluation components that meet the evaluation criteria are all aspects of the context component, discipline and learning process, while the components of batik teachers' education qualifications, the use of educational facilities and infrastructure standards, curriculum components, program financing, evaluation of learning outcomes, mastery of theoretical competencies, practices, and impacts to program participants have not been met. The batik skills training program needs to be continued with some improvement. It is recommended for the product components, especially on the impact aspects felt by graduates, to be improved. Keywords: training program implementation, batik skill, learning outcomes, institute of skills and training Permalink/DOI: https://doi.org/10.21831/reid.v5i2.23918 Introduction The development of batik in Indonesia today is impressive. Batik is very popular and growing rapidly in the country since the re- cognition of batik by UNESCO in 2009 as a world cultural heritage which is originated in Indonesia (Pancapalaga, Bintoro, Pramono, & Triatmojo, 2014). These developments affect and make people realize that batik can create jobs with special skills which are very prom- ising. It is reinforced by Data from Central Java Industry and Trade Department (2001) that there are 11,391 of batik unit productions spreading at 146 production centers with pro- duction values of €5.4 million (Hunga, 2011). Batik is a decorated textile made using the wax-resist method. Wax can be used to create intricate designs using an instrument called canting, and batik made using this vessel is called batik tulis (Lee, 2016). Batik develop- ment in Indonesia experienced various obsta- cles. One of the inhibiting factors in the de- velopment of batik industry using a printing machine. It happens because of advancements in information technology and knowledge which result in instability in all aspects of in- dustrial life. It is necessary to use management theory to characterize and understand the na- ture of turbulence that occurred (Anderson, Mason, Hibbert, & Rivers, 2017). One of the management theory imple- mentations in the education field is the devel- opment of batik skill practice in the form of non-formal education. Non-formal education as part of the education system has the same task as formal education, which provides the best service to the community. Alternative services programmed outside the education can serve as a replacement, enhancer, and or complement to the formal education system. https://doi.org/10.21831/reid.v5i2.23918 Evaluation of the implementation of Batik-skills... Hendro Prasetyono, Dedeh Kurniasari, & Laila Desnaranti Copyright © 2019, REiD (Research and Evaluation in Education), 5(2), 2019 - 131 ISSN 2460-6995 According to Field in Hoppers (2006, p. 21), non-formal education is rarely used, the term lifelong learning has increasingly gained currency when referring to the totality of edu- cational activities outside the school system. In the third form of education, the instructor and the learner are not in direct contact with each other. The learning material is provided for students by post and is written in a very simple language so that students can easily un- derstand and comprehend the material. Dia- grams and exercises are included in these courses to support and guide the students ac- cordingly. Some materials are communicated through TV channel programs and the inter- net (Saif, Reba, & Din, 2017). Non-formal education is usually not documented by a certificate or transcript. It occurs in educational institutions or public or- ganizations, clubs, circles, as well as during individual sessions with teachers or coaches (Ivanova, 2016). Non-formal forms of educa- tion are often characterized by the strong par- ticipation of volunteers, viewed as an expres- sion of the Protestant teaching of the priest- hood of all believers (Schweitzer, 2017). The targets of non-formal education are increasingly diverse, such as serving the poor, those who have not completed basic educa- tion, dropouts and dropping out of formal education, people who do not have access to formal education such as; isolated tribes, rural communities, border areas, and outer island communities and the development of a spe- cial skill or skill in the form of training. The existence of non-formal education such as courses and training institutions helps the community to develop. To improve the quality of education and quality of human resources in the field of education, the gov- ernment has issued a policy. Law of Republic of Indonesia No. 20 of 2003 on the national education system and the Regulation of the Minister of National Education No. 19 of 2005 on the national education standards, clause 26 verse 5 state that courses and train- ing are held for people who need knowledge, skills, life skills, and attitudes to develop themselves, professions, work, independent business, or continue their education to a higher level. One of non-formal education form which is widely adopted by the community is training. Mahapatro (2010, p. 252) states that training is the systematic development of the attitude, knowledge, skill pattern required by a person to perform a given task or job ade- quately. Mahapatro also state that develop- ment is ‘the growth of the individual in terms of ability, understanding, and awareness.’ Training is required to properly motivate and prepare the workers for operating these mechanisms effectively (White, 2004, p. 641). There are three specific training objec- tives according to Stredwick (2005, p. 376): (1) to develop the competencies of employees and improve their performance, (2) to help people grow within the organization in order that, as far as possible, in the future, the needs for human resources can be obtained from within the organization, (3) to reduce learning time for employees starting in new jobs on appointment, transfer or promotion, and en- sure they become fully competent as quickly and economically as possible. Besides, three components must be prepared in training: academic preparation, pedagogical skills, and teaching practice (Younus & Akbar, 2017). The training approach can be used in batik development since the training delivery approach, containing a combination of tech- nical and entrepreneurial skills, was relevant in responding to the needs and objectives of adult trainees (Mayombe, 2017). Training can address gap issues between education and in- dustry needs. Perhaps one of the most ambi- tious challenges currently that instructors of introductory management courses have to deal with is their desire to make their courses more relevant and meaningful for today’s learners, including students with little or no work experience (Wright & Gilmore, 2012 in Durant, Carlon, & Downs, 2017). It is in accordance with Kyriakides and Campbell in Govender, Grobler, and Mestry (2016) that school quality improvement seems supporting a culture of school audits, the phi- losophy of which notes that governments are now formally placing increased expectations on school leadership/management teams to integrate self-evaluation into both their strate- gic and operational structures and procedures. Evaluation of the implementation of Batik-skills... Hendro Prasetyono, Dedeh Kurniasari, & Laila Desnaranti 132 - Copyright © 2019, REiD (Research and Evaluation in Education), 5(2), 2019 ISSN 2460-6995 The batik training program is designed to accommodate training participants with knowledge, workability, designing, making, organizing, and packaging batik products needed by the industry. Batik will be pre- served and in accordance with the needs of the industrial world. Hence, it is necessary to conduct research to evaluate the success rate of the batik skills training program. It is in accordance with the opinion of Posavac and Carey (1980, p. 8) regarding the six reasons for the need of evaluation: '(1) fulfillment of accreditation requirements; (2) accounting funds; (3) answering requests for information; (4) making administrative decisions; (5) assist- ing staff in program development; (6) learning about unintended effects'. An evaluation is a systematic process that gives out information about program achievement (Dewi & Kartowagiran, 2018). Program evaluation is a systematic and on- going process to collect, describe, interpret and present the information to be used as a basis to make decisions, develop policies, and preparing the next program (Prasetyono, 2016). Meanwhile, evaluation in training ac- cording to Torrington, Hall, and Taylor (2005, p. 402) 'Evaluation is straight forward when the output of the training is clear to see, such as reducing the number of dispatch errors in a warehouse or increasing someone’s typing speed'. The essence of 'evaluation is a com- parison, and surveys are one (but not the on- ly) way to collect information useful for com- paring programs or for comparing individual performance’ (Langbein & Felbinger, 2006, p. 192). Evaluation is about a particular initia- tive. It is generally carried out to assess the initiative, and the results are not generalizable. Evaluations are designed to improve an ini- tiative and to provide information for deci- sion making at the program or policy level; the research aims to prove whether there is a cause and effect relationship between two entities in a controlled situation (Harris, 2010, p. 2). Evaluation is different from monitor- ing. According to Singh (2007, p. 54), evalu- ation is different from monitoring in many ways. Monitoring usually provides informa- tion regarding the performance of process indicators, whereas evaluation assesses the performance of impact indicators. Monitoring is an internal process where all concerned project staff devises a monitoring system, while evaluation is usually done by an external agency to assess the project’s achievements. Evaluation is a selective exercise that attempts to systematically and objectively assess pro- gress towards the achievement of an outcome. The purpose of the evaluation accord- ing to Edwards, Scott, and Raju (2007, p. 58) is 'the ultimate goal for the evaluation team is to deliver the most useful and accurate infor- mation to key stakeholders most cost-effec- tively and realistically possible'. Program eval- uation is a comprehensive approach that in- volves three general steps: (1) developing pro- gram theory, (2) formulating and prioritizing evaluation questions, and (3) answering evalu- ation questions (Donaldson & Scriven, 2008, pp. 109–110). The purpose of this research is to eval- uate the implementation of the batik skills training program as a recommendation for improving the batik training program in Indonesia based on the program evaluation result with Context, Input, Process, Product (CIPP) model. This CIPP model is suitable for evaluating formal and non-formal edu- cation programs, such as training (Madaus & Stufflebeam, 2002) because evaluation can an- swer the variation of the statement and deter- mine the success in viewing the quality of education (Prasasti & Istiyono, 2018). Method The research method used a qualitative approach. Qualitative research is defined as the collection, analysis, and interpretation of comprehensive narrative and visual (i.e., no numerical) data to gain insights into a parti- cular phenomenon of interest (Gay, Mills, & Asian, 2012, p. 7). The program evaluation model selected is CIPP. Corresponding to the letters in acronym CIPP, the model's core concepts are Context, Input, Process, and Product evaluation. Context evaluation asses- ses needs, problems, and opportunities as a base for defining goals and priorities and judging the significance of outcomes. Input Evaluation of the implementation of Batik-skills... Hendro Prasetyono, Dedeh Kurniasari, & Laila Desnaranti Copyright © 2019, REiD (Research and Evaluation in Education), 5(2), 2019 - 133 ISSN 2460-6995 evaluation assesses alternative approaches to meetings need as a means of plans to guide activities and later to help explain outcomes. The evaluative method was used in order to evaluate the process of product development (Istiyani, Zamroni, & Arikunto, 2017). Prod- uct evaluation identifies intended and unin- tended outcomes both to help keep the proc- ess on track and determine effectiveness (Stufflebeam, Madaus, & Kellaghan, 2002, p. 279). The research was conducted on six Skills and Training Institutions (STI) under the Asosiasi Profesi Batik dan Tenun Nusantara or Association of Batik and Tenun Nusantara Professions 'Bhuana' (APBTN 'Bhuana'). The APBTN 'Bhuana' innovates in the planning of non-formal education programs for batik courses according to national education stan- dards. APBTN 'Bhuana' under the Director- ate of Community Education and Training Course of the Ministry of National Education of the Republic of Indonesia has a great influence and authority in conducting batik training in Indonesia. The six STIs are STI Cosmos (Bekasi City), STI Lesha (Bekasi Regency), STI Tradisiku (Bogor Regency), STI Asri (Depok City), STI Kris (DKI Jakarta), and STI Datik (Tangerang Regency). The total key informants in the interviews were 18 people and the respondents for the questionnaire were 120 people. The data were collected using inter- views, questionnaires, observations, and docu- ment studies. The research instrument was developed based on four components in the CIPP evaluation model. The component and aspect of program evaluation with the CIPP model are presented in Table 1. The program evaluation criteria were divided into points of the interview, question- naire, observation statements, and document review. The total is 66 items. Expert analysis was used in testing the instrument validity and reliability. Table 1. Evaluation component and aspect of batik training program Component Aspect 1. The purpose of batik skills training a. Formulation of goals b. Basic formulation c. The foundation of objective formulation deliberation 2. The Program Design a. Formulation of the problem b. The use of standard learners c. The use of educational standards d. The use of educator and of Education staff standard e. The use of the curriculum program f. The determination of material program g. The design of learners tasks h. Financing 3. Program Implementation a. The rules for learner discipline b. Calendar of events and learning schedule c. Syllabus and learning process plan d. The learning process e. Program Supervision f. Implementation of learning evaluation g. Supervision of learning evaluation 4. Program Implementation Results a. Mastery of theoretical competencies b. Mastery of Practice Competencies c. Mastery of Attitudinal Competence d. Graduation certificates e. Opening new entrepreneurs f. Open your own training g. Become an educator h. Being a batik assessor Evaluation of the implementation of Batik-skills... Hendro Prasetyono, Dedeh Kurniasari, & Laila Desnaranti 134 - Copyright © 2019, REiD (Research and Evaluation in Education), 5(2), 2019 ISSN 2460-6995 Data obtained from interviews, docu- ment studies, and observations in the analysis using a qualitative approach. Qualitative data analysis is data reduction, display data, and verification (Miles & Huberman, 1992, pp. 16–17). The survey questionnaire consisted of four-point Likert scale items and qualitative items that were developed from and linked to the reviewed literature. The questionnaire was supervised by five teachers to get feedback and make necessary changes; however, their responses were not included in the collected data and they were asked not to take the survey again (Mcminn, Kadbey, & Dickson, 2015). The evaluation study of the unit ana- lysis program implementation focuses on how the process and quality training. According to Dane and Schneider and Dusenbury, et al. in Huff, Preston, and Goldring (2013), their coaches are differentiated in two key dimen- sions of implementation: dose and the quality of the program delivery. Findings and Discussion Findings The first component is the context component in the evaluation with the CIPP model consisting of three aspects of evalu- ation. The first aspect is the policy back- ground. Based on the results of interviews and study documents on the Law of Republic of Indonesia No. 20 of 2003, Regulation of the Minister of National Education No. 19 of 2005, and Regulation of the Minister of Na- tional Education No. 47 of 2010 on organ- izing courses, it is found that the batik skills program conducted by STI under the APBTN 'Bhuana' had a policy background in accor- dance with the law. The second aspect is the formulation of the objectives of the batik skills training program. Based on the results of interviews and document studies regarding the teaching design of each STI, the results of the formulation of program objectives are in line with the law and learning objectives. The third aspect is the objectives of the Batik Skills Program. Based on interviews and studies of STI administrative documents, the results of the process of formulating the ob- jectives of the program involved various cen- tral parties ranging from government ele- ments to representatives from all regions in Indonesia. Besides, the program's objectives are in accordance with community needs, es- pecially in increasing employment and cre- ating national batik business development. The second component is the Input component in the evaluation with the CIPP model consisting of six aspects of evaluation. The first aspect is the problem formulation of the batik skills program. Based on the results of interviews with managers, teachers, and participants of each STI, the results of the formulation of problems in each STI are adjusted to the conditions in the field and respond to challenges ahead, it's just not specific enough. The second aspect is the standard of prospective students in the batik skills pro- gram. Based on the results of interviews with the study participants' biodata documents, it was found that there was a mismatch between the minimum educational background of pro- gram participants on the condition of being a program participant. Thus, it can be con- cluded that the standard aspects of prospec- tive program participants 50% are met. The third aspect is the teacher's aca- demic qualifications for the batik skills train- ing program. Based on the results of inter- views and studies of training instructors' qual- ification documents, 67% of the educational qualifications of the trainers meet the educa- tional qualification standards and about 33% have not met the educational standards. All teachers have certificates as a batik instructor issued by the Ministry of Education and Culture also Ministry of Manpower but have no educational background in art majors. There are 80% of teachers who have high school educational background and 65% of teachers have experience as a batik instructor at least for three years. The fourth aspect is the use of facilities and infrastructure standards. Based on direct observations and photos of new infrastructure facilities, only 50% of STIs meet the stan- dards. The fifth aspect is the batik skills train- ing program curriculum. Based on the results of interviews and study of learning curriculum documents, the curriculum used have used Evaluation of the implementation of Batik-skills... Hendro Prasetyono, Dedeh Kurniasari, & Laila Desnaranti Copyright © 2019, REiD (Research and Evaluation in Education), 5(2), 2019 - 135 ISSN 2460-6995 the Indonesian National Qualification Frame- work or Kerangka Kualifikasi Nasional Indonesia (KKNI) up to level 3. The sixth aspect is the financing of the Batik Skills training program. Based on interviews and studies of STI finan- cial report documents, each STI still relied on the money from the training participants as the main source of income. The third component is the process component in the evaluation with the CIPP model consisting of three aspects of evalu- ation. The first aspect is the training rules. Based on the results of the distribution of questionnaires to 120 respondents from six STIs, interviews, and document studies, it is found that the results of the activity regu- lations have been implemented in accordance with activity planning, although not 100% of all STIs have implemented them. The second aspect is the learning proc- ess of the Batik Skills Program. Based on the results of the distribution of questionnaires to 120 respondents from six STIs, interviews, and participants observation, the sub-aspect of teachers provide instructional objectives when starting training gains the achievement rate of 70%. Teaching sub-aspect explained the material with discussion methods with the level of achievement of 87.5%. As for the sub-aspects of the instructor in explaining the material using learning tools, the achievement level is 100%. The third aspect is the evaluation of learning outcomes in the Batik Skills training program. Based on the results of the distri- bution of questionnaires to 120 respondents from six STIs, interviews, and participant observation, the highest answer for the sub- aspect of assignment at home is the STI Tradisiku, whereas the lowest number of answers is STI Lesha and STI Datik. The results of the recapitulation of the sub-aspects questionnaires show that the institution which most often achieved the highest number of answers to quizzes are STI Tradisiku, and the lowest number of answers is STI Datik and STI Cosmos. The fourth component is the product component in the evaluation with the CIPP model consisting of four aspects of evalu- ation. The first aspect is the mastery of theo- retical competence. Based on the results of the recapitulation of questionnaires, docu- ment studies, and interviews, the average score of quizzes and theory tests was 78. Meanwhile, for the sub-aspects of mastery of the theory of materials in making batik, 60% was achieved. Sub mastery of material aspects on various techniques in making batik by 42.57% was achieved. Sub mastery of material aspects regarding a variety of tools needed in the new batik process of 88.3% was achieved. The second aspect is the mastery of practice competencies. Based on the results of a recapitulation of questionnaires, observa- tions, and interviews, all participants get a minimum score of 'Good'. The third aspect is the proof of gradu- ation. Based on the results of the question- naire recapitulation, the study of documents, and interviews, all participants get a mark of competency test certificate graduation. The fourth aspect is the impact on program parti- cipants. Based on the results of the question- naire recapitulation, document studies, and interviews, it is found that the sub-aspects of the program graduates were able to open a batik production business with a percentage of the success rate of 20% with the highest number of alumni coming from STI Datik and STI Cosmos and the lowest number of alumni is from STI Kris and STI Asri. For the sub-aspect of 'the program graduates can open a batik selling business', the percentage of success rate is 50% with the highest number of alumni coming from STI Traditional and the lowest number of alumni is from STI Lesha. For the sub-aspect of 'the program graduates are able to open a batik course', the percentage of success rate is 16.7% with the highest number of alumni coming from STI Asri and the lowest number of alumni from STI Kris and STI Datik. For the sub-aspect of 'the program graduates can open a batik workshop', the percentage of success rate was 23.3% with the highest number of alumni coming from STI Asri and Cosmos while the lowest number of alumni from STI Kris and STI Datik. For the sub-aspect of 'the program graduates can open a tiered batik training', the percentage of the success rate is 16.7% with Evaluation of the implementation of Batik-skills... Hendro Prasetyono, Dedeh Kurniasari, & Laila Desnaranti 136 - Copyright © 2019, REiD (Research and Evaluation in Education), 5(2), 2019 ISSN 2460-6995 the highest number of alumni coming from STI Cosmos and the lowest number of alum- ni from STI Lesha and STI Tradisiku. For the sub-aspect of 'the program graduates can become batik educators in schools', the per- centage of success rate is 86.7% with the highest number of alumni coming from STI Lesha, Traditional, and Datik, while the low- est number of alumni from STI Asri and STI Cosmos. For the sub-aspect of 'the program graduates are financially capable', only 20% of the total respondents are financially capable and have started the business to open their own batik production, 50% of the total respondents who answered has opened a selling batik business and have their own store or cooperation with third parties; 13.3% of the total respondents who answered are finan- cially capable and had opened a batik course. The majorities are constrained by complex funds and permits; 23.3% of total respon- dents are financially capable and have opened batik training workshops; 16.7% of the total respondents who answered are financially ca- pable to open tiered batik training. The tough- est constraint is the license and standards that must be fulfilled; 86.7% of the total respon- dents answered they are capable and had become batik educators in schools; 93.3% of the total respondents answered they can and have become batik educators in the general public. Discussion The first component in the evaluation of the CIPP model is the context component. The first aspect of the context component is the policy background. According to Ball policies ‘create circumstances in which the range of options available in deciding what to do are narrowed or changed’ (Ward et al., 2016). The batik skills program has a clear legal basis. Law of Republic of Indonesia No. 20 of 2003 on National Education System, clause 26 verse 5 up to the Regulation of the Minister of National Education No. 19 of 2005, on National Standard of Education, clause 1 verse 18 explain that education evalu- ation is an activity of controlling, guarantee- ing, and determining the quality of education to various education component at every path, ladder, and type of education as a form of accountability of education. It is regulated in the implementation of Permit for Non- Formal Education and Early Childhood, Number: 421.10/1572/Dik/K.018. It is fol- lowed by all STIs which have a policy base in accordance with the law and the theory of (Thrupp & Robert, 2003, p. 195). The second aspect is the basic formu- lation of program objectives. The basic for- mulation of program objectives is based on the analysis of learning conducted by non- formal education. It is in accordance with the Law of Republic of Indonesia No. 20 of 2003 on National Education System clause 26 verse 5 that courses and training are held for people who need knowledge, skills, life skills, and attitudes to develop themselves, develop their profession, work, independent business, and or continue to higher education. It is also in line with the definition of objectives in the evaluation programs outlined by Edwards et al. (2007, p. 58) that it is the ultimate goal for accurate information to key stakeholders in the most cost-effective and realistic manner possible. The basic formulation of the pro- gram should be able to provide accurate in- formation based on the problems created in society. The third aspect is the purpose of the program. The process of formulating the goals of the batik skills program involved vari- ous parties ranging from the Training Direc- tor of Course and Training, Directorate Gen- eral of Early Childhood Education and Com- munity Education, chairman of the APTBN 'Bhuana', and representative of every STI in Indonesia. The objectives of the program are as follows: (1) preserving batik art, (2) im- proving the quality and quantity of batik, (3) developing batik as one of the professions in education like a professional teacher, (4) im- proving the welfare of batik with competency certificates, (5) preparing prospective profess- sional educators before working for the com- munity, (6) creating entrepreneurship and em- ployment opportunities for people in batik. Context evaluation measures the needs, based on objectives and priorities and also assesses the results significantly (Stufflebeam et al., Evaluation of the implementation of Batik-skills... Hendro Prasetyono, Dedeh Kurniasari, & Laila Desnaranti Copyright © 2019, REiD (Research and Evaluation in Education), 5(2), 2019 - 137 ISSN 2460-6995 2002, p. 279). Context evaluation assesses activities on batik skills which determine the situation and background that affect the types of strategic objectives to be developed in the system. The idea is inappropriate with Frye and Hemmer (2012) that context is a study that identifies and defines program goals and priorities by assessing needs, problems, assets, and opportunities relevant to the program. The first aspect is problem formulation. The problem formulation on each STI match- ed with the conditions in the field and is oriented to the future, only less specific. In brief, each STI requires the right strategy and plan. It is in line with the understanding of the strategy by Fidler (2002, p. 10) that strate- gy is the direction and scope of an organiza- tion over the long term goal which achieves advantages for the organization through its configuration of resources within a changing environment, to meet the needs of markets and to fulfill stakeholder expectations. Mean- while, strategy in management education ac- cording to De Kluyver and Pearce II (2009), Gamble, Thompson and Peteraf (2013), Mintzberg, Lampel, Quinn, and Ghoshal (2003), as well as Thompson, Strickland, and Gamble (2010) cited by Albert and Grzeda is described as a strategic framework that in- cludes an analytical process relying on several prescribed tools and expects the student to arrive at a list of strategic options and sub- sequent recommendations for implementation (Albert & Grzeda, 2015). This strategy is included in the plan which will be achieved by the training pro- gram graduates. The findings are in accor- dance with the concept of planning said by Yukl (2010, p. 72) that planning is a broadly defined behavior that includes making deci- sions about objectives, priorities, strategies, organization of the work, assignment of re- sponsibilities, scheduling of activities, and al- location of resources among different activ- ities according to their relative importance. The second aspect is the standard of prospective learners. Each STI has met the criteria in the standard of prospective learn- ers. The ages of prospective learners vary from the youngest in high school student grade 2 and the oldest is 53 years old. The educational background is diverse, ranging from in with a background of junior high school education to undergraduate (S1). It is in accordance with regulations where trainees are at least 17 years of age with diverse edu- cational backgrounds. The third aspect is the trainers’ educa- tional qualification. The findings, as compared to the prevailing regulations, need to be re- analyzed according to the Regulation of the Minister of National Education No. 19 of 2005 clause 29, verse 4, that educators at sen- ior high school or Madrasah Aliyah (Islamic- based senior high school), or other equivalent forms of education have: (a) a minimum edu- cation qualification of Diploma-four (D-IV) or Bachelor (S1); (b) a higher education back- ground with an education program appro- priate to the subject being taught; and (c) professional teacher certificate for senior high school. Whereas, it includes educators at the course and training skills institutions which consist of teachers, mentors, trainers or in- structors, and examiners. Therefore, it means that the faculty of the course and training institutions may come from faculty in higher education institutions. According to Sullivan, Mackie, Massy, and Sinha (2012, p. 24), higher education qualifies graduates for jobs or additional training as well as increasing their knowledge and analytic capacities. These ben- efits of undergraduate, graduate, and profes- sional education manifest as direct income effects, increased social mobility, and health, as well as other indirect effects. The fourth aspect is the condition of the facilities and infrastructure. The findings at the Institute of Skills and Education (STI) Cosmos, STI Lesha, and STI Tradisiku have met the minimum standards of educational facilities and infrastructure and the other STI has not met the criteria. When referring to the Regulation of the Minister of National Edu- cation No. 19 of 2005 clause 42 verse 1 and 2, the minimum standards of facilities and infra- structure that must be owned by universities as Institute of Teachers' Education or Lembaga Pendidikan Tenaga Kependidikan (LPTK) are as follows: (1) each educational unit shall have facilities including furniture, educational e- quipment, educational media, books, and Evaluation of the implementation of Batik-skills... Hendro Prasetyono, Dedeh Kurniasari, & Laila Desnaranti 138 - Copyright © 2019, REiD (Research and Evaluation in Education), 5(2), 2019 ISSN 2460-6995 other learning resources, consumables, and other equipment necessary to support a reg- ular and continuous learning process. (2) Each educational unit is required to have infrastructure covering land, classrooms, head unit room, educator room, administrative room, library room, laboratory space, work- shop space, production unit space, canteen, power and service installation, sports venues, places for worship, playgrounds, creative ven- ues, and other space or places needed to sup- port the regular and ongoing learning process. The findings are in line with the theo- ries of Fry, Ketteridge, and Marshall (2009, p. 308) that 'most educational organizations now work under the pressure of a system in which space in their buildings and infrastructure is measured and accounted for in relation to student numbers and activities'. Based on the aforementioned findings, the researchers con- clude that the condition of infrastructure from three of six STIs as the organizer of batik training has been feasible. The fifth aspect is the batik training program curriculum. All STIs have a Compe- tency-Based Curriculum design. The six STIs have a standard design of batik graduation levels: level 1, 2, and 3. All STIs have skill program material. Based on all these findings, it can be concluded if aspects of the program curriculum have met the criteria. The sixth aspect is training program financing. This finding is less relevant when compared to the existing provisions. Standard of financing is the standard that regulates the components and the amount of operating unit cost of education applicable for one year (Fry et al., 2009, p. 3). Meanwhile, the financing standard according to the Regulation of the Minister of National Education No. 19 of 2005 clause 62 is elaborated as follows: Edu- cational financing consists of investment cost, operating cost, and personal cost. (1) The investment unit cost of education covers the cost of providing facilities and infrastructure, human resource development, and fixed working capital. (2) The personal cost covers the tuition fees that must be paid by the learn- ers in order to be able to follow the learning process regularly and continuously. The oper- ating cost of the educational unit includes (a) educators and education personnel salaries and all allowances attached to the salary, (b) educational materials or equipment, and (c) indirect education operating costs in the form of electric power, water, telecommunication services, facilities maintenance and infrastruc- ture, overtime pay, transportation, consump- tion, taxes, insurance, and so forth. (3) The standard operating cost of the education unit shall be stipulated by the Minister of Regu- lation based on the BSNP proposal. Every program must have good finan- cial management. Shattock (2003, p. 30) states that financial management emphasizes inte- grity, frugality, a concern for the pennies rath- er than the pounds, and a reluctance to bor- row, the more it will command internal re- spect and provide a secure financial base for acting opportunistically and responding quick- ly to environmental change. Conservative fi- nancial control mechanisms, however, can create unnecessary layers of hierarchy and bureaucracy and can choke initiative. The input component of the evaluation on the batik skills program which consists of the problem formulation and the students’ standard has met the criteria. However, the batik teacher's qualification, the use of edu- cational facilities and education standards, the curriculum, and the program financing com- ponents have not been fulfilled the criteria. Evaluation at the stage of the process is to see the program achievements and the ob- stacles encountered by STI in running the batik program. The evaluation studied the program implementation which emphasizes the process components (Tuytens & Devos, 2014). The further evaluation process focuses on the coaching and professional growth of the teacher during at least two evaluation conferences (one formative and one summa- tive). At the end of this process, an evaluation report is handed to the teacher. In this report, the teacher receives a final conclusion (two possibilities, namely: satisfactory or unsatis- factory) about his or her performance. The first aspect is the discipline in train- ing. Each STI has discipline regulations for their trainees, but only 80% of the partici- pants who have received the socialization of the contents. Evaluation of the implementation of Batik-skills... Hendro Prasetyono, Dedeh Kurniasari, & Laila Desnaranti Copyright © 2019, REiD (Research and Evaluation in Education), 5(2), 2019 - 139 ISSN 2460-6995 The second aspect is the process of teaching and learning. Teachers provide in- structional goals when they start the training. It has a 70% achievement level, while for learning with an explanation with a two-way model, the achievement level is 87.5%. Be- sides, for the use of learning tools in teaching materials, the achievement level is 100%. The third aspect is learning evaluation. Not all STIs provide assignments at home. Only STI Tradisiku and STI Asri give a home assignment, and STI Tradisiku is the only one that always gives their trainee quizzes. All STIs conduct formative, summative, and also competency tests. Stufflebeam and Shinkfield (2007, p. 294) said that the evaluation process is an ongoing check on a plan’s implemen- tation plus documentation of the process, in- cluding changes in the plan and key omissions or poor execution of certain procedures. One of the goals is to provide staff and managers feedback about the extent to which staff is carrying out planned activities on schedule, as planned, and efficiently. This opinion is sup- ported by Patton (1980, p. 60) that the 'proc- ess' focus in an evaluation implies an empha- sis on looking at how product or outcome is produced rather than looking at the same product itself; that is, it is an analysis of the processes whereby a program produces the result it does. Process evaluation is develop- mental, descriptive, continuous, flexible, and inductive. Process components evaluation empha- sizes how the product is produced compared to the product itself. The process stage in this research involves the aspects of strategy im- plementation and the use of facilities or ca- pital or resources in real activities in the field. Thus, it is concluded that the components of the evaluation process from the batik skills training program implementation consist of the order and process of learning activities that have met the criteria, while the evaluation of learning outcomes has not met the evalu- ation criteria. Product evaluation is the next stage of the evaluation program in the CIPP model. It is directed to the things that show the changes that occur in the input and what kind of re- sults. Product evaluation serves to interpret success in achieving objectives, assessing the data set, comparing the established criteria with the results obtained in the field, and the considerations associated with the context, inputs, and processes and formulating a ra- tional interpretation. The products from this program are batik professionals so the meas- ured product component is the competence of batik skill training program graduates. The first aspect is the mastery of the batik theory of program graduates. All the STIs have the average score from the quiz and theoretical exams are above 70. Based on the previously-mentioned findings, the mas- tery of batik material and various technique theories have not been fulfilled except the mastery of theory about the tools required in batik which has met the criteria. The second aspect is the Competency Mastery of Program Practice Graduates. All STIs have implemented these criteria and the majority of trainees score above 80%. If there are participants who score below 80, then the STIs held the remedial exam. If the trainees did not pass the remedial exam, then they are required to repeat the training. There are 86.7% of participants from all STIs who can use canting, while 100% of participants can use batik cap, 100% of participants can prepare the wax, 87.5% of participants can perform the coloring process, and only 82.5% can per- form the process of pelorotan. The third aspect is the proof of grad- uation. All participants get a certificate of the graduate program but not all participants get a certificate of batik competence. If they want to get a certificate of batik competence, they must take the test conducted by Place Com- petency Test (PCT). It means that after pass- ing the program, a participant will not always be recognized as a batik professional. He/she needs to take another exam recognized by the Ministry of Education and Culture. The fourth aspect is the impact toward the participant of the batik skill program. It is the final stage of a series of evaluations pro- gram in product components. Impact evalu- ation is directed at focusing on the impact questions felt by the participants after the program. It serves to analyze the influence felt by the participants after the program. Evaluation of the implementation of Batik-skills... Hendro Prasetyono, Dedeh Kurniasari, & Laila Desnaranti 140 - Copyright © 2019, REiD (Research and Evaluation in Education), 5(2), 2019 ISSN 2460-6995 Rossi, Freeman, and Lipsey (1999, p. 25) believes that impact evaluation is directed at focusing on the impact questions felt by the participants after the program. The evaluation questions are about which impact assessment is organized that is related to such matters as whether the desired program outcomes were attained, whether the program was effective in producing a change in the targeted social con- ditions, and whether the program impact in- cluded to unintended side effects. These ques- tions assume a set of operationally defined objectives and criteria for success. It is also important to know whether the impact measured has a positive or nega- tive impact, the kind of impact, how big the impact is, and whether it can give a positive change for the participants for their future, so it can be analyzed for further program im- provement. Stufflebeam et al. (2002, p. 229) said that to assess performance beyond goals, evaluators need to search for unanticipated outcomes, both positive and negative. They might conduct hearings or group interviews to generate hypotheses about the full range of outcomes and follow these up with an effort to confirm or disconfirm the hypotheses. The findings on the impact aspect which is the most important thing for the in- dividual after a program is to get a better job or income. It is in line with Sullivan et al. (2012, p. 24) that the benefits of undergradu- ate, graduate and professional education ma- nifest as direct income effects, increased so- cial mobility, health, and also other indirect effects. Measures have been created to moni- tor changes in these outputs, narrowly de- fined: numbers of degrees, time to degree, de- gree mix, and the like. Attempts have also been made to estimate the benefits of educa- tion using broader concepts, such as the ac- cumulation of human capital. For estimating the economic returns to education, a starting point is to examine income differentials a- cross educational attainment categories and institution types, attempting to correct for other student characteristics. All the evaluation stages, starting from the evaluation of context, input evaluation, process evaluation, and product evaluation, are a unity that cannot be separated, but it has to be done together depending on the con- ditions in the field. The findings are in accor- dance with Stufflebeam and Shinkfield (2007, p. 294) that the purpose of product evaluation is to measure, interpret, and judge an enter- prise’s achievements. Its main goal is to ascer- tain the extent to which the evaluation met the needs of all the rightful beneficiaries. Based on the findings, the number of stakeholders who have been made as the re- spondent has met the requirements. Accord- ing to Bamberger, Rugh, and Mabry (2006, p. 271), program impact and quality cannot be determined without understanding the diverse experience of stakeholders. The perceptions of many must be search out. The impact aspect on the evaluation of the implementation of batik skills program has given many positive influences to the pro- gram participants but has not given certainty about the condition of their destiny in the fu- ture, because those who have passed the pro- gram have not been able to open their own job or looking for work as batik craftsman. Product components in the evaluation of the program which consist of mastery of theo- retical competence has not been fulfilled, mastery of practice competence and pro-gram pass mark has been fulfilled. The impact as- pect on the program participants has not pro- vided a promising future when the program participants have completed their training. Conclusion Referring to the research findings and discussion, all evaluation component in the implementation of the training program is carried out well, except the results of program implementation. The results that must be im- proved is the impact that the participants will get after attending the training at the STI. If STI graduates cannot work either opening a batik business or work as batik craftsman, the batik skills training program may not develop. If it does not develop, there is no generation who will continue their batik skills. It is re- commended for the Ministry of National Education to collaborate with the Ministry of Cooperatives and Small Medium Enterprises, local governments, and related agencies to provide business opportunities to STI gradu- Evaluation of the implementation of Batik-skills... Hendro Prasetyono, Dedeh Kurniasari, & Laila Desnaranti Copyright © 2019, REiD (Research and Evaluation in Education), 5(2), 2019 - 141 ISSN 2460-6995 ates in the form of soft loan assistance, entre- preneurship training, and facilitation in pro- viding business licenses. Acknowledgments The researchers would like to thank the Association of Batik and Tenun Nusantara Professions ‘Bhuana’ (APBTN ‘Bhuana’) for giving permission and supporting the authors to conduct this research. References Albert, S., & Grzeda, M. (2015). Reflection in strategic management education. Journal of Management Education, 39(5), 650–669. https://doi.org/10.1177/10525629145 64872 Anderson, L., Mason, K., Hibbert, P., & Rivers, C. (2017). Management education in turbulent times. Journal of Management Education, 41(2), 303–306. https://doi.org/10.1177/10525629166 82208 Bamberger, M., Rugh, R., & Mabry, L. (2006). Real world evaluation. London: SAGE Publications. Dewi, L. R., & Kartowagiran, B. (2018). An evaluation of internship program by using Kirkpatrick evaluation model. REiD (Research and Evaluation in Education), 4(2), 155–163. https://doi. org/10.21831/reid.v4i2.22495 Donaldson, S. I., & Scriven, M. (2008). Evaluating social programs and problems. Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Durant, R. A., Carlon, D. M., & Downs, A. (2017). The efficiency challenge: Creating a transformative learning experience in a principles of management course. Journal of Management Education, 41(6), 852–872. https://doi.org/10.1177/10525629166 82789 Edwards, J. E., Scott, J. C., & Raju, N. S. (2007). Evaluating human resources programs. San Francisco, CA: John Wiley & Sons. Fidler, B. (2002). Strategic management for school development: Leading your school’s improvement strategy. https://doi.org/ 10.4135/9781446219614 Fry, H., Ketteridge, S., & Marshall, S. (Eds.). (2009). A handbook for teaching and learning in higher education: Enhancing academic practice (3rd ed.). New York, NY: Routledge. Frye, A. W., & Hemmer, P. A. (2012). Program evaluation models and related theories: AMEE guide no. 67. Medical Teacher, 34(5), e288–e299. https://doi. org/10.3109/0142159X.2012.668637 Gay, L. R., Mills, G. E., & Asian, P. (2012). Educational research: Competencies for analysis and applications. Boston, MA: Pearson. Govender, N., Grobler, B., & Mestry, R. (2016). Internal whole-school evaluation in South Africa. Educational Management Administration & Leadership, 44(6), 996–1020. https://doi.org/ 10.1177/1741143215595414 Harris, M. J. (2010). Evaluating public and community health programs. San Francisco, CA: Jossey-Bass. Hoppers, W. (2006). Non-formal education and basic education reform: A conceptual review. Paris: International Institute for Educational Panning UNESCO. Huff, J., Preston, C., & Goldring, E. (2013). Implementation of a coaching program for school principals. Educational Management Administration & Leadership, 41(4), 504–526. https://doi.org/ 10.1177/1741143213485467 Hunga, A. I. R. (2011). Uncover the invisible: Home-workers in micro-small-medium industries based on “putting-out” system (The case study of the Batik and Batik convection industry in a Sragen- Surakarta-Sukoharjo cluster of Indonesia). The International Journal of Interdisciplinary Social Sciences, 5(9), 311– 322. Evaluation of the implementation of Batik-skills... Hendro Prasetyono, Dedeh Kurniasari, & Laila Desnaranti 142 - Copyright © 2019, REiD (Research and Evaluation in Education), 5(2), 2019 ISSN 2460-6995 Istiyani, D., Zamroni, Z., & Arikunto, S. (2017). A model of madrasa ibtidaiya quality evaluation. REiD (Research and Evaluation in Education), 3(1), 28–41. https://doi.org/10.21831/reid.v3i1.139 02 Ivanova, I. V. (2016). Non-formal education: Investing in human capital. Russian Education & Society, 58(11), 718–731. https://doi.org/10.1080/10609393.201 7.1342195 Langbein, L., & Felbinger, C. L. (2006). Public program evaluation. New York, NY: M. E. Sharpe. Law of Republic of Indonesia No. 20 of 2003 on National Education System. , (2003). Lee, T. (2016). Defining the aesthetics of the Nyonyas ’ Batik Sarongs in the straits settlements, late nineteenth to early twentieth century. Asian Studies Review, 40(2), 173–191. https://doi.org/ 10.1080/10357823.2016.1162137 Madaus, G. F., & Stufflebeam, D. L. (2002). Evaluation in education and human services. Basel: Springer Nature. Mahapatro, B. B. (2010). Human resource management. New Delhi: New Age International Limited. Mayombe, C. (2017). Integrated non-formal education and training programs and centre linkages for adult employment in South Africa. Australian Journal of Adult Learning, 57(1), 105–125. Retrieved from https://www.ajal.net.au/integra ted-non-formal-education-and-training- programs-and-centre-linkages-for-adult- employment-in-south-africa/ Mcminn, M., Kadbey, H., & Dickson, M. (2015). The impact of beliefs and challenges faced, on the reported practice of private school science teachers in Abu Dhabi. Journal of Turkish Science Education, 12(2), 69–79. https:// doi.org/10.12973/tused.10141a Miles, M. B., & Huberman, A. M. (1992). Qualitative data analysis. Beverly Hills, CA: SAGE Publication. Pancapalaga, W., Bintoro, V. P., Pramono, Y. B., & Triatmojo, S. (2014). The chrome- tanned goat leather for high quality of Batik. Journal of the Indonesian Tropical Animal Agriculture, 39(3), 188–193. https://doi.org/10.14710/jitaa.39.3.188 -193 Patton, M. Q. (1980). Qualitative evaluation methods. Beverly Hills, CA: SAGE Publication. Posavac, E. J., & Carey, R. G. (1980). Program evaluation: Methods and case study. Englewood Cliffs, NJ: Prentice-Hall. Prasasti, I. H., & Istiyono, E. (2018). Developing an instrument of national examination of equivalency education Package C of mathematics subject. REiD (Research and Evaluation in Education), 4(1), 58–69. https://doi.org/ 10.21831/reid.v4i1.15556 Prasetyono, H. (2016). Graduate program evaluation in the area leading educational, outlying and backward. Journal of Education and Practice, 7(36), 109–116. Retrieved from https:// www.iiste.org/Journals/index.php/JEP /article/view/34641 Regulation of the Minister of National Education No. 19 of 2005, on National Standard of Education. , (2005). Regulation of the Minister of National Education No. 47 of 2010, on the Competence Standard of Training Graduates. , (2010). Rossi, P. H., Freeman, H. E., & Lipsey, M. (1999). Evaluation: A systematic approach (2nd ed.). Thousand Oaks, CA: SAGE Publications. Saif, P., Reba, A.-, & Din, J. U. (2017). A comparitive study of subject knowledge of B.Ed graduates of formal and non- formal teacher education systems. Journal of Education and Educational Development, 4(2), 270–283. https://doi. org/10.22555/joeed.v4i2.1354 Schweitzer, F. (2017). Researching non- formal religious education: The example of the European study on confirmation Evaluation of the implementation of Batik-skills... Hendro Prasetyono, Dedeh Kurniasari, & Laila Desnaranti Copyright © 2019, REiD (Research and Evaluation in Education), 5(2), 2019 - 143 ISSN 2460-6995 work. HTS Teologiese Studies / Theological Studies, 73(4), 1–9. https://doi.org/ 10.4102/hts.v73i4.4613 Shattock, M. (2003). Managing successful universities. London: Open University Press. Singh, K. (2007). Quantitative social research methods. New Delhi: SAGE Publications. Stredwick, J. (2005). An introduction to human resource management. Oxford: Elsevier. Stufflebeam, D. L., Madaus, G. F., & Kellaghan, T. (2002). Evaluation models: Viewpoints on education and human services evaluation (2nd ed.). Boston, MA: Kluwer Academic Publisher. Stufflebeam, D. L., & Shinkfield, A. J. (2007). Evaluation theory, models and applications. San Francisco, CA: Jossey-Bass. Sullivan, T. A., Mackie, C., Massy, W. F., & Sinha, E. (Eds.). (2012). Improving measurement of productivity in higher education. https://doi.org/10.17226/ 13417 Thrupp, M., & Robert, W. (2003). Education management in managerialist times: Beyond the textual apologists. Philadelphia, PA: Open University Press. Torrington, D., Hall, L., & Taylor, S. (2005). Human resource management Sixth Edition (6th ed.). London: Pearson Education. Tuytens, M., & Devos, G. (2014). The problematic implementation of teacher evaluation policy. Educational Management Administration & Leadership, 42(4_suppl), 155–174. https://doi.org/ 10.1177/1741143213502188 Ward, S., Bagley, C., Lumby, J., Hamilton, T., Woods, P., & Roberts, A. (2016). What is ‘policy’ and what is ‘policy response’? An illustrative study of the implementation of the Leadership Standards for Social Justice in Scotland. Educational Management Administration & Leadership, 44(1), 43–56. https://doi. org/10.1177/1741143214558580 White, C. (2004). Strategic management. New York, NY: Palgrave Macmillan. Younus, F., & Akbar, R. A. (2017). Comparison of evaluation methods of teaching practice of formal and non- formal teacher education institutions of Punjab. Bulletin of Education and Research, 39(1), 159–173. Retrieved from http://pu.edu.pk/images/journal/ier/P DF-FILES/12_39_1_17.pdf Yukl, G. A. (2010). Leadership in organizations. Upper Saddle River, NJ: Prentice Hall.