REiD (Research and Evaluation in Education) ISSN 2460-6995 REiD (Research and Evaluation in Education), 3(2), 2017, 106-113 Available online at: http://journal.uny.ac.id/index.php/reid Research Article An evaluation of vocational high schools in Indonesia: A comparison between four-year and three-year programs * 1 Soenarto; 2 †Muhammad Mustaghfirin Amin; 3 Kumaidi *Faculty of Engineering, Universitas Negeri Yogyakarta Jl. Colombo No. 1, Depok, Sleman 55281, Yogyakarta, Indonesia *Email: soenarto@uny.ac.id Submitted: 28 November 2017 | Revised: 15 December 2017 | Accepted: 22 December 2017 Abstract The research aimed to gain insights into the quality of four-year program vocational high school (VHS) in Indonesia compared to four-year program VHS. This research was conducted based on the school graduate standard, business sector and industrial sector (or Dunia Usaha dan Dunia Industri (DUDI)) – or the performance of the graduates and alumni (the graduates’ satisfaction). The research was conducted using Discrepancy Evaluation Model using 16 VHSs (eight four-year program VHSs and eight three-year program VHSs). The result shows that from the standpoint of the school, the graduates of the four-year program VHS are higher in quality than those of the three-year program VHS. The four-year program VHS graduates are more qualified in seven as- pects: teamwork, discipline, tenacity, theoretical knowledge, confidence, creativity, and leadership. Meanwhile, using DUDI standpoint, the four-year program VHS graduates are also higher in quality than the three-year program VHS graduates. In addition, the four-year program VHS graduates are better in the quality of their discipline, tenacity, theoretical knowledge, practical skills, confidence, carefulness, creativity, and leadership. The four-year program VHS graduates have a higher level of satisfaction in terms of income than the three-year program VHS grad- uates. The higher quality of the four-year program VHS graduates has resulted from longer duration of the internship program (PKL) that provides them with reliable experience and skills concerning work-related problem-solving activities. Keywords: vocational high school, graduates, four-year program, three-year program How to cite item: Soenarto, S., Amin, M., & Kumaidi, K. (2017). An evaluation of vocational high schools in Indonesia: A comparison between four-year and three-year programs. REiD (Research and Evaluation in Education), 3(2), 106-113. doi:http://dx.doi.org/10.21831/reid.v3i2.17077 Introduction Education institution is a human re- source production house with the managerial competency related to human resource and other related resources. Thus, it is the duty of education institutions to keep the process of improvement going and to produce graduates who fulfill the needs of the society. The socie- ty needs evolve as time changes and in align- ment with the changes of circumstances. As Asean Free Trade Area (AFTA) and Asean Eco- nomic Society (MEA) were put into effect in 2003 and in 2015 respectively, the demand of business sector and industry sector (dunia usa- ha dan dunia industri or DUDI) for innovative and creative workforce is on the rise. On the other hand, free competition in the open mar- ket has made the distribution of goods, ser- vices, capital and market-ready skilled labor even more dynamic. To survive under such circumstances, Indonesia has to prepare itself http://dx.doi.org/10.21831/reid.v3i2.17077 REiD (Research and Evaluation in Education) 107 − REiD (Research and Evaluation in Education), 3(2), 2017 for upcoming chances and challenges in glob- al market. Alisjahbana (2014) states that in the free trade era, from the standpoint of popula- tion, manpower and human resource, Indone- sia has to pay more attention to three things: (1) keeping the demographic momentum, (2) improving the participation of manpower, and (3) improving the manpower productivity. The afore-mentioned action of keeping demographic momentum is an action of keep- ing the advantage of Indonesian demography conducted by pushing the fertility rate and driving migration. Migration is an effective strategy to keep the economic growth. The demographic momentum as the foundation of Indonesian economic strength has to be fol- lowed by the effort to improve the manpower participation by nurturing flexible and effi- cient working climate and driving the parti- cipation of women in improving the national economy. The area of manpower participa- tion is not the only area in need of improve- ment in Indonesia. Improvement is also need- ed in the area of manpower productivity, which can be a competitive advantadge that is able to improve the competitive edge of man- power in open market. Vocational high school (VHS – or SMK (Sekolah Menengah Kejuruan) is one of the edu- cation institutions responsible for producing skilled workers with the ability to adapt to the changes in the need of the society as the ef- fect of the dynamic international economy with the support of Indonesian demographic bonus. VHS can be a powerful weapon in im- proving manpower participation and produc- tivity by taking advantages of education pro- cesses. Pardjono, Sugiyono, and Budiyono (2015) state that ‘vocational education cannot be removed from existing workforce development’. Under the same light, in their research, Ramayani, Aimon, and Anis (2012) conclude that Indo- nesian government has to support the efforts made to improve manpower productivity by producing policy that focuses on education and health and providing more fund in the area related to human resource building. In the Law No. 20 of 2003 of Republic of Indonesia concerning National Education System, VHS is defined as the education insti- tution responsible for preparing students to work in certain fields of work. Dewey (1916) argues that ‘a vocation means nothing but such a direction of the life activities as renders them perceptibly significant to a person, because of the consequences they accomplish, and also usefull to his assocoate.’ Moreover, Thompson (1973) argues that vocational education improves students’ skills that eventually will improve their pro- ductivity. VHSs then play an important role in determining the competitive edge of Indo- nesian manpower by providing ready-to-work and high quality workers for national and also international needs. As stated by Komariah (2010), vocational high school is an education institution responsible to prepare students for labor market and nation-building effort. Prior to 1970, vocational high school and regular high school have the same study duration: three years. In 1970, as stipulated in through the First PELITA (Five-Year Build- ing, or Pembangunan Lima Tahun in Indonesian term) Program, Indonesian government built eight four-year program engineering vocation- al high schools under the banner of ‘Proyek Perintis Sekolah Teknologi Menengah Pembangunan’ (‘Development Engineering High School Ini- tiative Project’). In 1974, Indonesian govern- ment built four more four-year program voca- tional high schools – this time with agriculture as the concentration. The goal of this project is to prepare industrial technicians or workers with engineering skill possessing (1) initiative attitude, (2) ability to work and love the work, and (3) ability to understand, manage, and implement the ideas of engineers from upper level and to provide guidance to the technical workers from lower levels. The four-year pro- gram is expected to provide supports for vo- cational high schools in producing skilled workers with competitive edge. All of the goals of four-year program vocational high schools are in national education standard, specifically the standards for the graduates. In the Regulation of the Minister of Education and Culture No. 20 of 2016, the competence standard of the graduates (Standar Kompetensi Lulusan/SKL) is the formula of qualification criteria for graduates, which are achieved upon the completion of a series of programs and education in the area of attitude, knowledge, and skills. REiD (Research and Evaluation in Education) An evaluation of vocational high schools in Indonesia... - 108 Soenarto, †Muhammad Mustaghfirin Amin, & Kumaidi In order to be able to achieve the goals of the Development Engineering High School Initiative Project – which is now known as four-year program vocational high schools – the Directorate of Vocational High School Administration focuses on the improvement of curiculum, learning and teaching process, and evaluation process. To take everything one step further, the Directorate also focuses on the improvement of teacher profession- alism and builds cooperation with parties in- volved in business sector and industrial sector (DUDI). However, there were doubts related to the effectiveness of four-year program vocational high school as The National Statis- tics Board (Badan Pusat Statistik/BPS) released data related to the number of unemployment in Indonesia in 2014. The data show that there were 2.179 million unemployed gradu- ates of vocational high schools, which is 15.15% of the total number of unemployment in Indonesia for above-15-year-old workforce. The number is an accumulation of all un- employed graduates of four-year program vo- cational high schools and the graduates of three-year program vocational high schools. There were no distinction made between the graduates of four-year program vocational high schools and those of three-year program vocational high schools in the data presented by BPS although both of them do not follow the same education process. This phenome- non then made us wonder about the quality of the graduates of both programs and the differences. Table 1 shows the number of open unemployment with VHS education. Table 1. Vocational high school graduate unemployment data in 2011-2014 Year Total Number 2011 2,270,873 2012 2,085,474 2013 2,122,850 2104 2,179,886 The questions related to the worth or merit of four goals of the four-year program VHSs can only be answered through eval- uation. Stufflebeam and Shinkfield (1984) define evaluation as ‘systematic assessment of the worth or merit of some objects’. In this case, evaluation is conducted to define the worth or merit of the goals of four-year program voca- tional high schools. Stufflebeam, Madaus, and Kellaghan (2000) state that the process of evaluation should not be alien to the process of comparing. The evaluation of the worth or merit of the four-year program VHSs is conducted by comparing the competence of the graduates of both programs. The compe- tence of the graduates is measured with the standards set by schools of origin as the pro- vider of education services, the standards of DUDI (in terms of the performance of the graduates) as the employer of the graduates, and the personal standard of the graduates (level of satisfaction) related to their jobs. Method The goal of this evaluative research was to gain insights into the quality of education provided in both programs (three-year pro- gram and four-year program) of vocational high schools. The method applied in this re- search was Discrepancy Evaluation Model (DEM) developed by Provus. The Discrep- ancy Evaluation Model identifies discrepancy between the standards used as the basis of assessment and the performance in reality (Kaufman & Susan, 1982, p. 127). This re- search used three-year program vocational high schools’ graduates as the basis of assess- ment. The performance of the graduates of the three-year program vocational high school was set to be the basis or standards of assess- ment because it was the basis of the innova- tion that was known later as four-year pro- gram vocational high school. Inovation in this case is the production of something better than the existing product or program. This research was conducted in eight three-year program VHSs and eight four-year program VHSs. All of the selected four-year program VHSs were part of the early four- year program initiative. On the other hand, all of the selected three-year program VHSs were selected based on the similarities with the se- lected four-year program VHSs in terms of the area of the school location. The respon- dents included all parties involved in the man- agement of the vocational high schools, such as (1) head master, (2) vice head master, (3) REiD (Research and Evaluation in Education) 109 − REiD (Research and Evaluation in Education), 3(2), 2017 head of skill programs, (4) labor market coordinator, (5) guidance and counseling co- ordinator, (6) alumni, and (7) business sector and industrial sector. Table 2 shows detailed information about the respondents involved in the research. Table 2. Research respondents No Research Subjects four- year program three- year program Total 1 Head Master 8 8 16 2 Vice Head Master 32 32 64 3 Head of Skill Program 40 40 80 4 Special Labor Market Coordinator 8 8 16 5 Guidance and Counseling Coordinator 8 8 16 6 Alumni 40 40 80 7 DUDI 40 40 80 Total 176 176 176 The research data were collected using questionnaire, observation, interview and doc- umentation. The questionnaire in the data collection process was distributed to reach the opinions from schools and parties in DUDI about the competence of the graduates from both programs. In addition, the distribution of the questionnaire was also conducted to gain insights into the level of satisfaction of the graduates related to their jobs. The result of the validity test using a questionnaire show- ed that the instrument used was capable of measuring the data validly. The instrument reliability estimation shows that the question- naire had the reliability coefficient as much as 0.83 which can be categorized as reliable. In collecting the supporting data related to the graduates, this research used not only a ques- tionnaire but also observation, interview and documentation. The validity of the interview guide and observation guide was tested by ex- perts via expert judgement. In order to make the data fit to be pres- ented in the form of tables and diagrams, the collected data from various instruments were processed through tabulation and analysis process using the descriptive statistics. In this step, the qualitative data were projected as a support for quantitative descriptive findings. Findings and Discussion The result of the data analysis on the competence, performance, and level of satis- faction of the graduates of the four-year pro- gram vocational high schools in comparison with that of the graduates of the three-year program vocational high schools is described as follows. The Graduates’ Competence There are 11 aspects studied in this re- search, including team-work, discipline, ethics, tenacity, theoretical knowledge, practical skill, confidence, carefulness, creativity, sense of responsibility, and leadership. Figure 1 shows the competence of the graduates of four-year program and three-year program vocational high schoosl from the standpoint of school. Figure 1. The competence of vocational high school graduates from the standpoint of school Upon the analysis on the above-men- tioned aspects, from the standpoint of school – represented by head masters and vice head masters – all of the graduates of four-year program vocational high schools possess ‘very good’ competence. This result is better than the result for graduates of three-year program vocational high schools in which only 64% of them are in ‘very good’ category and the rest (36%) are in ‘good’ category. REiD (Research and Evaluation in Education) An evaluation of vocational high schools in Indonesia... - 110 Soenarto, †Muhammad Mustaghfirin Amin, & Kumaidi Figure 2. Aspects of competence of vocational high school graduates The result of the assessment on all of the aspects shows that the graduates of four- year program vocational high schools are better in seven out of eleven assessed aspects (teamwork, discipline, tenacity, theoretical knowledge, confidence, creativity, and leader- ship). On the other four aspects, the compe- tence of the graduates from both programs are in the same level. The superiority of the graduates of the four-year program vocational high schools on the seven aspects resulted from their rich experienced gained in a longer internship program (or Praktik Kerja Lapangan (PKL) in Indonesian term). This longer intern- ship program facilitated the students of four- year program vocational high schools with proficient time for in-class knowledge inter- nalization. In addition, the longer internship program made the students more experienced in problem-solving activities in the real daily work. The superiority of the four-year pro- gram vocational high school graduates in the seven aspects made them more competent at the business world. Figure 2 shows the com- petence of the graduates of both programs in every measured aspect. The Performance of the Graduates In this research, the performance of the graduates from both programs is also ana- lyzed from the standpoint of business sector and industrial sector (DUDI), specifically their aspects of competence. There are 11 aspects measured, including teamwork, discipline, ethic, tenacity, theoretical knowledge, practi- cal skills, confidence, carefulness, creativity, sense of responsibility, and leadership. Figure 3 shows the comparison in terms of perfor- mance of the graduates from both programs. Figure 3. The performance of vocational high school graduates The research result shows that 36% of the four-year program vocational high school graduates are in ‘very good’ category, whereas 64% of them are in ‘good’ category. The over- all performance of the graduates of four-year program VHSs is better than that of the grad- uates of three-year program VHSs since all of the graduates of three-year program VHSs are in ‘good’ category . REiD (Research and Evaluation in Education) 111 − REiD (Research and Evaluation in Education), 3(2), 2017 Figure 4. The performance of the graduates from the standpoint of DUDI According to the statement of the em- ployers or DUDI, the four-year program vo- cational high school graduates show supe- riority in eight aspects in terms of perfor- mance. They are superior in eight aspects - 72.72% of total aspects studied – including discipline, tenacity, theoretical skill, practical skill, confidence, carefulness, creativity, and also leadership (see Figure 4). As in the com- petence analysis from the standpoint of the schools, the superiority in these aspects is resulted from the longer internship programs which provide the students with richer and reliable experience. However, there is some- thing new and intriguing in this competence analysis from the standpoint of DUDI. In the aspect of teamwork, the graduates of the four-year program vocational high schools are inferior to that of the three-year program vo- cational high schools. This is driven by the fact that the graduates of the four-year pro- gram vocational high school have the ability to accomplish tasks individually since they are equipped with higher level of competence and experience. The Satisfaction of the Graduates The satisfaction of the graduates is an accumulation the graduates’ personal opinion about their jobs. There are nine aspects stu- died from this standpoint: income, working atmosphere, relationship with supervisors, re- lationship with co-workers, intention to get another job, working satisfaction, working fa- cilities, working environment, and health in- surance. The data of the satisfaction level of the graduates related to their jobs were col- lected by distributing questionnaires to the graduates of vocational high schools. Figure 5. Satisfaction level of the graduates Figure 5 shows the satisfaction level of the graduates of the both programs. The data show that 67% of the graduates of four-year program vocational high schools express ‘very good’ level of satisfaction toward their jobs. There are 22% of them who express ‘good’ satisfaction and 11% of them are in ‘low’ cate- gory. The level of satisfaction of the four-year program vocational high school graduates is higher than the level of satisfaction of the three-year program vocational high school graduates. There are 56% of the three-year program vocational high school graduates who express ‘very good’ level of satisfaction toward their jobs. As many as 33% of them are in ‘good’ category and 11% of them are in REiD (Research and Evaluation in Education) An evaluation of vocational high schools in Indonesia... - 112 Soenarto, †Muhammad Mustaghfirin Amin, & Kumaidi ‘low’ category. The most staggering difference is found in the aspect of income; the data show that the monthly income of the grad- uates of four-year program vocational high schools is between Rp 1,100,000.00 and Rp 5,000,000.00, while the income of the grad- uates of three-year program vocational high schools is between Rp 1,000,000.00 and Rp 2,500,000.00. The income of the graduates of four-year program vocational high schools is aligned with the level of competence and per- formance. Conclusion and Suggestions Conclusion In conclusion, the result of the research can be concluded as follows: (1) from the standpoint of school, the competence of the graduates of four-year program vocational high schools is superior in seven aspects, in- cluding: teamwork, discipline, tenacity, theo- retical knowledge, confidence, creativity, and leadership; (2) from the standpoint of the em- ployers of the graduates (DUDI), the grad- uates of four-year program vocational high schools are superior in eight aspects: disci- pline, tenacity, theoretical knowledge, prac- tical skill, confidence, carefulness, creativity, and leadership; (3) from the standpoint of personal satisfaction of the jobs obtained, the graduates of four-year program vocational high schools expressed a higher level of satis- faction, specifically in terms of incomes; (4) the competence superiority of the graduates of four-year program vocational high schools resulted from a longer internship program (PKL) which provided students with richer skills as well as experience related to problem- solving activities in real daily work. Suggestions Based on the conclusion, some sug- gestions are proposed as follows: (1) reconsid- er the role of Internship program in the stu- dents learning process. For optimized results, there should be more systematic, effective and efficient development, execution, and evalu- ation in the internship program of the four- year program vocational high schools; (2) for the internship program to be more efficient and effective, there should be unified evalu- ative efforts among all involved parties in schools and in business sector and industrial sector; (3) the result of the research shows that the graduates of the four-year program vocational high school are more superior than the graduates of the three-year program vo- cational high school in terms of working com- petence. Hence, there should be better appre- ciation for the graduates of the four-year pro- gram vocational high schools; (4) even though the graduates of the four-year program voca- tional high schools have sufficient skills to complete tasks presented individually, there should be more team-work-focused learning process for them. References Alisjahbana, A. S. (2014). Arah kebijakan dan program di bidang kependudukan, ketenagakerjaan, dan sumber daya manusia menghadapi globalisasi khususnya masyarakat ekonomi ASEAN. In Seminar Nasional Tantangan Kependudukan, Ketenagakerjaan, dan SDM Indonesia Menghadapi Globalisasi Khususnya Masyarakat Ekonomi ASEAN. Jakarta: Ikatan Praktisi dan Ahli Demografi Indonesia (IPADI). Dewey, J. (1916). Democracy and education: An introduction to the philosophy of education. New York, NY: Dover Publication. Kaufman, R., & Susan, T. (1982). Evaluation without fear. London: New View Points. Komariah, K. (2010). Memimpikan SMK di masa depan. In Seminar Nasional Prospek Pengembangan Pendidikan Vokasional dalam Era Globalisasi (pp. 127–132). Bandung: Culinary Education Study Program, FPTK, Universitas Pendidikan Indonesia. Law No. 20 of 2003 of Republic of Indonesia on National Education System (2003). Pardjono, P., Sugiyono, S., & Budiyono, A. (2015). Developing a model of com- petency certification test for vocational high school students. REiD (Research and Evaluation in Education), 1(2), 129– 145. REiD (Research and Evaluation in Education) 113 − REiD (Research and Evaluation in Education), 3(2), 2017 https://doi.org/http://dx.doi.org/10.2 1831/reid.v1i2.6517 Ramayani, C., Aimon, H., & Anis, A. (2012). Analisis produktivitas tenaga kerja dan pertumbuhan ekonomi Indonesia. Jurnal Kajian Ekonomi, 1(1). Retrieved from http://ejournal.unp.ac.id/index.php/ek onomi/article/view/738 Regulation of the Minister of Education and Culture No. 20 of 2016 on the competence standard of primary and secondary education graduates (2016). Republic of Indonesia. Stufflebeam, D. L., Madaus, G. F., & Kellaghan, T. (2000). Evaluation models: Viewpoints on educational and human services evaluation (2nd ed.). Boston, MA: Kluwer Academic Publishers. Stufflebeam, D. L., & Shinkfield, A. J. (1984). Systematic evaluation: A self-instructional guide to theory and practice. Dordrecht: Springer Netherlands. Thompson, J. F. (1973). Foundations of vocational education: Social and philosophical concepts. Englewood Cliffs, NJ: Prentice- Hall.