This is an open access article under the CC-BY-SA license. REiD (Research and Evaluation in Education), 6(2), 2020, 98-108 Available online at: http://journal.uny.ac.id/index.php/reid Factor analysis: Competency framework for measuring student achievements of architectural engineering education in Indonesia *1 Rihab Wit Daryono; 2 V. Lilik Hariyanto; 2 Husaini Usman; 2 Sutarto 1 Graduate School, Universitas Negeri Yogyakarta Jl. Colombo No. 1, Karangmalang, Depok, Sleman, Yogyakarta 55281, Indonesia 2 Faculty of Engineering, Universitas Negeri Yogyakarta Jl. Colombo No. 1, Karangmalang, Depok, Sleman, Yogyakarta 55281, Indonesia *Corresponding Author. E-mail: rihab0561pasca.2019@student.uny.ac.id Submitted: 25 June 2020 | Revised: 1 November 2020 | Accepted: 16 November 2020 Abstract This study aims to prove the validity test and estimate the instrument's reliability to measure student achievement in the department of architectural engineering education in Indonesia. The cluster random sampling technique was used to determine the number of students consisting of 103 vocational education students. This study uses a survey method to examine and analyze the structure of students' competency achievements factors. The collected empirical data were analyzed using descriptive and inferential statistics using Exploratory Factor Analysis (EFA). EFA test is intended to reveal factors that can be formed from instruments that have been established for measurement of achievements of vocational student compe- tencies. The analysis results using EFA showed that the instrument had good construct validity. The result of this research shows that the instrument test for measuring the achievement of student competencies has good reliability and consists of 30 competency items covering ten competency aspects, namely general competencies, technical drawing, statically structures, basic building construction, land measurement engi- neering, software application and building interior design, road and bridge construction, estimated con- struction costs, building construction and utility, creative and entrepreneurship product competencies. Keywords: achievement competency, vocational education, architectural engineering education, exploratory factor analysis How to cite: Daryono, R., Hariyanto, V., Usman, H., & Sutarto, S. (2020). Factor analysis: Competency framework for measuring student achievements of architectural engineering education in Indonesia. REiD (Research and Evaluation in Education), 6(2), 98-108. doi:https://doi.org/10.21831/reid.v6i2.32743. Introduction The quality of human resources (HR) is the key to the competitiveness of a nation to determine who can develop in global compe- tition and maintain survival. Innovative, crea- tive, technology literate, and having multiple intelligences are the hallmarks of superior hu- man resources (Slamet, 2013). Manpower is- sue is always related to human resources, thus the quality of human resources needs to be improved and developed to obtain a compe- tent workforce with high morale will main- tain the industry and strengthen the country's economy (Widodo & Pardjono, 2013). The 2019 Global Competitiveness Index database on the World Economic Forum (WEF) states that one of Indonesia's shortcomings is the 65 th skill pillar of 141 countries. The pillar in- cludes an index of the current and future components of the workforce skills that is still low to make Indonesia's competitiveness de- cline (Forum Económico Mundial, 2019). https://doi.org/10.21831/reid.v6i2.32743 https://doi.org/10.21831/reid.v6i2.32743 Rihab Wit Daryono, V. Lilik Hariyanto, Husaini Usman, & Sutarto Copyright © 2020, REiD (Research and Evaluation in Education), 6(2), 2020 - 99 ISSN: 2460-6995 (Online) Furthermore, based on the INSEAD data (2019) “The Global Talent Competi- tiveness Index”, Indonesia was ranked 67 th in 125 world countries. This also shows that the provision of human resources to improve the competitiveness of educational skills is still weak (Lanvin, & Monteiro, 2019). The data supported in Human Development Report 2019 show that the Indonesian HR quality in- dex ranks 111 th out of 189 countries in the world. The quality of human resources is im- portant, especially to obtain follow-up so that Indonesian human resources are able to com- pete in facing the era of globalization, tech- nological development, and other global chal- lenges (United Nations Development Pro- gramme, 2019). Thus, such condition that In- donesia does not have yet is quality human resources. The increasing number of the construc- tion sector makes Indonesia the construction market in the ASEAN country (Daryono et al., 2020). Indonesia has the largest construc- tion market compared to neigh boring coun- tries. The need for a professional workforce increase will affected by the rise in the con- struction market until 2025 (Kesai et al., 2018). Besides, the world of work continues to change creating new challenges for employ- ers and employees (Suarta et al., 2018). The progress of industry resulted in the advance- ment of the employment sector and the num- ber of workers in the construction sector. In- donesian employment data explains that the construction sector is in the top four with 18.98% in August 2019. Employment of the population of each business sector shows the ability in the construction sector in the labour absorption rate (Central Bureau of Statistics, 2020). Preparing vocational high school gradu- ates, according to the industrial qualifications and technological developments, is one of the goals for vocational education. Therefore, stu- dents must be equipped with competencies in line with the needs of the business and indus- trial world (Diwangkoro & Soenarto, 2020; Fakhri & Munadi, 2019). Educational devel- opment based on community needs will pro- duce competent graduates (Muaini, Zamroni, & Dwiningrum, 2019). Thus, quality vocation- al education is able to adjust economic de- velopment and the progress of science and technology. Essentially, in order to strengthen the global economy, there must be successful vocational education as support. Vocational high schools institutions in Indonesia are called Sekolah Menengah Kejuruan. Vocational high schools (VHS) are organized to prepare students for work after completing vocational education. Zhang (2009) explains that one of the basic goals for vocational edu- cation can be successful, namely by increasing students' skills. If after graduating students can work immediately, then the problems of unemployment in Indonesia will decrease. The research of Joo (2018) states that there are four premise factors that contribute to increasing the employment rate of voca- tional education graduates, including profes- sional teachers, industry-equivalent curricul- um, leadership spirit, and also link and match. Vocational education realizes the aim to strengthen education to become professional and improve economic and social develop- ment so that the increase in employment rates of vocational education graduates does not run optimally (Xiao, 2009). The first factor as a cause of absorption of Building Engineering graduates is the low teaching and learning process. The teaching and learning process of vocational schools that have not maximized the learning of soft skills and hard skills to- gether, the school emphasizes the learning of hard skills only for competencies that are re- quired to work. The learning process as ex- plained by Sutarto and Jaedun (2018) “em- phasizes authentic-learning and assessment that promote higher-order thinking skills: cre- ative, innovative, and problem-solving in real life”, to support the competencies of gradu- ates who are ready to work in the construc- tion industry. A nation can develop the new world, which is necessary to prepare superior and quality human resources with multiple and broad skills, as well as multilingual literacy to be able to develop sustainably (Widarto et al., 2012; Triyono et al., 2018). The second factor is the graduate com- petency standards applied in learning process. Some competencies are not applied due to time constraints in the implementation of the https://doi.org/10.21831/reid.v6i2.32743 Rihab Wit Daryono, V. Lilik Hariyanto, Husaini Usman, & Sutarto 100 - Copyright © 2020, REiD (Research and Evaluation in Education), 6(2), 2020 ISSN: 2460-6995 (Online) learning process. Research by Manap, Hassan, and Syahrom (2017) concludes that the con- straints include the lack of the equipment in VHS, teaching strategies to increase students' readiness to work in the industry, competent and unqualified workers carried out to the maximum. The competency framework is a tool that determines the competencies that are needed by individuals to reduce current chal- lenges and enforce sustainable development (Lai, Hamisu, & Salleh, 2019). This research was conducted due to the problem of compe- titiveness of vocational students. This situa- tion is influenced by the irrelevance of school competencies with the current state of devel- opment of the construction industry technol- ogy. The main solution is growing and devel- oping student competitiveness by increasing student competency through vocational edu- cation (Rahdiyanta, Nurhadiyanto, & Munadi, 2019), because this condition causes the low competitiveness of competencies in entrepre- neurial and technical skills in both national and international markets (Sukardi, Wildan, & Fahrurrozi, 2019; Ismail & Hassan, 2019). Thus, it is crucial to conduct research to determine the framework of vocational edu- cation competencies in developing techniques in accordance with industrial competencies. This is done to obtain ideal standard of work competency information that must be master- ed by vocational students so that they become competent workers in the field of construc- tion services and whether the competencies provided in vocational schools are in accor- dance with the competencies that are needed by the construction service industry. Method Participant Characteristics This research is based on a descriptive survey study, drawing conclusions from test- ing hypotheses to get answers to the problems studied (Creswell, 2012; Ingleby, 2012). This research was conducted at nine VHS in Cen- tral Java and Yogyakarta, Indonesia. Because the number of VHS in Central Java and Yog- yakarta are stratified, the sample was collected using cluster sampling and stratified random sampling (Creswell, 2014). Determination of the number of respondents of vocational edu- cation students in the department of architec- tural engineering was done by cluster random sampling technique, resulting in 103 students. The samples in the range of 100 are accept- able (MacCallum et al., 1999; Sugiyono, 2019). Measures and Covariates The questionnaire was prepared based on the Regulation of the Minister of Educa- tion and Culture No. 34 of 2018 concerning the national standards for vocational second- ary education, SKKNI No. 374 of 2013 for implementing buildings and public facilities, SKKNI No. 85 and 205 of 2015, and No. 340 of 2013. The questionnaire consisted of ten aspects of competence containing five com- petency indicator for each. The questionnaire consisted of ten questions measured on a four-point Likert scale at 4 'Very Good', 3 'Good', 2 'Not Good', and 1 'Very Not Good'. Competency aspects to measure the achieve- ment of student competencies in the depart- ment of architectural engineering are shown in Table 1. Table 1. The Instrument for Measuring the Achievement of Vocational Student Competencies No Competency Aspects Item 1 General competencies A 1-5 2 Technical drawing competencies B 1-5 3 Statically structures competencies C 1-5 4 Basic building construction competencies D 1-5 5 Land measurement engineering competencies E 1-5 6 Software application and building interior design competencies F 1-5 7 Road and bridge construction competencies G 1-5 8 Estimated construction costs competencies H 1-5 9 Building construction and utility competencies I 1-5 10 Creative and entrepreneurship product competencies J 1-5 Number of Item 50 https://doi.org/10.21831/reid.v6i2.32743 Rihab Wit Daryono, V. Lilik Hariyanto, Husaini Usman, & Sutarto Copyright © 2020, REiD (Research and Evaluation in Education), 6(2), 2020 - 101 ISSN: 2460-6995 (Online) Research Method Testing of construct validity was carried out using factor analysis with the employment of Explanatory Factor Analysis (EFA) meth- od (Barbara & Linda, 2010). Factor analysis using descriptive statistics is intended to en- sure as well as simplify any coherent variables and variables with each other in one factor (Hidayat et al., 2018). The reliability test was obtained based on the Cronbach alpha value. Furthermore, the component factor analysis using the EFA method was intended to en- sure the validity and confirmation of the con- struction. Data Analysis The first data analysis is a descriptive statistical test that includes multicollinearity, normality, and also data reduction using SPSS 25.0. The testing of the normality of data on the construct of the measurement model was carried out based on the value of skewness and kurtosis with a recommended value rang- ing from -1.96 to +1.96 at a significance level of 0.05 for each question item (Kline, 2005; Hair et al., 2010). The multicollinearity analy- sis can conclude the inter-change matrix with a value of ≤ 0.90 (Tabri & Elliott, 2012). In addition, all items were included in the factor analysis criteria and the data analyzed using EFA in order to determine the factors meas- uring the achievement of the student compe- tencies. Exploratory Factor Analysis (EFA) data analysis was performed using the SPSS in or- der to reveal how many factors can be formed so that it can find out the correlating factors and the contribution value of each variable in order to measure the factors (Kumar, 2012). The analysis results are based on the Kaiser- Meyer-Olkin (KMO) values, the Bartlett test values, the Measure of Sampling Adequacy (MSA), the communality values, the total vari- ance values that are described related to eigen- values, factor loading, and also the plot scree. Furthermore, for the assessment of the hypo- thesized construct position, the construct val- idity was carried out using the convergent and discriminant validity. Findings and Discussion Findings Preliminary Analysis The first result conducted in this study is a preliminary analysis intended to find out the data that has been obtained from the sur- vey results. The data were obtained from 103 vocational education students majoring in ar- chitecture engineering in Indonesia and cover- ing ten aspects consisting of five questions each. All 50 questions are called competency items. Table 2 shows the multicollinearity and normality data. Because 20 items other than those pre- sented in Table 2 get skewness and kurtosis values outside the range of -1.96 to +1.96 and a significance level with a value of ≤0.05 (Hair et al., 2010; Hidayat et al., 2018), then the 20 items are declared not normally distributed and excluded, and are not included in the next factor analysis. After the items were removed, 30 items of data that still survived or were normally distributed were analyzed again with descriptive statistics as presented in Table 2. There are 30 items that reach normality with the skewness values ranging from -1.612 until -0.641, and then the kurtosis values ranging from -0.935 to +1.857. Furthermore, the de- scriptive statistics reveal that the mean value ranges from 3.408 to 3.786. The value of devi- ation standard ranges from 0.412 to 0.760. The variance ranges from 0.170 to 0.577. The acquisition of total values ranged from 351.0 to 390.0 from the maximum value of 412.0. In the case of multicollinearity, the re- lationship between ten competency items ana- lyzed in the construction value ranges from 0.306 to 0.632. Sequentially, pearson correla- tions and sig. (2-tailed) on each variable A to J are as follows: (0.327; 0.001), (0.413; 0.000), (0.607; 0.000), (0.632; 0.000) (0.616; 0.000), (0.306; 0.002), (0.620; 0.000), (0.554; 0.000), (0.483; 0.000). These results indicate that the discriminant validity of each competency vari- able achieved because the inter-correlation matrix value is ≤0.90 (Kline, 2005; Hidayat et al., 2018) and the correlation is significant at the 0.01 level (2-tailed). https://doi.org/10.21831/reid.v6i2.32743 Rihab Wit Daryono, V. Lilik Hariyanto, Husaini Usman, & Sutarto 102 - Copyright © 2020, REiD (Research and Evaluation in Education), 6(2), 2020 ISSN: 2460-6995 (Online) Table 2. Results of Statistical Descriptive Analysis and Data Normality (N= 103) No Variable Mean Sd Var. Skew Kurtosis ∑ 1 A1 3.631 0.56 0.314 -1.221 0.543 374 2 A2 3.738 0.44 0.195 -1.098 -0.811 385 3 A3 3.476 0.65 0.428 -1.086 1.009 358 4 B1 3.408 0.76 0.577 -1.254 1.283 351 5 B2 3.447 0.73 0.544 -1.384 1.857 355 6 B3 3.534 0.62 0.389 -0.998 -0.029 364 7 C1 3.602 0.56 0.320 -1.074 0.185 371 8 C2 3.524 0.60 0.370 -1.163 1.695 363 9 C3 3.524 0.65 0.428 -1.268 1.372 363 10 D1 3.553 0.59 0.348 -0.943 -0.082 366 11 D2 3.621 0.54 0.296 -1.057 0.109 373 12 D3 3.505 0.62 0.390 -1.128 1.427 361 13 E1 3.563 0.53 0.288 -0.645 -0.797 367 14 E2 3.670 0.49 0.243 -0.982 -0.438 378 15 E3 3.689 0.50 0.255 -1.287 0.620 380 16 F1 3.563 0.53 0.288 -0.645 -0.797 367 17 F2 3.660 0.56 0.266 -1.118 0.151 377 18 F3 3.660 0.51 0.266 -1.118 0.151 377 19 G1 3.621 0.56 0.316 -1.170 0.415 373 20 G2 3.602 0.56 0.320 -1.074 0.185 371 21 G3 3.621 0.52 0.277 -0.916 -0.322 373 22 H1 3.660 0.53 0.285 -1.265 0.644 377 23 H2 3.689 0.50 0.255 -1.287 0.620 380 24 H2 3.563 0.66 0.445 -1.450 1.657 367 25 I1 3.786 0.41 0.170 -1.418 0.012 390 26 I2 3.767 0.44 0.200 -1.612 1.481 388 27 I3 3.728 0.44 0.200 -1.041 -0.935 384 28 J1 3.456 0.63 0.407 -0.987 1.012 356 29 J2 3.612 0.54 0.299 -1.009 -0.002 372 30 J3 3.476 0.63 0.409 -1.056 1.118 358 Table 3. Reliability Analysis of Competency Items No. Competency Aspects Item CA Overall CA 1 General competencies 3 0.7 0.9 2 Technical drawing competencies 3 0.7 3 Statically structures competencies 3 0.7 4 Basic building construction competencies 3 0.7 5 Land measurement engineering competencies 3 0.7 6 Software application and building interior design competencies 3 0.8 7 Road and bridge construction competencies 3 0.9 8 Estimated construction costs competencies 3 0.8 9 Building construction and utility competencies 3 0.8 10 Creative and entrepreneurship product competencies 3 0.7 Reliability of Instrument Reliability is the stability and suitability of each score found. It is said to be reliable if the question items get the same and identical scores when the instrument is tested to sev- eral clans and in different times and places (Hidayat et al., 2018). The reliability value of an item is based on the Cronbach Alpha val- ue. According to Lin (2002), if the Cronbach Alpha value is ≥ 0.7, then the item on the instrument is reliable, and vice versa, if the Cronbach Alpha value is < 0.7, then it is not reliable. The results of the instrument's relia- bility are shown in Table 3. The Cronbach Alpha results in this re- search instrument are in the reliable category. Overall, the Cronbach Alpha value obtained was ≥ 0.7, this value is included in the recom- mended value by Lin (2002). In general com- petencies, technical drawing, statically struc- tures, basic building construction, land meas- urement engineering, as well as creative and https://doi.org/10.21831/reid.v6i2.32743 Rihab Wit Daryono, V. Lilik Hariyanto, Husaini Usman, & Sutarto Copyright © 2020, REiD (Research and Evaluation in Education), 6(2), 2020 - 103 ISSN: 2460-6995 (Online) entrepreneurship product competencies get a value of α = 0.7, while software application and building interior design, estimated con- struction costs, building construction and util- ity competencies get a value of α = 0.8, and road and bridge construction get a value of α = 0.9 (≥0.7; Lin, 2002). Therefore, the instru- ment to measure the achievement of student competencies has a good level of reliability. Exploratory Factor Analysis (EFA) In this EFA test, the study considers testing items that have passed the multicol- linearity and normality and reliability tests for each item. All 30 items that passed were in- cluded in ten aspects of competency. EFA test criteria are based on the KMO Index values, Bartlett's Test, Measure of Sampling Adequacy (MSA), communalities, factor load- ing, eigenvalues, and plot scree. The results of the KMO Measure of Sampling Adequacy ob- tained a value of 0.886, that is more than 0.70, then the coverage of each factor is satisfac- tory. The Bartlett's Test of Sphericity Approx. Chi-Square obtained a value of 1932.501; df = 35; Sig. = 0.000. The scree plot pattern was used to reduce variance to several factors. The point at which the slope of the line begins to change is where the limit of the number of factors that can take. This point is called the inflection point. In Figure 1, after the 10th point, the line begins to change in tilt and the variations explained are less and less. Thus, it can reduce 30 items to ten factors. The next step of identifying the extrac- tion of community values, eigenvalues, per- centage variants, and loading factors is shown in Table 4. The value of communalities indi- cates the value of the variable under study whether it can explain the magnitude of the effective contribution (%) of each variant to the factor formed for each item. The results of the communalities in this instrument range from 0.751 to 0.909 (≥0.50) can be catego- rized as adequate variants in the instrument. MSA values range from 0.709 to 0.961 (≥ 0.70). The rotated component matrix shows the loading factor on each factor. The results of data analysis, it is recommended for all items to measure of competency achievement. This value is obtained from high loading fac- tors ranging from 0.461 to 0.899 (>0.40). In addition, Table 5 shows a summary of the re- sults of the EFA value of the competency framework to determine the competency at- tainment of architectural engineering students in Indonesia. Figure 1. Scree Plot of Achieving Student Competency Framework https://doi.org/10.21831/reid.v6i2.32743 Rihab Wit Daryono, V. Lilik Hariyanto, Husaini Usman, & Sutarto 104 - Copyright © 2020, REiD (Research and Evaluation in Education), 6(2), 2020 ISSN: 2460-6995 (Online) Table 4. MSA, Communalities, Factor Loading Var. Item MSA Comm. Factor Loading Components 1 2 3 4 5 6 7 8 9 10 A A1 0.741a 0.751 0.828 A2 0.709a 0.770 0.760 A3 0.795a 0.717 0.772 B B1 0.821a 0.888 0.899 B2 0.861a 0.820 0.739 B3 0.813a 0.870 0.837 C C1 0.882a 0.784 0.501 C2 0.926a 0.782 0.558 C3 0.875a 0.758 0.514 D D1 0.939a 0.767 0.705 D2 0.878a 0.909 0.794 D3 0.873a 0.880 0.850 E E1 0.887a 0.719 0.717 E2 0.846a 0.794 0.710 E3 0.872a 0.873 0.677 F F1 0.931a 0.764 0.503 F2 0.930a 0.779 0.461 F3 0.889a 0.762 0.600 G G1 0.925a 0.879 0.760 G2 0.865a 0.884 0.718 G3 0.865a 0.861 0.709 H H1 0.870a 0.829 0.590 H2 0.912a 0.864 0.649 H2 0.927a 0.803 0.711 I I1 0.961a 0.809 0.706 I2 0.948a 0.797 0.658 I3 0.883a 0.797 0.761 J J1 0.861a 0.885 0.785 J2 0.894a 0.804 0.667 J3 0.854a 0.816 0.775 Table 5. Measurement Indicators in the Exploratory Factor Analysis Test Index Value Results Recommendation Decision Index KMO 0.886 0.50 < x ≤ 0.8 Fit 0.80 < x ≤ 1.0 Fit Bartlett's Test p <0.000 p <0.05 Fit MSA 0.709 – 0.961 >0.07 Fit Factor Loading 0.461 – 0.899 0.4 – 0.9 Fit Eigenvalues >2.228 ≥1.0 Fit Discussion After EFA test was done to show that the analyzed student data involve ten factors: general competencies, technical drawing com- petencies, statically structures, basic building construction, land measurement engineering, software application and building interior de- sign, road and bridge construction, estimated construction costs, building construction and utility competencies, and creative and entre- preneurship product competencies. This is in line with the original fact structure, although 20 competency items have fallen from 20 i- tems since the data do not meet the descrip- tive statistical requirements and normality of data so there are still 30 items of competency items. This study is in line with previous stud- ies by Hidayat et al. (2018), Hazriyanto and Ibrahim (2019), Lai et al. (2019), and Nashir, Mustapha, Ma’arof, and Rui (2020). https://doi.org/10.21831/reid.v6i2.32743 Rihab Wit Daryono, V. Lilik Hariyanto, Husaini Usman, & Sutarto Copyright © 2020, REiD (Research and Evaluation in Education), 6(2), 2020 - 105 ISSN: 2460-6995 (Online) Table 6. Competency Framework for Measuring Student Achievements of Architectural Engineering Education Construct Item Competency General competencies A A1 Having a habit of behaving honestly in carrying out their job duties A2 Being able to complete work according to the criteria set in the workplace A3 Having the ability to generate innovative work ideas according to their expertise Technical drawing competencies B B1 Presenting the types, functions and engineering drawing tools B2 Drawing orthogonal (2D) and pictorial (3D) projections B3 Drawing symbol, notation, and dimensional rules on engineering drawings Statically structures competencies C C1 Presenting factors influencing the building structure based on design and loading criteria C2 Presenting various styles and ways of arranging styles in building structures C3 Calculating the forces (moment, shear, normal force, and rod force) on the building structure Basic building construction competencies D D1 Implementing occupational health and safety in building works D2 Presenting the types of construction work (buildings, roads, bridges and irrigation) D3 Carrying out concrete, steel, wood, earth and stone construction work Land measurement engineering competencies E E1 Carrying out measurement principles of land measurement techniques E2 Performing maintenance techniques and checking optical types E3 Carrying out the operation of tools for levelling, theodolite, and stake out work Software application and building interior design competencies F F1 Presenting data on the needs of interior design work F2 Creating 2D and 3D construction drawings with colour schemes and artificial lighting F3 Creating interior designs with elements, materials, models and accessories in every room Road and bridge construction competencies G G1 Presenting road and bridge classification G2 Presenting road and bridge pavement material specifications G3 Drawing of the detailed construction of roads and bridges Estimated construction costs competencies H H1 Presenting materials specifications for building, road and bridge construction work H2 Calculating the estimated cost of construction work H3 Checking the results of the estimated construction costs Building construction and utility competencies I I1 Creating floor plans, cuts, and building construction drawings I2 Creating detailed building construction drawings I3 Making isometric drawings of clean water and dirty water installations, electricity installations, air conditioners, and lightning rods Creative and entrepreneurship product competencies J J1 Presenting the attitudes and behaviour of entrepreneurs J2 Creating worksheets/work drawings for making prototypes of services J3 Creating and test product/service prototypes This study aims to test the instrument for measuring the achievement of vocational education student competencies from archi- tectural engineering study programs in Indo- nesia. The results of the instrument analysis in this study have a high level of overall reliabil- ity: the Cronbach Alpha value=0.9 (≥0.7) The value is calculated in the categorization criteria (Lin, 2002), the highest reliability value in ba- sic building construction competency is the Cronbach Alpha value=0.9, and the lowest value is the competence of creative and entre- preneurial products with a Cronbach Alpha value of 0.7, therefore, the competence of cre- ative and entrepreneurial products taught to be mastered by vocational students must be increased so the achievement of entrepreneur- ial competencies owned by students is deeper. Factor analysis shows ten factors with ten items formulated consisting of 30 compe- tency items. Each item shows a satisfactory loading with a value of 0.474 to 0.987 (≥0.40). Moreover, from the EFA analysis, the KMO index=0.886; Bartlett's Test = p <0.000; MSA = 0.709-0.961; Factor Loading = 0.461-0.899; Eigenvalues >2.228, these values meet the minimum criteria recommended (Hidayat et al., 2018; Hazriyanto & Ibrahim, 2019). The results of the analysis of convergent validity and discriminant validity have met the multi- https://doi.org/10.21831/reid.v6i2.32743 Rihab Wit Daryono, V. Lilik Hariyanto, Husaini Usman, & Sutarto 106 - Copyright © 2020, REiD (Research and Evaluation in Education), 6(2), 2020 ISSN: 2460-6995 (Online) variate analysis requirements. Thus, the devel- opment instrument is suitable for measuring the level of mastery of vocational education student competencies in the building depart- ment consisting of architectural engineering study programs which mainly involve these ten factors. Further studies are expected to develop a competency framework model and be able to apply and evaluate applications in continuous learning. The list of competencies to measure student achievement in architec- tural engineering education in Indonesia is shown in Table 6. Conclusion This study confirms the validity and re- liability of the measurement instrument for the achievement of the competency level of students of architectural engineering voca- tional schools in Indonesia. This research shows that the instrument test for measuring the achievement of student competencies has good reliability. The EFA analysis shows that the instrument has good construct validity consisting of 30 competency items covering ten competency aspects, namely general com- petencies, technical drawing, statically struc- tures, basic building construction, land meas- urement engineering, software application and building interior design, road and bridge con- struction, estimated construction costs, build- ing construction and utility, creative and en- trepreneurship product competencies. References Barbara, G. T., & Linda, S. F. (2010). Using multivariate statistics (6 th ed.). Pearson. Central Bureau of Statistics of Indonesia. (2020) Berita resmi statistik: Keadaan ketenagakerjaan Indonesia Agustus 2020. Central Bureau of Statistics. Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches. SAGE Publications. Creswell, J. W. (2012). Planning, conducting, and evaluating quantitative and qualitative research (Vol. 66). Pearson. Daryono, R. W., Yolando, A. P., Jaedun, A., & Hidayat, N. (2020). Competency of vocational schools required by construction industry in consultants’ supervisor. Journal of Physics: Conference Series, 1456(1), 1–10. https://doi.org/ 10.1088/1742-6596/1456/1/012057 Diwangkoro, E., & Soenarto, S. (2020). Development of teaching factory learning models in vocational schools. Journal of Physics: Conference Series, 1456(1), 1–5. https://doi.org/10.1088/1742-6596 /1456/1/012046 Fakhri, A. A., & Munadi, S. (2019). The evaluation of industrial internship for vocational school of mechanical engineering in Tegal. American Journal of Educational Research, 7(11), 806–809. htt ps://doi.org/10.12691/education-7-11-8 Forum Económico Mundial. (2019). Índice Global de Competitividad 2019. World Economic Forum. Retrieved from http: //www3.weforum.org/docs/WEF_The GlobalCompetitivenessReport2019.pdf Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis (7th ed.). Prentice Hall. Hazriyanto, & Ibrahim, B. (2019). The factor analysis of organizational commitment, job satisfaction and performance among lecturers in Batam. Journal of Technical Education and Training, 11(1), 151–158. https://doi.org/10.30880/jtet.2019.11.0 1.19 Hidayat, R., Zamri, S. N. A. S., & Zulnaidi, H. (2018). Exploratory and confirmatory factor analysis of achievement goals for Indonesian students in mathematics education programmes. Eurasia Journal of Mathematics, Science and Technology Education, 14(12), 1–12. https://doi.org/ 10.29333/ejmste/99173 Ingleby, E. (2012). Research methods in education. Professional Development in Education, 38(3), 507–509. https://doi. org/10.1080/19415257.2011.643130 Ismail, A. A., & Hassan, R. (2019). Technical competencies in digital technology towards industrial revolution 4.0. Journal https://doi.org/10.21831/reid.v6i2.32743 Rihab Wit Daryono, V. Lilik Hariyanto, Husaini Usman, & Sutarto Copyright © 2020, REiD (Research and Evaluation in Education), 6(2), 2020 - 107 ISSN: 2460-6995 (Online) of Technical Education and Training, 11(3), 55–62. https://doi.org/10.30880/jtet.20 19.11.03.008 Joo, L. (2018). Vol. 1: The excellence of Technical Vocational Education and Training (TVET) institutions in Korea: Yeungjin College case study. International Education Studies, 11(7), 136. https://doi. org/10.5539/ies.v11n7p136 Kesai, P., Soegiarso, R., Hardjomuljadi, S., Setiawan, M. I., Abdullah, D., & Napitupulu, D. (2018). Indonesia position in the globalization of construction industry. Journal of Physics: Conference Series, 1114(1), 012133. https: //doi.org/10.1088/1742-6596/1114/1/ 012133 Kline, R. B. (2005). Principles and practice of structural equation modeling. The Guilford Press. Kumar, K. (2012). A beginner’s guide to structural equation modeling (3rd ed.). In Journal of the Royal Statistical Society: Series A (Statistics in Society), 175(3), 828- 829. https://doi.org/10.1111/j.1467-985 x.2012.01045_12.x Lai, C. S., Hamisu, M. A., & Salleh, K. M. (2019). Development of competency framework for Nigerian TVET teachers in tertiary TVET institutions. Journal of Technical Education and Training, 11(1), 11– 18. https://doi.org/10.30880/jtet.2019. 11.01.002 Lanvin, B., & Monteiro, F. (Eds.) (2019). The Global Talent Competitiveness Index 2019: Entrepreneurial talent and global competitiveness. INSEAD, the Adecco Group, and Tata Communications. Lin, Y. K. (2002). Using minimal cuts to evaluate the system reliability of a Stochastic-Flow network with failures at nodes and arcs. Reliability Engineering and System Safety, 75(1), 41–46. https://doi. org/10.1016/S0951-8320(01)00110-7 MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999). Sample size in factor analysis. Psychological Methods, 4(1), 84–99. https://doi.org/10.1037/1082-989X.4.1. 84 Manap, N., Hassan, N. S., & Syahrom, N. S. (2017). Preparation of vocational college graduates as skilled workforce in the local construction industry. Journal of Technical Education and Training, 9(2), 69– 80. Muaini, M., Zamroni, Z., & Dwiningrum, S. I. A. (2019). The concept of vocational high school development industry-based in Central Lombok Regency. Journal of Physics: Conference Series, 1179(1), 1–5. https://doi.org/10.1088/1742-6596/11 79/1/012054 Nashir, I. M., Mustapha, R., Ma’arof, N. N. M. I., & Rui, T. J. (2020). Modified Delphi technique: The development of measurement model for innovative instructional leadership in technical and vocational education systems. Journal of Technical Education and Training, 12(1 Special Issue), 24–37. https://doi.org/ 10.30880/jtet.2020.12.01.003 Rahdiyanta, D., Nurhadiyanto, D., & Munadi, S. (2019). The effects of situational factors in the implementation of work- based learning model on vocational education in Indonesia. International Journal of Instruction, 12(3), 307–324. https ://doi.org/10.29333/iji.2019.12319a Regulation of the Minister of Education and Culture No. 34 of 2018 on the National Standards for Vocational Secondary Education/Vocational Madrasah Aliyah. , (2018). Slamet, P. (2013). Pengembangan SMK model untuk masa depan. Jurnal Cakrawala Pendidikan, 5(1), 14–26. https://doi.org/ 10.21831/cp.v5i1.1256 Suarta, I. M., Suwintana, I. K., Sudana, I. G. P. F. P., & Hariyanti, N. K. D. (2018). Employability skills for entry level workers: A content analysis of job advertisements in Indonesia. Journal of Technical Education and Training, 10(2), 49- 61. https://doi.org/10.30880/jtet.2018. 10.02.005 Sugiyono. (2019). Metode penelitian kuantitatif, kualitatif, dan R&D. Alfabeta. https://doi.org/10.21831/reid.v6i2.32743 Rihab Wit Daryono, V. Lilik Hariyanto, Husaini Usman, & Sutarto 108 - Copyright © 2020, REiD (Research and Evaluation in Education), 6(2), 2020 ISSN: 2460-6995 (Online) Sukardi, S., Wildan, W., & Fahrurrozi, M. (2019). Vocational education: A missing link for the competitive graduates? International Education Studies, 12(11), 26- 35. https://doi.org/10.5539/ies.v12n11 p26 Sutarto, H. P., & Jaedun, M. P. D. (2018). Authentic assessment competence of building construction teachers in Indonesian vocational schools. Journal of Technical Education and Training, 10(1), 91– 108. https://doi.org/10.30880/jtet.2018. 10.01.008 Tabri, N., & Elliott, C. M. (2012). Principles and practice of structural equation modeling. Canadian Graduate Journal of Sociology and Criminology, 1(1). https://doi. org/10.15353/cgjsc-rcessc.v1i1.25 Triyono, M. B., Trianingsih, L., & Nurhadi, D. (2018). Students’ employability skills for construction drawing engineering in Indonesia. World Transactions on Engineering and Technology Education, 16(1), 29–35. United Nations Development Programme. (2019). Human development report 2019: Inequalities in human development in the 21st century Jordan (pp. 1–10). United Nations Development Programme. Retrieved from http://hdr.undp.org/en/data Widodo, W. N., & Pardjono, P. (2013). Pengembangan model pembelajaran soft skills dan hard skills untuk siswa SMK. Jurnal Cakrawala Pendidikan, XXXI(3), 409–423. https://doi.org/10.21831/cp. v0i3.1139 Xiao, X. (2009). Construction and practice of the new business dpecialty talent cultivation mode. International Education Studies, 2(4), 56–60. https://doi.org/ 10.5539/ies.v2n4p56 Zhang, W. (2009). Issues of practical teaching in vocational-technical schools in China and their countermeasures. International Education Studies, 2(4), 75–78. https:// doi.org/10.5539/ies.v2n4p75