144 https://doi.org/10.21009/JISAE JISAE (Journal of Indonesian Student Assessment and Evaluation) ISSN : P-ISSN: 2442-4919│E-ISSN: 2597-8934 Vol 8 No 2 (2022) Website : http://journal.unj.ac.id/unj/index.php/jisae APPLICATION OF PRINCIPAL COMPONENT ANALYSIS METHOD ON TUTOR PERFORMANCE ASSESSMENT INSTRUMENTS WITH JEFFREYS'S AMAZING STATISTICS PROGRAM FORMULA Ahmad Fakultas Ilmu Pendidikan Universitas Negeri Malang, Indonesia ABSTRACT Producing tutor performance appraisal instruments is a practical step to improve the quality of human resources and the quality of non-formal education units. This instrument development research aims to produce a standard instrument for assessing tutor performance by involving 250 tutors as respondents when conducting an empirical test by filling in 33 questions representing 11 variables. Data on instrument filling was then analyzed using the formula of Jeffreys's Amazing Statistics Program (JASP) version 16.3. After the instrument met the requirements of factor analysis, an analysis was carried out using the PCA method the results in general the tutor performance assessment instrument had formed eleven main components with various sub-variables of each component such as seven sub-variables, three sub-variables, two sub-variables and one sub variable with matrix values. high correlation, so that the tutor's performance assessment instrument is suitable to be used to assess the tutor's performance. Keywords: Assessment, Tutor Performance, Non-Formal Education, PCA Address for Correspondence: Jl. Semarang 5 Malang, Kota Malang, Indonesia Email: ahmad.fip@um.ac.id INTRODUCTION Non-formal unit educators hereinafter referred to as tutors, not only carry out the function of transferring knowledge to students but also function to instill values and build student character on an ongoing basis. A tutor is a professional position that has the main task in the learning process (Undang-Undang Republik Indonesia Nomor 14 Tahun 2005 Tentang Guru Dan Dosen, 2005) Therefore, as a profession, a tutor must meet the minimum requirements including (1) Recruited from superior seeds from various potentials, (2) Educated well so that they master all competencies, not only have professional competence, but also master other competencies (pedagogic, social, and personality), (3) Having good motivation, educating is a call to the soul, and in its implementation using the 'heart', not only fulfilling the number of teaching hours and not only delivering standardized material, (4 ) Equipped with good and adequate facilities and infrastructure (Ahmad; Faisal Madani; M. Ishaq; Lasi Purwito; Ratih Permata Sari., 2022) (Siswantari, 2011) To find out, if the tutor has worked well, then an assessment is carried out periodically with a good assessment tool. One of the indicators of the assessment is that it can be seen from the suitability of the process with what is planned, the suitability of achieving goals through good management, effective and efficient use and utilization of resources, as well as the ability to guarantee the suitability of the process and the achievement of goals. Tutor performance assessment is a systematic step that assesses the real work of the tutor's work and behavior in realizing Pancasila-cultured students based on predetermined standards (The American Educational Research Association, the American Psychological http://journal.unj.ac.id/unj/index.php/jisae mailto:ahmad.fip@um.ac.id 145 |JISAE (Journal of Indonesian Student Assessment and Evaluation) |Volume 8 Number 2 Association, 2014) (Chirchir & Letangule, 2021)(Atieno Otieno et al., 2021). The indicators used in the development of the tutor's performance appraisal model are divided into pedagogic, professional, social, and personality competencies. Tutor performance appraisal activities are not only seen from cognitive mastery as measured by test formulations,(Chanda Sichinsambwe, Lilian Lialabi, Alexina Muyenga, 2021) but more on the assessment of affective and psychomotor aspects is also a concern, to get the full profile of the tutor. Tutors as professionals must meet standards by their expertise, skills, skills from an ethical point of view, and work ethic. On that basis, the work must be relevant to the duties of the profession. Discussions related to professional tutors are generally defined into two parts, namely: (1) A person who holds a profession. This means that professional tutors are identified as those who work according to their expertise and devote themselves to those who need expertise, and (2) In the form of tutor performance in completing work by professional assignments. Tutor performance assessment is a continuous effort made to determine what is being carried out by monitoring the results achieved and if there are deviations from the predetermined criteria, an improvement is immediately made, so that all the results achieved can be as planned. Tutors not only carry out the function of transferring knowledge but also function to instill values and build the character of students in a sustainable manner. (Darling- hammond et al., 2017) but social reality shows that there are still many tutors who have not performed optimally. One of the causes of this problem is that not all non-formal institutions fully have home-based tutors while working tutors are resources who devote themselves voluntarily and not permanently. In general, those who become tutors get additional assignments and even side jobs from their main job. This is a serious phenomenon in staffing in non-formal units. Indeed, as a tutor, they must carry out their duties in totality so that they can shape the character of Indonesian children with Pancasila culture. Tutor performance can be measured by learning planning, learning management procedures, the ability to build social relations in learning, the ability to evaluate, and the ability to complete non-academic tasks. (Direktorat Tenaga Kependidikan Ditjen PMPTK Depdiknas., 2008)(Besim Enes Bicak, Cornelia Eleonore Borchert and Kerstin Höner., 2021)(Atieno Otieno et al., 2021). With these indicators, at least the tutor in carrying out his duties as a learning facilitator who inspires students is not oriented to the function of the learning material provider, to create student creativity. This is what is expected in the context of independent learning where the absolute requirement of performance is the adaptation and high competence of a tutor. To carry out the adaptation process, a tutor must deeply understand the characteristics of students to have life skills and work skills. In addition, tutors must also be creative learners, able to teach, educate, inspire and be role models for students and society in general. Meanwhile, from the aspect of competence, tutors must be able to teach students to have numeracy literacy skills, science, information, finance, culture, and citizenship, be able to think critically, reason, be creative, be able to communicate, be able to collaborate, and have skills in problem-solving. So, tutors in the era of disruption must be the drivers of change by transforming knowledge, skills, and attitudes centered on students who are independent in learning, able to innovate and all learning processes contribute greatly to learning change for learning citizens. Therefore, it is possible to measure the performance of tutors in non-formal education units differently from the point of view of the context, input, process, output, and impact. However, these differences, it becomes their strength in developing a standardized tutor performance appraisal model. To measure the performance of tutors, a performance appraisal instrument based on pedagogic, personality, professional and social, technology and information, and literacy was developed. The dimensions of assessing the tutor's performance require an objective 146 refresher to produce an instrument that meets high validity and reliability. If the instrument meets these requirements, a needs analysis is carried out by the unit manager. Then, the head of the unit recommends the tutor to do a self-assessment and recommends other tutors to do a performance appraisal of the tutor being assessed. The results of the assessment are accumulated with the results of the observer and panelist assessments, to obtain recommendations for the results of the tutor's performance assessment so that it is useful for the self-development of tutors and non-formal education units. For the record, the team of observers and panelists is the element in charge of assessing the performance of tutors in non-formal education units, this can come from the leadership or other resources that are considered to meet the criteria that have been set. Research has been done on the professional competence profile of tutors in Java, the results of the study show that the teaching and researching competence of a tutor is still not optimal so it needs to be improved, besides the problem of tutor competence still needs to be improved in supporting tasks in daily life. (Rahmat et al., 2021). Ten years ago, a study was conducted that the competence of tutors was low, one of the causes was the low understanding of the community regarding non-formal education (Siswantari., 2011) The results of other studies show that from the aspect of tutor competence, they are categorized as quite good. (Moh Muzaqi. Dwi Sudarmanto. Widya Ayu P.Yuni Ekawati., 2008). Based on the results of the study, in general, the performance of tutors is still low, so it is necessary to carry out periodic assessments to formulate competency improvement policies. The novelty of this research compared to other previous studies is to try to develop a tutor performance assessment instrument that is formulated into various complementary components in assessing the tutor's profile, so that the quality picture according to the competency standards set can be fulfilled optimally. Therefore, it is important to conduct this research to provide solutions to improve the quality of tutors so that the demands of work ethic and work ethic are well met. To produce a tutor's performance appraisal instrument, this study uses the Principal Component Analysis (PCA) method, which is a technique for reducing various dimensions to form new variables.(Zarachoff et al., 2022)(Jansson et al., 2022) The way it works is to determine the concept of eigenvalues and eigenvectors through the calculation of variance and principal components. PCA has benefits such as being able to overcome multicollinearity, reducing variables into models, practically limited variables forming simple models, and speeding up the computational process. So by using the PCA model, it can illustrate that the tutor's performance assessment instrument can be developed and has a high standard. METHOD This research on the development of the tutor's performance appraisal instrument is intended to produce a tutor's performance appraisal instrument that is obtained based on an empirical process through several tests. The standard instrument is a measuring instrument compiled by experts, always being tested, analyzed, and revised. (Aiken, 2020)(Gronlund, 1998) The development of standardized instruments is practically carried out in the following stages: (1) Identification of the underlying philosophy as a guide for the development process, (2) Developing basic theories, (3) Planning of prototype models by conducting a meta-analysis of theories, and (4) Validating by trying and revise one or more of the assessments.(Yalow & Popham, 1983)(Popham, 2016) This study involved 250 tutors who worked in non-formal education units in East Java and involved 5 expert panelists/validators in the field of measurement. (Crocker, Linda, Algina, 1986)(Crocker & Algina, 1986a)(Tanujaya et al., 2017)(Popham, 1999) The stages of this research include (1) Observation and interviews with tutors and directly compiling the initial product, (2) Conducting construct tests and empirical tests to obtain valid and reliable final 147 |JISAE (Journal of Indonesian Student Assessment and Evaluation) |Volume 8 Number 2 instruments, and (3) Implementation of the results of assessment instruments the performance of non-formal education unit tutors. The criteria for the tutor's performance assessment instrument must meet the following requirements: (1) Can be generalized, (2) By actual conditions, (3) Measure more than one output, (4) Can be taught, (5) Objective, (6) Can be implemented realistically, and (7) Can be scored accurately. (Popham, 2009)(Popham, 2004)(Popham, 2014)(Crocker & Algina, 1986b)(Crocker & Algina, 1986a)(Crocker, Linda, Algina, 1986) To produce a standard instrument, the data from the experimental construct as well as empirically, then analyzed by factorial analysis technique using the formula Jeffreys's Amazing Statistics Program (JASP) version 16.3.(Pasaribu et al., 2021)(Zulfachri, 2021) RESULTS AND DISCUSSION Before discussing the results of the study, it is necessary to explain that the research instrument has been carried out with empirical tests involving 250 respondents to respond to 33 questionnaires divided into 11 dimensions of tutor performance assessment. The 250 people are all tutors who are and work at the Community Learning Activity Center and the Non-formal Education Unit of the Community Learning Activity Center in East Java. Factor analysis in this study uses Principal Component Analysis (PCA). The results of this study are as follows. Factor analysis using PCA is a more concise analytical technique so that it quickly reduces the results of item analysis without losing the information contained in the original/initial data. PCA steps include selection and measurement of variables, preparation of correlation matrix, extraction of factors from correlation matrix, and rotation of factors to improve interpretation and interpretation of results. The results of the chi-squared test show that the tutor's performance instrument items have formed a varied model, where the p-value <.001 means that there are variations in the instrument items presented as shown in the following table. Table 1. Chi-squared Test Value df p Model 13750.071 220 < .001 Because the data assumption is that there is a fairly strong influence between the existing variations, this result is relevant to the loading factor value of the instrument items as follows. Table. 2 Component Loadings PC1 PC2 PC3 PC4 PC5 PC6 PC7 PC8 PC9 PC10 PC11 Uniqueness F2 0.755 0.228 F3 0.707 0.309 E2 0.563 0.452 E1 0.552 0.400 E3 0.549 0.401 F1 0.539 0.460 D2 0.522 0.375 H2 0.670 0.420 I2 0.618 0.350 J2 0.539 0.376 G2 0.909 0.052 K1 0.909 0.052 K3 0.909 0.020 H1 0.909 0.020 K2 0.823 0.068 G3 0.823 0.068 A3 0.757 0.290 B2 0.655 0.340 A2 0.578 0.469 148 Table. 2 Component Loadings PC1 PC2 PC3 PC4 PC5 PC6 PC7 PC8 PC9 PC10 PC11 Uniqueness J3 0.810 0.249 J1 0.744 0.314 C2 0.679 0.276 C3 0.626 0.383 A1 0.547 0.323 C1 0.815 0.233 B3 0.783 0.240 D3 0.834 0.197 G1 0.798 0.259 B1 0.512 0.369 D1 0.378 H3 0.352 I1 0.394 I3 0.451 Note. The applied rotation method is varimax. The concentration of factors or commonly referred to as factor loading on the developed tutor performance assessment instrument has fulfilled 11 main components with an average value of factor loading between cells/factors above 0.5 as the lower limit/standard criteria. This shows that the main components of the item factors of the instrument used are feasible to be used as standard instruments in this study. This can be explained by the specific characteristics of the factor with the high characteristic eigenvalue of a vector/cell. The lambda value or eigenvalue of the three components is positive, meaning that there is an influence of an item on the formation of the matrix from factor analysis. Descriptively presented as follows. Table 3. Component Characteristics Unrotated solution Rotated solution Eigenvalue Proportion var. Cumulative SumSq. Loadings Proportion var. Cumulative Component 1 10.066 0.305 0.305 3.693 0.112 0.112 Component 2 2.576 0.078 0.383 2.662 0.081 0.193 Component 3 2.033 0.062 0.445 2.404 0.073 0.265 Component 4 1.500 0.045 0.490 2.136 0.065 0.330 Component 5 1.301 0.039 0.530 2.080 0.063 0.393 Component 6 1.161 0.035 0.565 2.056 0.062 0.455 Component 7 1.122 0.034 0.599 2.047 0.062 0.518 Component 8 1.022 0.031 0.630 1.854 0.056 0.574 Component 9 0.962 0.029 0.659 1.760 0.053 0.627 Component 10 0.858 0.026 0.685 1.409 0.043 0.670 Component 11 0.834 0.025 0.710 1.334 0.040 0.710 Based on the results of the principal factor loading on the results of the 33-item empirical test filled in by 250 respondents, it can be described as the following sree plot. 149 |JISAE (Journal of Indonesian Student Assessment and Evaluation) |Volume 8 Number 2 Scree plot Referring to the results of the PCA analysis for the eleven components that measure the tutor's performance appraisal variable, empirically the instrument can be used to measure the quality of tutors. However, this result requires adjustments between items. This is due to the unequal distribution of grains in each of the existing components. In this discussion, it is necessary to re-explain the eleven components formed in the research, including Mastery of technology and information (PC1), Skills in writing scientific papers (PC2), Mastering literacy and numeracy (PC3), Management of learning time (PC4), Management of student behavior (PC5), Presentation of learning (PC6), Monitoring of Learning (PC7), Feedback in learning (PC8), Facilitating learning (PC9), Effective communication (PC10), and Additional tasks (PC11). The explanation of each of these components is as follows: 1. PC1 with the highest eigenvalue of 10,066 with its seven sub-variables has a characteristic high correlation matrix value. 2. PC2, PC6, and PC8 with high eigenvalues and three sub-variables with high correlation matrix values. 3. PC3, PC4, PC5, PC7, PC9, and PC11 on average have two sub-variables that are part of each group with high eigenvalues and correlation matrix. 4. PC10 with an eigenvalue of 0.858 has one sub-variable with a high correlation matrix value. So, each component has various sub-variable characteristics, so that it can reduce variables and form the main variable. PCA can reduce data easily to simplify the observed variables by shrinking their dimensions.(Fenyvesi & Horváth, 2022)(Badri & Sari, 2021)(Wangge, 2021) This step is done to eliminate the correlation between variables by forming new sub-variables. CONCLUSION Based on the results of research using factor analysis using the PCA method, it is concluded that the tutor's performance assessment instrument has formed eleven main components with various sub-variables of each component such as seven sub-variables, three sub-variables, two sub-variables, and one sub-variable with a high correlation matrix value so that the tutor's performance assessment instrument can be declared to have a high level of validity and reliability so that it is appropriate to be used to assess the tutor's performance. REFERENCE Ahmad; Faisal Madani; M. Ishaq; Lasi Purwito; Ratih Permata Sari. (2022). Evaluasi Kebijakan Merdeka Belajar Pada Satuan Pendidikan Nonformal. AKSARA: Jurnal Ilmu Pendidikan Nonforma, 8(2), 1143–1154. 150 Aiken, L. R. (2020). Research on attitudes toward mathematics. The Arithmetic Teacher, 19(3). https://doi.org/10.5951/at.19.3.0229 Atieno Otieno, T., Phylisters, M., & Ursulla, O. (2021). Professional Development Appraisal And Teacher Performance Among Secondary Schools In Migori County. International Journal of Scientific Research and Management, 9(11), 1973–1982. https://doi.org/10.18535/ijsrm/v9i11.el02 Badri, F., & Sari, S. U. R. (2021). Penerapan Metode Principal Component Analysis (PCA) Untuk Identifikasi Faktor-Faktor yang Mempengaruhi Sikap Mahasiswa Memilih Melanjutkan Studi ke Kota Malang. Building of Informatics, Technology and Science (BITS), 3(3), 426–431. https://doi.org/10.47065/bits.v3i3.1139 Besim Enes Bicak, Cornelia Eleonore Borchert and Kerstin Höner. (2021). Measuring and Fostering Preservice Chemistry Teachers’ Scientific Reasoning Competency. Education Sciences, 11(496), 1–23. Chanda Sichinsambwe, Lilian Lialabi, Alexina Muyenga, R. M. (2021). Influence of Performance Appraisals on Promotion of Teachers in the Ministry of Education, Kabwe District of Zambia. International Journal of Human and Behavioral Science, 6(2), 12–25. https://doi.org/10.19148/ijhbs.775226 Chirchir, K. M., & Letangule, S. (2021). The Influence of Teacher Performance Appraisal and Development Implementation on the Secondary Students’ Examination Scores in Public Schools in Kenya. European Journal of Education and Pedagogy, 2(6), 34–44. https://doi.org/10.24018/ejedu.2021.2.6.199 Crocker, Linda, Algina, J. (1986). Ch. 5 Test Scores as Composites. In Introduction to classical and modern test theory. Vol. 6277. Crocker, L., & Algina, J. (1986a). Ch. 6 Reliability and The Classical True Score Model. In Introduction to Classical and Modern Test Theory. Crocker, L., & Algina, J. (1986b). Statistical Concepts for Test Theory. In Introduction to Classical and Modern Test Theory. Darling-hammond, L., Hyler, M. E., & Gardner, M. (2017). Effective Teacher Professional Development. 1(April), 1–8. https://learningpolicyinstitute. Direktorat Tenaga Kependidikan Ditjen PMPTK Depdiknas. (2008). Penilaian Kinerja Guru. In Direktorat Tenaga Kependidikan Ditjen PMPTK Depdiknas. (pp. 1–49). https://doi.org/10.1088/0031-9155/48/8/305 Fenyvesi, B., & Horváth, C. (2022). Identification of Turbomachinery Noise Sources via Processing Beamforming Data Using Principal Component Analysis. Periodica Polytechnica Mechanical Engineering, 66(1), 32–50. https://doi.org/10.3311/PPME.18555 Gronlund, N. E. (1998). Assessment of student achievement. In Assessment of Student Achievement. Jansson, N. F., Allen, R. L., Skogsmo, G., & Tavakoli, S. (2022). Principal component analysis and K-means clustering as tools during exploration for Zn skarn deposits and industrial carbonates, Sala area, Sweden. Journal of Geochemical Exploration, 233(May 2021). https://doi.org/10.1016/j.gexplo.2021.106909 Undang-undang Republik Indonesia Nomor 14 Tahun 2005 Tentang Guru dan Dosen, 54 (2005). Moh Muzaqi. Dwi Sudarmanto. Widya Ayu P.Yuni Ekawati. (2008). Analisis Kompetensi Tutor Pendidikan Kesetaraan Program Paket B di Provinsi Jawa Timur. Jurnal Ilmiah VISI PTK- PNF, 3(1), 5–15. Pasaribu, A. M. N., Candra, J., Kamsurya, R., Zainur, Nurkadri, & Laksana, A. A. N. P. (2021). The relationship of motor skills with performance: Meta-analysis study. Journal Sport Area, 6(3), 315–324. https://doi.org/10.25299/sportarea.2021.vol6(3).6647 Popham, W. J. (1999). Why Standardized Tests Don ’ t Measure Educational Quality A 151 |JISAE (Journal of Indonesian Student Assessment and Evaluation) |Volume 8 Number 2 Standardized Test ’ s Assessment Mission. In Educational Leadership (Vol. 56, Issue 6). Popham, W. J. (2004). Curriculum, Instruction, and Assessment: Amiable Allies or Phony Friends? Teachers College Record, 106(3), 417–428. https://doi.org/10.1111/j.1467- 9620.2004.00345.x Popham, W. J. (2009). Assessment literacy for teachers: Faddish or fundamental? Theory into Practice, 48(1), 4–11. https://doi.org/10.1080/00405840802577536 Popham, W. J. (2014). Criterion-referenced measurement: Half a century wasted? Educational Leadership, 71(6). Popham, W. J. (2016). Standardized tests purpose is the point. Educational Leadership, 73(7). Rahmat, A., Mobo, F. D., Robby, F., Tallar, Y., Ode, W., Zaharah, S., & Karmila, A. (2021). Merdeka Belajar. In A. Rahmat (Ed.), Mengukur Performance PKBM dengan IPV: Penerapan Akreditasi dengan SISPENA (p. 175). ZAHIR PUBLISHING. https://www.google.co.id/books/edition/MERDEKA_BELAJAR/dPwxEAAAQBAJ?hl=id& gbpv=1&dq=inauthor:%22M.+Ishaq%22&printsec=frontcover Siswantari. (2011). Kompetensi Pendidik Dan Tenaga Kependidikan Pada Pendidikan Nonformal. Jurnal Pendidikan Dan Kebudayaan, 17(5), 539–553. https://doi.org/10.24832/jpnk.v17i5.47 Siswantari. (2011). Kompetensi Pendidik Dan Tenaga Kependidikan Pada Pendidikan Nonformal. Jurnal Pendidikan Dan Kebudayaan, 17(5), 539. https://doi.org/10.24832/jpnk.v17i5.47 Tanujaya, B., Mumu, J., & Margono, G. (2017). The Relationship between Higher Order Thinking Skills and Academic Performance of Student in Mathematics Instruction. International Education Studies, 10(11), 78. https://doi.org/10.5539/ies.v10n11p78 The American Educational Research Association, the American Psychological Asso- ciation, and the N. C. on M. in E. (2014). Standards for Educational and Psychological Testing. American Educational Research Association. Wangge, M. (2021). Penerapan Metode Principal Component Analysis (PCA) Terhadap Faktor-faktor yang Mempengaruhi Lamanya Penyelesaian Skripsi Mahasiswa Program Studi Pendidikan Matematika FKIP UNDANA. Jurnal Cendekia : Jurnal Pendidikan Matematika, 5(2), 974–988. https://doi.org/10.31004/cendekia.v5i2.465 Yalow, E. S., & Popham, W. J. (1983). Content Validity at the Crossroads. Educational Researcher, 12(8). https://doi.org/10.3102/0013189X012008010 Zarachoff, M. M., Sheikh-Akbari, A., & Monekosso, D. (2022). Non-Decimated Wavelet Based Multi-Band Ear Recognition Using Principal Component Analysis. IEEE Access, 10, 3949– 3961. https://doi.org/10.1109/ACCESS.2021.3139684 Zulfachri, B. & R. U. (2021). Pengaruh Kompetisi Dalam E-Procurement Terhadap Nilai Pada Layanan Pengadaan Secara Elektronik Kota Pengolahan data menggunakan dalam penelitian menggunakan JASP ( Jeffreys ’ s Amazing. CASH, 91–110.