EVALUATION OF TEACHING MATERIALS OF EVALUATION OF LEARNING COURSES FOR PRIMARY SCHOOL TEACHERS EDUCATION STUDY PROGRAM OF UNIVERSITAS KATOLIK MUSI CHARITAS Bernadine Ajeng Indriasari Universitas Katolik Musi Charitas indriasari@ukmc.ac.id ABSTRACT This study aims to determine the needs of teaching materials for evaluasipembelajaran courses that support the ability of students to carry out assessment of learning outcomes in accordance with the development of current assesments in Education for Primary School Teachers study program UniversitasKatolikMusiCharitas (UKMC). This research is descriptive study using several documents in the form of RPS, Academic manuals and HOTS Guide modules as well as articles from official BNSP website relates to the research topic. The analysis showed that recommended teaching materials for evaluasipembelajaran courses were (1) the concept of measurement, test, evaluation dan assessment; (2) type and function of assessment; (3) basic concept of test and non-test; (4) item analysis; (5) development of test and non-test measuring instrument; (6) the quality of measuring instrument which include validity and reliability; and (7) scoring and following up. Keyword: analyze, evaluation, teaching materials, PGSD INTRODUCTION Evaluasipembelajaran courses (GPB2302) is one of the subjects determined as the Course of Work Behavior (MPB: mata kuliah Perilaku Berkarya) Educatiaon for Primary School Teacher Education (PGSD) study Programs Universitas Katolik Musi Charitas (UKMC). This course has the goal of being able to carry out an assessment of the learning process and outcomes in accordance with the learning objectives. Based on the College Curriculum Compilation Guide, the formation of a College course (PT) is done by selecting study material and compiling a matrix between study materials with the formulation of Graduates Learning Achievement (CPL) that has been determined. So at least the subjects that are determined as courses of study programs have elements of study material that are scientifically integrated and effective to meet the CPL of the study program. One effort that can be done to see the achievement of a course against CPL is to conduct an evaluation. This is in line with Permenristek dikti No. 62 Tahun 2016 tentang Sistem Penjaminan Mutu Pendidikan Tinggi (SPM Dikti). According to Pasal 53 and Pasal 52 ayat (4) UU Dikti, one of the SPM Dikti consists of an SistemPenjaminanMutu Internal (SPMI) implemented by universities. Sistem Penjaminan Mutu Pendidikan Tinggi is a systemic activity to improve the quality of higher education in a planned and sustainable manner. SPMI is planned, implemented, evaluated, controlled, and developed by each tertiary institution. The quality of higher education is the level of conformity between the implementation of tertiary education and Dikti Standards consisting of SN Dikti and Dikti 94 | JISAE. Volume 6 Number 1 February 2020. Copyright ©LPPM Universitas Negeri Jakarta 95 Standards set by each tertiary institution. Evaluation can be interpreted as a process of measuring the effectiveness of strategies used in an effort to achieve goals (companies). Educatiaon for Primary School Teachers study program, if likened to a company, is a printer of excellent elementary teacher candidates. The process of evaluating the subject matter of learning evaluation is one of the efforts to measure the effectiveness of the material that has been determined to achieve the expected goals of graduates. Educatiaon for Primary School Teachersstudy program has become one of the study programs at the Faculty of Business and Accounting (FBA) which has been running for three years. During the three years of the study program, EvalusiPembelajaran courses have been given three times to students. However, there has never been an evaluation of subject teaching material. The researcher, who is also a lecturer in the subject of EvaluasiPembelajaran, compiles a RPS based on previous RPS and results of discussions with the head of the study program as well as fellow Educatiaon for Primary School Teachers study program lecturers. The material is arranged based on the needs that arise along with the course of study, such as the addition of material about making HOTS (High Order Thingking Skills), psychomotor problems, affective questions and visual analysis and readability questions. These materials were not delivered in previous learning evaluation courses. The need for the material mentioned above arises after the holding of a public lecture related to the HOTS question in the mid-semester even of the 2018/2019 school year. In accordance with the academic manuals the material should not be given to students. Based on the academic guidelines the learning objectives only want students to be able to carry out assessments in accordance with learning in the classroom. In addition to issues regarding the novelty of the material, it is also found the need for continuity of teaching evaluation subject matter learning with Research methodology courses. This problem was discovered when students began taking thesis courses. Many students still do not master the validity of instrument testing and data processing techniques. Research methodology courses require students' ability to understand the various types of validity tests of research instruments and data processing, both the results of instrument validation and research data. This material has the appropriate characteristics to be given through EvaluasiPembelajaran courses but based on the formulation in the academic manual this material is not in accordance with the objectives of the course. Based on these findings, learning evaluation subject matter needs to be reviewed. It is necessary to evaluate teaching material to measure the suitability and achievement of CPL of study programs that have been determined. And formulate the exact purpose of the course. METHOD This research is a descriptive qualitative research. Syaodih (2010: 72) states that descriptive research is research that describes the phenomena that exist, both scientific and human engineering. This research can be done by collecting or analyzing official documents, documents whose validity and validity are guaranteed, both the legal and policy documents and the results of the research. 96 | JISAE. Volume 6 Number 1 February 2020. Copyright ©LPPM Universitas Negeri Jakarta The documents required in the study include, 2.1. RPS of evaluasi pembelajaran courses for the fisrtlearning year 2017/2018 2.2. RPS of evaluasi pembelajaran coursesfor the second learning year 2017/2018 2.3. RPS of evaluasi pembelajaran courses for the fisrtlearning year 2017/2018 2.4. Undang-Undang RI No. 12 Tahun 2012 tentang Pendidikan Tinggi 2.5. Peraturan Presiden No. 8 Tahun 2012 tentang Kerangka Kualifikasi Nasional Indonesia 2.6. Peraturan Menteri RisetTeknologi No. 44 Tahun 2015 tentang Standar Nasional Pendidikan Tinggi (SN Dikti) 2.7. Peraturan Menteri Pendidikandan Kebudayaan No. 49 Tahun 2014 tentang Standar Pendidikan Nasional Pendidikan Tinggi 2.8. Peraturan Menteri Pendidikan dan Kebudayaan No. 73 Tahun 2013 tentang Penerapan Kerangka Kualifikasi Nasional Bidang Pendidikan Tinggi 2.9. Learning Achievement of Education of Primary School Teachers study program of Universitas Katolik Musi Charitas. Data collection techniques in this study were carried out by steps (1) Gathering the documents that have been described in the previous paragraph. (2) Read the entire data. (3) Investigate findings. Data analysis techniques in this study were carried out by steps (1) Analyzing the inventory of findings in data collection; (2) Re-examine the documents that have been collected; and (3) make conclusions. Data validity uses document triangulation. Document triangulation is a data validity technique that explores the truth of information through various methods and data sources. Like, the use of written documents, archives, historical documents, official records, notes or personal writing. In this study, the documents are in the form of nine documents that have been mentioned in the previous sub-section. Each document is analyzed to provide information. Different information collected from each document is analyzed to produce conclusions. RESULT AND DISCUSSIONS Education of Primary School Teachers study program of Universitas Katolik Musi Charitasis a new study program that has been running for 3 years. Then there are 3 lesson evaluation RPS courses that will be analyzed in this study. The 1st is the first 2016/2017 academic year. The second is second of academic year 2017/2018. The 3rd is first of academic year 2018/2019. The recapitulation of the results of the analysis is in table 1 below. Table 1.Recapitulation of RPS Analysis Result of Evaluasi Pembelajaran Courses of Education of Primary School Teachers Study Programs No Guide book RPS 1 RPS 2 RPS 3 1. Learning evaluation aspect 2. Basic concepts of test and non- 1. Definition of test, measurement and assesment 2. Classification of measuring 1. Concepts of evaluation, tests, measurement and 1. Concepts of measurement, evaluation, test and assessment 2. Types and 97 No Guide book RPS 1 RPS 2 RPS 3 test, measurement, assessment and evaluation 3. Types and function of assessment and learning 4. Development of test and non- test measuring instruments 5. Management and interpretation of measurement results 6. Quality of measuring instruments 7. Grading ang following up on the result of measurement instruments 3. Test the description 4. Write the description test 5. Test scoring guidelines 6. Make a test lattice 7. Objective tests 8. Development of non-test assessment tools 9. Validity and reliability of measurements 10. Analysis of items 11. Compile objective tests 12. Non-test instruments 13. Checking and processing measurement data 14. Statistic of measurement 15. Empiric validation process with trials assesements 2. Assesment, function, procedures objectives, assessment technique 3. Test technique 4. Non-test technique 5. Analysis of test and non- test items 6. Make a lattice 7. Making test results for learning outcomes 8. Validity and reliability of the items 9. Analyzing and correting test results of learning outcomes 10. Processing learning outcomes 11. Designing remidial 12. Designing an assessment of learning process function of assessment in learning 3. Basic test and non-test concepts 4. Developing of test and non-test measuring instruments 5. Data processing and interpretation 6. Quality of measuring instruments 7. Grading and following up. The recapitulation results of the above material are then analyzed for their compatibility with the academic manual. Based on the 2017 academic guide for the Education of Primary School Teachers study program of UniversitasKatolikMusiCharitas, the aim of the EvaluasiPembelajaran course is that students master the ability to carry out an assessment of the learning process and outcomes in accordance with the objectives. Teaching material covers aspects of learning evaluation, basic concepts of tests and non-tests, measurement, assessment, and assessment; types and functions of assessment and learning; development of test and non-test measuring instruments; processing and interpretation of measurement results data, quality of measuring instruments, grading of values and further assessment results. In table 4.1 you can see the instructional materials specified in the academic manuals and changes in teaching materials for each academic year. The determination of this material refers to the objectives of the Education of Primary School Teachers study program of UniversitasKatolikMusiCharitascourse in accordance with the 2017 academic manual. The first material in accordance with the academic manual is the evaluation aspect of learning. Since the first academic year the material has not been used because it was considered unnecessary. The material should refer to the concept of assessment, test and measurement and evaluation. This material is more relevant to the need for understanding of assessment and its differences with the terms test, measurement and evaluation rather than merely aspects of learning evaluation. The evaluation aspects of learning can be explained briefly after students know the concept of assessment. The learning evaluation aspect contains material that shows how broad and in the evaluation it is carried out in elementary school. This material is implicit in the concepts of assessment, tests, measurements and evaluations. So the first meeting material in RPS 2 and RPS 3 is exactly in accordance with the objectives of the Education of Primary School Teachers study program of UniversitasKatolikMusiCharitascourse even though it is not in accordance with the academic manuals. The second material is in accordance with the academic manual of the basic concepts of test and non-test, measurement, assessment and assessment. This material was covered at the first meeting of RPS 1, 2 and 3. This material is relevant to the objectives of the Education of Primary School Teachers study program of UniversitasKatolikMusiCharitaslearning evaluation course and continues to be used in each academic year. But in a different order according to the lecturer design. The third material is in accordance with the material in the academic manuals about the types and functions of learning assessment. This material is in line with the objectives of the Education of Primary School Teachers study program of UniversitasKatolikMusiCharitaslearning evaluation course and is used in each academic year. The fourth material is in accordance with the material in the academic manual, namely the development of test and non-test measuring instruments. Students are required to be able to develop their own measurement tools for both test and non-test assessments. This material is used in each academic year in a different order of meetings. This material is in line with the objectives of the Education of Primary School Teachers study program of UniversitasKatolikMusiCharitaslearning evaluation course and is in accordance with academic guidelines. It should be emphasized that in the material development of test and non-test measuring devices, each student does not need to develop all types of test and non- test measuring instruments but can be chosen randomly from a number of students to 98 | JISAE. Volume 6 Number 1 February 2020. Copyright ©LPPM Universitas Negeri Jakarta 99 develop different measuring instruments. Until all types of measuring devices can be made but not charged to one student. But complementary between students. The fifth material in accordance with the material in the academic manual is the management and interpretation of measurement results. This material relates to scoring guidelines and assessment rubrics. Including interpretation of measurement results. For example, graduating or not graduating. This material is consistent with the learning objectives of the Education of Primary School Teachers study program of UniversitasKatolikMusiCharitaslearning evaluation course regarding the ability to carry out assessments even though the objectives are not explained in detail about the management and interpretation of the assessment. But doing an assessment means giving the final results in the form of values and their interpretations. The sixth material according to the academic manual is the quality of the measuring instrument. This material relates to the validity and reliability of measuring instruments. Over time, this material was changed to validity and reliability in accordance with the development of science. The order of material varies in each RPS in each academic year. This material is in accordance with not explained in detail in the learning objectives but it is clearly needed to ensure the measuring instruments used are in accordance with the standards and can measure students' abilities well. It should be emphasized on the quality of the measuring instrument which includes validity and reliability, that the validity that is studied more leads to the validity of the content including the question construct, sentence length, sentence effectiveness, clarity of words and punctuation, suitability of competence measured, and clear visuals. Does not emphasize testing the validity of the subject because it is a type of question used in the assessment of small-scale learning outcomes in the classroom so it does not require detailed testing such as questions used for national or larger scale exams. The seventh material in accordance with the Education of Primary School Teachers study program of UniversitasKatolikMusiCharitasacademic guidelines is the grading and follow-up of the results of the assessment. This material can actually be explained in the fifth material about data processing and interpretation. Processing and interpretation of data will ultimately provide the final value which is the meaning of value in this seventh material. It only needs to be added regarding the follow up assessment. This material is relevant to the learning objectives. This material is also in accordance with the relevance of the needs in the field regarding the follow-up that the teacher needs to do after obtaining the student's final grade. This follow up can be in the form of repetition (remedial) or enrichment. In addition to the seven materials that are in accordance with the book of warfare above, there are some additional material that can be seen in each RPS as in the table. The material is a translation of the seven materials that have been determined in the guidebook. Each supporting lecturer develops and narrows according to the needs and designs that have been prepared for one semester of learning. It should be noted that the item analysis material is limited to the analysis of qualitative items that focuses on the rules of writing questions rather than quantitative analysis based on empirical evidence. By reason of limited use of the questions in the small scope and complexity of the validation of the questions if only used for daily tests in class. This is consistent with the material validity and reliability that is limited in its suitability to the topic of learning. One note for the purpose of the subject of the evaluation of the word learning "goal" that hangs at the end of the sentence. In accordance with the academic manual of learning evaluation subject goals, students master the ability to carry out an assessment of the learning process and outcomes in accordance with the objectives. The researcher recommends adding the word "learning" to become a student mastering the ability to carry out assessments of the learning process and outcomes in accordance with learning objectives. So that the sentence becomes clear and does not hang. In addition to the analysis discussed in the previous paragraph, an analysis was carried out on the material regarding its suitability with HOTS questioning skills. Material about HOTS questions can be given when the material is about developing test and non-test measuring instruments. Besides that, it can also be given to the subject matter of the quality of measuring instruments. This material synergizes because of the quality of the measuring instrument, as explained in the discussion of the previous paragraph that the validity of the study is more directed towards content validity including the construct of the questions, sentence length, effectiveness of sentences, clarity of words and punctuation, suitability of competence measured, and visual clear. This material is closely related to the preparation of HOTS questions which must pay close attention to the level of knowledge measured by the questions. Choosing the right words to give clear meaning to students and in accordance with the intent that the educator wishes to convey. So the composition of the material designed by lecturers supporting the evaluation of learning remains in accordance with the academic manual by adding material about HOTS questions. Analysis of the RPS on the subject of Education of Primary School Teachers study program of UniversitasKatolikMusiCharitasevaluation of the assessment standard articles issued by BSNP is as follows. According to the BNSP article it is stated that the guidelines for assessment by educators cover 3 things. (1) general assessment standards which include selection of assessment techniques, conformity with content standards and graduate competency standards, including attitude assessment, checking and providing feedback as well as having a cumulative record; (2) assessment planning standards which include educational obligations to make an integrated assessment plan, develop achievement criteria, assessment techniques according to BC, make a grid and make instruments; (3) standard implementation of the assessment which includes analysis of the quality of the instrument, examining the work of students and feedback; (4) standard processing and reporting of results which includes the granting of scores, determination of one value, determination of the results of assessments and submission to related parties; and (5) results utilization standards which include the classification of completeness in achieving standards, providing 100 | JISAE. Volume 6 Number 1 February 2020. yright ©LPPM Universitas Negeri Jakarta 101 follow-up in the form of remedial and enrichment and evaluating the effectiveness of learning activities based on assessment. The results of the analysis of the RPS of the evaluation evaluations of the subjects and the five (5) standards described in the previous paragraph can be concluded that the subject of Education of Primary School Teachers study program of UniversitasKatolikMusiCharitasstudy evaluasipembelajaran materials is in line with the expectations of the BSNP assessment standards. Expected material based on BSNP general assessment standards covering assessment techniques, developing questions based on SI and SKL and providing feedback have appeared in the RPS even though in a different order. The material on the standard assessment planning briefly focused on the making of the questionnaire and the development of instruments has also appeared on the material in the RPS which is material on the development of test and non-test questions. Standards for conducting the assessment relate to material quality instruments and feedback. In the RPS explained in the topic of validity and reliability. The material included in the standard processing and reporting of results is given in the RPS on the topic of processing and interpretation. Then the standard for utilizing results includes follow-up assessments. Then it can be concluded that the material in the RPS is in accordance with BNSP provisions. Thus it is also in accordance with the juridical basis of learning assessment, including Undang- Undang No. 20 Tahun 2003 pasal 57 dan 58 dan PP No. 19 tahun 2005 pasal 63. CONCLUSION Based on the discussion it was found that teaching materials needed by Education of Primary School Teachers study program of UniversitasKatolikMusiCharitas students to be able to have the ability to carry out learning outcome assessments are as follows: a. The concept of measurement, test, evaluation and assessment b. Assessment type c. Assessment function d. Basic concepts of tests and non-tests e. Item analysis f. Development of test and non-test measuring instruments g. Quality of measuring instruments (validity and reliability) h. Scoring and following up Based on the findings of the researcher submit the following suggestions to lecturers supporting the learning evaluation course. a. Emphasis should be given to the development of test and non-test measuring instruments, that each student does not need to develop all types of test and non-test measuring instruments but can be chosen randomly from a number of students to develop different measuring instruments. 102 | JISAE. Volume 6 Number 1 February 2020. Copyright ©LPPM Universitas Negeri Jakarta b. It is recommended that the quality of the measuring material cover validity and reliability, that the validity learned is more directed towards content validity (qualitative) which includes the question construct, sentence length, sentence effectiveness, clarity of words and punctuation, suitability of competence measured, and clear visuals. Does not emphasize testing the validity of the subject (quantitative) because it is a type of questions used in the assessment of small-scale learning outcomes in the classroom so it does not require detailed testing such as the questions used for national or larger scale exams. c. The item analysis material should be limited to the analysis of qualitative items that focus on the rules of writing questions rather than quantitative analysis based on empirical evidence. By reason of limited use of the questions in the small scope and complexity of the validation of the questions if only used for daily tests in class. This is consistent with the material validity and reliability that is limited in its suitability to the topic of learning. REFERNCES Direktorat Pembinaan Sekolah Menengah Atas Direktorat Jendral Pendidikan Dasar dan Menengah Kementrian Pendidikan dan Kebudayaan. 2017. Modul Penyusunan Soal Higher Order Thingking Skills (HOTS ). Jakarta: Kemedikbud. Nasution. 2003. Metode Penelitian Naturalistik Kualitatif. Bandung: Tarsito. Pedoman Akademik Universitas Katolik Musi Charitas 2015. Pedoman Akademik Universitas Katolik Musi Charitas 2017. Pedoman Akademik Universitas Katolik Musi Charitas 2018. Peraturan Menteri Pendidikan dan Kebudayaan Republik Indonesia No. 23 Tahun 2016 tentang Standar Penilaian Pendidikan. Bnsp-indonesia.org. Poerwanti, Endang. Asesmen Pembelajaran di SD. http://storage.kopertis6.or.id/kelembagaan/Applied%20Approach/MATERI/D rs.%20Suwarno,%20M.Si/1-Konsep-Dasar-Asesmen-Pembelajaran.pdf . Pada 1 Agustus 2019. Standar Penilaian BSNP. http://staff.unila.ac.id/ngadimunhd/files/2012/03/2-Standar- Penilaian-Sesuai-BSNP.pdf. Diunduh pada 1 Agustus 2019. Syaodih, Nana. 2010. Metode Penelitian Pendidikan. Bandung: Remaja Rosdakarya.