LANGUAGE CIRCLE: Journal of Language and Literature 14(1) October 2019 p-ISSN 1858-0165 Available online at http://journal.unnes.ac.id e-ISSN 2460-853X 67 Evaluating the Reading Section of English National Standardized School Exams (USBN): (A Field Study in SMK NU Ungaran) Rizky Mega Susanti Magister of Linguistics Diponegoro University – Semarang, Indonesia Email: mega92rizky@gmail.com Abstract With regard to the role of English in Indonesia that is mainly used for academic purpose, it is important to sharpen the two skills, reading and writing, more than the other two skills, listening and speaking. Based on that fact, this study focuses on reading skills, especially in reading comprehension skills. Reading comprehension skill is closely related to literacy levels that nowadays considered important by educational institution. The data was taken from reading comprehension questions in English USBN question sheet and the students’ answer sheets. The participants are students of XII grade of SMK NU Ungaran that are randomly chosen. The result of this study showed that Out of 50 students, 59% of them are best cope with editing task, followed by 56% in dealing with reading comprehension task, 50% of gap-filling task, 48% with short answer task and the least is 22% in vocabulary task. This meant that the proficiency level of reading comprehension of students in XII grade SMK NU Ungaran still needs to elevate. They were still lack in term of vocabulary and understanding the ideas implicitly delivered in the text. Keywords: reading skills, reading comprehension, USBN INTRODUCTION Reading is regarded as as one of the most important skills in a language. Students can continuesly develop other skills such as writing and speaking or other language components through reading comprehension. This is in line with Nunan (1991, p. 82) stated that four language skills have close interaction in which reading gives more opportunities for learners to develop other skills. Therefore, the students are encourage to interact with various kind of texts in order to develop other skills. The ability to understand and use written text is an important precondition for developing personal knowledge and skill. It is also important to participate in cultural and social life; reading makes many areas of knowledge and life accessible. Text does not only convey information and facts, but they also deliver ideas, moral concepts and cultural contents. This makes reading crucial to further education and lifelong learning. Besides, reading comprehension requires the reader and text interaction which results in the successful of reading comprehension. Reading comprehension is one of language competences emphasized by the government for the English teachers to deal with. It is stated in PP No. 19 Year 2005 that language education should develop language competence with special emphasis on reading and writing according to the literacy level set up for every level of education. Besides, students are expected to be able to comprehend various kinds of texts, such as descriptive, recount, narrative procedure text according to the content of Curriculum of LANGUAGE CIRCLE: Journal of Language and Literature, 14(1) October 2019 68 2013 as current curriculum. As English in Indonesia is mainly used for academic purpose, it is important to sharpen those two skills, reading and writing, more than the other two skills, listening and speaking. Based on those reasons, this study focuses on reading comprehension skills. The 2013 Curriculum is competency and character-based curriculum. Based on the Ministry of Culture and Education Regulation No. 59 Year 2014 stated that 2013 Curriculum is one of the government's efforts to resolve the various problems being faced by the world of education today. The theme of 2013 Curriculum is generating Indonesian people which are: productive, creative, innovative, affective; through the strengthening of attitudes, skills, and knowledge which are integrated. Based on the theme, the implementation of 2013 curriculum is expected to produce a productive, creative, and innovative human. In addition, the 2013 Curriculum also becomes the guidelines in designing various kinds of exam, including final exam or USBN. To design a standardized test as USBN test, teachers need to consider at least three things; core standards (KI), competence standards (KD) and graduate competence standards (SKL) as they are mentioned in the 2013 Curriculum. Those three components become the guidelines to create the learning materials, design the lerning methodologies, and develop the assessment which then realized in syllabus and lesson plan. While in designing a reading assessment, the teachers need to involve literacy level and types of reading performance. In SMA/K context, there are four Core Competence that must be reached by the students at the end of learning process as stated in Permendikbud No. 59 Year 2014. As it is realized on the curriculum, the CC1 was attached in the content of the textbook implicitly to make the students realize about themselves as God’s creature to do good things toward the nature and others.With regard to CC2, the learning materials and exepriences were provided in a form of independent activity, activity in pairs, and group discussion which aimed to develop students’ social skills. While, to reach CC3 and CC4, the classroom activities were carried out in an integrated manner through listening, reading, speaking and writing which have been developed based on scientific approach. Hence, the practical core competences that have to be designed by the teacher are core competence 3 (factual knowledge, conceptual knowledge, and procedural knowledge) and core competence 4 (metacognitive knowledge) Here are the core competence and basic competence for English subject of XII grade students in SMA/K based on Permendikbud 24/2016: Table 1. Core Competence and Basic Competence of XII Grade (based on Permendikbud 24/2016) KOMPETENSI INTI 3 (PENGETAHUAN) KOMPETENSI INTI 4 (KETERAMPILAN) 3. memahami, menerapkan, menganalisis dan mengevaluasi pengetahuan faktual, konseptual, prosedural, dan metakognitif berdasarkan rasa ingin tahunya tentang ilmu pengetahuan, teknologi, seni, budaya, dan humaniora dengan wawasan kemanusiaan, kebangsaan, kenegaraan, dan peradaban terkait penyebab fenomena dan kejadian, serta menerapkan pengetahuan prosedural pada bidang kajian yang spesifik sesuai dengan bakat dan minatnya untuk memecahkan masalah 4. mengolah, menalar, menyaji, dan mencipta dalam ranah konkret dan ranah abstrak terkait dengan pengembangan dari yang dipelajarinya di sekolah secara mandiri serta bertindak secara efektif dan kreatif, dan mampu menggunakan metoda sesuai kaidah keilmuan LANGUAGE CIRCLE: Journal of Language and Literature 14(1) October 2019 p-ISSN 1858-0165 Available online at http://journal.unnes.ac.id e-ISSN 2460-853X 69 KOMPETENSI DASAR KOMPETENSI DASAR 3.1 menerapkan fungsi sosial, struktur teks, dan unsur kebahasaan teks interaksi interpersonal lisan dan tulis yang melibatkan tindakan menawarkan jasa, serta menanggapinya, sesuai dengan konteks penggunaannya. (Perhatikan unsur kebahasaan May I help you?, What can I do for you? What if ...?) 4.1 menyusun teks interaksi interpersonal lisan dan tulis sederhana yang melibatkan tindakan menawarkan jasa, dan menanggapinya dengan memperhatikan fungsi sosial, struktur teks, dan unsur kebahasaan yang benar dan sesuai konteks 3.2. membedakan fungsi sosial, struktur teks, dan unsur kebahasaan beberapa teks khusus dalam bentuk surat lamaran kerja, dengan memberi dan meminta informasi terkait jati diri, latar belakang pendidikan/pengalaman kerja, sesuai dengan konteks penggunaannya 4.2 surat lamaran kerja 4.2.1 menangkap makna secara kontekstual terkait fungsi sosial, struktur teks, dan unsur kebahasaan teks khusus dalam bentuk surat lamaran kerja, yang memberikan informasi antara lain jati diri, latar belakang pendidikan/pengalaman kerja 4.2.2 menyusun teks khusus surat lamaran kerja, yang memberikan informasi antara lain jati diri, latar belakang pendidikan/pengalaman kerja, dengan memperhatikan fungsi sosial, struktur teks, dan unsur kebahasaan, secara benar dan sesuai konteks 3.3 membedakan fungsi sosial, struktur teks, dan unsur kebahasaan beberapa teks khusus dalam bentuk teks caption, dengan memberi dan meminta informasi terkait gambar /foto /tabel/grafik/ bagan, sesuai dengan konteks penggunaannya 4.3 teks penyerta gambar (caption) 4.3.1 menangkap makna secara kontekstual terkait fungsi sosial, struktur teks, dan unsur kebahasaan teks khusus dalam bentuk caption terkait gambar/foto/tabel/grafik/bagan 4.3.2 menyusun teks khusus dalam bentuk text caption terkait gambar/foto/tabel/grafik/bagan, dengan memperhatikan fungsi sosial, struktur teks, dan unsur kebahasaan, secara benar dan sesuai konteks 3.4 membedakan fungsi sosial, struktur teks, dan unsur kebahasaan beberapa teks news item lisan dan tulis dengan memberi dan meminta informasi terkait berita sederhana dari koran/radio/TV, sesuai dengan konteks penggunaannya 4.4 menangkap makna secara kontekstual terkait fungsi sosial, struktur teks, dan unsur kebahasaan teks news items lisan dan tulis, dalam bentuk berita sederhana koran/radio/TV 3.5 menerapkan fungsi sosial, struktur teks, dan unsur kebahasaan teks interaksi transaksional lisan dan tulis yang melibatkan tindakan memberi dan meminta informasi terkait pengandaian diikuti oleh perintah/saran, sesuai dengan konteks penggunaannya. (Perhatikan unsur kebahasaan if dengan imperative, can, should) 4.5. menyusun teks interaksi transaksional lisan dan tulis yang melibatkan tindakan memberi dan meminta informasi terkait pengandaian diikuti oleh perintah/saran, dengan memperhatikan fungsi sosial, struktur teks, dan unsur kebahasaan yang benar dan sesuai konteks LANGUAGE CIRCLE: Journal of Language and Literature, 14(1) October 2019 70 membedakan fungsi sosial, struktur teks, dan unsur kebahasaan beberapa teks prosedur lisan dan tulis dengan memberi dan meminta informasi terkait manual penggunaan teknologi dan kiat-kiat (tips), pendek dan sederhana, sesuai dengan konteks penggunaannya 4.6 teks prosedur 4.6.1 menangkap makna secara kontekstual terkait fungsi sosial, struktur teks, dan unsur kebahasaan teks prosedur lisan dan tulis, dalam bentuk manual terkait penggunaan teknologi dan kiat-kiat (tips) 4.6.2 menyusun teks prosedur, lisan dan tulis, dalam bentuk manual terkait penggunaan teknologi dan kiat-kiat (tips), dengan memperhatikan fungsi sosial, struktur teks, dan unsur kebahasaan, secara benar dan sesuai konteks 3.7 menafsirkan fungsi sosial dan unsur kebahasaan lirik lagu terkait kehidupan remaja SMA/MA/SMK/MAK 4.7 menangkap makna secara kontekstual terkait fungsi sosial dan unsur kebahasaan lirik lagu terkait kehidupan remaja SMA/MA/ SMK/MAK According to Wells (1987, pp. 18-19), literacy level includes performative, functional, informational and epistemic. In performative level, the learners are targeted for having ability to communicate in English to cope with the demands of everyday life or for survival purposes. In a functional level, the learners are supposed to have ability using language to meet their daily needs such as, reading newspaper, using telephone, writing letters, etc. In an informational level, the learners are expected to be able to get involved in communication using English spoken as well as written, not only for transactional and interpersonal purposes but also for accessing information and knowledge. In an epistemic level, the learners are expected to be able to express the knowledge in an intended language. The epistemic grade is considered as the most difficult grade for the learners. At this level, the learners are targeted to have available ways of acting upon the language. According to PISA (The OECD Programme for International Student Assessment) in 2015 conducted a study showed that more than 86% of Indonesian 15 year olds read at PISA Level 2 or below – that is, they are unable to consistently perform Level 3 skills such as ‘locat[ing] and… recognis[ing] the relationship between several pieces of information in a text. Therefore, the literacy level of Indonesian students with average age 15 years old are still need to be improved. Brown (2004) identifies four types of reading performance that serve as organizers of various assessment tasks. They are: 1. Perceptive. This type is fundamental and basic for second language learners. At first, a learner needs to recognize alphabetic symbols, capitalized and lowercase letters, punctuation, words and grapheme-phoneme correspondences. Assessment on this basic reading skill can be conducted in different ways: reading aloud, written responses, multiple-choice, and picture-cued items. 2. Selective. This category deals with lexical and grammatical aspects of language. Some possible tasks to assess those aspects are: multiple-choice with form-focused criteria, editing task, picture-cued task, and gap-filling task. 3. Interactive. It has a combination of form-focused and meaning-focused objectives in which meaning gets more focus. Texts given in this task are little longer, from a paragraph to a page in length. The format of charts, graphs and other graphics may be much more complex. Kinds of questions can be: cloze-task, impromptu reading plus comprehension questions, short-answer task, LANGUAGE CIRCLE: Journal of Language and Literature 14(1) October 2019 p-ISSN 1858-0165 Available online at http://journal.unnes.ac.id e-ISSN 2460-853X 71 editing, scanning, ordering, and reading charts/graphs/diagrams. 4. Extensive. Extensive reading involves longer texts, such as journal articles, technical reports, longer essays, short stories and books. The assignments for this category can be skimming task, summarizing or responding, note-taking and outlining. The table below illustrates the level of literacy and types of reading performance in relation with the educational level in Indonesia: Table 2. Literacy Levels, Reading Performance Types and Educational Level in Indonesia No. Level of Literacy Types of Reading Performance Educational Level 1. Performative Perceptive Elementary School (SD) 2. Functional Selective Interactive (introduction) Junior High School (SMP) 3. Informational Selective Interactive Extensive (introduction) Senior High School (SMA/K) 4. Epistemic Extensive University level Based on the table above, students of XII grade which are in SMA/K, should have informational literacy level and their reading performance should be selective reading, interactive reading and a slight introduction to extensive reading. An evaluation on the implementation of English National Examination Exercise once has been done by Widya et al. in 2015. They conduct the research in Ex RSBI Private SMA in Yogyakarta. They use Stake’s Evaluation Model, thus they gather the information of the input (antecedents) of English tryout, document the process (transactions) of the implementation and the results (outcomes). The results are (1) the input of the implementation is good if it is viewed from the dimensions of the preparation that include the orientation of question writing, committee, schedule and supervision; (2) the process is good if it is viewed from the dimensions of the implementation that included security matter, objectivity of the implementation, and students’ perceptions; (3) results show that question items are good in terms of difficulty levels, distractor functioning and reliability. English tryout gives good benefits and there are no obstacles in the implementation of English tryout. Although the evaluation done in this research has been described thoroughly, the proficiency level of four basic skills is not mentioned in detail. Therefore, the writer assumed taht this present research was different in topics as this study only focus on reading proficiency reflected by the students’ test result on reading section of USBN question sheets. SMK NU Ungaran is one of the largest vocational high schools in Ungaran. It has five departments with approximately 1,500 students. As one of the favorite schools in Semarang Regency, it has good reputation in National Exam result year by year and it has graduated its students almost 100% per year. Concerning to its great achievement, the writer decided to conduct my research on reading comprehension in its English USBN in order to investigate the students’ reading proficiency level. This research attempts to answer the following questions: 1) Have the reading comprehension questions given to the students fulfilled the competence standards in the curriculum? 2) Have the questions considered the literacy level and types of reading performance? 3) How is the students’ reading proficiency reflected in the test result? LANGUAGE CIRCLE: Journal of Language and Literature, 14(1) October 2019 72 METHODOLOGY The participants of this research were XII grade students of SMK NU Ungaran as the population. They were randomly taken as the sample. The data were reading comprehension questions taken from question sheet of English USBN as well as the students’ answer sheets. The method used in analyzing the data and presenting the result was qualitative descriptive with some tables and chart to make some part of the analysis clearer. The writer tried to pair competence standards and core standards set by the government with the question forms given to the students. Except that, the writer also considered the four levels of literacy and types of reading performance in examining the accuracy of the question forms for the students. FINDINGS AND DISCUSSION Text Genre and Task Type Reading section that the writer analyzed consists of various kinds of text: job application, procedure text, narrative, descriptive, tourism leaflet, dialogues, short conversation and announcement. Those kinds of texts can explore informational literacy level needed by SMA/K students. In informational level, students are expected to have an ability to use language to access knowledge and to use it in both formal and informal situation. The table below showed the genre text used in English exam and their relations to basic competence, literacy level and type of reading performance. Table 3. Text Genre in English Exam (USBN) Text Genre Basic Competence (KD) Literacy Level Type of Reading Performance Job-related Reading ------------------------------------------- ------------------------ ----------------- Job application KD 4.2. Job application letter Informational Interactive Procedure Text KD 4.6. Procedure text Informational Interactive Announcement KD 4.4. Grabbing contextual meaning related to text structure and language items Informational Interactive Tourism Passage KD 4.4. Grabbing contextual meaning related to text structure and language items Informational Interactive Personal Reading ------------------------------------------- ------------------------ ----------------- Narrative KD 4.4. Grabbing contextual meaning related to text structure and language items Informational Interactive Descriptive KD 4.4. Grabbing contextual meaning related to text structure and language items Informational Interactive Dialogue KD 4.1 spoken/written simple interpersonal text KD 4.5. transactional text Functional Selective, interactive LANGUAGE CIRCLE: Journal of Language and Literature 14(1) October 2019 p-ISSN 1858-0165 Available online at http://journal.unnes.ac.id e-ISSN 2460-853X 73 Text Genre Basic Competence (KD) Literacy Level Type of Reading Performance Short Conversation KD 4.1 spoken/written simple interpersonal text KD 4.5. transactional text Functional Selective, interactive As we can see, the organization of text genre involved in English USBN has fulfilled the basic competence, literacy level and types of reading performance. The table above also indicate that the Core Competence and Basic Competence set by the government has appropriately adapted the literacy level as well as the types of reading performance proposed by Brown (2004). Thus, the writer could barely say that the reading section of English USBN has good quality and it supposed to be able to measure the students reading proficiency based on the level of SMA/K. The questions given to the students are varied from reading comprehension questions, short-answer task, editing task and gap-filling task. They consist of two basic forms of questions: 30 multiple-choice task and 5 essay task. The composition of each task type is shown on the chart below: Chart 1. Composition of Reading Task Type in English USBN Question Sheets As shown in the chart above, reading comprehension task gets the largest portion. It is interesting to see that the gap between reading comprehension task and the other task types is quite large. If we look back at the core competence that have been presented in the previous part of this paper, it is stated that XII grade students supposed to have an ability to understand, apply, analyze and evaluate factual, conceptual, procedural and metacognitive knowledge based on their curiosity in science, technology, art, culture and humanities along with humanity, nationality, statehood, and civilization insights related to causes of certain phenomenon and events, and also apply procedural knowledge of specific fields according to their talents and interests to solve problems. Based on this core competence, reading comprehension task is the most advantageous in promoting the students understanding of certain topics 0% 10% 20% 30% 40% 50% Question Forms 49% 14% 26% 6% 6% reading comprehension task short-answer task gap-filling task editing task vocabulary task LANGUAGE CIRCLE: Journal of Language and Literature, 14(1) October 2019 74 concerned in the passages. Reading comprehension tasks in the reading section of English USBN have represented a sample of the test specifications for TOEFL reading passages, which are derived from research on a variety of abilities good readers’ exhibit. Many of them are consistent with strategies of effective reading: skimming for main idea, scanning for details, guessing word meaning from context, using inferences, using discourse markers, etc. Shortly, reading comprehension task is a ‘one fit all’ task type that can be effectively used to assess learner’s reading comprehension. That is why the portion of this kind of task is the largest among the other reading task types. Gap-filling tasks in this reading section are presented in dialogue. It is in the second place after reading comprehension task. I would like to highlight the literacy level here; vocational high school students should be in informational level in literacy. It means that they are supposed to be able to use language in formal and informal situation. By giving them gap-filling task in dialogue forms, students need to explore their mental experience related to expression used in such situation illustrated in the dialogue. The least three types, short-answer task, editing and vocabulary task, are closely related to the students’ knowledge of language items both words relation and sentence relation as a whole. Actually, language items have been covered in reading comprehension task, so that these three task types are not much exposed in reading section. Analysis on Students’ Answer Sheets Dealing with various kinds of task types in reading section has made students struggle in answering the task. Ideally, the result of the test reflects the students’ proficiency. In this following section, the writer examined the results of the test on reading section that have been done by the students. The following chart exposed that the students mainly have best performance in editing task. This kind of task is obviously helpful for those who lack of vocabulary as it mainly deals with grammatical aspect of the language. By memorizing the grammatical pattern, such as tenses, a student can easily answer this task type. Chart 2. Percentage of Correct Answer on each Question Forms Reading comprehension task is in the second place. Students seem to have adequate skills in skimming and scanning text to cope with the task. However, the high percentage of its correct answer may be influenced by the number of questions. Due to its large numbers, the percentage of correct answer shown will also be as high as the number of Percentage of Students' Correct Answers 56% 48% 50% 59% 22% reading comprehension task short-answer task gap-filling task editing task LANGUAGE CIRCLE: Journal of Language and Literature 14(1) October 2019 p-ISSN 1858-0165 Available online at http://journal.unnes.ac.id e-ISSN 2460-853X 75 questions. The same indication is also prevailed in gap-filling task. I cannot assume that the students have adequate knowledge on using language in certain context, but at least by looking at the chart above I can say that they might succeed in exploring their mental experience to such situations because the dialogues given are relatively familiar to them. Students are mostly lack on vocabulary items relating to synonym. Although the word meaning is inferred in the passage, they seem to have difficulties in guessing the similar meaning. Lack of reading interest can be the major cause in this case. Vocabulary item is about nothing but memorizing. The key to make them have billion vocabs is exposing students to reading and listening as many as possible in comprehensible and meaningful way to make it effective. Based on the findings reflected in the graph above, students’ reading proficiency was still lack of score. Therefore, teachers need to examine their teaching methods and materials to be delivered to the students in order to enhance the level of their skills. Further, teachers need to identify what makes students encounter difficulties in reading section. In spite of just improve the teaching methods and materials, it would be more effective if the teachers also take account students’ problems in understanding the materials. CONCLUSION Task presented in reading section of English USBN in SMK NU Ungaran has fulfilled the core competence and basic competence set in Permendikbud 24/2016. It also takes literacy level and types of reading performance into consideration to organize the reading section. However, the ratio between reading comprehension task with the other task is too large. It will be more effective if each task type is fairly distributed so that the result will be more valid and reliable. The result of reading section shows that students of XII grade in SMK NU Ungaran still have lack proficiency in reading skills. Out of 50 students, 59% of them are best cope with editing task, followed by 56% in dealing with reading comprehension task, 50% of gap-filling task, 48% with short answer task and the least is 22% in vocabulary task. As we can see, the percentage of correct answer is not even close to 70% for each task type. It means that students’ reading proficiency is absolutely in need to be improved. The improvement measures can be carried out in teaching and learning activity. Teacher must ensure to prepare a proper reading material along with the task and deliver it to the student in the most effective ways regarding the students’ needs, lacks and wants. Notice that teacher should make the learning activity runs in a communicative and meaningful way to make the students motivated in learning English as foreign language. REFERENCES: Permendikbud, No. 19 (tentang Standar Nasional Pendidikan 2005). Permendikbud, No. 24 (tentang Kompetensi Inti dan Kompetensi Dasar Pelajaran Kurikulum 2013 2016). Afflerbach, P. (2016, February). Reading Assessment: Looking Ahead. The Reading Teacher, 413- 419. Brown, D. H. (2004). Language Assessment: Principles and Classroom Practices. New York: Pearson Education Inc. LANGUAGE CIRCLE: Journal of Language and Literature, 14(1) October 2019 76 Klingner, J. K. (2004). Assessing reading comprehension. Assessment for Effective Intervention (pp. 59-70). Michigan: Michigan State University. Marin, J. H. (2009). Exploring teacher's practice for assessing reading comprehension abilities in English as a foreign language. PROFILE, 71-84. Nunan, D. (1991). Language eaching methodology: A textbook for teachers. London: Prentice Hall. PISA. (2015) Results (Volume I): Excellence and Equity in Education. Retrieved from http://dx.doi.org/10.1787/9789264266490-en Widya, N., & Aman. (2015). An Evaluation of the implementation of English national examination exercise at ex-RSBI private SMA. Jurnal Penelitian dan Evaluasi Pendidikan, 217- 229.