Microsoft Word - 6. 158 Investigating Washback of National Standardized School Examination on Junior High School Students and Teachers Alief Imani Sumpada  English Department, Faculty of Languages and Arts, Universitas Negeri Semarang, Indonesia Article Info ________________ Article History: Received in 5 August 2019 Approved in 29 November 2019 Published in 30 November 2019 ________________ Keywords: washback; learning strategy; teachers’ strategies; USBN ____________________ Abstract ___________________________________________________________________ This final project aimed to investigate the washback of English National Standardized School Examination on Ninth Graders and Teachers. The research used descriptive qualitative method as its research methodology. The results showed that English National Standardized School Examination or Ujian Sekolah Berstandar Nasional (USBN) brought positive and negative effect. The students applied some learning strategies to develop their reading and writing skills to face English USBN. However, they gave less attention to listening and speaking skills. Also, the students focused to get good result on the examination because it determined their graduation and their higher level school. The teachers prepare the students to take the examination by taught the material more thoroughly. They used various sources and media to support the teaching and learning activities. However, they gave less attention on listening and speaking skills and lose instructional time. To summarize, there were trends of positive and negative washback of English USBN on the students and teacher at SMP N 1 Kertek in academic year of 2018/2019. © 2019 Universitas Negeri Semarang  Correspondent Address: ISSN 2252-6706 B8 Building FBS Unnes Sekaran, Gunungpati, Semarang, 50229 E-mail: e.teacher.ichsan@gmail.com ELT FORUM 8 (2) (2019) Journal of English Language Teaching http://journal.unnes.ac.id/sju/index.php/elt Alief Imani Sumpada / ELT Forum 8 (2) (2019) 159 INTRODUCTION In some recent years, the Ujian Nasional (UN) is not used as main factor to determine students’ graduation. In 2016 the minister of education and culture ministry, Muhajir Effendi, proposed the national examination moratorium in front the tenth commission of parliament. Nevertheless, the national Examination is still needed to be conducted for some reasons. Based on The Cabinet Secretary, Pramono Anung, Indonesia President Joko Widodo, decided to defend the national examination in Indonesia. The national examination is still needed to defend Indonesia’s ranking in Program for International Student Assessment (PISA). PISA is an international survey that observe achievement of 15 years old students on, reading literature, mathematic, and science. However, there is another well-known and nation-wide standardized test in Indonesia which is called Ujian Sekolah Berstandar Nasioinal (USBN). In the year academic 2018/2019, USBN is used as determiner of the students’ graduation. Nonetheless, the students and teacher change their focus to USBN. USBN is a measuring activity on students’ competence conducted by educational unit based on students’ competence specification, in order to obtain admission of academic achievement (BSNP, 2018). However, the USBN requires the students and teacher do some extra efforts to anticipate it. These extra efforts and special treatment given by the teacher may bring some effects to the students and the lesson. There are two kinds of effects that could possibly appear, those are positive and negative effect. The positive effect may result on teaching and learning purpose to develop English skill. In other way, the negative effect may result the teaching and learning to a test, thus lesson in class is dominated by test practice and reviewing. The effect of a test in language learning is commonly called as “washback”. The term “washback” refers to the influence of testing on teaching learning (Bailey, 1996). The word can be found in certain dictionaries and defined as “the unwelcome repercussions of social actions” by the New Webster’s Comprehensive Dictionary of the Language, and “unpleasant after-effects of an event or situation” by Collin’s Cobuild Dictionary of English Language. Also based on Cheng (2005, p. 26), washback is an educational phenomenon derived from research studies into (a) the relationship between teaching, learning, and testing. In his book Brown (2004, p. 29) defines washback as the effects of an assessment on teaching and learning prior to assessment itself, that is, on preparation for the assessment. Based on Bailey (1996, p.259) washback is generally defined as the influence of testing on teaching and learning. Wall & Alderson (1993) in Bailey (1999, p. 1) stated that It is common to claim the existence of washback (the impact of a test on teaching) and to declare that tests can be powerful determiners, both positively and negatively, of what happens in classrooms. Buck (1988) in Bailey (1999, p. 3), described washback as a natural tendency for both teachers and students to tailor their classroom activities to the demands of the test, especially when the test is very important to the future of the students, and pass rates are used as a measure of teacher success. This influence of the test on the classroom (referred to as washback by language testers) is, of course, very important; this washback effect can be either beneficial or harmful. Since washback is an effect of a test, so the effect itself can be a beneficial effect or harmful effect. Taylor (2005, p. 154) washback is generally perceived as being either negative (harmful) or positive (beneficial). Positive washback as Taylor (2005, p. 154) explained is a result when a testing procedure encourages ‘good’ teaching practice; for example, an oral proficiency test is introduced in the expectation that it will promote the teaching of speaking skills. On the other hand, the harmful or negative washback as Pan (2009) stated that washback is where the teachers will usually teach to the test, narrow the curriculum and only focus on what will be tested. Noble and Smith (1994) in Cheng, Watanabe, & Curtis (2008, p. 9) also found that high- stake testing could affect teachers directly and negatively, and that teaching test-taking skills and drilling on multiple-choice worksheets is likely to boost the scores but unlikely to promote general understanding. Pan (2009, p. 261) summarized the effect of washback as follows; Table 1 Positive Effect of Washback Positive Washback Classroom settings 1. Tests induce teachers to cover their subjects more thoroughly, making them complete their syllabi within the prescribed time limits. 2. Tests motivate students to work harder to have a sense of accomplishment and thus enhance learning. Alief Imani Sumpada / ELT Forum 8 (2) (2019) 160 3. Good tests can be utilized and designed as beneficial teaching-learning activities so as to encourage positive teaching-learning processes. Educational/societal system Decision makers use the authority power of high-stakes testing to achieve the goals of teaching and learning, such as the introduction of new textbooks and new curricula. Table 2. Negative Effect of Washback Negative Washback Classroom settings 1. Tests encourage teachers to narrow the curriculum and lose instructional time, leading to “teaching to the test.” 2. Tests bring anxiety both to teachers and students and distort their performance. 3. Students may not be able to learn real-life knowledge, but instead learn discrete points of knowledge that are tested. 4. Cramming will lead students to have a negative positive toward tests and accordingly alter their learning motivation. Educational/societal system Decision makers overwhelmingly use tests to promote their political agendas and to seize influence and control of educational systems. Thus, the writer wanted to observe the students’ learning strategies to learn English for preparing the School National Standardized Test. Based on Ellis (2003) successful learners use more strategies than unsuccessful learners. By comparing many students learning strategies and their achievement, the writer believes that results in this study can be useful for many parties. Not only the students, but also the teachers’ strategies is the focus in this study because the teachers take important role to guide the students comprehend the material. Spratt (2005, p. 8-21) stated that teacher attitude towards an exam would seem to play an important role in determining the choice of methods used to teach exam classes. METHODS The writer focuses in describing and explaining washback related to teachers’ strategies and students’ learning strategies of English USBN at SMP N 1 Kertek in the academic year of 2018/2019. There were 16 students in this study from class IX A until IX G. The students were selected without any sampling technique, however the writer considered to choose the students based on their achievement in the class. There were two teachers as the respondents in this study. Both teachers taught English for 9th grade students. The teachers have taught English for more than a decade. Due to confidentiality, the writer uses initial name in this study for Mrs ES and Mrs WRDW. In this research, the writer used descriptive qualitative method. The descriptive qualitative method is aimed to describe phenomenon and its characteristics (Nassaji, 2015, p. 129). He further explained that this research is more concerned on “what” rather than “how” or “why” something has happened. Also, one fundamental characteristic of this method is naturalistic data involvement. In addition, this research method attempts to study language learning and teaching in their naturally occurring setting without any intervention or manipulation of variables. Gall et al. (2007) in Nassaji (2015, p. 129), stated that observation and survey tools are often used to gather data. The writer used some tools to gather the data. One is a summative rating scale, popularly known as a questionnaire, an inventory, or (less accurately) a survey. The most often used strategy scale around the world at this time is the Strategy Inventory for Language Learning (SILL, Oxford, 1986-1990) in (Oxford, 1995). The SILL (Oxford 1986-present) was first designed as an instrument for assessing the frequency of use of language learning strategies by students at the Defense Language Institute in Monterey, California. There are two revised versions of the SILL, one for foreign language learners whose native language is English (80 items) and the other for learners of English as a second or foreign language (ESL/EFL, 50 items) (Oxford, 1995). In this study, the writer used 50 items version of SILL. Alief Imani Sumpada / ELT Forum 8 (2) (2019) 161 The writer translated the original version (English) into Bahasa Indonesia in order to anticipate misunderstanding. The items were reduced and modified to 22 items, however the 6 sub- scales (memory, cognitive, compensation, metacognitive, affective, and social) were maintained to keep the core of the SILL. The writer also used interview guidance. The questions asked to the students were related to their learning strategies. Bahasa Indonesia was used to anticipate misunderstood. It was ranged from questions related to their English background and habitual related to English to more specific questions related to their learning strategies. There were 20 items and 6 sub-items in the students’ interview guidance. The questions asked to the teachers were related to their teachers’ strategies. The questions were also in Bahasa Indonesia. It was also ranged from questions related to their opinion toward USBN to their method in developing the students’ reading and writing skill. The interview guidance consisted of 20 questions. The writer adopted a data analysis procedure proposed by Miles and Huberman (1984). This analysis procedure consist of three steps, those steps are data reduction, data display, and conclusion. Data reduction is process of selecting, focusing, simplifying, abstracting, and transforming the ‘raw’ data that appear in written-up field notes. The writer screened the less valuable data and picked valuable data, then the writer processed the data into understandable data. The data were divided into students’ learning strategy and teachers’ teachers’ strategies. Data display is an organized assembly of information that permits conclusion drawing and action taking. The collected data from the instruments were presented in figures, and tables to grasp easier the data meaning. The conclusion drawing and verification is to decide what things mean, is noting regularities, patterns, explanations, possible configurations, causal flows, and propositions. The reader writer explained in sentences the data meaning based on the processed with supporting tables and charts of the data from previous step. According to Miles and Huberman (1984, p.23) the diagram of data analyzing procedure as follows; Figure 1. Diagram of data analyzing procedure Descriptive technique was used in this study to report the data in. The basic procedure in reporting the results of a qualitative study is to develop description and themes form the data to present these description and themes that convey multiple perspectives from participant and detailed description of the setting or individuals (Creswell, 1994, p. 204). Based on Loeb et al. (2017), descriptive analyses are central to almost every research project. She continued to explain, whether the goal is to identify and describe trends and variation in populations, create new measures of key phenomena, or simply describe samples in studies aimed at identifying causal effects, descriptive analyses are part of almost every empirical paper and report. The writer described the interview and observation in sentences to explain the response of the subject of study. The statistical results were presented in sentence after processing by simple statistical method. Data collection Data reduction Data display Conclusion/ Drawing Verificaiton Alief Imani Sumpada / ELT Forum 8 (2) (2019) 162 FINDINGS AND DISCUSSION Washback of the National Standardized School Examination on Teachers’ strategies Based on the data gained from the questionnaire, interview, and teachers’ lesson plan, the washback related to teachers’ strategies to prepare the students of SMP N 1 Kertek to take English USBN in the academic year of 2018/2019 is both positive and negative. Pan (2009) stated that tests induce teachers to cover their subjects more thoroughly, making them complete their syllabi within the prescribed time limits. The teachers’ responses on the questionnaire shows that USBN preparation is used to teach the materials more thoroughly. Some materials related to reading and writing skills were taught in detail, since reading and writing skills were dominant used in English USBN. Both teachers used different method to teach the students. Mrs. WRDW used Genre Based Approach to help the students understand many kind of text deeper. The approach is based on a teaching-learning cycle where strategies such as modelling texts and joint construction (Firkins et al., 2007). The GBA approach was elaborated with the Kurikulum 2013 which requires the students to be more active learner. Mrs. WRDW argued that group discussion was effective to develop students’ skills. She also stated that reading and writing skills were developed more intensively in the extra class. The material both in regular and extra class were integrated, thus the students can comprehend the material deeper. Mrs. ES, however, used different approach to teach her students. She used scientific approach to guide the students to comprehend the material more systematically based on the students’ book. The students were guided to comprehend the material in their book and discuss it in detail including some aspects of English related to the text. Both teachers stated that they taught the material based on SK & KD in regular class and did more exercises in the extra class. In the extra class, they taught less theory and gave the students more exercises. Some early meetings were used to refresh students’ understanding on some materials and the rest of the meetings were used to discuss and do exercises. Both teachers used some media to teach the students. For example, Mrs WRDW used laptop and LCD screen to conduct the lesson. Moreover, the school suggested the students to have exercise books of four national examination subject, which are Bahasa Indonesia, English, Mathematic, and Science. The books were often discussed in the extra class, also in some meetings in regular class after SK & KD materials were given. The students’ books were also used to give more examples and exercises on the given material. Mrs ER used some books to give the students more variations of the question item. Some previous test papers were also used by both teacher. The test papers were given to the students in order to help the students connect the material with question item. The test paper were used in the extra class program and it was only used in some regular class meetings after all material based on SK & KD were given. Furthermore, the teacher stated that the USBN preparation did not interrupt the regular class. The teachers taught the students based on the curriculum, syllabi, Standar Kompetensi & Kompetensi Dasar. USBN preparation was used to gain students’ deeper understanding on the material, since the English USBN covered the material form 7th until 9th grade. However, there was also trend of negative washback on teachers’ strategies. Based on (Pan, 2009), negative washback happens when the teacher have to narrow the curriculum and lose the instructional time leading to “teaching to the test”. The teachers gave less attention to the materials of listening and speaking skills because both skills were not tested in English USBN. Also, some meetings in the regular class were used to do exercises drilling because the teacher considered that they should give more exercises for the students to prepare the USBN. Furthermore, the school decided to start the Extra class program from the middle of first semester. In addition, some meetings in regular class were used as USBN preparation which was focusing on exercises drilling. Mrs ES stated that students’ comprehension on previous year materials were not good enough. She argued that the USBN specification were quite high and the Kurikulum 2013 did not cover the specification in detail. Based on her experience, years ago when she taught 8th grade students, she thought that Kurikulum 2013 were designed to grow students’ interest on English and the learning activity somehow were not supposed to discuss the material comprehensively. Thus, when the students have to face the UN & USBN, the teacher had to guide their students to catch up wide gap between students’ understanding and the specification of the test. Mrs WRDW also argued that the old same problem which the students dealt with was vocabulary. Not only the low motivated students, but also high motivated students experience the Alief Imani Sumpada / ELT Forum 8 (2) (2019) 163 same. Thus, the extra classes were grouped based on the students’ achievement on the try out test in order to give precise treatment for the students. Mrs WRDW admitted that she ever asked the students to bring a text with some question items based on text. Then, she asked the students to exchange their texts, of course the students have to understand their own text before they give it to other student. As a result, the students could get many texts and exercises without spending much time to find it. In addition, both teachers agreed if English was tested nationally, however they considered that the specification of the test was quite high compared to the students’ skills. The teacher also admitted that they have to do extra effort to teach the students. The extra effort had to be done since the gap between students’ material comprehension with USBN specification was quite wide. Yet, some low motivated students did not make extra effort to study harder. For example, some low motivated students had low motivation to develop their vocabulary, as a result the students found difficulty to understand a text and question related to the text. Washback of the National Standardized School Examination on Students’ Learning Strategies Based on the data gained from the questionnaire and interview, the washback on students’ learning strategies to develop reading and writing skills at SMP N 1 Kertek in academic year of 2018/2019 is both positive and negative. According to Pan (2009), tests motivate students to work harder to have a sense of accomplishment and thus enhance learning. The result from questionnaire and interview showed that they studied harder to develop their reading and writing skills, even though English was not their favorite subject. English is a foreign language in Indonesia and it was considered challenging subject for the students. The time for them to practice or use English were not much because English was rarely used in the daily communication. The lesson in the school also limited based on the lesson plan. Based on that reason, the students admitted that English USBN was needed extra preparation. Their strategy to prepare English USBN were mostly by study harder either in the school or outside school. USBN result was used as consideration of their graduation and to enrol to Senior High SchoolSome students admitted that they dealt with vocabulary. Most of them had difficulty when they found new word and since the word was a key to answer the question. Most of them used the dictionary to find out the meaning, then the students memorized the words by repeating it a lot. When they found new word and it was not allowed to open dictionary, guessing the meaning based on the sentence and context was the strategy. In addition, trying to find the synonym of the word was also applied by the students. The second common difficulty was grammar which is related to tenses. Some students admitted that they experienced the difficulty on grammar. Most of the students answered that they reviewed the tenses to evaluate their mistake and did a lot of practices to memorize the formula of the tense. Moreover, many students responded that they study English outside the school from many media, such as Movies, television, music and online application. In some cases, they connected some words which are similar to words in Bahasa Indonesia. In addition, when they did not understand meaning of a word, they would guess the word by paying attention to the context or sentence. Also, they would use another similar word if they could not find the correct one. The students sometimes reviewed the material related to USBN specification. Most of the students “collaborated” with other people to help them preparing English USBN. Many students admitted that they joint tutoring class, eventhough the school have provided extra class program to help them face USBN. The students did not only actively consulting with the teacher or tutor within the lesson, but also outside the lesson. By doing so, they could get extra time to deal with the difficulty in learning USBN material. The students also showed good time management to balance their time for studying and refreshing. The effective time to learn was different among them, however they spent 1 hour up to 1.5 hours to study for English. In the weekend, they spent the time for refreshing by watching movies, playing games, doing sports, and many activities to relieve the stress, but some students also spent the time to study after refreshing. They also appreciated themselves for every learning accomplishment reached. It is important to keep their motivation and confidence. However, negative washback also appeared when students may not be able to learn real-life knowledge, but instead learn discrete points of knowledge that are tested (Pan, 2009). Since the students were motivated to get good mark on USBN, they focused only on developing their reading Alief Imani Sumpada / ELT Forum 8 (2) (2019) 164 and writing skills. Thus, listening and speaking were given less attention because both skills were not tested in USBN. Most of them only learnt listening and speaking skills on the regular class which is based on syllabi, SK and KD. They also did not have enough time to practice both skills after school. Even though many students admitted they watch movies and listening music in English to develop their vocabulary, it was not real-life knowledge that was expected. The exercises given to the students also triggered negative washback. Based on Pan (2009), cramming will lead students to have a negative positive toward tests and accordingly alter their learning motivation. Based on the interview a month before USBN, most of the students admitted that they were about 60-70% readiness to prepare USBN. They also said that they had some difficulties and needed to study again to prepare USBN, since the results of USBN was used to determine their graduation. The result of USBN was combined with UNBK as a requirement to enroll in higher level school. The better the result, the opportunity to enroll at favorite School was also higher. Thus, the students’ intention was not purely to understand the material instead to get good mark on the examination. CONCLUSION The ninth grade students of SMP N 1 Kertek used vary strategies to improve their reading and writing skills to face USBN. According to Pan (2009), tests motivate students to work harder to have a sense of accomplishment and thus enhance learning. The students did extra effort to develop their reading and writing skills by spending more time to study, for example many students joint tutoring program outside the school even though the school have provided the extra class program. The students also tried to develop some aspects of English such as vocabulary and grammar by memorizing new words and doing a lot of exercises. They also managed their time to balance between studying and refreshing, thus they could handle pressure of the examination. However, the students also showed trend of negative washback on English USBN. According to Pan (2009) negative washback also appeared when students may not be able to learn real-life knowledge, but instead learn discrete points of knowledge that are tested. Even though the students studied hard to develop their reading and writing skills, they only focus on English USBN preparation. The strategies they applied were supposed to make them ready face English USBN. Also, listening and speaking were given less concern because English USBN only tested reading and writing skills. They only practice both skills in the regular class which were based on SK and KD. Thus, it could be seen that the students did not consider to practice their English skills in real-life context. In addition, the numbers of exercises which were given to students also triggered negative washback. Based on Pan (2009) cramming will lead students to have a negative positive toward tests and accordingly alter their learning motivation. The students motivation to develop their reading and writing skills was to get good score as possible on English USBN, thus it alter their motivation which was supposed to develop the real-life skill of reading and writing. Pan (2009) stated that tests induce teachers to cover their subjects more thoroughly, making them complete their syllabi within the prescribed time limits. Both teacher used English USBN preparation to teach the materials in detail, also English USBN preparation also did not interrupt the lesson based on SK and KD. The English teacher used methods to teach the students which were supposed to develop students’ reading and writing skills. Both teachers used different teachers’ strategies to develop students’ reading and writing skills. MR ES used scientific approach to develop students’ reading and wiritin skills, however MR WRDW used Genre Based Approach. However, there was a trend of negative washback on teachers’ strategies. According to Pan (2009), negative washback happens when the teacher have to narrow the curriculum and lose the instructional time leading to “teaching to the test”. The teachers gave less attention to the materials of listening and speaking skills because both skills were not tested in English USBN, also the school decided to start the extra class program from the middle of first semester. In addition, some meetings in regular were used as USBN preparation which was focusing on exercises drilling. REFERENCES Alderson, & Wall. (1993). Does Washback Exist? Applied Linguistic, 115-129. Bailey, K. M. (1996). Working for Washback: A Review of The Washback Concept in Language Testing. SAGE, 257-279. Alief Imani Sumpada / ELT Forum 8 (2) (2019) 165 Bailey, K. M. (1999). TOEFL Monograph Series Washback in Language Testing. New Jersey: Educational Testing Service. Brown, H. D. (2004). Langauge Assesment Principle and Classroom. In H. D. Brown, Langauge Assesment Principle and Classroom (p. 29). New York: Pearson Longman. BSNP. (2018, February 7). Retrieved April 8, 2018, from http://bsnp- indonesia.org/2018/02/07/POS-USBN-2017-2018 Cheng, L. (2005). Changing language teaching trough language tesing: A washback study. Cambridge: Cambridge University Press. Cheng, L., Watanabe, Y., & Curtis, A. (2008). Washback in Language Testing : research content and method. New Jersey: Lawrence Erlbaum Associates. Creswell, J. W. (1994). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Thousand Oaks, CA, US: Sage Publication, Inc. Ellis, R. (2003). Second Language Acquisition. Oxford: Oxford University Press. Firkins et al. (2007). Teaching Writing to Low Proficiency Students. ELT Journal, 341-352. Loeb, S. (2017). Descriptive Analysis in Education: A Guide for Researchers. Washington, DC: National Center for Education Evaluation adn Regional Assistance. Messick, S. (1996). Validity and Washback in Language Testing. Educational Testing Service. Miles & Huberman. (1984). Qualitative Data Analysis: A Sourcebook of New Methods. California: SAGE Publicaiton Inc. Nassaji, H. (2015). Qualitative and descriptive research: Data type versus data analysis. Language Teaching Research, 129-132. Oxford, R. L.-S. (1995). Assessing The Use of Language Learning Strategies Worldwide with The ESL/EFL Version of The Strategy Inventory for Language Learning (SILL). Elsevier, 1-23. Pan, Y.-C. (2009). A Review of Washback and Its Pedagocical Implication. VNU Journal of Science, 257-263. Spratt, M. (2005). Washback and the classroom; the implications for teaching and learning of studies of washback from exams. Language Teaching Research, 5-29. Taylor, L. (2005). Washback and impact. ELT Journal, 154-155.