21 EEJ 9 (4) (2019) 541 - 557 English Education Journal http://journal.unnes.ac.id/sju/index.php/eej The Formative Assessment Backwash in English Instruction at Kristen Nusantara Vocational School Kresna Rahma Aji1, Rudi Hartono2 1. SMK Kristen Nusantara Kudus, Indonesia 2. Universitas Negeri Semarang, Indonesia Article Info ________________ Article History: Recived 20 June 2019 Accepted 13 August 2019 Published 23 December 2019 ________________ Keywords: Formative Assessment Conversation Cycle, Elicitation, Feedbacks ____________________ Abstract ___________________________________________________________________ This case study focuses on covert backwash of formative assessment in English instruction and aims to provide framework of teacher’s instruction realizations in administering formative assessment.The subject was a teacher selected by certain established categories. Data collection techniques are observational recording, interview, and questionnaire. The instruments of data collection are checklist of lesson plan, interview, observational recording, and questionnaire. The findings consist of twenty claims. The conclusions are (1) teacher’s elicitation as key point, (2) elicitations to develop cognitive, (3) numbers of elicitation depending on the existence of students’ responses, (4) deeper involvement by teacher’s feedbacks, (5) no gap during grammar class orientation, (6) slow response, (7) active and interactive demands for the teacher, and (8) life on-going process of learning. The research suggests English teachers (1) to implement formative assessment conversation (2) to implement the claim because it is helpful in developing student cognition; (4) to provide sometimes of FACC absence for students to get ready into the next step and (5) the non-verbal attributes seen on the teacher facilitated the realization of formative assessment conversation to be understood by the students. This research is only limited on teacher without seeing the backwash on the students’ sides. Since it was sought to see the covert backwash, then the unit of analysis was classroom activity, specifically in formative assessment conversation cycle, in which it was administered orally. Further investigation is expected to see the backwash on other types of formative assessment administration. © 2019 Universitas Negeri Semarang  Correspondence Address: Sunggingan, Central District, Kudus (59317) E-mail: ajisakatranslation@gmail.com p-ISSN 2087-0108 e-ISSN 2502-4566 Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 542 INTRODUCTION The use of assessment in learning is always connected each other, even to teaching and learning. It links between a state of content standard to realization of the content (Hakuta and Jacks, 2009). Assessment results may be used for various purposes, included to form students’ understanding or knowledge construction (Sheerman et al., 2008). It is seen as diagnosis to find out students’ learning difficulites and to improve students achieving better competences and masteries by using both incoming stimuli and existing knowledge instead of merely absorbing transmitted knowledge (Berlak, 1992; Gipps, 1994; Wiliam, 1994 in Bell and Cowie, 2001a). The assessment having such benefit is called formative assessment (Evans, Zeun, and Stanier, 2014). Other benefit of formative assessment deals with its formal and informal natures without requiring any written recording during its implementation by eliciting students and immediating use of knowledge in instruction – denoted to create learning (Ruiz- Primo and Furtak, 2007; William, 2010 in Vingsle, 2014). Ruiz and Furtak formulate a model to notify the use of informal formative assessment known as ESRU cycle to seek information during teaching an learning through interaction between teachers and students, and then to be used in the teaching and learning process simultaneously to foster knowledge of the student. It is frequently an unplanned occurrence done by teacher through his professional knowledge and skill to mediate, keep, help, and create situation for students to practice (Vingsle, 2014). Thus, it becomes very important for English teachers of vocational high school to find well-planned and organized way to facilitate learning process (Derakhshan & Shirmohammadli, 2015; Ministry of National education, 2006 in Newhouse & Suryadarma, 2011). Situations due to formative assessment implementation as explained previously are called backwash of formative assessment. Backwash is defined as negative effect and or direct or indirect of certain assessment practices used to change students’ learning or teaching methods, generally categorized into positive or negative (Luke, 1995; Othman, 2007; Watkins, Dahlin, and Ekholm, 2005). When the assessment is used to reflect the skills and content taught in classroom, it is considered as positive backwash (Bedford, 1995) The implementation of formative assessment is actually suggested by 2013 curriculum. Through the curriculum, vocational school students are educated to seek the knowledge by themselves through observing, questioning, exploring and experimenting, associating and analyzing, and communicating, known as scientific approach (Kemendikbud 2013 in Nidhra & Dondeti, 2012). Its learning activity can be done into some models such as discovery learning, problem based approach, and project based approach (Purnawarman, Ratnaningsih, & Gunawan, 2017). The implementation of formative assessment and the expectations of 2013 curriculum are done by teacher instructions. Several studies undertaking teacher instruction: Y. huei Wang, Chao, and Liao (2011), Doiz, Lasagabaster, and Sierra (2011), Pawan and Craig (2011), Torres-Olave (2012), and Yang, Gamble, Hung, & Lin (2014) and studies about backwash of assessment and formative assessment by Hult & Liljeström (2011), Falk (2012), and Syafei (2012) have not investigated backwash of formative assesment to teacher instruction. Dealing with backwash, according to Wang, Yan, and Liu (2014), there are three backwash models from experts to ease further researchers’ understandings started from: Hughes’ basic model (1989), Anderson and Wall’s model (1993), and Prodromous’ Overt- Covert model (1995). The backwash of assessment to teacher may influence pedagogic principles occurring in the class, Overt-covert model (Prodromou, 1995). Formative assessment is understood to be applicable in every kind of assessment inside of learning process, applied both written or oral inside of teacher – students’ interaction (Weurlander, Söderberg, Scheja, Hult, and Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 543 Wernerson, 2012). In classroom interaction, formative assessment dynamically involves teacher and student talks adjusted to best meet the current learning needs for their students through informal, discrete, and frequent interaction (Broussard, 2014). The quality of the assessment is strongly depended on the quality of classroom interaction (Antón, 2015). In another hand, backwash is defined as negative effect and or direct or indirect of certain assessment practices used to change students’ learning or teaching methods, generally categorized into positive or negative (Luke, 1995; Othman, 2007; Watkins, Dahlin, and Ekholm, 2005). When the assessment is used to reflect the skills and content taught in classroom, it is considered as positive backwash (Bedford, 1995). In this thesis, formative assessment is assumed to have backwash toward English instructions of the teacher seen from formative assessment conversation and teacher’s non- verbal communication. Formative assessment conversation is defined as daily instructional dialogues embedding assessment into an activity already occurred in the classroom, consisting of teacher elicit question, students’ responses, teacher recognition on students’ responses, and information collected uses to learning in which continuously modified while learning is taking place (Duschl, 2003; Duschl and Gitomer, 1997 in Ruiz-Primo, Furtak, and Araceli Ruiz-Primo, 2006). This cycle will be used as methodological triangulation for selecting classroom talk categorization system called Fiander Interaction Analysis System (FIACS). In another hand, this thesis also anticipates any non-verbal behavior in the class contributing to the backwash of formative assessment. Therefore, an analysis system, Zoric’s and Schmid’s non-verbal category (2007), dealing with non-verbal behavior having the backwash will be also needed. METHOD This case study had purposes investigate certain phenomenon in depth and to provide framework, in this case, the backwash of formative assessment in English instruction (Given, 2008; Yin, 2014). Therefore, the researcher had role as a human instrument to observe, to interview, to mediate data collection, to analyze, to encode, and to report (Watson, 2010). The main unit of analysis is classroom activity, such as instruction, classroom talks, teacher and student activities (Lom, 2012). The data are formative assessment conversation (Morrison, 2015) and nonverbal communication (Krauss, Chen, and Chawla, 1996). The research subject was a teacher selected by established categories of good teacher from various combinations of possible traits (Scates and Douglas, 1950). To ease the process of selecting the teacher, there was a need to examine whether the subject fitted the study objectives (Martínez-Mesa, González-Chica, Duquia, Bonamigo, and Bastos, 2016). To protect her privacy, a pseudonym was given (Pan and Li, 2013). Data collection techniques are observational recording, interview, and questionnaire. The instruments of data collection are checklist of lesson plan, interview, observational recording, and questionnaire. Checklist was used to confirm the essential elements and facilitated researcher ensuring compeleteness in carrying out task ( Garloch, 1947; Frels, Sharma, Onwuegbuzie, Leech, and Stark, 2011; Hagströmer, Ainsworth, Kwak, and Bowles, 2012). Dealing with observation, a pilot study was initially done (Bartlett, 2013; J. Fraser, Fahlman, Arscott, and Guillot, 2018; Teijilingen and Hundley, 2001). On the pilot test, this thesis employed observer as participant to collect the information better in non-threatening situation (B. Kawulich, 2014). However, participant observation had weaknesses such as probability of the observer to lack of interest or to miss a certain occasion (B. B. Kawulich, 2005), then it was decided to use video recording. This observational recording was considered to be more flexible and facilitate the investigation to get detail and precise data (Bowman, 1994; Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 544 Halimaa, 2001; Penn-Edwards, 2004). The observational guideline used was H. M. Inamullah et al guideline (2011) done by writing down the categories of FIACS developed by Bailey (1974) and Zoric, Smid et al (2007) nonverbal communication categories. The procedure to collect the data by recording was also adopted from Inamullah et al’s participant observation guideline: Standardized open-ended interview employs worded identical questions to allow participant contributing much detailed information as they desire and allow researcher asking probing questions as means of follow-up (Turner, 2010; Gall, Gall, and Borg, 2003 in Turner, 2010). Turner (2010) suggests to prepare interview by selecting participants, constructing effective research questions, and using useful questions without making earlier assumption. The questionnaire was carefully designed started from its question wording, choices of response formats, question sequences, questionnaire formats, and aspects of presentation (McColl et al., 2001). In this research, the questionnaire was used to ease the process of selecting teacher as the subject. Techniques of analyzing data were transcribing (Roberts, 2006) and co-ocurrence coding (Guest & Mclellan, 2003). The coding covers three analysis systems: FIACS, ESRCU, Zoric and Smid’ model. Triangulation was done by using expert judgment adopted from Retnawati and Mujiyanto (2015). The content of this study is presented based on SRQR model by by O’Brien, Harris, Beckman, Reed, and Cook (2014). RESULTS AND DISCUSSION Based on observational recording, transcription, and interview, twenty claims were found: 1) the student readiness, 2) the student prior knowledge, 3) teacher recognition, 4) positive student engagement as feedback, 5) teacher personal feeling, 6) series of elicitation, 7) cognitive ability and cognitive source, 8) direct instruction, 9) discourse filler, 10) instructional orientation and teacher’s attitudes to silent period, 11) student idea development, 12) teacher’s question tag, 13) theme based instruction, 14) lowering complexity of elicitation question, 15) discourse marker, 16) purposes of elicitation, 17) purpose of calling student name, 18) revised language instruction, 19) absence of formative assessment conversation cycle, and 20) formative assessment conversation cycle to check current cognitive level. The Students’ Readiness The teacher spent time directing, sharing information, and delivering instructional orientation to brainstorm the students about the given material (Observational recording, 03/09/18; Transcription, 23/10/18). When they know what they learn, they will be motivated and ready to learn. It is in line with study done by Kırmızı (2015) and the teacher statement based on interview (Interview, 05/11/18). This finding about students’ readiness is in line with findings of Agherdien, Mey, and Poisat (2018) and Kapur (2015). Students’ Prior Knowledge The teacher to find the current understanding level of the students after listening to her explanation so she knew what to do in the next teaching sequence by an elicit question(Observational recording, 03/09/18; Transcription, 23/10/18). It is in line with Aksan and Çelikler (2015) telling about preliminary knowledge of students before teaching. The teacher also argued that the students’ prior knowledge influenced learning outcome in the end of the class (Interview, 05/11/18). It is also in line with Hailikari, Katajavuori, and Lindblom-ylanne (2008). Teacher’s Recognition The teacher’s recognition had purpose to positively reinforce the students, as for example, by saying Good (Observational recording, 03/09/18; Transcription, 23/10/18). This finding about positive reinforcement in teacher’s recognition is also supported by Kelly and Pohl (2018) and Rumfola (2017), showing that positive reinforcement made students Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 545 feeling better and being ready to learn. Positive Student Engagement as Feedback The teacher directed a student individually to answer her elicit question (Observational recording, 03/09/18; Transcription, 23/10/18). It is a positive feedback engagement by involving one of the student to think, making him as the model for the others to actively engage (Hattie and Timperley, 2007; Rivers et al., 2017). Teacher Personal Feeling The teacher accepted HR’s feeling then she expressed her personal feeling by saying “You are brilliant” (Observational recording, 03/09/18; Transcription, 23/10/18). This realization of teacher’s recognition is influenced by the previous step of formative assessment cycle by expressing her personal feeling (Saunders, 2013; Zembylas, 2003). It is also a positive feedback, reinforcement, and encouragement to an individual. Series of Elicitation When an elicitation was not answered by the students, the teacher tried to elicit again (Observational recording, 03/09/18; Transcription, 23/10/18). It influences the cycle of formative assessment conversation to have series of elicitations, proving well-structured questions to get a certain variable, the response of the student in this case (Gosling, 2018). In this case, the response of the students was uncertain and could not be predicted (Quigley, Colson, Aspinall, and Cooke, 2018). Cognitive Ability and Cognitive Source One student at a time shared his response while the others keeping silent (Observational recording, 03/09/18; Transcription, 23/10/18). It happened since the others had not developed cognitive abilities and resources to bear and to effect on the subject matter (Berkowitz and Stern, 2018; Gill and Prowse, 2014). Direct Instruction Based on (Observational recording, 03/09/18; Transcription, 23/10/18), the direct instruction was simply directing the students to look at the example consisting new information, maintaining her instructional goal and promoting the process of knowledge instruction (Boleware, n.d.; Kenny, 1980; Liem and Martin, 2013). Discourse Filler A discourse filler appeared (Observational recording, 03/09/18; Transcription, 23/10/18) with no intention to mediate the learning but instead to plan the next utterance and manage the interaction of classroom sequence(Erten, n.d.; Navarretta, 2015). Instructional Orientation and Teacher’s Attitude to Silent Period The teacher expressed instructional orientation to review current knowledge, allowing the students to engage in learning and to understand more complex matter (Celal Akdeniz, 2016; Lazarides and Rubach, 2016; Marzano, Gaddy, and Dean, 2000). She asked question and directed the whole class to share their opinion (Observational recording, 03/09/18; Transcription, 23/10/18). A silent period occurred (Observational recording, 03/09/18; Transcription, 23/10/18), it had purposed to allow the students understanding and assimilating the message of the question (Gilmore in King, p.39, 2011; Vassilopoulos and Konstantinidis, 2012). Developing Students’ Idea There was a formative assessment conversation part – using students’ response – also functioned as elicitation in the end of teacher’s utterance, it influenced to the subsequent instructions (Observational recording, 03/09/18; Transcription, 23/10/18). The English instructions had purpose to encourage one of the students volunteering himself, to get his identity, and to develop his own idea and to provide chance to be active in the class (Noor, 2014; Sharpe, 2008). Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 546 Tag Question of the Teacher On observational recording (03/09/18) and Transcription (23/10/18), the teacher asked a tag question - a follow up based on the teacher’s previous statement - to confirm the students’ understanding by providing space for student to contribute (Jovanović and Pavlović, 2014; Kimps, Davidse, and Cornillie, 2014). The tag question in which was influenced by the formative assessment conversation had purposes to develop critical thinking skills and construct the students’ knowledge (Atwood, Turnbull, and Carpendale, 2010). Theme Based Instruction After the teacher assumed the students to have already understood the linguistic features of the material, she then continued to construct it into a complete text (Observational recording, 03/09/18; Transcription, 23/10/18). However, before she got into the complete text, she showed Prambanan temple picture as a theme of the learning material (Observational recording, 03/09/18; Transcription, 23/10/18). According to Alptekin, Erçetin, and Bayyurt (2007), this kind of instruction, known as theme based instruction or content based instruction, focuses on meaning rather than form so it will develop cognitive and linguistic factors simultaneously. Moreover, theme based instruction also provide more integrated activities and tasks (Cheung & Yang, 2009). Lowering Complexity of Elicitation Questions Teacher’s repeated question occured since there was no response noticed, to lead and get better understanding, the teacher adjusted the question by lowering the level or complexity of elicit question into tag question – involving, making, enabling, and training them thinking to higher order thinking (Abosalem, 2016; Arnellis, Jamaan, and Amalita, 2018; Fianti and S, 2017; Freahat and Smadi, 2014; Riandari, Susanti, and Suratmi, 2018; Sari, Budiyono, and Slamet, 2018; Suprapto, Fahrizal, Priyono, and K., 2017; Tikhonova and Kudinova, 2015; Wun and Harun, 2017). Then, she repeated it to ease the students in answering the question (Observational recording, 03/09/18; Transcription, 23/10/18). Discourse Marker Another backwash was in the form of discourse marker of the teacher (Observational recording, 03/09/18; Transcription, 23/10/18). It imposed a relationship between some aspects of separated discourse segment, such as a turn taking signal or beneficially guiding for explaining the intention in communication (Alraddadi, 2016; Chapet, 2009; B. Fraser, 1991). This embedded discourse marker in English instruction also has textual function, aiding the speaker to structure and organize the sequence of teaching, contributing to the development of a higher-level explicature and encoding a procedural constrain on context selection (Schourup, 2011; Talebinejad and Namdar, 2011). Purposes of Elicitation The backwash of formative assessment are mostly seen after initial elicitation of each cycle (Observational recording, 03/09/18; Transcription, 23/10/18). An elicitation is not only a teacher-posted question to stimulate, provoke, and make students listening carefully, analyzing their thought, thinking critically, initiating discussion, and reviewing materials but also to incorporate students’ thought into teachers’ instructions realized in five categories (Fitriati, Isfara, and Tristanti, 2017; Jafari, 2013; Kopf and Effelsberg, 2007; Qi and Sykes, 2016; Sahin, 2007). Thus, it allows teacher to express more instructions until the purposes of elicitation categorized as: to inform, confirm, agree, commit, repeat, and clarify are achieved (Jafari, 2013). Purposes of Calling Student Name This claim could be seen when there was a student raising his hand, the teacher’s talk had intention to instruct the student to start speaking (Observational recording, 03/09/18; Transcription, 23/10/18). This instruction was done by calling the student’s name to foster the learning and to make the student feeling Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 547 appreciated and exist (Glenz, 2014; Middendorf and Osborn, 2002). Revised Language Instruction This claim could be seen when the teacher gave chance to the student by asking a question, any other? (Observational recording, 03/09/18; Transcription, 23/10/18) Then, in this elicitation, the teacher indirectly criticized the whole class, by expecting the students to be sensitive to the criticism, by purpose to let them coping with the problem and to improve (Mizokawa, 2013, 2014; Skipper and Douglas, 2015). There was also unsuccessful bit of language instruction revision (Observational recording, 03/09/18; Transcription, 23/10/18). It had indirect purpose to let the students recognizing the correct language structure in the question (Tomková, 2013; W. Yang, 2010). While revising her instruction, it was accounted to have spaced-repeated question, separated by other turn takings, provided processing time to boost their performance (Observational recording, 03/09/18; Transcription, 23/10/18; Horness, 2016; Rouhi, Nabavi, and Mohebbi, 2014; Smith, 2012). This repetition was responded well by the student’s answer although after that the teacher requested a clarification to rephrase the answer clearly by encouraging him to speak again (Baradeyah and Farrah, 2017; Purver, 2004). Absence of Formative Assesment Conversation Cycle The condition where the teacher gave the students work to perform and demonstrate their past experience story in front of the class (Observational recording, 03/09/18; Transcription, 23/10/18). In this part, there was not any FACC observed and accounted but some teacher corrections, teacher – student silent periods, and the use of discourse marker existed (Observational recording, 03/09/18; Transcription, 23/10/18). FACC to Check Current Cognitive Level Based on Observational recording (03/09/18) and Transcription (23/10/18), there was only one FACC to check the current cognitive level of the student. Since, there was a slight of silent period, the teacher uttered an instruction as a clue for the student. Then, after the student answered as expected, the teacher recognized the response by encouraging him, inviting the whole class to give applause (Observational recording, 03/09/18; Transcription, 23/10/18). In this cycle, it was not found any using student’s response step since no gaps were found during previous steps of the cycle (Observational recording, 03/09/18; Transcription, 23/10/18). The Realization of Teacher’s Instruction in Administering Formative Assessment The teacher’s instruction realizations in administering formative assessment were done indirectly. It was initially done to the whole class in each FACC. The forms of these indirect instructions were elicit questions. Those questions were mostly in the form of WH- questions, functioning to get the information from the students. However, when the first elicitations were not responded well, the teacher added more-simpler indirect questions, or would change the instructions into direct instructions to get the responses. Then, when the responses were as expected, the teacher appreciated the students by using indirect instructions. All of those episodes were backwash of formative assessment conversation in teacher’s English instruction to decrease students’ apathy emergence (Abbas and Thaheem, 2018). There were also some FACCs in which were entailed by other FACCS. They were not considered to have gaps but instead the teacher intended to involve the students into deeper discussion. By playing this role as what Winters and America (n.d.) refers as a circle keeper could students connecting ideas by thinking in a new ways or insight. The used of discourse markers were also noted in the findings. They were uttered by the teacher to give her chance in moving to the next sequence of teaching, activity, or giving her chance to think. Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 548 Teacher’s Management in Following Up the Gaps There was no gap or unexpected responses found in the findings. It happened due to what Marashi and Tehran (2018) called as traditional-grammar oriented class, causing no content information exchange. However, there were some slow responses of the students and silent periods. To overcome these, the teacher facilitated and eased the students by orally uttering clues. The Realization of Nonverbal Communication Attributes According to Zoric, Smid et al (2007), there are 10 categories of nonverbal communication. After analyzing, there were found: chronemics, kinesics, oculesics, physical appearance, proxemics, silence, and vocalics. Among those categories, the most frequent appearing categories were chronemics and kinesics. In terms of chronemics, the finding of this thesis aligns with Muchemwa (2013) showing that the teacher perceived time, structured it, and reacted to it to set the stage of classroom communication. The teacher admitted she intentionally spoke with normal speech rate to let the student understand the instruction and the delivered material. Meanwhile, in case of kinesics, she admitted it was her habit to do so. Therefore, she did not realize for what specific purposes she did it. However, due to teacher’s kinesics, it was believed she could manage the class well and formed students’ communicative competence in nature of paralinguistic (Antes, 1996; Barabar and Caganaga, 2015). CONCLUSION The conclusions are (1) they key point of formative assessment was teacher’s elicitation because teacher knew the ideal cognitive level of students to have understood the material given; (2) the elicitations were not only merely asking questions but they were intended to involve the students thinking and developing their cognitive level; (3) the numbers of elicitation might vary, depending on the existence of students’ responses. When the was an absence of the response or students’ apathy, it might trigger more elicitations to keep the students following the learning; (4) the feedbacks given by teacher were not only limited by gaps found during formative assessment cycle, but they were also given to involve the students into deeper discussion; (5) the gap was no gap found during the learning since it implemented traditional- grammar oriented class; (6) instead of gaps, the problems appearing on students’ responses were slow responses; (7) in this thesis, formative assessment demands the teacher to be active and interactive to communicate with the students; and (8) the formative assessment was a life on- going process of learning. The research suggests (1) to implement formative assessment conversation, according to the findings, teacher should be more aware and literate to deal with the on-going cognitive level, preliminary knowledge, and readiness of the students; (2) positive student engagement, teacher recognition, teacher’s personal feeling, direct instruction, instructional orientation, teacher’s attitudes to silent period, theme based instruction, lowering complexity of elicitation question, calling student name, developing student’s idea, tag question of the teacher, revising language instruction, teacher’s criticism, space-repeated question, and teacher’s request at student to clarify could help teacher to deal with the on-going learning problems in the class; (3) question was not only used to be answered but it could be used to help the students in developing their cognition; (4) moments of lack or absence of FACC or the emergence of discourse filler provided time for students to get ready into the next step of teaching as well as providing teacher’s time to move to the next sequence of teaching; and (5) the non-verbal attributes seen on the teacher facilitated the realization of formative assessment conversation to be understood by the students. This article investigates the covert backwash of formative assessment toward English instructions by analyzing the teacher’s formative assessment conversation regardless Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 549 from positive and negative backwash. It was done by using formative assessment conversation cycle called as ESRU cycle to see teacher’s feedbacks. However, it was too broad so that FIACS was needed to analyze what was going within the cycle. Unfortunately, these research instruments were not tested first to check their reliability. However, by using interview to crosscheck the claims, it could bridge and close the gap. In another hand, there is also limitation by using Zorich’s model (2007) to investigate non-verbal attributes entailing English instructions of the teacher. This model only covers the attributes attached on the teacher but it does not give further categories to indicate the purposes of the attributes. To overcome the problem, the interview is used to reveal what is going on within the attributes. The use of interview is important for this research to keep consistency of the findings and claims of the research. Therefore, it is believed this research still reliable to contribute. This article focuses on covert backwash of formative assessment in English instructions. Therefore, it is only limited on teacher without seeing the backwash on the students’ sides. Since it was sought to see the covert backwash, then the unit of analysis was classroom activity, specifically in formative assessment conversation cycle, in which it was administered orally. Further investigation is expected to see the backwash on other types of formative assessment administration. REFERENCES Abbas, S., & Thaheem, S. S. (2018). Washback Impact on Teachers ’ Instruction Resulting from Students ’ Apathy Washback Impact on Teachers ’ Instruction Resulting from Students ’ Apathy. Research on Humanities and Social Sciences, 8(6). Abosalem, Y. (2016). Assessment Techniques and Students’ Higher-Order Thinking Skills. International Journal of Secondary Education, 4(1), 1–11. https://doi.org/10.11648/j.ijsedu.201604 01.11 Agherdien, N., Mey, M., & Poisat, P. (2018). Factors Impacting on Students’ Readiness for Higher Education. Africa Education Review, 15(1), 52–71. https://doi.org/10.1080/18146627.2016. 1224596 Akdeniz, Celal. (2016). Instructional Strategies. In C Akdeniz (Ed.), Instructional Process and Concepts in Theory and Practice. Springer Science and Business Media. https://doi.org/10.1007/978-981-10- 2519-8 Aksan, Z., & Çelikler, D. (2015). Determining the Level of Knowledge and Mental Models of Secondary School Students Regarding the Solar System. Journal of Environment and Earth Science, 5(13), 15– 22. Alptekin, C., Erçetin, G., & Bayyurt, Y. (2007). The effectiveness of a theme-based syllabus for young L2 learners. Journal of Multilingual and Multicultural Development, 28(1), 1–17. https://doi.org/10.2167/jmmd470.1 Alraddadi, B. M. (2016). The Effect of Teaching Structural Discourse Markers in an EFL Classroom Setting. English Language Teaching, 9(7), 16–31. https://doi.org/10.5539/elt.v9n7p16 Antes, T. A. (1996). Kinesics The Value of Gesture n Language and i n the Language Classroom. Foregin Language Annals, 29(3). Antón, M. (2015). Shifting trends in the assessment of classroom interaction. In The handbook of classroom discourse and interaction (pp. 74–89). https://doi.org/10.1002/9781118531242. ch5 Arnellis, A., Jamaan, E. Z., & Amalita, N. (2018). Efforts to Improve Mathematics Teacher Competency Through Training Program on Design Olympiad Mathematics Problems Based on Higher Order Thinking Skills in the Junior High School. IOP Conference Series: Materials Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 550 Science and Engineering 335, 335(1), 0–5. https://doi.org/10.1088/1757- 899X/335/1/012118 Atwood, S., Turnbull, W., & Carpendale, J. I. M. (2010). The construction of knowledge in classroom talk. Journal of the Learning Sciences, 19(3), 358–402. https://doi.org/10.1080/10508406.2010. 481013 Bailey, G. D. (1974). A Study of Classroom Interaction Patterns from Student Teaching to Independent Classroom Teaching. Research Reports, (December), 225–230. Barabar, A., & Caganaga, C. K. (2015). Using Nonverbal Communicaiton in EFL Classes. Cypriot Journal of Educational Sciences, 10(2), 136–147. Baradeyah, N., & Farrah, M. (2017). The Impact of Using Asking for Clarification and Circumlocution Speaking Strategies on Enhancing the Speaking Skill of EFL Students. Studies in Linguistics and Literature, 1(2), 86. https://doi.org/10.22158/sll.v1n2p86 Bartlett, L. (2013). Pilot Test for Reliability and Validity of a New Assessment Tool Measuring Relationships between Individual Health and Environmental Sustainability. Dalhousie University. Bedford, J. (1995). Washback – the Effect of Assessment on ESOL Teaching and Learning Literature Review : the Washback Effect. Applied Linguistics. Bell, B., & Cowie, B. (2001). The Characteristics of Formative Assessment. Science Education, 85(5), 536–553. https://doi.org/10.1007/0-306-47227-9_4 Berkowitz, M., & Stern, E. (2018). Which Cognitive Abilities Make the Difference? Predicting Academic Achievements in Advanced STEM Studies. Journal of Intelligence, 6(4), 48. https://doi.org/10.3390/jintelligence604 0048 Boleware, J. (n.d.). The Types of Instructions Used in the Classroom. Synonym. Retrieved from https://classroom.synonym.com/types- instructions-used-classroom-7609879.html Bowman, M. (1994). Using Video in Research. Scottish Council for Research in Education. Eidinburgh, Scotland. https://doi.org/10.1103/PhysRevLett.66 .3321 Broussard, M. J. S. (2014). Using games to make formative assessment fun in the academic library. Journal of Academic Librarianship, 40(1), 35–42. https://doi.org/10.1016/j.acalib.2012.12. 001 Chapet, C. M. (2009). The Use and Functions of Discourse Markers in EFL Classroom Interaction The Use and Functions of Discourse Markers in EFL Classroom Interaction. Profile Issues in Teachers` Professional Development, 57–58(11). Cheung, C., & Yang, R. (2009). Theme-based Teaching in an English Course for Primary ESL Students in Hong Kong. Electronic Journal of Foreign Language Teaching, 6(2), 161–176. Derakhshan, A., & Shirmohammadli, M. (2015). The Difficulties of Teaching English Language: The Relationship between Research and Teaching. International Journal of Linguistics, 7(1), 102. https://doi.org/10.5296/ijl.v7i1.6648 Doiz, A., Lasagabaster, D., & Sierra, J. M. (2011). Doiz, A,. Lasagabaster, D Sierra (2011).pdf, 30(3), 345–359. Erten, S. (n.d.). Teaching Fillers and Students ’ Filler Usage : A Study Conducted at ESOGU Preparation School. International Journal of Teaching and Educaiton, II(3), 67–79. Evans, D. J. R., Zeun, P., & Stanier, R. A. (2014). Motivating student learning using a formative assessment journey. Journal of Anatomy, 224(3), 296–303. https://doi.org/10.1111/joa.12117 Falk, A. (2012). Teachers learning from professional development in elementary science: Reciprocal relations between formative assessment and pedagogical Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 551 content knowledge. Science Education, 96(2), 265–290. https://doi.org/10.1002/sce.20473 Fianti, F. L. N., & S, L. (2017). Development of Open-Ended Problems for Measuring the Higher-Order-Thinking-Skills of High School Students on Global Warming Phenomenon. Journal of Physics. https://doi.org/10.1088/1742- 6596/824/1/012008 Fitriati, S. W., Isfara, G. A. V., & Tristanti, N. (2017). Teacher’s Questioning Strategies to Elicit Students’ Verbal Responses in EFL Classes at A Secondary School. English Review: Journal of English Education, 5(2), 217–226. Fraser, B. (1991). What are discourse markers ? Journal of Pramgatics, 31(December), 931– 952. https://doi.org/10.1016/S0378- 2166(98)00101-5 Fraser, J., Fahlman, D., Arscott, J., & Guillot, I. (2018). Online course enrollment in community college and degree completion: The tipping point. International Review of Research in Open and Distance Learning, 19(2), 282–293. https://doi.org/10.19173/irrodl.v19i2.34 60 Freahat, N. M., & Smadi, O. M. (2014). Lower- order and Higher-order Reading Questions in Secondary and University Level EFL Textbooks in Jordan. Theory and Practice in Language Studies, 4(9), 1804–1813. https://doi.org/10.4304/tpls.4.9.1804- 1813 Frels, R. K., Sharma, B., Onwuegbuzie, A. J., Leech, N. L., & Stark, M. D. (2011). The Use of a Checklist and Qualitative Notebooks for an Interactive Process of Teaching and Learning Qualitative Research. The Journal of Effective Teaching, 11(1), 62–79. Garloch, L. A. (1947). Tools of Research. Journal of Geography, 46(2), 55–62. https://doi.org/10.1080/0022134470898 6695 Gill, D., & Prowse, V. L. (2014). Cognitive Ability, Character Skills, and Learning to Play Equilibrium: A Level-k Analysis. IZA, (8236). https://doi.org/10.2139/ssrn.2448144 Given, L. M. (2008). The Sage Encyclopedia of Qualitative Research Methods Volumes 1&2. Thousand Oaks, California 91320: SAGE Publications, Inc. https://doi.org/10.4135/9781412963909 Glenz, T. (2014). The Importance of Learning Students’ Names. Journal on Best Teaching Practices. Gosling, J. P. (2018). SHELF: The Sheffield Elicitation Framework. In L. C. Dias, A. Morton, & J. Quigley (Eds.), Elicitation the Science and Art of Structuring Judgement (pp. 61–93). Springer International Publishing. Guest, G., & Mclellan, E. (2003). Distinguishing the Trees from the Forest : Field Methods, 15(2), 186–201. https://doi.org/10.1177/1525822X03251 188 Hagströmer, M., Ainsworth, B. E., Kwak, L., & Bowles, H. R. (2012). A Checklist for Evaluating the Methodological Quality of Validation Studies on Self-Report Instruments for Physical Activity and Sedentary Behavior. Journal of Physical Activity and Health, 9(s1), S29–S36. https://doi.org/10.1123/jpah.9.s1.s29 Hailikari, T., Katajavuori, N., & Lindblom- ylanne, S. (2008). The Relevance of Prior Knowledge in Learning and Instructional Design The Relevance of Prior Knowledge in Learning and Instructional Design. American Journal of Phramaceutical Education, 72(5). https://doi.org/10.5688/aj7205113 Hakuta, K., & Jacks, L. L. (2009). Guidelines for the assessment of English language learners. Educational Testing Service. Standofrd University. Retrieved from http://scholar.google.com/scholar?hl=en &btnG=Search&q=intitle:Guidelines+for +the+Assessment+of+English+Language +Learners#0 Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 552 Halimaa, S.-L. (2001). Video Recording as a Method of Data Collection in Nursing Research. Nursing Science and Research in Nordic Countries, 21(2), 21–26. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research2, 77(1), 81–122. https://doi.org/10.1111/j.1365- 2923.2009.03542.x Horness, P. (2016). Pacific Rim Objective Measurement Symposium (PROMS) 2015 Conference Proceedings. Pacific Rim Objective Measurement Symposium (PROMS). https://doi.org/10.1007/978- 981-10-1687-5_25 Hult, A., & Liljeström, M. (2011). Formative assessment in peer review settings online. The University of the Fraser Valley Research Review, 4(1), 53–64. Retrieved from http://www.diva- portal.org/smash/record.jsf?aq2=%5B%5 B%5D%5D&c=24&af=%5B%22publicati onTypeCode%3Aarticle%22%5D&search Type=SIMPLE&query=Formative+Asse ssment&language=en&pid=diva2%3A43 4989&aq=%5B%5B%5D%5D&sf=all&aq e=%5B%5D&sortOrder=author_sort_asc &onlyFull Inamullah, H. M., Hussain, I., & din, M. N. ud. (2011). Direct Influence Of English Teachers In The Teaching Learning Process. College Teaching Methods & Styles Journal (CTMS), 4(4), 29–36. https://doi.org/10.19030/ctms.v4i4.5544 Ismail Pratt, I., Bikoo, M., Liao, L., Conway, G., & Creighton, S. (2007). Normalization of the Vagina by Dilator Treatment Alone in Complete Androgen Insensitivity Syndrome and Mayer- Rokitansky-Kuster-Hauser-Syndrome. Human Reproduction, 22(7), 2020–2024. Jafari, J. (2013). Elicitation Questions in English and Persian Written Texts : A Comparative Study. World Journal of English Language, 3(2), 34–44. https://doi.org/10.5430/wjel.v3n2p34 Jovanović, V. Ž., & Pavlović, V. (2014). The Use of Question Tags with Male and Female Speakers of English and Serbian. Jezici i Kulture u Vremenu i Prostoru III (Languages and Cultures in Time and Space III), 491– 504. https://doi.org/10.2139/ssrn.2967908 Kapur, S. (2015). Andragogy: The Adult Learning Theory. Indian Journal of Adult Education, 76, 50–60. Kawulich, B. (2014). Collecting Data Through Observation. Katalog BPS, XXXIII(2), 81– 87. https://doi.org/10.1007/s13398-014- 0173-7.2 Kawulich, B. B. (2005). Participant Observation as Data Collection Method. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 6(2). Kelly, J., & Pohl, B. (2018). Using Structured Positive and Negative Reinforcement to Change Student Behavior in Educational Settings in order to Achieve Student Academic Success. Multidisciplinary Journal for Education, Social and Technological Sciences, 5(1), 17. https://doi.org/10.4995/muse.2018.6370 Kenny, D. T. (1980). Direct instruction : An overview of theory and practice. Journal of the Association of Special Education Teachers, 15(12). Kimps, D., Davidse, K., & Cornillie, B. (2014). The speech functions of tag questions and their properties. A comparison of their distribution in COLT and LLC. https://doi.org/10.1075/scl.63.21kim King, J. E. (2011). Silence in the Second Language Classroom. The University of Nottingham. Kırmızı, Ö. (2015). The Influence of Learner Readiness on Student Satisfaction and Academic Achievement in an Online Program at higher Education. The Turkish Online Journal of Educational Technology, 14(1), 133–142. Kopf, S., & Effelsberg, W. (2007). New Teaching and Learning Technologies for Interactive Lectures. Advanced Technology for Learning, 4(2). https://doi.org/10.2316/Journal.208.200 7.2.208-1082 Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 553 Krauss, R. M., Chen, Y., & Chawla, P. (1996). Nonverbal Behavior and Nonverbal Communication: What do Conversational Hand Gestures Tell Us? Advances in Experimental Social Psychology, 28(C), 389–450. https://doi.org/10.1016/S0065- 2601(08)60241-5 Lazarides, R., & Rubach, C. (2016). Instructional Characteristics in Mathematics Classrooms : Relationships to Achievement Goal Orientation and Student Engagement. Mathematics Education Research Group of Australia. https://doi.org/10.1007/s13394-017- 0196-4 Liem, G. A. D., & Martin, A. J. (2013). Direct Instruction and Academic Achievement. International Guide to Student Achievement. Lom, B. (2012). Classroom Activities: Simple Strategies to Incorporate Student- Centered Activities within Undergraduate Science Lectures. The Journal of Undergraduate Neuroscience Education (JUNE), 11(1), A64–A71. https://doi.org/10.1080/10601325.2013. 802196 Luke, P. (1995). The backwash effecf: from testing to teaching. ELT Journal, 49(January), 13–25. Marashi, H., & Tehran, C. (2018). Using Information-Gap Tasks to Improve Reading : An Analysis of Cognitive Using Information-Gap Tasks to Improve Reading : An Analysis of Cognitive. Journal of Language Horizons, Alzahra University, 2(1). https://doi.org/10.22051/lghor.2018.205 94.1082 Martínez-Mesa, J., González-Chica, D. A., Duquia, R. P., Bonamigo, R. R., & Bastos, J. L. (2016). Sampling: How to select participants in my research study? Anais Brasileiros de Dermatologia, 91(3), 326–330. https://doi.org/10.1590/abd1806- 4841.20165254 Marzano, R. J., Gaddy, B. B., & Dean, C. (2000). What Works In Classroom Instruction What Works In Classroom Instruction. McColl, E., Jacoby, A., Thomas, L., Soutter, J., Bamford, C., Steen, N., … Bond, J. (2001). Design and Use of Questionnaires: a Review of Best Practice Applicable to Surveys of Health Service Staff and Patients. Health Technology Assessment (Vol. 5). Middendorf, J., & Osborn, E. (2002). Learning Student Names. In National Teaching and Learniing Forum, 28. Mizokawa, A. (2013). Relationships between maternal emotional expressiveness and children’s sensitivity to teacher criticism. Frontiers in Psychology, 4, 1–7. https://doi.org/10.3389/fpsyg.2013.0080 7 Mizokawa, A. (2014). Theory of Mind and Sensitivity to Teacher and Peer Criticism among Japanese Children. Infant and Child Development, (1). https://doi.org/10.1002/icd.1877 Morrison, D. (2015). Formative assessment and equity: An exploration of opportunities for eliciting, recognizing, and responding within science classroom conversations. ProQuest Dissertations and Theses, 274. Retrieved from https://search.proquest.com/docview/16 90900183?accountid=10673%0Ahttp://o penurl.ac.uk/redirect/athens:edu/?url_ve r=Z39.88- 2004&rft_val_fmt=info:ofi/fmt:kev:mtx:d issertation&genre=dissertations+%26+th eses&sid=ProQ:Education+Database&ati tle=&title=Formative Muchemwa, S. (2013). Use of Nonverbal Communication in the Classroom as a Way of Enhancing Classroom Teaching: A Case Study of Solusi High School, Zimbabwe. Procedia - Social and Behavioral Sciences, 103(November 2013), 1279– 1287. https://doi.org/10.1016/j.sbspro.2013.10 .457 Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 554 Navarretta, C. (2015). The Functions of Fillers , Filled Pauses and Co-Occurring Gestures in Danish Dyadic Conversations. In 3rd European Symposium on Multimodal Communicaiton (pp. 55–61). Newhouse, D., & Suryadarma, D. (2011). The value of vocational education: High school type and labor market outcomes in Indonesia. World Bank Economic Review, 25(2), 296–322. https://doi.org/10.1093/wber/lhr010 Nidhra, S., & Dondeti, J. (2012). How to Write a L iterature R eview, 2(2), 29–50. https://doi.org/10.1016/j.profnurs.2010. 10.006 Noor, A. M. (2014). Teaching Thinking Skills : Redesigning Classroom Practices Teaching Thinking Skills : Redesigning Classroom Practices, (April). O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine, 89(9), 1245–1251. https://doi.org/10.1097/ACM.00000000 00000388 Othman, N. B. T. (2007). School-Based Oral English Test : the, 1–3. Pan, Y., & Li, J. (2013). Cooperative pseudonym change scheme based on the number of neighbors in VANETs. Journal of Network and Computer Applications, 36(6), 1599–1609. https://doi.org/10.1016/j.jnca.2013.02.0 03 Pawan, F., & Craig, D. A. (2011). ESL and Content Area Teacher Responses to Discussions on English Language Learner Instruction. TESOL Journal, 2(3), 293– 311. https://doi.org/10.5054/tj.2011.259956 Penn-Edwards, S. (2004). Visual Evidence in Qualitative Research: The Role of Videorecording. The Qualitative Report, 9(2), 266–277. Retrieved from http://nsuworks.nova.edu/tqr%0Ahttp:/ /nsuworks.nova.edu/tqr/vol9/iss2/5 Purnawarman, P., Ratnaningsih, S., & Gunawan, M. H. (2017). Scientific Approach of 2013 Curriculum: Teachers Implementation in English Language Teaching. English Review: Journal of English Education, 6(1), 33. https://doi.org/10.25134/erjee.v6i1.768 Purver, M. (2004). The Theory and Use of Clarification Requests in Dialogue. King’s College University of London. Qi, Y., & Sykes, G. (2016). Eliciting Student Thinking: Definition, Research Support, and Measurement of the “ETS”® National Observational Teaching Examination (NOTE) Assessment Series. Research Memorandum No. RM-16-06. Educational Testing Service, (August), 1–56. Retrieved from https://www.lib.uwo.ca/cgi- bin/ezpauthn.cgi?url=http://search.proq uest.com/docview/1968430151?accounti d=15115%0Ahttp://vr2pk9sx9w.search.s erialssolutions.com?ctx_ver=Z39.88- 2004&ctx_enc=info:ofi/enc:UTF- 8&rfr_id=info:sid/ERIC&rft_val_fmt=inf o:ofi/fmt:kev: Quigley, J., Colson, A., Aspinall, Wi., & Cooke, R. M. (2018). Elicitation in the Classical Model. In L. C. Dias, A. Morton, & J. Quigley (Eds.), Elicitation the Science and Art of Structuring Judgement (pp. 15–36). Retnawati, S., & Mujiyanto, Y. (2015). Code swithcing used in conversation by an American student of the darmasiswa program. Journal of Language and Literature, 10(1), 29–35. Retrieved from https://journal.unnes.ac.id/nju/index.ph p/LC/article/download/4156/3690 Riandari, F., Susanti, R., & Suratmi. (2018). The Influence of Discovery Learning Model Application to the Higher Order Thinking Skills Student of Srijaya Negara Senior High School Palembang on the animal kingdom subject matter. Journal of Physics: Conference Series, 1022(1). https://doi.org/10.1088/1742- 6596/1022/1/012055 Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 555 Rivers, J., Smith, A., Higgins, D., Mills, R., Maier, A. G., & Howitt, S. M. (2017). Asking and Answering Questions: Partners, Peer Learning, and Participation. International Journal for Students as Partners, 1(1), 1–10. https://doi.org/http://dx.doi.org/10.151 73/ijsap.v1i1.3072 Roberts, C. (2006). Part One : Issues in Transcribing Spoken. Qualitative Research Methods and Transcription, 1–36. Rouhi, A., Nabavi, S., & Mohebbi, H. (2014). The Effects of Previewing Questions, Repetition of Input, and Topic Preparation on Listening Comprehension of Iranian EFL Learners. Iranian Journal of Language Teaching Research, 2(2), 73–85. Ruiz-Primo, M. A., & Furtak, E. M. (2007). Exploring Teachers’ Informal Formative Assessment Practices and Students’ Understanding in the Context of Scientific Inquiry. Journal of Research in Science Teaching, 44(1), 57–84. https://doi.org/10.1002/tea Ruiz-Primo, M. A., Furtak, E. M., & Araceli Ruiz-Primo, M. (2006). Informal Formative Assessment and Scientific Inquiry: Exploring Teachers’ Practices and Student Learning. Educational Assessment, 11(3 & 4), 205–235. https://doi.org/10.1207/s15326977ea110 3&4_4 Rumfola, L. (2017). Positive Reinforcement Positively Helps Students in the Classroom. State University of New York. Retrieved from http://digitalcommons.brockport.edu/eh d_theses/786 Sahin, A. (2007). Teachers’ Classroom Questions. School Science and Mathematics, 107(1), 369–370. https://doi.org/10.1080/0007679070129 4964 Sari, P. P., Budiyono, & Slamet, I. (2018). Cooperative learning model with high order thinking skills questions: An understanding on geometry. Journal of Physics: Conference Series. https://doi.org/10.1088/1742- 6596/1013/1/012123 Saunders, R. (2013). The role of teacher emotions in change: Experiences, patterns and implications for professional development. Journal of Educational Change, 14(3), 303–333. https://doi.org/10.1007/s10833-012- 9195-0 Scates, D. E., & Douglas, E. (1950). The Good Teacher: Establishing Criteria for Identification. Journal of Teacher Education, 1(2), 346–353. https://doi.org/10.1177/0022487150001 00210 Schourup, L. (2011). The Discourse Marker Now : A Relevance-Theoretic Approach. Journal of Pragmatics, 43(8), 2110–2129. https://doi.org/10.1016/j.pragma.2011.0 1.005 Sharpe, T. (2008). How Can Teacher Talk Support Learning? Linguistics and Education, 19, 132–148. https://doi.org/10.1016/j.linged.2008.05 .001 Sheerman, B., Adam, A., Annette, B., Dawn, B., Douglas, C., David, C., … Lynda, W. (2008). Testing and Assessment (Vol. I). Skipper, Y., & Douglas, K. (2015). The Influence of Teacher Feedback on Children’s Perceptions of Student- Teacher Relationships. British Journal of Educational Psychology, 85(3), 276–288. https://doi.org/10.1111/bjep.12070 Smith, A. S. (2012). An Exploration of the Negative Effects of Repitition and Testing on Memory. The Yale Review of Undergraduate Research in Psychology, (2012), 78–91. Retrieved from http://www.yale.edu/yrurp/issues/Smit h2013 Suprapto, E., Fahrizal, F., Priyono, P., & K., B. (2017). The Application of Problem- Based Learning Strategy to Increase High Order Thinking Skills of Senior Vocational School Students. International Education Studies, 10(6), 123. https://doi.org/10.5539/ies.v10n6p123 Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 556 Syafei, M. (2012). Backwash Effects of Portfolio Assessment in Academic Writing Classes. TEFLIN Journal, 23(2), 15–30. https://doi.org/10.15639/TEFLINJOU RNAL.V23I2/206-221 Talebinejad, R., & Namdar, A. (2011). Discourse Markers in High School English Textbooks in Iran. Theory and Practice in Language Studies, (1). https://doi.org/10.4304/tpls.1.11.1590- 1602 Teijilingen, E. R. V., & Hundley, V. (2001). The importance of study design. Department of Sociology University of Surrey, 33(3–4), 105– 106. Tikhonova, E., & Kudinova, N. (2015). Sophisticated Thinking: Lower Order Thinking Skills. https://doi.org/10.5593/SGEMSOCIAL 2015/B12/S3.117 Tomková, G. (2013). Error Correction in Spoken Practice. Masaryk University. Torres-Olave, B. M. (2012). Imaginative geographies: Identity, difference, and English as the language of instruction in a Mexican university program. Higher Education, 63(3), 317–335. https://doi.org/10.1007/s10734-011- 9443-x Turner, D. W. (2010). Qualitative interview design: A practical guide for novice investigators. The Qualitative Report, 15(3), 754–760. https://doi.org/http://www.nova.edu/ss ss/QR/QR15-3/qid.pdf Vassilopoulos, S. P., & Konstantinidis, G. (2012). Teacher Use of Silence in Elementary Education. Journal of Teaching and Learning, 8(1), 91–105. Vingsle, C. (2014). Formative assessment: Teacher knowledge and skills to make it happen. Wang, C., Yan, J., & Liu, B. (2014). An empirical study on washback effects of the internet-based college English test band 4 in China. English Language Teaching, 7(6), 26–53. https://doi.org/10.5539/elt.v7n6p26 Wang, Y. huei, Chao, C. Y., & Liao, H. C. (2011). Poststructural feminist pedagogy in English instruction of vocational-and- technical education. Higher Education, 61(2), 109–139. https://doi.org/10.1007/s10734-010- 9327-5 Watkins, D., Dahlin, B., & Ekholm, M. (2005). Awareness of the backwash effect of assessment: A phenomenographic study of the views of Hong Kong and Swedish lecturers. Instructional Science, 33(4), 283– 309. https://doi.org/10.1007/s11251- 005-3002-8 Watson, C. W. (2010). Conducting Research in Conservation. Conducting Research in Conservation: Social Science Methods and Practice (Vol. 29). Routledge. https://doi.org/10.4324/9780203846452 Weurlander, M., Söderberg, M., Scheja, M., Hult, H., & Wernerson, A. (2012). Exploring formative assessment as a tool for learning: students’ experiences of different methods of formative assessment. Assessment & Evaluation in Higher Education, 37(6), 747–760. https://doi.org/10.1080/02602938.2011. 572153 Winters, A., & America, N. (n.d.). Using Talking Circles in the Classroom. Heartland COmmunity, 1, 1–8. Wun, K. P., & Harun, J. (2017). The Effect of Scenario-Epistemic Game on Higher Order Thinking Skills among High School Chemistry Students in Malaysia. Proceedings - 5th International Conference on Learning and Teaching in Computing and Engineering, LaTiCE 2017, 16–22. https://doi.org/10.1109/LaTiCE.2017.1 0 Yang, W. (2010). A Tentative Analysis of Errors in Language Learning and Use. Journal of Language Teaching and Research, 1(3), 266– 268. https://doi.org/10.4304/jltr.1.3.266- 268 Yang, Y. T. C., Gamble, J. H., Hung, Y. W., & Lin, T. Y. (2014). An online adaptive learning environment for critical-thinking- Kresna Rahma Aji & Rudi Hartono/ EEJ 9 (4) (2019) 541 - 557 557 infused English literacy instruction. British Journal of Educational Technology, 45(4), 723–747. https://doi.org/10.1111/bjet.12080 Yin, R. K. (2014). Case Study Research Design and Method. Sage Publication (5th ed.). Thousand Oak. https://doi.org/10.1111/j.1540- 4781.2011.01212_17.x Zembylas, M. (2003). Teachers and Teaching : Theory and Emotions and Teacher Identity. Teachers and Teaching: Theory and Practice, 9(July 2013), 213–238. https://doi.org/10.1080/1354060032000 116611