LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 21, No. 2, October 2018 LLT Journal: A Journal on Language and Language Teaching http://e-journal.usd.ac.id/index.php/LLT Sanata Dharma University, Yogyakarta, Indonesia 199 LANGUAGE ASSESSMENT LITERACY DEVELOPMENT: A STUDENT-TEACHER’S EXPERIENCES IN TEACHING PRACTICE PROGRAM Agustinus Hardi Prasetyo Iowa State University hardi@iastate.edu DOI: doi.org/10.24071/llt.2018.210209 received 13 June 2018; revised 22 August 2018; accepted 1 September 2018 Abstract The development of a student-teacher’s language assessment literacy was viewed through her experiences in conducting teaching practice in one of the senior high schools in Yogyakarta, Indonesia. An interview was conducted to explore the experiences taking place in the teaching-practice in relation to the language assessment literacy development. It was found out that the experiences were understood as mostly dealing with administrative tasks and there is a need of more knowledge in administrative matter in teaching and assessment. From the results of the study it was obvious that further studies need to be conducted to explore the role of teaching-practice in the development of student-teaching language assessment literacy. More participants and multiple data collection methods in a longitudinal study are needed to help student-teachers to be more assessment literate, which in turn helps them to be better teachers. Keywords: administrative tasks, language assessment literacy, teaching practice Introduction Assessment is an integral part of teaching. However, studies have shown that many teachers do not have enough knowledge and skills on how to assess their students. The teachers’ lack of knowledge and skills on assessment will disadvantage their students and render teachers less accountable towards stakeholders. It is therefore necessary to explore the teachers’ educational measurement knowledge and the skills to apply that knowledge to measure students’ achievement. This study is aimed to discover the Language Assessment Literacy/LAL of student-teachers of English Language Education Study Program, Sanata Dharma University, Indonesia. These particular participants and their setting were chosen since not many studies have been done to explore their language assessment literacy. This study therefore will fill in the gap in the literature. The term assessment literacy was first coined by Stiggins (1991) who defined it as a fundamental understanding of educational assessment and skills to apply such knowledge to measure student achievement. In the field of language assessment, Fulcher (Fulcher, 2012) proposes the most detailed working definition of Language Assessment Literacy (LAL). To date, despite the agreement that LAL is necessary for language teachers, there has been little consensus among assessment experts on LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 21, No. 2, October 2018 200 what knowledge, skills, and principles (Davies, 2008) that teachers should master in order to be assessment literate, or what components of LAL should be taught and prioritized (Harding & Kremmel, 2016; Taylor, 2009). Assessment literacy has gradually been considered as an integral part of teachers’ professionalism (Popham, 2004, 2011; Schafer, 1993; Stiggins, 1995) due to the pivotal role of assessment in students’ learning (Black & Wiliam, 1998, 2010; Xu & Brown, 2017). Popham (2004) even argued that assessment illiteracy is a form of professional suicide considering the vital role assessment plays in students learning. However, many studies have shown that teachers are unconfident in assessing their students (Volante & Fazio, 2007) and lacking in knowledge and skills of educational assessment, and they are in the need of proper training (DeLuca, LaPointe-McEwan, & Luhanga, 2016; Herrera & Macías, 2015; Malone, 2011; Stiggins, 1999). In order to address this problem, several studies have been conducted to find out what training and knowledge are needed for teachers to be assessment literate. Using the survey form they designed for three types of stakeholders (language teachers, language teachers training, and language testing experts. i.e. who are involved in item writing) in Language Testing and Assessment (LTA), Hasselgreen, Carlsen, and Helness (2004) identified training needs of those stakeholders in the countries across Europe to offer subsequent training in the field. The researchers reported that there is a need for more formal education and training in language testing and assessment among those stakeholders. They also reported that most LTA activities (using statistics, assessing culture, integrated skills, establishing validity, and assessing productive skills) were carried out by teachers who have no training in these activities. In response to Hasselgreen et al.’s call for more research on teachers’ assessment literacy, Vogt and Tsagari (2014) conducted a similar study built on Hasselgreen et al.’s research by including additional European countries and exclusively targeted foreign language teachers (see also Tsagari & Vogt, 2017). Their results show that according to the participants, the LTA literacy of foreign language teachers across Europe is not very well-developed. The majority of those teachers had received either “a little” or “no” training at all, and they believe that their training has not sufficiently prepared them for their work. This study’s results then corroborate Hasselgreen et al.’s study. Several studies conducted to study both pre-service and in-service teachers’ assessment literacy also yield similar results (Mertler & Campbell, 2005; DeLuca, Chavez, Bellara, & Cao, 2013; Campbell, C., Murphy, J. A. & Holt, J. K. , 2002, Mertler, C. A., 2003, and Plake, 1993 as cited in Mertler, 2009) which show that those teachers do not seem to possess high, or even adequate, levels of assessment literacy. Researchers have also noted that there is comparatively little research on teachers’ current assessment practices from which professional learning programs to encourage teacher assessment literacy can be constructed (Brindley, 2001; Harding & Kremmel, 2016; Mertler, 2009). Moreover, little research has been conducted on teacher candidates’ assessment practice when they are conducting their field experiences in schools. While in fact, finding out what teacher candidates experience in terms of designing and implementing classroom-based assessment in LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 21, No. 2, October 2018 201 their teaching practice or field experience will help avoid the phenomena of testing the students the way they were tested (DeLuca et al., 2013). Teacher candidates’ field experiences can influence their perception of assessment and their assessment decisions, as well as many other instructional decisions (Clark, 2015; Heafner, 2004). This case study therefore is aimed at describing and interpreting what happens during teaching practice or field experience in secondary schools in relation to teacher candidates’ language assessment literacy development. As I mentioned in the beginning of this paper, assessment is an integral part of teaching. Teachers will not be able to help their students or themselves to learn, if they do not have the knowledge and related skills to assess their own and their students’ learning. Therefore, teachers should be assessment literate. Even though assessment literary has a vital role in teachers’ teaching and students’ learning, many studies have shown that teachers do not have the required knowledge and skills to be considered assessment literate. The same studies also shown that teachers need training and support to be assessment literate. Therefore, in this study, I would like to explore how student-teachers of English Language Education Study Program (ELESP) develop, design, and implement their assessment plan. In that way then I will be able to see whether they have the knowledge and skills to design an appropriate assessment plan. Later, I will observe them doing their teaching practice when I can observe the way they assess their students. Since pragmatism is the paradigm of this study, I will use multiple methods to answer the research questions. The following are the research questions guiding this study: Central question: What happens in teaching practice or field experience in secondary schools in relation to teacher candidates’ language assessment literacy development? Subquestions: 1) How do student-teachers of English Language Education Study Program develop their assessment plan (formative and summative assessment) in their teaching practice in the local secondary school? 2) How do student-teachers of English Language Education Study Program implement their assessment plan? 3) In what ways do development and implementation affect assessment literacy, if at all? 4) How does the school as a system, where those student-teachers do their teaching practice, support their professional development for assessment literacy? Method Interview was employed to answer the research questions above. One student- teacher of English Language Education Study Program, Sanata Dharma University, Indonesia who has just finished her field experience (teaching practice) was chosen to be interviewed. The interview was conducted in Bahasa Indonesia via text-based WhatsApp application. Considering the distance and time difference between the interviewee and interview, all the questions were all at once posted in the WhatsApp for the interviewee to answer whenever it is possible for her to respond. This LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 21, No. 2, October 2018 202 technique allows the interviewee to have more time to think about her responses and allows the interviewer to ask further questions on her responses that need further clarification. The interview protocol that I used consists of two parts. The first part contains the consent form asking the interviewee whether she is willing to voluntarily participate in this study. Once she confirmed that she was willing to participate, then the second part was posted in the application. The following is the interview protocol: (1) Dear participant, First of all, I need to get your consent for this interview. Before that, let me give you some information about this study. This is a small study looking at English Language Education Study Program (ELESP) students’ language assessment literacy and the role of teaching practice/PPL in the development of ELESP students’ language assessment literacy. Some questions will be about your experiences designing and implementing assessment when you are doing your teaching practice. Some other questions will be about the role of teaching practice, ELESP lecturers, and the courses you have taken before you took teaching practice. Your identity will not be revealed in any document published related to this data gathering. Do you agree to participate in this interview? (2) The following are the questions you need to respond: 1. I believed that you have just finished your teaching practice program. Would you please describe your responsibilities in the program? 2. In terms of assessment, how do you go about planning and implementing your assessment plan? I mean, would you please share your experiences in designing and implementing your assessment (formative and summative) plan? 3. Do your lecturers and teacher-supervisor guide you in designing and implementing your assessment plan? How do they go about doing it? 4. Do you think you are ready to do your teaching practice especially in relation to conducting assessment for your students’ learning? Why? It took two days for the interviewee to respond to these questions. Once she replied, I followed up with several questions for further clarification and examples. The complete transcript of the WhatsApp interview as well as its translation can be found in the appendix. Thematic analysis was applied to analyze the transcript of the interview. I followed Braun and Clarke's (2006) step by step procedure of thematic analysis to analyze the interview transcript. First, I transcribed the interview. This was easy since the interview was conducted using text-based WhatsApp desktop application, so what I need to do just copied and pasted the interview/chat into word processor. Then, I translated the interview transcript into English. This helped me to understand it deeply since I had to read the original transcript over and again to ensure that I did not miss the original meaning of the conversation. Then, using MAXQDA 12 software, I coded the transcript and found 48 codes. The codes were informed by the research questions that I have formulated, so this kind of coding was deductive in nature. I then compiled and collated those codes into some themes LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 21, No. 2, October 2018 203 in order to provide insights and deeper understanding on what happened during teaching practice in relation to student-teacher language assessment literacy development. Findings and Discussion Based on the themes found in the interview transcript, what happened in the teaching practice program was that this student-teacher dealt with mostly administrative tasks. From the day one she entered the school, until the last months of the teaching-practice program, most of the tasks were administrative. Even in their teaching and in the assessment design and implementation, she dealt with administrative matter. Her concern in the assessment design was how to design rubric to assess attitudes, skills, and knowledge correctly using the template given by the vice headmaster in curriculum. She thought that she was not prepared enough to design a lesson plan and its rubric since her lecturer did not teach her. She even suggested that ELESP lecturers should equip their student-teacher with more detailed knowledge on how to write lesson plan, especially in terms of designing and writing those rubrics. In terms of designing the assessment, she received little help from her teacher supervisor and lecturer since she was considered competent enough to design it. She was also trusted to teach Grade XII students. Usually, student-teachers generally were given opportunity to teach Grade X students. In fact, some secondary schools only allow student-teacher to teach Grade X students. It shows that she gained trust from her teacher supervisor, so she was given the opportunity to teach not only one but two classes of Grade XII. In terms of assessment implementation, she implemented her assessment as mostly formative assessment in the form of quizzes, comprehension questions, and assignments to check her students’ understanding of the materials. She became more competent in her assessment literacy, even though only in terms of understanding the template to design rubric to assess attitudes, skills, and knowledge through collaboration with fellow student-teachers from other study programs, who happened to be more well-informed than her in terms of some administrative tasks dealing with teaching (writing yearly program and semester program) and assessment (writing rubrics). In terms of the teacher supervisor and lecturer’s support toward the student- teacher’s language assessment literacy development, the student-teacher in this particular school received little support since she was considered competent enough to conduct her own teaching and her own assessment. It can be seen from the interview that only minor suggestions or revision were provided by teacher supervisor and lecturer concerning her assessment design. One of the limitations of the study is that only one participant was involved in this study which might not provide a complete picture of what was happening during teacher practice in terms of ELESP student-teachers’ language assessment literacy development. This participant conducted her teaching practice program in one of the most favorite state senior high schools in Yogyakarta. It is situated in a rural area, and it is considered as one of the best schools in terms of the implementation information technology. Other contexts of ELESP teaching- LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 21, No. 2, October 2018 204 practice program (rural vs. urban schools, senior vs. junior high schools, senior vs. vocational high schools, private-based vs. state-owned schools) were not represented. Conclusion More qualitative studies need to be conducted to explore and better understand the role of teaching practice in the development of student-teachers’ language assessment literacy. Multiple methods of data collection (interview, lesson plans, assessment materials, video-recorded teaching performances, observations, reflective journals, among others) and a longitudinal study must be employed to provide a more complete picture of what is happening in the teaching practice program. It is necessary therefore to conduct more studies on this area since teacher candidates’ field experiences can influence their perception of assessment and their assessment decisions, as well as many other instructional decisions (Clark, 2015; Heafner, 2004). The other limitation was the data collection technique. The data were collected through interview using text-based WhatsApp application which might limit the interaction and depth of the interview. Since the participant was quite busy (it was toward the end of the semester with a lot of deadlines) and considering the time difference, the distance as well as the quality of the internet will not guarantee the smooth quality of long-distance video or even phone call, so text-based chat was chosen. However, the synchronous nature of text-based chat allows both the interviewee and interviewer ample time to think about the responses and further follow-up questions which can facilitate more meaningful interaction. References Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74. https://doi.org/10.1080/0969595980050102 Black, P., & Wiliam, D. (2010). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 92(1), 81–90. https://doi.org/10.1177/003172171009200119 Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa Brindley, G. (2001). Outcomes-based assessment in practice: Some examples and emerging insights. Language Testing, 18(4), 393–407. https://doi.org/10.1191/026553201682430102 Clark, J. S. (2015). “My assessment didn’t seem real”: The Influence of field experiences on preservice teachers’ agency and assessment literacy. Journal of Social Studies Education Research, 6(2), 91–111. https://doi.org/10.17499/jsser.91829 Davies, A. (2008). Textbook trends in teaching language testing. Language Testing, 25(3), 327–347. https://doi.org/10.1177/0265532208090156 DeLuca, C., Chavez, T., Bellara, A., & Cao, C. (2013). Pedagogies for preservice assessment education: Supporting teacher candidates’ assessment literacy LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 21, No. 2, October 2018 205 development. Teacher Educator, 48(2), 128–142. https://doi.org/10.1080/08878730.2012.760024 DeLuca, C., LaPointe-McEwan, D., & Luhanga, U. (2016). Approaches to classroom assessment inventory: A new instrument to support teacher assessment literacy. Educational Assessment, 21(4), 248–266. https://doi.org/10.1080/10627197.2016.1236677 Fulcher, G. (2012). Assessment literacy for the language classroom. Language Assessment Quarterly, 9(2), 113–132. https://doi.org/10.1080/15434303.2011.642041 Harding, L., & Kremmel, B. (2016). Teacher assessment literacy and professional development. In D. Tsagari & J. Banerjee (Eds.), Handbook of second language assessment (pp. 89–104). Berlin: Walter de Gruyter Inc. Hasselgreen, A., Carlsen, C., & Helness, H. (2004). European survey of language testing report. Part one: General findings. Retrieved from http://www.ealta.eu.org/resources.htm Heafner, T. (2004). Assessment as a magnification of internal, parallel, and external reflection. Action in Teacher Education, 25(4), 14–19. https://doi.org/10.1080/01626620.2004.10648292 Herrera, L., & Macías, D. (2015). A call for language assessment literacy in the education and development of teachers of English as a foreign language. Colomb. Appl. Linguist. J, 17(2), 302–312. https://doi.org/http://dx.doi.org/10.14483/udistrital.jour.calj.2015.2.a09 Malone, M. E. (2011). Assessment literacy for language educators. Center for Applied Linguistics Digests, (October). Retrieved from http://www.cal.org/resources/digest/digest_pdfs/assessment-literacy-for- language-educators.pdf Mertler, C. A. (2009). Teachers’ assessment knowledge and their perceptions of the impact of classroom assessment professional development. Improving Schools, 12(2), 101–113. https://doi.org/10.1177/1365480209105575 Mertler, C. A., & Campbell, C. (2005). Measuring Teachers’ knowledge & application of classroom assessment concepts: Development of the assessment literacy inventory. American Educational Research Association, 27. Retrieved from http://files.eric.ed.gov/fulltext/ED490355.pdf Popham, W. J. (2004). Why assessment illiteracy is professional suicide. Educational Leadership, 62(1), 1–2. Popham, W. J. (2011). Assessment literacy overlooked: A teacher educator’s confession. Teacher Educator, 46(4), 265–273. https://doi.org/10.1080/08878730.2011.605048 Schafer, W. D. (1993). Assessment literacy for teachers. Theory Into Practice, 32(2), 118–126. https://doi.org/10.2753/JEI0021-3624440303 Stiggins, R. J. (1991). Assssment literacy. Phi Delta Kappan, 72(7), 534–539. Stiggins, R. J. (1995). Assessment literacy for the 21st century. Phi Delta Kappan, 77(3), 238–245. Stiggins, R. J. (1999). Evaluating classroom assessment training in teacher education programs. Educational Measurement: Issues and Practice, 18(1), 23–27. https://doi.org/10.1111/j.1745-3992.1999.tb00004.x LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 21, No. 2, October 2018 206 Taylor, L. (2009). Developing assessment literacy. Annual Review of Applied Linguistics, 29, 21–36. https://doi.org/10.1017/S0267190509090035 Tsagari, D., & Vogt, K. (2017). Assessment literacy of foreign language teachers around Europe: Research, challenges and future prospects. Papers in Language Testing and Assessment, 6(1), 41–63. Retrieved from http://arts.unimelb.edu.au/__data/assets/pdf_file/0010/2349928/6_1_SI3Tsag ariVogt.pdf Vogt, K., & Tsagari, D. (2014). Assessment literacy of foreign language teachers: Findings of a European study. Language Assessment Quarterly, 11(4), 374– 402. https://doi.org/10.1080/15434303.2014.960046 Volante, L., & Fazio, X. (2007). Exploring teacher candidates’ assessment literacy: Implications for teacher education reform and professional development. Canadian Journal of Education, 30(3), 749–770. https://doi.org/10.2307/20466661 Xu, Y., & Brown, G. T. L. (2017). University English teacher assessment literacy : A survey-test report from China. Papers in Language Testing and Assessment, 6(1), 133–158.