This study is intended to understand teaching quality of English student teachers when they conduct their teaching practicum. Teaching quality is conceptualized based on the principles of effective teaching resulted by teacher effectiveness studies. Thes IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 124 Teachers of English Assessment Literacy: Unveiling What They Know and Do SONI MIRIZON* Abstract Conducting assessment is one of the crucial responsibilities every teacher has to do. A teacher needs to assess his students both during and at the end of instruction; therefore he needs to have good assessment literacy in order to be able to carry out assessment successfully. This study investigated teachers’ of English assessment literacy at two different senior high schools in Palembang. This case study involved six teachers of English as participants. The data were collected using questionnaires, semi-structured interview, classroom observation, and documentation. Data obtained from questionnaires were analysed using descriptive statistics, while data obtained from the semi-structured interview were analysed using thematic analysis through coding. Data obtained from classroom observation and documents were used to verify the ones from the questionnaire and interview. The findings revealed that teachers of English were in fairly literate category of assessment literacy; not every teacher carried out formative assessment regularly; they encountered difficulties in assessing students learning due to lacked understanding in interpreting the basic competence of the curriculum which led to inappropriate indicators formulation; assessment items did not match with the formulated indicators; and lacked of variety in constructing question items for formative assessment. These difficulties are due to that they lacked of experience in constructing assessment items and lacked of formal training related to test items construction. These findings lead to the conclusion that teachers assessment literacy needs to be enhanced so that they could carry out accountable assessment to support their students learning. Keywords Assessment literacy, formative assessment, teachers of English * Associate Professor, Faculty of Teacher Training and Education, Sriwijaya University, Indonesia; smirizon@unsri.ac.id mailto:smirizon@unsri.ac.id IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 125 Introduction It is almost impossible for students to have attended school without having been exposed to a wide variety of educational assessment. Assessment has become a part of student learning process. It is an essential portion of teaching and learning activities. In this case, conducting assessment is one of the crucial responsibilities every teacher has to do in education process. Particularly, a teacher needs to regularly carry out classroom assessment as part of his or her duty to improve student learning. Classroom assessment plays central role since students achievement in learning is likely influenced by such assessment. Butler and McMunn (2006) point out that research reveals that when the assessment sets clear standards for learning, provides good feedback to students related to their performance, and is conducted continuously to monitor student learning progress, it is likely to increase student achievement. Classroom assessment has been much discussed in educational research for the last thirty years. A lot of literatures put more emphasis on teacher accountable classroom assessment (Black & Wiliam 1999) and statement about effectiveness of classroom assessment for student learning has been much acknowledged (Black & Wiliam 1998). Russel and Airasian (2012, p.3) assert, ―Classroom assessment is the process of collecting, synthesizing, and interpreting information to aid in decision making.‖ In this case, assessment becomes a continuous part of classroom life. ―Classroom assessment that involves students in the process and focuses on increasing learning can motivate rather than merely measure students‖ (Chappuis & Stiggins, 2002, p.1). Classroom assessment can be used to measure the three domains: cognitive, affective, and psychomotor. Although cognitive domain tends to receive more attention, teachers make assessment decisions for all three domains throughout the school year. What is necessary in classroom assessment is whether a teacher carries out assessment regularly. It is sad to say that most teachers find it not easy to give effective assessment that could give good impact on student learning. The difficulty is caused by limited direct assistance for classroom assessment (Black & William, 1998), complex nature of such assessment, and dispute between the practice of classroom assessment and existing policy (Martin-Kniep, 1998). Consequently, teachers have appeared sluggish to take action. Teachers are required to understand classroom assessment, standard to follow, and prototype to imitate, critics on the attempt they make, and continuous back up. In learning English, a teacher of English needs to carry out classroom assessment to assess his or her students both during the instruction (formative) and at the end of instruction (summative). These two assessment approaches contribute to student learning in vastly different ways. The former promotes assessment for learning. Assessment for learning requires that assessment occurs regularly and that the information gained is used to mould teaching and learning. This kind of assessment helps students recognize what they can or cannot do so that teacher can anticipate what best to do. In other words, it focuses on providing feedback for student improvement. The latter, also known as assessment of learning, does not emphasize on providing feedback for student achievement but rather focus on rating or comparing students’ achievement. To get a balance on the two is important. Unfortunately, most classroom assessment tends to focus on the latter. IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 126 Whichever assessment is dealt with, a teacher is required to be assessment literate in order that he is able to carry out assessment successfully. A teacher competence in carrying out assessment on student learning will give great influence on student achievement. In relation to this, Stiggins (2004, p. 26) states, ―a teacher needs to know and understand the principles of sound assessment‖. In other words, a teacher needs to be assessment literate. Popham (2009) asserts that an adequate level of assessment literacy is necessarily needed for teachers. Further, Popham (2011) points out, ―assessment literacy consists of an individual’s understanding of the fundamental assessment concepts and procedures deemed likely to influence educational decisions‖. In line with this, Stiggins (1991) argues that assessment-literate teacher knows what he assesses, why he assesses, how to assess, what obstacles may present in assessment, and how to apprehend the obstacles. In addition, Gamire and Pearson (2006) claim that assessment literacy include ―knowledge about assessment, critical thinking and reflective judgement skills, and capabilities in the use of content knowledge to solve practical problems‖. Research in assessment demonstrate that teachers of English still lack of assessment literacy in various countries such as in Iran (Jannati, 2015), in United States (Popham, 2011; Steadman, 1998), in Greek and Cypriot (Tsagari & Vogt, 2017). In Indonesian context, similar findings were also found (Azis, 2012; Bilmona, 2013; Mirizon, 2009; Saefurrohman & Balinas, 2016). There has been much research on teacher assessment literacy but the one related to teacher of English is still limited, especially in Indonesian context. This study focuses on the assessment literacy of the teachers of English at two senior high schools in Palembang; particularly teachers’ literacy in formative assessment in English teaching and learning from the perspective of Indonesian 2013 Curriculum. Therefore, this study aims at finding out: (1) to what extent the teachers of English at the two schools are literate on the formative assessment, (2) how the teachers of English at the two schools carry out formative assessment, and (3) difficulties encountered by the teachers of English in assessing students learning at the two schools. Literature Review The 2013 Curriculum is adopted as theoretical framework of this study. The data analysis was seen through the lens of formative assessment according to 2013 Curriculum. In 2013 Curriculum, classroom assessment has been set to fulfil the following principles: valid (measure what has to measure), objective (based on clear criteria and procedure), fair (neither benefited nor unbenefited students), integrated (an unseparated part of learning activity), open (assessment procedure, criterion, and rationale of decision making are accessible to others), holistic and continuous (cover all aspect of competencies), systematic (planned and gradual), criterion-referenced (based on set criteria), and accountable (can be accounted for the technique, procedure and result). The assessment scope and instrument have also been determined in 2013 Curriculum. The assessment scope covers attitude, knowledge, and skill, while the assessment instrument includes test, observation, individual/group assignment, or other assessment forms appropriate for certain competence characteristics. Assessment to measure learning outcomes covers formative and summative. The former is used to monitor learning IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 127 progress and to detect the need for improvement in the learning process, while the latter is used to find out the level of mastery of competencies, to determine the completeness of mastery of competencies, and to determine improvement or enrichment programs based on the level of competency mastery. The 2013 Curriculum is manifested in some syllabus of different subjects. According to the Minister of National Education and Culture Regulation No.59 Year 2014, the syllabus is developed by three parties: the central government, the school, and the subject teacher. The national government sets the Core Competence (Kompetensi Inti, KI) and the Basic Competence (Kompetensi Dasar, KD), while the school and the teacher are responsible to determine the Indicator of Competence Achievement, commonly known as Learning Indicator (Indikator Pencapaian Kompetensi, IPK) in which later developed into learning objectives, learning material, learning activities, learning resources/media, time allocation, and assessment. This syllabus needs to be developed by the school/teacher in order to address students’ needs, to actualize the institutional goal, and to develop content standard (core competence and basic competence into teachable materials used in learning activities of the school). Figure 1 below illustrates the process how Learning Indicator is developed from Basic Competence. Figure 1. Determination of competency achievement indicator The learning indicator is developed from basic competence to measure the cognitive, psychomotor, and affective domain and is formulated using operational verbs (kata kerja operasinal, KKO). Table 1 is the example of formulating learning indicators from the basic competence of Year 10 learning material in knowledge area. Basic competence has been set by the central government but the learning indicators have to be developed by the schools or the teachers of English. The learning indicators, which are derived from the basic competence, consist of two—the bridging and targeted indicators. Learning indicators 3.4.1 to 3.4.3 are the bridging indicators which function as the bridge to reach indicator 3.4.4, the targeted indicator. The targeted learning indicator has to have higher cognitive domain than the bridging indicators. Learning indicator 3.4.4 is in the C4 (analysis) level of the cognitive domain, while learning indicators 3.4.1, 3.4.2, and 3.4.3 are in lower level (C1, C2, C3, knowledge, comprehension, application respectively) of the cognitive domain. Competence HOTS Learning Indicator Material Basic Competence LOTS IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 128 Table 1. Formulation of learning indicator Determining the Learning Indicator Basic Competence K.D. 3.4 Differentiating the social function, text structure, language features of some descriptive texts by asking and giving information related to famous tourism destination and historical buildings based on the given context Subject Matter Social function, text structure, language features of descriptive texts related to tourist destination and famous historical buildings Learning Material Learning Indicators 3.4.1. Identifying the social function, text structure, language feature of descriptive text related to famous tourism destination. 3.4.2. Explaining the social function, text structure, language feature of descriptive text related to famous tourism destination. 3.4.3. Using text structure, language feature of descriptive text related to famous tourism destination. 3.4.4. Differentiating the social function, text structure, language feature of descriptive text related to famous tourism destination. Social Function: sell, introduce, identify, criticize, justify, stress Text Structure: could cover 1. Identify (name), 2. Character (size, color, number, shape, etc.), 3. Function, benefit, action, habit. Language Features: 1. Vocabulary and related terminology to tourist destination and famous historical buildings. 2. Adverbs, such as quite, very, extremely, etc. 3. Declarative and interrogative sentences in correct tenses. 4. Using comparison with other object. Referring to the basic competence 3.4 in Table 1 above, it is expected that students are able to differentiate the social function, text structure, language features of some descriptive texts by asking and giving information related to famous tourism destination and historical buildings based on the given context, in which then derived and developed into four learning indicators 3.4.1, 3.4.2, 3.4.3, and 3.4.4. These learning indicators determine what learning materials to learn, how the learning materials are taught in the classroom, what learning sources can be used, what teaching media are appropriate, the time allocated to teach learning materials, and how learning materials are measured in the formative and summative assessment. The assessment is made to measure the students’ mastery of the learning materials which are derived from the learning indicators. In other words, assessment has to agree with the learning indicator because if it is, then the assessment is valid, which means it measures what it has to measure. So, learning indicators determine what to measure in the classroom assessment both formative and summative. Methodology Research design, site, and participants A case study was adopted as the design of the research. This study involved two different high schools in Palembang, one public and one private. These two schools shared similar characteristics, such as both schools were located in the same city district, were accredited A, and had been established for more than twenty years. The public school had six teachers of English, while the private one had five teachers of English. They were all invited to participate in the study, but not all were available and able to take part in the study. IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 129 Some of them were taking leave, were out of school attending in house training, and others were busy studying for higher degree. As a result, only six teachers of English participated as the subject of the study. Each school was represented by three teachers of English teaching at year 10, 11, and 12 respectively. Data collection and analysis The data of the study were collected using questionnaires, semi-structured interview, classroom observation, and document. Two questionnaires were used to find out the subjects’ understanding of (formative) assessment literacy. The first questionnaire was Classroom Assessment Literacy Inventory (CALI) developed by Mertler (2003) consisting of 30 items. It was used to see the teachers’ of English classroom assessment literacy in general. The second questionnaire was adopted from Pratiwi (2015) consisting of 20 items. It was used to see the teachers’ of English classroom assessment literacy related to the current curriculum implemented in Indonesia—Curriculum 2013. This questionnaire has three constructs: (1) concept of assessment (5 items), (2) assessment principles and coverage (5 items), (3) assessment techniques and instruments (5 items), and (4) scoring guide (5 items). Semi-structured interview was used to explore how the (formative) assessment literacy was put into practice by the teachers of English in the classroom. The questions were adopted from classroom assessment literacy inventory. Classroom observation was conducted to see whether the teachers of English really applied formative assessment as what they reported in the interview. This observation was conducted for twelve times during four months period of data collection. Document was used to see the written evidence of the implementation of the formative assessment carried out by the teachers during teaching and learning process, such as the syllabus, lesson plans, teacher’s book, students’ books and workbooks, and teacher-made tests that were used for formative assessment. The quantitative data obtained from questionnaire were analyzed using descriptive statistics, while the qualitative data obtained from the semi-structured interview and classroom observation note were analyzed using thematic analysis through coding. To check and ensure the consistency of the analysis, transcriptions of the interview were given to the subjects in order to confirm their answers and to keep the trustworthiness of the study. They could see the interview transcription in order to recheck the originality of the answers and avoid the ambiguity or uncertainty of the interview results. Data obtained from documentation were used to crosscheck the ones from the questionnaires and semi-structured interview. In short, triangulation of the data obtained from questionnaires, semi-structured interview, classroom observation, and documentation was carried out to get the credible and accurate conclusion. Findings The questionnaires were used to find out to what extent the teachers of English at the two schools were literate on the formative assessment. Findings from the questionnaires revealed that teachers of English were fairly literate in formative assessment which was indicated by the score of the responses as summarized in Table 2 and Table 3. IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 130 Table 2. Teachers’ of English assessment literacy based on Mertler (2003) questionnaire Score Category* Senior High School 1 (Private) Teacher 1# Teacher 2 Teacher 3 10 12 22 Not literate Fairly literate literate Senior High School 2 (Public) Teacher 4 Teacher 5 Teacher 6 16 10 25 Fairly literate Not literate literate Note: *Scoring category: 1-30, where 1-10 is not literate, 11-20 is fairly literate, and 21-30 is literate. #Teacher 1: BA, 1-5 years teaching experience Teacher 4: MA, 1-5 years teaching experience Teacher 2: BA, 6-10 years teaching experience Teacher 5: BA, 6-10 years teaching experience Teacher 3: MA, 11-15 years teaching experience Teacher 6: MA, 11-15 years teaching experience Table 3. Teachers’ of English assessment literacy based on Pratiwi (2015) questionnaire Score Category* Senior High School 1 (Private) Teacher 1 Teacher 2 Teacher 3 17 38 41 Not literate Fairly literate literate Senior High School 2 (Public) Teacher 4 Teacher 5 Teacher 6 39 19 43 Fairly literate Not literate literate Note: Scoring category: 1-60, where 1-20 is not literate, 21-40 is fairly literate, and 41-60 is literate. Based on the data in Table 2 and Table 3 above, it can be seen that teachers of English at both schools fell in the fairly literate category in average. In particular, teachers of English at each school were in not literate, fairly literate, and literate categories and the score of their responses were quite similar at both questionnaires. Finding from the interviews The interview was used to explore the teachers of English formative assessment literacy, how it was put into practice in conducting classroom assessment, and what difficulties encountered by the teachers of English in assessing students learning at the two schools. Five key findings emerged from the analysis of interview data: (1) teachers’ fairly literate category of assessment literacy, (2) not every teacher carried out formative assessment regularly; It was only given after teachers finished teaching one basic competence, (3) teachers lacked of understanding in interpreting the basic competence of the curriculum which led to inappropriate indicators formulation, (4) most of assessment items did not match with the formulated indicators, and (5) lacked of variety in construction question items for formative assessment. These difficulties are due to that they lacked of IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 131 experience in constructing assessment items and lacked of formal training related to test items construction. When all the subjects were asked about the essence of assessment in education, especially formative assessment, they reported that assessment is compulsory to help student learning as indicated below. ―… I believe assessment is important and needed in education. We use it to see how far students make progress in studying English. If there is no assessment, how do we know they are progressing or regressing? …‖ [Teacher 3] ―You know, I usually assess or test my students when we finish discussing a chapter or a topic. Besides to know whether they meet the instructional objectives in studying English, it can also be used to know how successful I teach them …‖ [Teacher 6] Although those teachers of English believed that good assessment literacy is crucial and is required in supporting student learning, not all of them seemed to have good assessment literacy when they were asked about the concept and procedures of assessment, as indicated by the following interview quotes. ―I think assessment is the test… yes only using test… I don’t think teachers can assess student learning without test… how can we decide the score?... [Teacher 1] ―What I know test is the best way to assess student learning… you know what… using test teachers can measure student progress, I only use test when I want to assess my students…‖ [Teacher 5] The above statements indicate that those teachers were not literate toward the assessment. However, other teachers of English had better understanding about assessment. They knew what they assessed, why they assessed, how to assess, and what obstacles might present in assessment, as reported below. ―Every subject matter teachers teach needs to be assessed in order to know students’ progress. Assessment should be carried out periodically… so that students are encouraged to learn…assessment can be done using test, portfolio…‖ [Teacher 6] ―I sometime gave test to assess my students understanding of what I taught… but sometime I also used observation when I wanted to know their understanding … even test was more frequently used…‖ [Teacher 4] ―When I gave oral test such as speaking, I found it difficult to do because there were so many students to test in a class. Since it would take much time if every students was tested individually, then I asked them to work in groups… by doing so, it was easier to assess them in group…‖ [Teacher 3] IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 132 Compared to Teachers 1 and 5, Teachers 3, 4, and 6 appeared to have better assessment literacy. They knew the assessment concept and procedures better than Teacher 1 and 5. When it is examined, it seems that teaching experience and educational qualification affect Teachers 3, 4, and 6 responses. These teachers have been working for ten years or more, while Teachers 1 and 5 have shorter teaching experience. Usually the longer the teachers work, the more experience and knowledge they get. In addition, Teachers 3, 4, and 6 have better educational qualification compared to Teachers 1 and 5. The former are master’s degree graduates whereas the latter are bachelor degree graduates. Academic qualification seems to affect their competence. In relation to how frequent the teachers of English at the two schools carried out formative assessment, it was found that even they did the assessment but not all of them conducted formative assessment every time they finished teaching a topic. In other words, the holistic and continuous principle of classroom assessment is not fulfilled, as reported in the interview quotes below. ―I know a teacher has to assess student learning every time he or she finishes teaching a given topic… since it has been prescribed so I did it regularly…‖[Teacher 6] ―… assessing student learning regularly is very important… that’s what I know and put in mind. If it is not done regularly, how do we know students’ progress or at least how do we know whether they got what we taught? ―[Teacher 3] ―… for me, sometime I gave formative assessment once I finished discussing the targeted topic… but sometime I didn’t… especially if the time was not enough…‖ [Teacher 2] ―I tried my best to give formative assessment every time I finished teaching one basic competence in the syllabus… but… due to some reasons such as limited time available, not all students seemed to understand the material, and students’ extracurricular activities… I did not do it…‖ [Teacher 4] Interview excerpts above revealed that actually those teachers realized that conducting formative assessment is required every time a teacher has finished teaching a given topic in one basic competence. They were aware that such an assessment was useful to measure student learning to see whether students were able to achieve the targeted learning objectives or not and whether they were successful in teaching or not. However, not every of those teachers could make it due to some reasons such as limited time available, not all students seemed to understand the material taught so that they deserved extra explanation, or due to other reason such as some students had extracurricular activities to do during the teaching hours so that not every students could join the formative assessment. When those teachers of English formative assessment literacy was further explored, especially the one related to the underlying competence to do classroom assessment based on the curriculum 2013, it was found that not all teachers understood what exactly to assess IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 133 from the topic discussed. It was due to their lack of understanding in interpreting the basic competence of the curriculum which led to inappropriate learning indicators formulation. It is plausible that it might be not easy to derive the basic competence into the learning indicators and develop them correctly. Teachers need to be able to understand the basic competence, know appropriate operational verbs, and consider the level of the cognitive domain in order to formulate the indicators. Moreover, the assessment of the student learning has to be in line with the indicators. When teachers lack of understanding of how the indicators are derived from the basic competence and do not know how to formulate the indicators, they would probably make mistakes in constructing appropriate assessment items that match with the indicators. This was apparent from the interview quotes below. ―What I know… teachers need to assess what they teach… and it should be based on the targeted competence stated in the curriculum. The problem is… you know… it is not easy to derive the basic competence into the learning indicators… we need to know the learning domain and appropriate operational verbs to use…‖ [Teacher 1] ―I think I get difficulties in formulating the learning indicators from the basic competence… it’s confusing… most of the time I just copied from colleagues from other schools…‖ [Teacher 5] Such responses indicate that those teachers had problems related to the understanding of the curriculum 2013. Their lack of understanding in interpreting the basic competence and indicators formulation affected their ability in constructing the assessment items. In other words, the assessment items they constructed were not in line with or did not match with the indicators which were stated in the lesson plan. As the result, the assessment items did not assess what they had set in the lesson plan. It has been proved that most of assessment (test) items those teachers made did not match with the formulated indicators. In this case, the test items are not valid since they did not measure what they had to measure. The following quotes reported such limitation. ―When I assessed the students learning after I finished teaching a topic or a chapter, I sometime gave them formative test. In deciding what to test I usually selected some questions from the students’ workbook and also tried to find the related ones from the internet… you know… not easy to make ones…‖ [Teacher 1] ―I also made assessment items such as a test by myself. Usually I checked what the students have learned. I tried to do my best… when I found it difficult to make some test items; I just took it from a collection of test items from the compiled test book…‖ [Teacher 2] The above quotes indicate that those teachers were ignorance about what exactly to assess in giving the formative assessment. They should have checked the learning indicators and targeted basic competence and constructed the test items based on them. This was for the IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 134 sake of constructing test items that were in line with what to achieve as stated in the learning indicators. Instead of doing this, they only selected some from the students’ workbook or found ones from the internet or even took them from a compiled test book, which were not necessarily related to and matched with the basic competence set in the syllabus and learning indicators written in the lesson plan. If that is the case, the assessment did not actually assess what it had to assess. In other words, the instructional objectives which were derived from the indicators would not have been able to achieve. Limitation in writing or constructing assessment or test items did not end in the above issue but also in providing the variety of test question types. Instead of using a variety of test question types such as multiple choices, true-false, matching, rearrangement, cloze procedure, completion, etc. in the objective test question types and short-answer essay, extended-response essay, problem solving and performance items in the subjective test, they preferred using a few of those question types. The most commonly used question type in the objective test was multiple choices, while the most commonly used one in the subjective test was short answer question type. It was apparent that the teachers lacked of variety in writing or constructing question items for formative assessment. The following interview quotes reveal such a limitation. ―To assess students’ knowledge, I preferred using multiple choices question types when I gave formative test. It’s easy to check and to score… besides, there were many stocks of such type available I could choose…‖ [Teacher 2] ―… yes multiple choices… it’s quite easy to assess and score, of course. Students could answer them relatively easy. The best thing is availability of question items… I could find it from many sources… so I did not need to make it by myself…‖ [Teacher 3] ―If I wanted to assess students’ language skills using essay test, I mostly used short answer type. The students could answer it relatively easy and I could check their answers in relatively easy as well…‖ [Teacher 4] ―As one criteria of good test is practicality, so I just wanted to be practical in assessing students’ work in formative test… I always gave them multiple choice questions… easy to check. I know it’s not easy to make the good ones but I could choose the test items from books and internet…‖ [Teacher 6] As it is known every question type has weaknesses and strengths. If the same question type is used all the time, its weaknesses cannot be covered by other question types. Besides, one question type cannot measure all of the levels in cognitive and psychomotor domains. It is recommended to use a variety of question types in both of the objective and subjective tests since the weaknesses of a certain type can be covered by other question types. In addition, students’ various levels of cognitive and psychomotor domains can be measured. When it was further investigated, those teachers of English limitation in constructing assessment items that matched the indicators was caused by two factors, namely: teachers IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 135 lacked of experience in constructing assessment items and teachers lacked of training related to test items construction. From those six teachers of English interviewed, four of them got difficulties in constructing test items that matched with the learning indicators, while the other two could relatively construct test items required, as shown by interview quotes below. ―I’ve been teaching for almost five years now... so far I tried to do my job by myself… in preparing a lesson… teaching the lesson…, including preparing assessment of student learning. I learned from documents such as syllabus, sample lesson plan, internet… and tried to do it by myself… I never had experience in constructing test by myself…‖ [Teacher 1] ―When I needed to do formative assessment, all I did… I tried my best to make the test item by myself… when I got stuck… when I found it difficult to make some items; I just took it from a collection of test items from the compiled test book… well I was not experienced…‖ [Teacher 2] ―… not easy… I tried already… I preferred selected test items from last year’s documents made by colleagues… even sometime I made it by myself, but I was not sure.‖ [Teacher 4] ―… I was not experienced in making the test items… but I always tried and tried, but I had no idea whether the items matched with the indicators, I think yes, but I didn’t know…‖ [Teacher 5] Such responses reported above are apparent. They lacked of experience in constructing assessment items. Even they have tried to construct the formative assessment items by themselves but in many parts the items were not matched with the learning indicators stated in the lesson plans. When further asked why such a limitation happened, they reported that they lacked of training related to test items construction, as reported below. ―I’ve been working as teacher for quite long time, but I never had a training dealing with test item writing. What I had was knowledge I learned when I studied in undergraduate education… not enough I think… (Teacher 5) There was a number of in house training I joined during my employment as a teacher of English, such as teaching methodology, ICT in teaching, etc.… but I never had one about assessment or language testing…‖ [Teacher 2] ―So far… It’s not easy to get a chance for joining training, may be because I am still junior. I’ve been working for less than five years… wait for an opportunity for professional development like training about assessment or testing… but there are other seniors who also need such opportunity for professional development…‖ [Teacher 1] IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 136 ―… no… not yet… I hardly had a chance for PD… such as training, especially about test item construction for assessment… I don’t know why… It seems that not every teacher can get a chance…‖ [Teacher 4] The above statements indicate that those teachers were in need for academic recharging such as training about assessment in general or language testing, especially the one related to test items construction which could support them in conducting formative assessment using test. Unfortunately such an opportunity was rare. Without a chance to have one, they would not have sufficient knowledge of how to construct good test items for classroom assessment. Lack of this knowledge may cause the teachers cannot construct valid test items for assessment. Finding from the classroom observations Classroom observation was held to examine whether the six teachers of English applied formative assessment as what they reported in the interview. Therefore the focus was on seeing how the teachers of English at the two schools planned and carried out the formative assessment when they were teaching in the classroom. Findings from this classroom observation uncovered that not every teachers consistently conducted formative assessment when they finished teaching a targeted topic of one basic competence. In other words, not every of them gave the assessment regularly. Sometime they assessed students learning and sometimes they did not. They should have assessed students learning every time they have finished teaching a targeted topic of one basic competence. In addition, when they gave assessment after teaching the targeted topic, they lacked of variety in writing or constructing test question types. For example, in giving an objective test, instead of using a variety of ways in assessing different language skills or aspects such as multiple choices, true-false, fill-in, matching, rearrangement, etc. they tended to use multiple choices. This is in line with the finding from the interview. These findings also support the findings from the questionnaires where most of the teachers of English from the two schools were fairly literate in assessment. In other words, they were not really assessment literate. Their fairly literate category in assessment literacy was reflected in the assessment items they constructed. Finding from the documents The focus of the documentation was to review the written evidence of formative assessment implementation made and used by the teachers in teaching and learning process. In this case, the English syllabus, lesson plans, and assessment instrument such as test items that the teachers constructed were paired to see whether they matched each other or not. Besides, teacher’s book, student’s book, and student’s workbook were also examined to see whether the test items written in the lesson plans and used in the formative assessment were actually taken from those documents or were written by the teachers themselves. The data obtained from the document analysis showed that not all items used in the formative assessment were made by the teachers themselves. Also, the test items used for the IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 137 formative assessment lacked of variety. Only certain question types such as multiple choices and short answer were used. Besides, non-test assessment instrument such as portfolio, observation, individual/group project were hardly used. Moreover, not all test items constructed were in line with the indicators or instructional objectives stated in the lesson plan. Ideally test items which were constructed should have assessed the knowledge or skills which had been taught. These findings supported the ones obtained from the questionnaires where teachers of English were fairly literate in formative assessment literacy, semi-structured interview where teachers encountered difficulties in assessing student learning, and classroom observation where not all teachers gave formative assessment after they finished teaching a given topic. Discussion It goes without saying that teacher competence is a vital constituent of student achievement, including teacher’s assessment literacy. Teacher good assessment literacy is one of the competences a teacher must have. Looking at the findings of the first objective of this study, it is apparent that in average teachers of English from the two schools were in fairly literate category of formative assessment literacy. From six respondents only two were literate (Teacher 3 and Teacher 6); the rest were not literate (Teachers 1 and Teacher 5) and fairly literate (Teacher 2 and Teacher 4) respectively, as shown in Table 2 and Table 3 above. When further examined it was found that teachers who worked longer and had better educational qualification (Teacher 3 and Teacher 6) were more assessment literate than that of others who had shorter teaching experience and lower educational qualification (Teachers 2, 4, 1, and 5). It seems that length of working experience and higher educational qualification gave benefit to their competence. This finding cohered with previous research reporting that length of working experience gave teachers more experience and made them confidence in their own abilities (Dickson, McMinn, & Kadbey, 2019); short experience of novice teacher in teaching affect their efficacy in performing expected task where guidance was still needed (Faez & Valeo, 2012). However, other studies informed that self-efficacy of teachers contribute more on students’ achievement rather than teachers’ length of teaching experience and qualification (Guo, Connor & Morrison, 2012) and teachers’ advanced degrees did not necessarily affect students’ achievement although teachers teaching experience may influence students achievement (Buddin & Zamarro, 2009). These imply that teachers’ length of working experience and/or education qualification does not necessarily determine their performance, including having good assessment literacy. In other words, teachers need to have good competence in content knowledge and pedagogical knowledge in order to be able to carry out their job regardless of the length of working experience and qualification they have. Referring to the findings of the second objective of this study, it is plausible that teachers of English did not regularly conduct formative assessment after they finished teaching one topic in a meeting but rather after finish teaching one basic competence (usually after three to four meetings) of a given syllabus. It was due to teaching time constraint or student mastery learning was not yet reached or not all students were available in the classroom. This action is certainly not recommended as formative assessment has to IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 138 be carried out based on some principles, such as unseparated part of learning activity (integrated), planned and gradual (systematic), and cover all aspect of competencies (holistic and continuous). Absence in assessing student learning in a meeting reflects teachers’ illiteracy of the principle of formative assessment, while a teacher willy-nilly needs to know and understand the principle of sound assessment as reminded by Stiggins (2004). Besides, it is detrimental to student learning as it did not provide feedback that students need related to their performance, and is not conducted continuously to monitor student learning progress (McNunn, 2006). Indeed, in this case an adequate level of assessment literacy is inevitably needed for teachers (Popham, 2009). In regard to the findings of the third objective of this study, it is reasonable that the teachers of English of the two schools encountered difficulties in assessing students learning. As it was reported, they lacked of understanding in interpreting the basic competence of the curriculum which led to inappropriate indicators formulation, assessment items did not match with learning indicators, and lacked of variety in constructing question items. It was admitted that these difficulties are due to their lacked of experience in constructing assessment items and lacked of formal training related to test items construction. When further learned, it was uncovered that the difficulties they confronted were definitely related to their illiteracy in assessment for learning. Ideally a competent teacher is free from such shortcoming, as argued by Stiggins (1991) that assessment-literate teacher knows what he assesses, why he assesses, how to assess, what obstacles may present in assessment, and how to apprehend the obstacles. Lack of teachers’ of English assessment literacy found in this study are in the same sound as the ones reported in Azis (2012), Bilmona (2013), Jannati (2015), Popham (2011), Saefurrohman and Balinas (2016), Steadman (1998), and Tsagari and Vogt (2017) that assessment illiteracy were also experienced by teachers in many parts of the world. Conclusion, Implication, and Recommendation This article aimed at reporting teachers’ of English assessment literacy, especially formative assessment at two different senior high schools in Palembang. Particularly it investigated to what extent the teachers of English at the two schools were literate on the formative assessment, how they carried out formative assessment, and what difficulties they encountered in conducting formative assessment at the two schools. Findings from the questionnaire revealed that in average teachers of English at the two schools fell in fairly literate category of assessment literacy. Only two of six teachers were literate, the other four were not literate and fairly literate respectively. Their illiteracy was apparent as they hardly had profound knowledge about assessment in general and formative assessment in particular regardless of number of years they devoted their time working as teachers of English at the schools and further education some of them had. Findings from the interview were in line with the ones from the questionnaire where respondents in general showed similar trend of literacy in the formative assessment. In particular, teachers of English from the two senior high schools encountered difficulties in assessing students learning indicated by their lacked of understanding in interpreting the basic competence of the curriculum which led to inappropriate learning indicators IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 139 formulation, most of assessment items they wrote did not match with the formulated learning indicators, and their lacked of variety in constructing question items for formative assessment. Findings from the classroom observation indicated that not every teacher carried out formative assessment regularly after they finished teaching one topic in a meeting. They sometime gave formative assessment in case the time allocated was still available, while the findings from documentation showed that not all assessment items were made by the teachers; some were taken from various sources. Besides, those items lacked of variety; only certain question types were used and items were not in line with the learning indicators stated. The findings of the study imply that teachers of English of the two schools in general were relatively not competent in assessing student learning as one main job of a teacher apart from planning and teaching. Good competence merely in planning and teaching is not sufficient. It has to be completed with good competence in assessing student learning as it determines whether students are successful in learning or not in which also reflects the success or failure of the teacher in planning and teaching the lesson. In order to be competent in assessing student learning a teacher needs to have good assessment literacy, especially formative assessment. Thus, assessment literacy is a compulsory competence a teacher of English has to have. The findings of this study lead to the conclusion that teachers assessment literacy needs to be enhanced so that they could carry out accountable assessment to support their students learning. In relation to that, some recommendations are offered. First, teachers of English need to really understand 2013 English Curriculum, especially the components stated in the syllabus such as the core competence, basic competence, learning indicators, learning objectives, learning materials, learning resources/media, assessment, and the relationship of each component to another so that problems such as lack of understanding in interpreting the basic competence of the curriculum which lead to inappropriate indicators formulation, assessment question items which are not in line with the indicators, lack of variety in conducting formative assessment would not happen. Second, They need to have continuing professional development related to assessment such as workshop on formative assessment, test item specification, test item construction, etc. so that they would be assessment literate. Sadly, such continuing professional development was hardly provided; mostly those related to teaching and learning English issue in general was discussed. Besides, not every teacher has equal opportunity to take part in the continuing professional development. Since this study focused on limited number of schools and participants, the results do not represent other teachers of English from other schools although similar trend may likely be to happen. Therefore, further research on formative assessment involving larger number of participants or schools focusing on specific issues related to English skills and aspects are necessary. Other possible issues to investigate and explore are teachers’ literacy in summative assessment (assessment of learning) and assessment as learning. Disclosure statement The author declares that there is no conflict of interest in relation to the publication of this article. IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 140 Acknowledgments The author would like to extend his sincere appreciation and gratitude to the Indonesian Research Journal in Education (IRJE) for serving as an avenue for the publication of this article. References Azis, A. (2012). Teacher conception and use of assessment in student learning. Indonesian Journal of Applied Linguistics. 12 (1), 41-51. Bilmona, H. (2013). The school English teachers’ perception on assessment toward teaching programs at UPT’S SPM laboratorium percobaan Bandung. Journal of English Language Teaching in Indonesia,1, 56-68. Black, P., & D. Wiliam. 1998. Inside the black box. Raising standards through classroom assessment. Phi Delta Kappan, 80 (2), 139–48. Black, P., and D. Wiliam. 1999. Assessment for learning. Beyond the black box. Assessment Reform Group. http://www.assessment-reform-group.org/publications. html. Buddin, R., & Zamarro, G. (2009). Teacher qualifications and student achievement in urban elementary schools. Journal of Urban Economics, 66(2), 103-115. Butler, S. M., & McMunn, N. D. (2006). A Teacher’s Guide to Classroom Assessment: Understanding and Using Assessment to Improve Student Learning. San Francisco, CA: Jossey-Bass. Chappuis, S., & Stiggins, R. J. (2002). Classroom assessment for learning. Educational Leadership, 60(1), 40-43. Dickson, M., McMInn, M., & Kadbey, H. (2019). Do years of teaching experience make a difference for teachers working in Abu Dhabi government schools? Cypriot Journal of Educational Science, 14(4), 471-481. Faez, F., & Valeo, A. (2012), TESOL teacher education: Novice teachers' perceptions of their preparedness and efficacy in the classroom. TESOL Quarterly, 46, 450-471.https: //doi.org/10.1002/tesq.37 Gamire, E., & Pearson, G. (Eds.). (2006). Tech tally: Approaches to assessing technological literacy (Free executive summary). Retrieved February 23, 2007, from http://www.nap.edu/catalog/ 11691.html. Guo, Y., Connor, C. M., & Morrison, F. (2012). The Effects of Teacher Qualification, Teacher Self-Efficacy, and Classroom Practices on Fifth Graders' Literacy Outcomes. The Elementary School Journal, 113(1), 3-24. Jannati, S. (2015). ELT teachers’ language assessment literacy: Perceptions and practices. The International Journal of Research in Teacher Education,6(2), 26-37. Martin-Kniep, G. O. (1998). Why Am I Doing This? Purposeful Teaching through Portfolio Assessment. Portsmouth, NH: Heinemann. Mertler, C. A. (2003). Classroom Assessment Literacy Inventory. Retrieved from http://pareonline. net/htm/v8n22/cali.htm http://www.assessment-reform-group.org/publications https://doi.org/10.1002/tesq.37 https://doi.org/10.1002/tesq.37 http://www.nap.edu/catalog/ IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 141 Mirizon, S. (2009). Communicative language testing: How teachers understand and apply it in constructing test items. Forum Kependidikan, 29(1), 83-89. Peraturan Menteri Pendidikan dan Kebudayaan Nomor 59 Tahun 2014 tentang Kurikulum 2013 SMA/MA. [Minister of National Education and Culture Regulation No.59 Year 2014] Popham, W. J. (2009). Assessment literacy for teachers: Faddish or fundamental? Theory into Practice, 48(1), 4-11. Popham, W. J. (2011). Assessment literacy overlooked: A teacher educator's confession. The Teacher Educator, 46(4), 265-273. Pratiwi, I. (2015). Tingkat pemahaman guru terhadap penilaian hasil belajar berdasarkan kurikulum 2013 pada mata pelajaran ekonomi di sekolah menengah atas (SMA) negeri se-kabupaten Sleman. (Undergraduate’s thesis). UNY. FE, Yogyakarta. Indonesia. Russel, M. K., & Airasian, P.W. (2012). Classroom Assessment: Concept and Applications. New York, NY: McGraw Hill. Saefurrohman & Balinas, E. S. (2016) English teachers classroom assessment practice. International Journal of Evaluation and Research in Education, 5 (1), 82-92. Stiggins, R. J. (1991). Assessment literacy. Phi Delta Kappan, 72(7), 534-39. Stiggins, R. J. (2004). New assessment beliefs for a new school mission. PhiDelta Kappan, 86, 22-27. Steadman, M. (1998). Using classroom assessment to change both teaching and learning. New Directions for Teaching and Learning, 75, 23-35. Tsagari, D., & Vogt, K. (2017). Assessment literacy of foreign language teachers around Europe: Research, challenges and future prospects. Papers in Language Testing and Assessment, 6(1), 41-63. Biographical note SONI MIRIZON is an Associate Professor at the English Education Study Program, Language and Arts Education, Faculty of Teacher Training and Education, Sriwijaya University, Palembang, Indonesia. He obtained his doctoral degree in English Learning and Instruction at Flinders University, Australia. His research interest includes assessment in education. He has been in the academe for more than 28 years. He is currently the Head of Master’s Degree in Language Education Study Program, Faculty of Teacher Training and Education, Sriwijaya University. Email: smirizon@unsri.ac.id mailto:smirizon@unsri.ac.id