IEEE Paper Template in A4 (V1) Linguistic, English Education and Art (LEEA) Journal Volume 5 Nomor 2, Januari-Juni 2022 e-ISSN : 2597-3819 p-ISSN : 2597-9248 DOI : https://doi.org/10.31539/leea.v5i2.3534 183 ENGLISH TEACHERS’ CHALLENGES IN CONSTRUCTING REMOTE ENGLISH SUMMATIVE ASSESSMENTS DURING COVID-19 PANDEMIC Nurwahidah1 UIN Syarif Hidayatullah Jakarta Maulana Rizki Hamidi2 UIN Syarif Hidayatullah Jakarta Nida Husna3 UIN Syarif Hidayatullah Jakarta Muhammad Farkhan4 UIN Syarif Hidayatullah Jakarta Didin Nuruddin Hidayat5 UIN Syarif Hidayatullah Jakarta Alek6 UIN Syarif Hidayatullah Jakarta nurwahidah20@mhs.uinjkt.ac.id1 Submit, 01-03-2022 Accepted, 22-05-2022 Publish, 01-06-2022 ABSTRACT This study aims to investigate English teachers’ challenges in constructing remote English summative assessments for junior high school students during the COVID- 19 Pandemic. The method used in this research is a qualitative method with descriptive research in case study design. The results of this study obtained from 33 English teachers of Junior High schools from 8 big provinces in Indonesia highlighted technical problems and the validity and reliability of the test become the main challenges when the teachers construct a remote summative assessment for junior high school students. Furthermore, the researchers concluded that during the COVID-19 Pandemic, most junior high school teachers face many challenges and can overcome those problems by being tech-savvy teachers. Keywords: COVID-19, English Teacher Challenges, Remote Learning, Summative Assessment INTRODUCTION The deadly Coronavirus has devastated the world, which originated in Wuhan, China. The damage caused detain in all aspects of human activities overall countries. As a result, the World Health Organization (WHO) proclaimed the https://doi.org/10.31539/leea.v5i2.3534 mailto:nurwahidah20@mhs.uinjkt.ac.id 2022. Linguistics, English Education and Art (LEEA) Journal 5 (2):183-193 184 COVID-19 outbreak a pandemic on March 11, 2020 (Bozkurt & Sharma, 2020) due to its significance of spreading. The impact of this Coronavirus automatically brings many changes in many aspects of life, such as economic, social, political, tourism, and the educational sphere (Bartik et al., 2020). One of the most affected factors is the educational field. According to the United Nations Educational, Scientific, and Cultural Organization (UNESCO), by mid-May 2020, more than 1.2 billion students will have stopped attending face-to-face classes (Daniel, 2020). During the Pandemic, all efforts are made to find ways out of the difficulties. For this reason, the government has begun implementing online or virtual instruction in place of face-to-face instruction. An urgent regulation was created due to this development, with additional benefits and drawbacks emerging (Siahaan, 2020). It can be seen that in higher education, online education affects all aspects of teaching and learning (Lee & Hidayat, 2019). As a result, the government has a critical role in resolving this issue, notably by releasing an emergency education curriculum. Because it is a simplification, the curriculum has a lot in common with the National Curriculum (Sanjaya, 2020). Furthermore, this emergency curriculum creates many challenges in educational sectors, especially for the teachers. The teachers’ teaching competence should be improved because it has become one of the adaptation needs of online education (Zhang et al., 2020). In other words, the teachers should master technology to engage with their students and measure the students’ comprehension. They should apply many teaching applications during virtual learning to create an effective teaching and learning process that will lead to the students’ good comprehension. Many researchers researched the challenges in assessing students during the COVID-19 Pandemic, such as conducted by Bozkurt & Sharma (2020), Zenki- dalipi et al., (2021), Perwitasari et al., (2021), Jankowski (2020), Guangul et al., (2020), but the kind of assessment (summative assessment) and the grade level (Junior high school) of the English teachers become the newest thing on this research which differs to the previous related study. Thus, based on the previous studies, this research is essential to be conducted concerning assessment in English education study since there is still a lack of references about it to the present year. LITERATURE REVIEW Assessment is an effort to obtain data or information from the learning process. It determines how sound students, classes or courses, or study programs are compared to particular learning objectives or criteria and outcomes. Therefore, it is essential to create an excellent assessment to collect data and information from various types and assessment techniques to get a picture of student learning progress. Hence, assessment is one of the most crucial aspects that must be 2022. Linguistics, English Education and Art (LEEA) Journal 5 (2):183-193 185 conducted. In this ongoing world pandemic, the learning process and evaluation of learning outcomes cannot be carried out directly. It puts the teacher in the phenomenon that most teachers have difficulty measuring learning outcomes. Teachers were challenged to develop fresh ideas and techniques for monitoring children's learning during the COVID-19 crisis. It would be a challenge for the teachers to control their students virtually. Online assessments are common in online learning, with teachers limited to proxy oversight of students, making it impossible to supervise and manage to cheat (Adedoyin & Soykan, 2020). They might cheat while doing the test, such as googling or asking someone else to make the assessment invalid and unreliable. However, constructing assessment during a pandemic would be difficult for the teachers, especially the summative one, which will measure graduation standards. Summative assessment can conclude students' level of mastery against predetermined competencies. Creating a qualified summative assessment needs some criteria that should be fulfilled to assess the student’s real comprehension. However, the pandemic situation discussed above leads the teachers to change their assessment methods since virtually teaching and learning are done. It puts the teacher in the phenomenon that most teachers have difficulty measuring learning outcomes. Teachers have to follow new protocols for distance learning, online and the like, that require teachers to change the way they teach, including rapidly assigning daily tasks. In the end, during the COVID-19 crisis, instructors were asked to develop new ideas and techniques to measure children's learning. Different methods may be needed to meet these challenges. Teachers may experience difficulties due to a lack of training and experience, especially in digital learning platforms. As a result, teachers and students could not improve the process and learning outcomes maximally due to the Pandemic. Hence, this study intends to explore the challenges faced by teachers in constructing assessments for junior high school students in a pandemic situation. Summative Assessment In the academic environment, assessment refers to testing or evaluation in general., Its goal is to realize what learners can do through grading, also known as summative assessment (Black et al., 2010). Designing the students’ assessment can help teachers clarify the learning objectives and content to help the teacher determine the parts of the standard curriculum (Daniel, 2020). However, Summative assessment assesses students' abilities as they approach the completion of a course (or at the end of a unit). Summative findings are expected to show a more thorough knowledge of learning outcomes than formative results. While according to Abduh (2021), summative evaluation is used to ensure that students have met their learning objectives or have attained the required levels of competence. 2022. Linguistics, English Education and Art (LEEA) Journal 5 (2):183-193 186 According to Guangul et al., (2020), criteria make an assessment qualified and valuable. When assessing the students, the teachers should consider the following criteria: validity, reliability, clarity, and avoiding susceptibility to technical problems. Due to the primary goal of assessment being to evaluate students' learning on a specific learning outcome, the assessment method should be appropriate for assessing their achievement of learning objectives, or, we can say, validity. Another notion is reliability. The most challenging aspect of remote assessment is maintaining academic honesty. Preventive measures should be included in the assessment design to ensure that the evaluation is free of cheating and fraud. Preventive methods should be used during the assessment process and detection mechanisms following submission. The next is clarity. Because there will be no face-to-face interaction throughout the assessment period, and students may not ask questions, the assessment should be clear to students. When students are unfamiliar with the new evaluation approach, it is vital to explain the procedures and expectations and present relevant examples thoroughly. It is true even if the questions are asked higher than usual. Before distributing the evaluation, make a list of sample questions and, if possible, go over them with the students online. The last is avoiding susceptibility to technical problems. Assessment of COVID-19 Pandemic The change from offline learning to virtual one also assaulted the student assessment due to COVID-19's abrupt strike. This outbreak has significantly impacted educational and assessment systems Andreou et al., (2021). Due to its complicated properties, the assessment becomes more difficult to reform and implement in virtual environments. Modifying students' natural attitudes toward learning and time management make it difficult to assure transparency, objectivity, impartiality, validity, and dependability during the evaluation process. It is often misunderstood and relies on first impressions during this time (Zenki-dalipi et al., 2021) According to Guangul et al., (2020), there are two types of assessment when the teachers assess their students during Emergency remote teaching and learning. The first is remotely proctored exams (time-constrained), and the second is open- ended assessments. All the teachers need to know the most appropriate types of remote assessments that should be conducted. They have to sharpen their knowledge and comprehension in choosing the proper strategies to assess their students to get valid and reliable ones. Proctored exams are examinations where the time is limited, and the teachers control the testing process. In contrast, an open- ended assessment is when the time is not limited or at a specific time but not exactly when the exam is done. Moreover, the teachers here do not directly control the students. 2022. Linguistics, English Education and Art (LEEA) Journal 5 (2):183-193 187 RESEARCH METHOD This study takes a qualitative approach and employs a descriptive research design. The descriptive study method is the method used to analyze the ongoing phenomenon. The type of descriptive study used in this research is a case study that focuses intensively on a particular object and learns it as a case. The research instrument used in this research is a questionnaire using google form. This questionnaire uses an Open-ended question and optional choices. Data collection techniques were carried out through a questionnaire by google form. The questionnaire was created and disseminated via Google Forms to English teachers, and 33 replies were received. The questionnaires were divided into three sections: the affirmative question of the challenge; section two, assessment tools and assessment types; section three, challenges they face in designing and constructing the assessment. Thirty-three respondents participated in this study from 8 various provinces in Indonesia. Hence, the data analysis procedure used in this research is the qualitative data analysis technique of Miles and Huberman, known as the interactive model. The last is the techniques of data analysis which process consists of data reduction, data presentation, and deriving conclusions or verification. FINDING Affirmative Question It is presented in Fig. 1, all respondents agreed that the spread of COVID- 19 creates many challenges for English teachers, especially in constructing summative assessments for students. The first question was asked to participants, related to their agreement and perception about facing challenges in constructing assessment during COVID-19. The affirmative question is about whether they agree or not that the spread of COVID-19 creates many challenges for English teachers, especially in constructing summative assessments for students. Figure 1. Affirmative Question of Challenges Agree 100% Disagree 0% Do you agree that the spread of COVID -19 creates many challenges for english teachers, especially in constructing summative assessments for students ? Agree Disagree 2022. Linguistics, English Education and Art (LEEA) Journal 5 (2):183-193 188 From the diagram above, it implied that all participants of different provinces (33 respondents) agreed that all of them face challenges in constructing summative assessments during the COVID-19 Pandemic due to the real situation that happened in Indonesia. Assessment Tools and Assessment Types The next question is about assessment tools and assessment types. The respondents are asked to choose one of them. It is presented in Fig. 2, that there are two assessment methods for conducting a remote summative assessment. The first is a question-based assessment that should be completed in a reasonable amount of time (maximum 3 hours). The second type is an evaluation based on an assignment or a project that can be completed and submitted over a long time (from 1 to 4 weeks). Fifteen respondents (43.7 %) used Question-based assessment in conducting a remote summative assessment, while 18 (56.3%) of them used the assessment-based assessment type. Hence, most of the respondents used assessment-based judgment. Figure 2. Types of Assessment Methods It is presented in Fig. 3, surprisingly, 21 of the respondents (65.06 %) chose Open-Ended Assessment in their remote summative assessment, while only 12 respondents (34.04 %) chose remotely proctored exams (time-constrained). Figure 3. Types of Remote Assessment 34.04% 65.06% R E M O T E L Y P R O C T O R E D E X A M S ( T I M E - C O N S T R A I N E D ) O P E N - E N D E D A S S E S S M E N T S T Y P E S O F R E M O T E A S S E S S M E N T Responses 43,70% 56.30% Q U E S T I O N - B A S E D A S S E S S M E N T A S S I G N M E N T - B A S E D A S S E S S M E N T ASSESSMENT METHOD 2022. Linguistics, English Education and Art (LEEA) Journal 5 (2):183-193 189 It is presented in Fig. 4 below, in constructing remote summative assessment, 23 of the respondents (71.9%) provided a test to the students, 6 (18.08 %) chose Portfolio, 4 (12.05%) chose Journal, 11(36.7%) chose Self-Assessment, and 5 (16.06%) chose Learning log, 12 (37.05%) chose self-assessment, 7 (21.09%) chose Role-play, 7 (25%) used others than mentioned above. Figure 4. Types of Assessment Challenges and the Ways to Overcome the Challenges In this part of the question, the respondents are given two types of questions, they are open-ended questions and optional ones. The open-ended question is the question that they can freely answer the five challenges they faced when they constructed a remote summative assessment. The optional question is they are asked to opt for one or more optional challenges that the writer list based on the criteria of constructing a good summative assessment. It is found from the respondents’ responses to an open-ended question of challenges that finding the exciting application for the students becomes the most challenge they face. Some of the students will trigger themselves when the project is very interactive to fulfill the assessment given by the teacher. Another challenge is finding the most accessible way to reach the students because some students live in rural areas where no signal becomes the problem or even do not have a cell phone. The third challenge is checking the validity and reliability of the test made. The fourth one is choosing the appropriate assessment, the fifth one is adjusting the difficulty level of the question, and the last is the teachers' creativity to create various types of tests and the ways of designing speaking assessments. Another issue discovered is computer and internet accessibility. In the next type of question which is an optional one, the respondents are asked to choose which challenges they faced during constructing the summative assessment and they may choose more than one challenge. The result shows that 50 % of the respondents answer that technical problems are the most challenging 71.09% 18.08% 37.05% 15.06% 40.60% 12.05% 21.09% 25% T E S T P O R T O F O L I O S E L F - A S S E S S M E N T L E A R N I N G L O G Q U E S T I O N I N G J O U R N A L R O L E - P L A Y O T H E R S T Y P E O F A S S E S S M E N T Responses 2022. Linguistics, English Education and Art (LEEA) Journal 5 (2):183-193 190 criteria in constructing a remote summative assessment. The same percentage is also found in the challenge of the validity of the test (50%), followed by the reliability of the test (14 participants) or 43.8%. Meanwhile, the last is the clarity of the test, which is only 4 participants or (12.5%) of the participants as shown in Fig. 5. Figure 5. Criteria in Constructing Remote Summative Assessment DISCUSSION The findings above show that this research is appropriate to be conducted since all the participants agree that running a summative assessment is a challenge for them. It is in line with the statement of Bozkurt & Sharma, (2020) that with the COVID-19 virus outbreak; it has become clear that the educational system is vulnerable to outside threats. Furthermore, the study conducted by Lukas & Yunus, (2021) found that one of the challenges that most EFL teachers found during a pandemic is assessing their students. Evriana, (2020) also stated that teachers found online assessment inconvenient and irrelevant, in line with this finding. Most students may cheat by googling or asking another person, making the online test invalid and reliable. However, most of the respondents used assessment-based assessment for their summative assessment, the reasons they chose might be due to the difficulty of question-based assessment being lower than for assignment/project-based assessment. The assessment is expected to be finished quickly, and proctoring could be considered if the infrastructure is available during the assessment period. However, the finding of this study is in line with the study conducted by (Guangul et al., 2020), which showed that most of their respondents preferred to use assignment/project-based assessment (68%), and only 32% of them used question- based assessment as their assessment methods. 50% 43.8 % 12.5 % 50 % CRITERIA IN CONSTRUCTING REMOTE SUMMATIVE ASSESSMENT Validity of the test Reliability of the test Clarity of the test Technical problem when you create the test 2022. Linguistics, English Education and Art (LEEA) Journal 5 (2):183-193 191 In addition, most respondents chose open-ended assessments for the types of Remote assessments. This finding is in line with the study conducted by Guangul et al., (2020) that 37% of the respondents chose remotely proctored exams. In other words, it can be said that most of the respondents chose an open-ended assessment. Many respondents opted not to proctor students while they were writing the exam due to their concerns about the infrastructure needed to do so. Various assignments or projects could be produced and provided to students for assignment/project- based evaluation. Because the assessment type and evaluation method may differ, assignment and project-based assessments were also addressed independently. As a result, particular questions were developed for this study to examine the assessment techniques of assignment and project-based work independently. For assignment-based assessment, respondents were given a variety of submission and evaluation methods to prioritize based on their relevance to their courses. In part of the types of assessments, the teacher may select more than one option (test, portfolio, Journal, or other.). It is also relevant to the statement that designing the students’ assessment can help teachers clarify the learning objectives and content to help the teacher determine the parts of the standard curriculum (Daniel, 2020). It is better to use the easiest one in an emergency. However, there are many challenges mentioned in the findings above, and those challenges are also found in the research conducted by Evriana, (2020). She also found that some teachers are new to technology, and the majority of them are using online assessments for the first time. As a result, teachers will require training to feel comfortable using an online assessment system. The school should give those children a fully equipped lab and internet access to remedy this problem. The technical problem relates to the teachers’ competence in using technology, online platforms, or appropriate applications. This finding is in line with Perwitasari's et al., (2021) statement that through this pandemic situation, the challenge of using technology gives the teachers the chance to obtain new experiences by studying various types of online media applications to create a good impact on online learning and assessment. Assessing students during this emergency remote teaching period seems complicated for the teachers. Thus, the teachers should be mindful and wise in overcoming the challenges they face. Referring to the challenges they faced, the respondents also expressed their ways to overcome them to learn from each other. What is more, the teachers should improve their competency in preparing for constructing the test. Hence, a study conducted by Jankowski, (2020) revealed that 42% of 116 respondents stated that all of their professional development needs, such as constructing an assessment, are needed during Emergency Remote Teaching (ERT). Besides, they should improve their capability or competence to check the validity and reliability of the test they made and determine the most 2022. Linguistics, English Education and Art (LEEA) Journal 5 (2):183-193 192 appropriate types and methods of assessments. Also, they have to improve their creativity in choosing the tools for doing the test would be the solution for the challenges they faced in constructing emergency remote summative assessments for the junior high school students. CONCLUSION The technical problem while creating the test is one of the challenges found in this research. Other challenges are finding the most appropriate application and checking the validity and reliability of the test. The next challenge is a technical problem while constructing the summative assessment and it became the most challenging for English teachers in Junior High School. Hence, to overcome the challenges that English teachers face, they have to enrich their knowledge and competency in using the media or platform to make the assessment run well and succeed. REFERENCES Abduh, M. Y. M. (2021). Full-Time Online Assessment during COVID-19 Lockdown: EFL Teachers’ Perceptions. Asian EFL Journal, 28(11), 26–46. https://www.researchgate.net/publication/349103573_Full- time_Online_Assessment_during_COVID_- 19_Lockdown_EFL_Teachers’_Perceptions Adedoyin, O. B., & Soykan, E. (2020). Covid-19 Pandemic and Online Learning: The Challenges and Opportunities. Interactive Learning Environments, 0(0), 1–13. https://doi.org/10.1080/10494820.2020.1813180 Andreou, V., Peters, S., Eggermont, J., Wens, J., & Schoenmakers, B. (2021). Remote Versus On-Site Proctored Exam: Comparing Student Results in A Cross-Sectional Study. BMC Medical Education, 21(1), 624. https://doi.org/10.1186/s12909-021-03068-x Bartik, A. W., Bertrand, M., Cullen, Z., Glaeser, E. L., Luca, M., & Stanton, C. (2020). The Impact of COVID-19 on Small Business Outcomes and Expectations. Proceedings of the National Academy of Sciences of the United States of America, 117(30), 17656–17666. https://doi.org/10.1073/pnas.2006991117 Black, P., Harrison, C., Hodgen, J., Marshall, B., & Serret, N. (2010). Validity in Teachers’ Summative Assessments. Assessment in Education: Principles, Policy & Practice, 17(2), 215–232. https://doi.org/10.1080/09695941003696016 Bozkurt, A., & Sharma, R. (2020). Emergency Remote Teaching in A Time of Global Crisis Due to CoronaVirus Pandemic. Asian Journal of Distance Education, 15(April). https://doi.org/10.5281/zenodo.3778083 Daniel, S. J. (2020). Education and The COVID-19 Pandemic. Prospects, 49(1–2), 91–96. https://doi.org/10.1007/s11125-020-09464-3 Evriana, L. (2020). The Implementation of Online Assessment in English Lesson at 2022. Linguistics, English Education and Art (LEEA) Journal 5 (2):183-193 193 10. 1–67. http://repository.iainpurwokerto.ac.id/id/eprint/7601 Guangul, F. M., Suhail, A. H., Khalit, M. I., & Khidhir, B. A. (2020). Challenges of Remote Assessment in Higher Education in The Context of COVID-19: A Case Study of Middle East College. Educational Assessment, Evaluation, and Accountability, 32(4), 519–535. https://doi.org/10.1007/s11092-020-09340-w Jankowski, N. A. (2020). Assessment during A Crisis: Responding to A Global Pandemic. National Institute for Learning Outcomes Assessment. Kebritchi, M., Lipschuetz, A., & Santiague, L. (2017). Issues and Challenges for Teaching Successful Online Courses in Higher Education. Journal of Educational Technology Systems, 46(1), 4–29. https://doi.org/10.1177/0047239516661713 Lee, J. Y., & Nuruddin Hidayat, D. (2019). Digital technology for Indonesia’s young people: The significance of SNS use and digital literacy for learning. MedienPädagogik: Zeitschrift Für Theorie Und Praxis Der Medienbildung, 35(Media literacy), 20–35. https://doi.org/10.21240/mpaed/35/2019.10.17.X Lukas, B. A., & Yunus, M. M. (2021). ESL Teachers’ Challenges in Implementing E-Learning During COVID-19. International Journal of Learning, Teaching and Educational Research, 20(2), 330–348. https://doi.org/10.26803/IJLTER.20.2.18 Perwitasari, F., Astuti, N. B., & Atmojo, S. (2021). Online Learning and Assessment: Challenges and Opportunities during Pandemic COVID-19. Proceedings of the International Conference on Educational Assessment and Policy (ICEAP 2020), 545(4), 133–137. https://doi.org/10.2991/assehr.k.210423.077 Sanjaya, J. B. (2020). Implementasi Kurikulum Darurat. 1, 161–174. https://doi.org/10.18326/jil.v1i2.161-174 Siahaan, M. (2020). Dampak Pandemi Covid-19 Terhadap Dunia Pendidikan. Jurnal Kajian Ilmiah, 1(1), 73–80. https://doi.org/10.31599/jki.v1i1.265 Zenki-dalipi, A., Dogani, B., & Ameti, M. (2021). Assessment of Students Achievement during The COVID. 834, 9–18. https://www.ceeol.com/search/article-detail?id=985134 Zhang, W., Wang, Y., Yang, L., & Wang, C. (2020). Suspending Classes without Stopping Learning: China’s Education Emergency Management Policy in The COVID-19 Outbreak. Journal of Risk and Financial Management, 13(3), 55. https://doi.org/10.3390/jrfm13030055