International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – vol. 13, No. 8, 2019 Short Paper—Digital Assessment Resources in Primary and Secondary School Classrooms: Teachers’ … Digital Assessment Resources in Primary and Secondary School Classrooms Teachers’ Use and Perceptions https://doi.org/10.3991/ijim.v13i08.10730 Sri Kusuma Ningsih University of Muhammadiyah, Prof. Dr Hamka, Jakarta, Indonesia Herri Mulyono (*) University of Muhammadiyah, Prof. Dr Hamka, Jakarta, Indonesia hmulyono@uhamka.ac.id Abstract—This short article reports on teachers’ use and perception of digi- tal assessment recourses in primary school classroom. A total of eighteen pri- mary school teachers participated in the study where they were asked to experi- ence of using Kahoot and ZipGrade as digital assessment resources. Self- reflection survey was distributed to the teacher participants to capture their atti- tude and perception about the two applications. Findings revealed that teachers were positive towards the application of two assessment technology in class- room practice. The findings highlighted some factors that encouraged teachers to use Kahoot! and ZipGrade as digital assessment tools including the creation of fun learning environment, practicality, automated scoring and direct feed- back. Despite these benefits, two critical challenges were addressed by teachers when incorporating the digital application at school such as the school context as well as teachers. Keywords—Digital assessment, assessment resources, teacher attitude and per- ception. 1 Introduction The advancement of information and communication technology for teaching and learning has influenced the way teachers evaluate students’ learning performance. Digital technology for educational assessment enhances assessment capabilities, of- fering teachers an opportunity “for improvement and diversification in the evaluation of learners, including addressing written communication skills, cooperation, team- work, and reflective thinking” [1, p. 40]. Alderson [2] also argues that technology enables teachers to address the diversity of learners when undergoing classroom as- sessment. Moreover, assessment technological tools provide teachers with technical support to create tests, deliver students’ responses to test items, allow automatic scor- ing and reporting [3]. iJIM ‒ Vol. 13, No. 8, 2019 167 https://doi.org/10.3991/ijim.v13i08.10730 https://doi.org/10.3991/ijim.v13i08.10730 https://doi.org/10.3991/ijim.v13i08.10730 Short Paper—Digital Assessment Resources in Primary and Secondary School Classrooms: Teachers’ … This study aimed to explore teachers’ perception of incorporating two digital appli- cations for classroom assessment, Kahoot! and ZipGrade. Kahoot! is a free web and android based learning platform, a game-based student response system (GSRS) that combines the game-based learning approach and student response system or SRS [4], [5]. In the context of educational assessment, the combination of a game-based learn- ing approach and SRS has enabled both teachers and students to engage with “game- like pre-made or impromptu quizzes, discussions and surveys” [6, p. 49]. Although many authors have perceived Kahoot! as an online learning platform such as Bicen and Kocakoyun [7], Dellos [6], Graham [8], Plump and LaRosa [4], Zarzycka-Piskorz [9], there is evidence of the incorporation of Kahoot! in classroom assessment prac- tice in the literature, among others Ismail and Mohammad [10], Iwamoto, Hargis, Taitano and Vuong [11] and King [12]. Technically, Kahoot! is regarded as a simple and user-friendly application allowing instructors to use it at their ease [4]. In Kahoot!, teachers play their role as game show hosts and students act as players in a fun game show competing to earn points through answering various questions correctly [5]. Using the application, teachers are enabled to project the questions and alternative answers on a large screen, then students are asked to respond by clicking/pressing the colour and symbol of the correct answer on their own digital device, such as smartphone or tablet. Feedback of students’ answers is displayed between questions. The students’ score is evaluated not only through their correct answers but also by the amount of time they had spent to think of the possible answer and press the button in the applicaton. The other application, ZipGrade, is a grading application that enables teachers to accelerate the grading process [13]. LeHew [14] suggests that as a digital application, ZipGrade helps teachers scan students’ responses on ZipGrade generated answer sheets using their smartphone or tablet. As students’ responses are appropriately scanned, ZipGrade automatically analyses the responses and imports the result into a digital grade book in the application [15]. Teachers then can obtain the scores with additional details, such as score distribution charts and statistical analysis for each test item. Despite the benefits offered by the two digital applications as in the literature, our early observation prior to the study had revealed that only few Indonesian teachers in primary education have incorporated such applications to facilitate their classroom assessment. Lack of time, teachers’ insufficient experience, or doubts regarding the scholarly advantages of such application use in the classroom may be contributing factors to teachers’ reluctance to incorporate the applications in classroom practice [4]. 2 Method This current study aimed to explore teachers’ perception of the incorporation of two digital applications for classroom assessment: Kahoot! and ZipGrade. To this end, a total of eighteen primary and secondary school teachers participated in the study and were asked about their experience of using Kahoot! and ZipGrade as digital 168 http://www.i-jim.org Short Paper—Digital Assessment Resources in Primary and Secondary School Classrooms: Teachers’ … assessment resources. Most participants were females (N=15), with few males (N=3) aged between 25 and 45 years old. Participants were designated by number to main- tain anonymity (e.g. Teacher 12). Teacher participants were observed to have basic computer competence and internet browsing skills, allowing them to operate several Windows-based applications, such as Microsoft Word, Excel and Powerpoint. Teach- ers were also able to perform online activities such as email correspondence, upload- ing and downloading files to/from online databases and web browsing, activities which were believed to benefit teachers’ online assessment using Kahoot! and ZipGrade. 2.1 Research procedure Teacher participants were asked to take part in a one-day workshop on digital as- sessment methods for classroom practice held by the school. The workshop comprised two sessions, the first of which was a 90-minute seminar to provide teachers with knowledge regarding the nature of educational evaluation, classroom assessment and the role of technology to facilitate teachers’ classroom assessment. The second ses- sion was a teacher workshop in which they were introduced to the two digital applica- tions, Kahoot! and ZipGrade. Kahoot was introduced to facilitate online assessment in classroom practice, while the other application was aimed to help teachers with the scoring process in classroom settings. During the second session, teachers were given an opportunity to experience Kahoot! and ZipGrade with both teacher and student roles. After the workshop sessions, teachers were given three weeks to exercise digital assessment using Kahoot! and ZipGrade in classroom practice. 2.2 Data collecting method and analysis Data were collected using observation and a self-evaluation survey. The researcher observed the workshop of eighteen participants where they practised using Kahoot! and ZipGrade to capture teachers’ perceptions and attitudes towards the use of the applications for classroom assessment. In addition to observation, teacher participants were asked to complete a self-evaluation survey after three weeks of using the digital assessment tools in the classroom. Specifically, teachers were asked to respond to three questions: • What do you think of Kahoot! and ZipGrade as digital assessment resources in the classroom? • What benefits do you obtain when using Kahoot! and ZipGrade for classroom assessment? • What challenges do you encounter when using the two digital assessment tools? The collected data then were analysed qualitatively. iJIM ‒ Vol. 13, No. 8, 2019 169 Short Paper—Digital Assessment Resources in Primary and Secondary School Classrooms: Teachers’ … 3 Findings and Discussion Findings from both observation and self-evaluation survey revealed that teachers were positive towards the incorporation of technology for classroom assessment. Furthermore, teachers thought that Kahoot! and ZipGrade were of benefit to their classroom assessment practices as evidenced by Teacher 2 comment in the survey, “Kahoot! would be beneficial for the daily exams and it promoted the use of gadget for students’ learning”. Teachers considered that these digital assessment tools helped to create a fun learning environment, were practical, and they thought that the auto- mated scoring and direct feedback were positive factors encouraging their application. Teacher 4 said that students were excited to perform tests with Kahoot! She suggest- ed, “children were conditioned as they were playing a fun game instead of learning”. Furthermore, Teacher 1 and 2 mentioned the practicality aspects of the applications. Kahoot! and ZipGrade were considered easy to use, with icon menus and functions helpful in guiding navigation of the application. In addition, teachers affirmed that they allowed automatic scoring, providing stu- dents with direct feedback, which benefited teachers by speeding up their scoring, particularly for those teaching parallel classes at school (Teacher 1). Teacher 2 also commented that “ZipGrade application would help [her] in correcting students’ ex- am”. Such benefits have also been reported in Llamas-Nistal et al. [3]. Despite teachers’ positive perception of assessment technology, the findings re- vealed several challenges that teachers encountered when using Kahoot! and ZipGrade, such as school infrastructure, school policy, assessment types, teachers’ lack of knowledge and inappropriate scanning results. Some teachers felt that internet access was not reliable to support the use of Kahoot! and ZipGrade for classroom assessment. Teacher 3 affirmed that she was not using the two assessment tools fre- quently due to unreliable internet connection. She said that the poor internet connec- tion had resulted errors in Kahoot!, thus had prevented her from using the application. In addition, teachers mentioned that the incorporation of Kahoot! and ZipGrade in classroom assessment practice was restricted by school policy on smartphones. For many schools in Indonesia, the policy regarding whether smartphones are allowed to be used in the classroom varied and there has been dispute among teachers them- selves. In the current study, teachers said that the school administration had restricted students bringing smartphones to school, so teachers did not use Kahoot! particularly in the primary school classroom setting. Teacher 1 wrote in her reflection: “I felt that Kahoot! benefited me in ‘ice breaking’ session. Using Kahoot was fun and students enjoyed it. But, it was rather difficult to use Kahoot! so frequent. [it was because] students were restricted to bring smartphone to school. I had to obtain ad- ministrator’s permission [if I want] to use Kahoot!” Other challenges relevant to school policy concerned the answer sheet format. As discussed earlier, the ZipGrade application generated a formatted answer sheet to allow the application to scan students’ responses [13]. Unfortunately, the school ad- ministration had assigned teachers to use pre-prepared school answer sheets. Teacher 2 emphasised that “we have not yet agreed on the model for school answer sheet and whether or now we can use Kahoot! model of answer sheet”. 170 http://www.i-jim.org Short Paper—Digital Assessment Resources in Primary and Secondary School Classrooms: Teachers’ … Teachers’ reluctance to use Kahoot! was also driven by teachers’ lack of knowledge related to the application. For instance, Teacher 14 mentioned her igno- rance of using Arabic in Kahoot! had prevented her from not using the application for classroom assessment. However, according to the Kahoot! website, this application allows multiple languages (see Figure 1). Fig. 1. Language support in Kahoot! (Image source: https://create.kahoot.it/create#/new/quiz/description) In addition, teachers encountered inappropriate scanning results in ZipGrade! Four teachers reported that ZipGrade did not appropriately scan students’ answers, result- ing in some test items not being correctly scored. Teacher 6, for example, reported that she found some items were not scanned by ZipGrade: “when I did manual scor- ing, students’ score was different from in ZipGrade’s grade book”. A similar situation was experienced by Teacher 16, assuming that “I was thinking if pauses in the scan- ning process may affect the ability of ZipGrade in producing correct scan”. This is a critical issue for the ZipGrade developer to address in their further version develop- ment. The challenges that teachers have experienced during the incorporation of assess- ment technology were interesting, though not surprising. Discussion on teachers’ challenges in incorporating information and communication technology abound in the literature [16]–[19]. A literature review by Groff and Mauza [17] summarised several factors that affect the incorporation of technology in classroom contexts, two of which include school context and teacher factor. As the current study found, the school con- text, such as school culture, lack of support and resources, discouraged teachers from iJIM ‒ Vol. 13, No. 8, 2019 171 https://create.kahoot.it/create#/new/quiz/description https://create.kahoot.it/create#/new/quiz/description Short Paper—Digital Assessment Resources in Primary and Secondary School Classrooms: Teachers’ … integrating technology in classroom instruction practice. Teachers also reflected that they had insufficient knowledge and competence to use technology, with school ad- ministrators not responding to such a situation, neglecting to provide teachers with technological support and responses. Moreover, teachers felt they were restricted to obtain equitable access to classroom technology or other school resources. Without full support from school administrators, teachers will achieve little success in incorpo- rating classroom technology in their instructional practices [17], [19], [20]. 4 Conclusion The current study evaluated teachers’ perceptions of the incorporation of two digi- tal applications for classroom assessment: Kahoot! and ZipGrade, finding that teach- ers had a positive attitude and perception of the use of Kahoot! and ZipGrade in class- room practice. Teachers perceived that these applications benefited them in the crea- tion of a fun and enjoyable learning environment, enabling automatic scoring and direct feedback. Nonetheless, the issue of the school culture related to the use of smartphones in the classroom as well as little technological support and resources needs to be addressed for successful implementation of such applications, as these issues may constrain teachers from achieving the success in the use of technology for assessment activity in the classroom context. 5 Acknowledgement This current study was funded by the institute of community service and empow- erment, University of Muhammadiyah Prof. DR. HAMKA, Jakarta, Indonesia No. 589/H.04.02/2018. 6 References [1] L. Eyal, ‘Digital assessment literacy—The core role of the teacher in a digital environ- ment’, J. Educ. Technol. Soc., vol. 15, no. 2, pp. 37–49, 2012. [2] J. C. Alderson, ‘Technology in testing: The present and the future’, System, vol. 28, no. 4, pp. 593–603, 2000. [3] M. Llamas-Nistal, M. J. Fernández-Iglesias, J. González-Tato, and F. A. Mikic-Fonte, ‘Blended e-assessment: Migrating classical exams to the digital world’, Comput. Educ., vol. 62, pp. 72–87, 2013. https://doi.org/10.1016/j.compedu.2012.10.021 [4] C. M. Plump and J. LaRosa, ‘Using Kahoot! in the classroom to create engagement and ac- tive learning: A game-based technology solution for eLearning novices’, Manag. Teach. Rev., vol. 2, no. 2, pp. 151–158, 2017. https://doi.org/10.1177/2379298116689783 [5] A. I. Wang, ‘The wear out effect of a game-based student response system’, Comput. Educ., vol. 82, pp. 217–227, 2015. [6] R. Dellos, ‘Kahoot! A digital game resource for learning’, Int. J. Instr. Technol. Distance Learn., vol. 12, no. 4, pp. 49–52, 2015. 172 http://www.i-jim.org https://doi.org/10.1016/j.compedu.2012.10.021 https://doi.org/10.1016/j.compedu.2012.10.021 https://doi.org/10.1177/2379298116689783 https://doi.org/10.1177/2379298116689783 Short Paper—Digital Assessment Resources in Primary and Secondary School Classrooms: Teachers’ … [7] H. Bicen and S. Kocakoyun, ‘Perceptions of students for gamification approach: Kahoot as a case study’, Int. J. Emerg. Technol. Learn., vol. 13, no. 02, pp. 72–93, 2018. https://doi. org/10.3991/ijet.v13i02.7467 [8] K. Graham, ‘TechMatters: Getting into Kahoot!(s): Exploring a game-based learning sys- tem to enhance student learning’, LOEX Q., vol. 42, no. 3, p. 4, 2015. [9] E. Zarzycka-Piskorz, ‘Kahoot it or not? Can games be motivating in learning grammar?.’, Teach. English with Technol., vol. 16, no. 3, pp. 17–36, 2016. [10] M. A.-A. Ismail and J. A.-M. Mohammad, ‘Kahoot: A promising tool for formative as- sessment in medical education.’, Educ. Med. J., vol. 9, no. 2, 2017. [11] D. H. Iwamoto, J. Hargis, E. J. Taitano, and K. Vuong, ‘Analyzing the efficacy of the test- ing effect esing KahootTM on student performance.’, Turkish Online J. Distance Educ., vol. 18, no. 2, pp. 80–93, 2017. https://doi.org/10.17718/tojde.306561 [12] A. King, ‘Using Kahoot!’, Aust. Math. Teach., vol. 73, no. 4, pp. 35–37, 2017. [13] N. Muslu, ‘Exploring and conceptualizing teacher formative assessment practices and digi- tal applications within a technology-enhanced high school classroom’. Doctoral disserta- tion. The University of Missouri, 2017. [14] M. LeHew, ‘Technology Note: Efficiently analyzing in-class assessment data with ZipGrade’, J. Acad. Excell., vol. 5, no. 4, p. 7, 2018. [15] T. Chermer, C.-Y. Lee, A. Fegely, and L. Santaniello, ‘A detailed rubric for assessing the quality of teacher resource apps’, J. Inf. Technol. Educ. Innov. Pract., 2016. https://doi.org/ 10.28945/3527 [16] A. Al-Bataineh and L. Brooks, ‘Challenges, advantages, and disadvantages of instructional technology in the community college classroom’, Community Coll. J. Res. &Practice, vol. 27, no. 6, pp. 473–484, 2003. https://doi.org/10.1080/713838180 [17] J. Groff and C. Mouza, ‘A framework for addressing challenges to classroom technology use’, Assoc. Adv. Comput. Educ. J., vol. 16, no. 1, pp. 21–46, 2008. [18] R. Woods, J. D. Baker, and D. Hopper, ‘Hybrid structures: Faculty use and perception of web-based courseware as a supplement to face-to-face instruction’, Internet High. Educ., vol. 7, no. 4, pp. 281–297, 2004. https://doi.org/10.1016/j.iheduc.2004.09.002 [19] H. Mulyono, ‘Technology enhanced collaborative writing in Indonesian EFL classroom’. Doctoral dissertation. University of York, 2016. [20] Y. Zhao, K. Pugh, S. Sheldon, and J. L. Byers, ‘Conditions for classroom technology in- novations’, Teach. Coll. Rec., vol. 104, no. 3, pp. 482–515, 2002. https://doi.org/10.1111/ 1467-9620.00170 7 Authors Sri Kusuma Ningsih is a lecture at University of Muhammadiyah Prof. DR. HAMKA, Indonesia. She holds a master degree in English Education with research focus on teaching English as a foreign language. E-Mail sri_kusuma@uhamka. ac.id Herri Mulyono is a senior lecture at University of Muhammadiyah Prof. DR. HAMKA, Indonesia. He received his Ph.D from University of York. UK. His re- search interests include teaching English as a foreign language, computer assisted language learning (CALL), and teacher professional development. Article submitted 2019-04-24. Resubmitted 2019-06-02. Final acceptance 2019-06-03. Final version published as submitted by the authors. iJIM ‒ Vol. 13, No. 8, 2019 173 https://doi.org/10.3991/ijet.v13i02.7467 https://doi.org/10.3991/ijet.v13i02.7467 https://doi.org/10.17718/tojde.306561 https://doi.org/10.17718/tojde.306561 https://doi.org/10.28945/3527 https://doi.org/10.28945/3527 https://doi.org/10.1080/713838180 https://doi.org/10.1080/713838180 https://doi.org/10.1016/j.iheduc.2004.09.002 https://doi.org/10.1016/j.iheduc.2004.09.002 https://doi.org/10.1111/1467-9620.00170 https://doi.org/10.1111/1467-9620.00170 https://doi.org/10.1111/1467-9620.00170