ENGLISH REVIEW: Journal of English Education p-ISSN 2301-7554, e-ISSN 2541-3643 Volume 10, Issue 1, December 2021 https://journal.uniku.ac.id/index.php/ERJEE 297 THE EFFECT OF USING QUIZIZZ TO EFL STUDENTS’ ENGAGEMENT AND LEARNING OUTCOME Ahmad Munawir English Education Department, Teacher Training and Education Faculty, University of Sulawesi Barat, Majene/West Sulawesi, Indonesia Email : ahmadmunawir@unsulbar.ac.id Nurul Pratiwi Hasbi MAN 1Majene, Majene/West Sulawesi, Indonesia Email: nurulpratiwi880@gmail.com APA Citation: Munawir, A., & Hasbi, N. P. (2021). The effect of using quizizz to EFL students’ engagement and learning outcome. English Review: Journal of English Education, 10(1), 297-308 https://doi.org/10.25134/erjee.v10i1.5412 Received: 04-10-2021 Accepted: 27-11-2021 Published: 31-12-2021 INTRODUCTION Creating a learning environment through a gamification approach is not a very new concept. There are several studies are using this approach towards the classroom. One of them is a study by Bicen & Kocakoyun (2018). It was found that through the gamification approach, the students' interest was increased and their ambition for success was formed (Bicen & Kocakoyun, 2018). In gamification studies, internal motivation goals and competence have important effects on work engagement. (Muntean, 2011). It has been examined that it provided a positive impact on the motivational affordance and psychological/behavioral affordance (Hamari, Koivisto, Sarsa, 2014). It is a necessity to solve the problem of students faced in their learning experience –emotional and behavioral problems and learning difficulties (Evolution, n.d.). The problems are such as limited class hours in practicing language skills, limited authentic material resource and too many students in a class. For this reason, the e-learning model is an innovation to facilitate students in improving their learning abilities (Evolution, n.d.). Quizizz is a free online-based of a formative assessment tool that allows the teacher to conduct both assessments in class and as homework. It tends to be effectively used, helped to stimulate meta- cognitive of the student, liveliness in class and student attendance in advanced education which Abstract: The objectives of the research are to find out the extent of students' engagement in EFL learning before using Quizizz application, to find out students' score results from the test using Quizizz application, to find out the extent of students' behavior during the EFL learning and to find out the effect after using Quizizz to students' engagement and learning outcomes in EFL learning. This research applies a one-shot case study on explanatory quantitative research design to describe a phenomenon on the effectiveness of student engagement and learning outcomes after using Quizizz application in the EFL classroom of the 5th semester at the English Education Department in Universitas Sulawesi Barat. There are four classes and the researcher took one class purposively. The data are collected by employing preliminary and final surveys, treatments, direct observation, and tests. Structural equation modeling analyses show that using Quizizz application gave constructively contribution to some of the dependent variables. The effect of using Quizizz has the highest contribution or influence on the students' psychomotor outcomes. Next is to the effect of using Quizizz to the students' cognitive and affective outcomes. Moreover, student engagement is little or not significantly affected by the use of Quizizz. The conclusion highlights the importance, although currently ignored, ways that students contribute depends on the flow of the instruction they received. Moreover, the learning environment encourages students to think. As a concern to the teacher that students need motivational elements than a scoring system to increase their motivation. Keywords: Quizizz, gamification approach; students’ score; students’ behavior; students’ engagement; learning outcomes; EFL learning Ahmad Munawir & Nurul Pratiwi Hasbi The effect of using quizizz to EFL students’ engagement and learning outcome 298 demanding the limited participation of lecturers and students (Bicen and Kocakoyun, 2018). Although, some previous studies describing the benefits of using Quizizz and the students' perception of Quizizz application in the classroom, this study examines Quizizz application from a review of students' abilities: knowledge (cognitive), attitude (affective) and psychomotor in English as foreign language classroom. This study applies Quizizz application as a gamification approach towards the class to engage students by analyzing those students' abilities in being able to learn EFL. It is to find how the students' engagement and their learning outcomes are affected in terms of those students' abilities using Quizizz in learning EFL. Student engagement Studies on the engagement of the student in the teaching-learning process showed that there is no consensus about the definition of it (Fredricks, Blumenfeld, Paris, 2004). The concept was found its strong connection to a psychological and behavioral component (Beer, Clark, Jones, n.d.). Regarding engagement means to discuss students' behavior in the school (Sullivan et al., 2009). To the students who are in their adolescent development in the age of 15-19, the school experience is considered as not an opportunity to succeed, but rather as an obstacle to maturity (Papalia, Olds, Feldman, 2009). This obstacle occurs due to many problems students often faced such as emotional and behavioral problems and learning difficulties (Battin-Pearson, et al., 2000). The tension becomes more intense because of demands as rigid rules related to education in school (Appleton, Christenson, Furlong, M.J, 2008). Students often experienced boredom in school and few students took advantage of their study time, both inside and outside the classroom. Above emotional, behavior problems and learning difficulties are at risk of inhibiting the learning process. It can reduce academic achievement. Students need to increase involvement in learning including aspects: emotion; behavior and cognition to achieve academic achievement. Student engagement is the time and effort created for learning activities based on the goal or outcome of the school to encourage students to participate in these activities (Kuh, 2009). According to literature National Association of Independent School (NAIS) Report on the 2016 High School Survey of Student Engagement, there are three dimensions of engagement: cognitive, behavioral, and emotional engagement. Student cognitive engagementdescribes how students regulate themselves, create steady and deep technique for learning in their learning activities – the students' works do and the students' ways to go about what their work. Behavioral engagement includes students' interaction with their social, extracurricular and non-academic activities in the school. This involvement results in the positive attitude of the students during these activities. Emotional engagement is about the emotional reaction and affective relationship of the students during the lesson or other activities that composed as the school environment. Reactions such as excitement, well-being, anxiety, frustration, and disgust are the example. It deals with a feeling to the people, working ways and where they are in the school 'placed' (Joseph Corbett & Amanda Torres (2016) on NAIS Report on the High School Survey of Student Engagement," n.d.). Learning outcomes Learning outcomes is an expression of educational goals, which is the statement of what is expected, known, understood and can be done by students after completing a learning period (KKNI Team, 2015). Learning outcomes is more focused on what is expected to be carried out by student during or at the end of a learning process. Learning outcomes must be accompanied by proper assessment criteria that can use to assess the expected learning outcomes that have been achieved. Learning outcomes together with assessment criteria specifies the credit requirements (Butcher, Davies, Highton, 2006). Bloom's Taxonomy is one of the bases for categorizing educational goals, test preparation, and curriculum that has been used for almost half a century (Gunawan & Palupi, 2016). This taxonomy was first compiled by Benjamin S. Bloom in 1956 in his book Taxonomy of educational objectives and the learning domain is categorized into several domains and each of them is divided into levels/complexities of ability: Cognitive, affective, and psychomotor outcomes. Cognitive Outcomes contain behaviors that emphasize intellectual aspects, such as knowledge, understanding and thinking skills (solving complex problems). Knowledge called complete and board ENGLISH REVIEW: Journal of English Education p-ISSN 2301-7554, e-ISSN 2541-3643 Volume 10, Issue 1, December 2021 https://journal.uniku.ac.id/index.php/ERJEE 299 adapted to the context or needs. In this domain, the term Low Order Thinking Skills (LOTS) and High Order Thinking Skills (HOTS) is referred to which refers to the level of cognitive thinking ability according to the taxonomy developed by Bloom. Affective Outcomes emphasized the aspect of belief, ideas and attitude, such as interest, attitude, appreciation, and ways of adjustment. La Pierre incited Crano & Prislin, (2006), the meaning of word attitude can generally be described as mental tendencies or to be actualized in affective tendencies, both in a more positive or negative direction. It formulated in the form of attitude, affective tendencies are commonly expressed of likes – dislikes, agree – disagree, love-hate, like- dislike, and so on. Psychomotor Outcomes is the acquisition of physical skills that includes behavior that emphasizes aspects of motor skills such as handwriting, typing, swimming and/or operating machines. According to Bloom, psychomotor domains include physical skills, both physical (muscle) and complex coordination between the brain and muscles (Oermann, 1990). Quizizz Quizizz is an educational software example using game-based pedagogy that provides online questionnaires, discussions or exams. It can be designed in different formats (true or false, multiple choices, etc.). The teacher can set the quizzes as public or private. For public mode, the teacher can share his/her quizzes with other educators around the world. A free account is created on https://Quizizz.com/. To join the game, students do not need to sign up. Students can participate in the game by accessing https://joinmyquiz.com, enter the game code and type their nickname. The password given is randomly created which is shown on-screen that connects to the projector. A new password will be created automatically for each game. The teacher can also share the link via Google Classroom or remind. METHOD This research uses a one-shot case study on the explanatory form of quantitative research. Quantitative research is a systematic scientific on parts and phenomena and their relationships. It develops and uses mathematical models, theories, and or hypotheses relating to natural phenomena as their goals. Therefore, inferential statistics is the main tool to analyze data (Žukauskas, Vveinhardt, Andriukaitienė, 2018). The explanatory research method uses one group as the sample. The measurements of student engagement and learning outcomes are taken at first as primary data and references for the final survey and then we do certain treatments for a while. Once doing the treatment on the same group, the measurements are made as a final survey a second time. The research requires five instruments to collect the data and information needed. They are student engagement questionnaire (SEQ), Quizizz-based- test, report documentation, observation form, and learning outcomes questionnaire (LOQ). Participants This survey was administrated to the student of the Educational English department in the fifth semester at Universitas Sulawesi Barat. There were four classes where each class consists of approximately 25 students. In explanatory research, the sample is intended to explain the generalization of the sample to its population (Creswell, 2014). All the participants in this research are from class D. In this semester, students had studied ICT courses so it was easier to promote the Quizizz application as a learning media later. The selected students joined a class where the learning use Quizizz app as a mediating tool. All of the participants were male and female with a proficiency level of EFL is intermediate. They had the same level of proficiency (intermediate). Measures The questionnaire of student engagement (SEQ) in achieving expected learning outcomes is adapted from Reeve and Tseng, (2011) who used Wolters' (2004) learning strategies derived from Pintrich and DeGroot's (1990) widely-used Motivated Strategies for learning questionnaire. Pintrich and DeGroot's (1990) study written by Wolters (2004) about the use of learning strategies found a positive relation between seventh-grade students' endorsement of intrinsic goals and their use of cognitive and metacognitive-learning strategies. This research adapted measure features two subscales, one with items to assess the use of sophisticated (elaboration-based) learning strategies (items 1–4 in Table 3.1 indicator of the research) Ahmad Munawir & Nurul Pratiwi Hasbi The effect of using quizizz to EFL students’ engagement and learning outcome 300 and a second with items to assess the use of metacognitive self-regulation strategies such as planning before study, monitoring on tasks study, and revising one's work after studying. Learning outcomes' questionnaire (LOQ) is used to the survey of students learning outcomes which are conducted in the last meeting of the study. Its items indicators are adapted from Bloom's taxonomy and Bloom's Digital Revised Taxonomy. The contribution of the researcher is to change the context of the items to ensure students respond to their level of learning outcomes. Table 1. Questionnaire items to assess the aspects of student engagement learning outcomes Items to assess students’ cognitive engagement 1. When I am doing schoolwork, I try to relate what I am learning to what I have already known. 2. When I study in class, I try to connect what I am learning with my own experience. 3. When I study in class, I try to fit all the ideas so it makes sense 4. I make up my own example so it is easy to understand the important material. 1. Before I begin to study, I think about what I want to get done. 2. When I am working on my schoolwork, I stop once, in a while and go over what I have been doing. 3. When I study, I focus on how much I can understand, not only if I am getting the right answer. 4. If I am learning on difficult to understand, I change the way I learn the material. Items to assess students’ emotional engagement 1. I enjoy learning through Quizizz app in the class. 2. When I am learning through Quizizz app, I feel interested. 3. When I study by using Quizizz, I feel curious about what we are learning. 4. Quizizz makes learning fun! Items to assess students’ behavioural engagement 1. I listen to the material carefully. 2. The first time the teacher talks about a new topic, I pay a lot of attention. 3. I am difficult to be accustomed when we begin to learn something new in class. 4. I ask question to teacher when I meet difficulty in learning. 5. During in learning class, I express my preferences and opinions about the material to my teacher. 6. I pay attention to my teacher explanation in front of class. 7. I try to understand the material as I can while learning in class. Items to assess students’ cognitive outcomes 1. I have been able to make each example of all types of Sentence Structures without need to look at a notebook 2. I am able to describe the differences of the four types of Sentence Structures directly when asked by the teacher 3. I am adept at operating the Quizizz app well 4. I can get used to submit assignments through the Google Classroom app and will recommend it to other teachers using this app. 5. I can post question (quizzes) that I made myself to be played by my classmates Items to assess students’ affective outcomes 1. I pay attention to the subject as well about Sentence Structures because it is very important in English courses 2. I enjoy learning Sentence Structures through the Quizizz app 3. I organize my notebook to be completed and contains teacher’s explanation and my understanding of this material 4. When working in group, I express my ideas and get opinions from group members 5. I enjoy assignments both individually and in groups through Quizizz app given by teacher. Therefore, I get good scores 6. I am interested in learning English even harder. This is because applying Quizizz app on the learning Items to assess students’ psychomotor outcomes 1. Using Quizizz is effective in enhancing my engagement in the class. ENGLISH REVIEW: Journal of English Education p-ISSN 2301-7554, e-ISSN 2541-3643 Volume 10, Issue 1, December 2021 https://journal.uniku.ac.id/index.php/ERJEE 301 2. Doing in-class exercises using Quizizz is more helpful in my learning than doing in-class exercise on paper. Quizizz-based test is a test on English as a Foreign Language by using the Quizizz app. Quizizz reports documentation is real-time data of students' scores of the test accomplishment. The teacher can view the progress of the game use these data reports. Observation Form which the data is obtained by observing directly and indirectly by using video support. Data collection We do a preliminary survey to measure the extent of students' engagement before the study. The preliminary survey aims to gain a deep understanding of the conditions related to the students' engagement. Thus becoming an indicator refers to the measurement of students' engagement after the study, whether to be the same or different (increased). Quizizz is applied for one class where the sample (25 students) as the representation of all populations of the fifth semester. Quizizz is used frequently throughout the study. The purpose of researching using Quizizz is to investigate how student engagement influences their learning outcomes which emphasize the frequency of using Quizizz as a media tool of learning EFL. We collect data using a secondary data source. The data collection techniques are carried out by observing and giving questionnaires. We do a test that would later be the data to be used as a reference to measure the extent of student engagement in the classroom. It is intended as the reference to the final survey indicator where there is a significant effect or not. We give tests that we had named the Quizizz- Based Test (Q-BT) to all participants of Quizizz learning class in sections 1, 2 and 3. We teach Sentence Structure first before giving the test. Once teaching the material, we execute a test on Quizizz application named Quizizz-Based Test (Q-BT) in the 20 minutes-left of each section which first we explain the tutorial of Quizizz application. The tests are given frequently in sections 1, 2 and 3. Therefore, we evaluate their engagement during the treatment in-class by filling out an observation form. Evaluations are done at the end of sections. We survey at section 3 to examine their engagement and learning outcomes during the study. Data analysis Students' engagement questionnaire consists of 27 items which factors loaded to be measured consist of 9 indicators. They are (1) learning strategies and meta-cognitive self-regulation strategies loaded on cognitive engagement; (2) fun, interest, curiosity, and enjoyment loaded on emotional engagement; (3) expression of students' on-task attention, lesson involvement, and effort loaded on behavioral engagement. We analyze data of the result of student engagement preliminary survey by using a 5-point Likert Scale ranging from 1 to 5. We use a Google Form application to collect the preliminary survey data. It is displayed by diagram and statistical analysis. After analyzing the data, we do the validity and reliability test to the overall items of the questionnaire. We use Bivariate Pearson Correlation and Cronbach's Alpha technique to analyze the data survey on IBM SPSS Statistics 25. Direct observation is used to see the student behavioral and emotional involvement among individuals and particular situations (dealing with the class situation and/or other students). The data is displayed and arranged in the form of tabulated data. The item of each indicator in the observation form is recorded using the tally method. This relates to the frequency of the same indicator that we observe when in-class activities. Students' engagement questionnaire on this final survey is consisted of 26 items which factor loaded to be measured also consist of 9 indicators, same with the students' engagement questionnaire on the preliminary survey. The students' learning outcomes consist of 22 items which factors loaded consist of 11 indicators to be measured. They are (1) doing, connecting, applying, conceptualizing, and evaluating loaded onto cognitive outcomes; (2) receiving, responding, assessing, processing, and appreciating loaded on affective outcomes and (3) psychomotor outcomes. Both questionnaires also consist of positive and negative statements that use a 5-point Likert scale ranging from 1 to 5. Students fill out the questionnaires on the Google Form application. We also do validity and reliability test which measuring with the same techniques on the questionnaire on the preliminary survey. After the preliminary and final survey data are collected, we do a paired sample t-test. Before doing the test, it is necessary to ensure that the data Ahmad Munawir & Nurul Pratiwi Hasbi The effect of using quizizz to EFL students’ engagement and learning outcome 302 distribute normally by the test of normality. Then, data also are tested on a paired sample correlation test to know whether there significant correlation or not of both data. Once the data collected, the analysis technique of Structural Equation Modeling (SEM) is used to examine the effect of using Quizizz application to variables (student engagement and learning outcomes and EFL learning). Research model through SEM enables the researcher to answer the research question that is both regressive and dimensional (Hooper et al., n.d., 2008). SEM is a combination of two separate statistical models namely factor (regression) analysis and path analysis into one comprehensive statistical method (Bungin, 2005). RESULTS AND DISCUSSION After administering the preliminary survey, we did the treatment at three sections where each section was held per week from 4th to 18th November 2019. At the end of the third section where the treatment was finished, we administered a final survey to know the effect after using Quizizz application as media learning in their learning activities. The questionnaires contain statements related to student engagement and learning outcomes before and after using the Quizizz application. Student engagement on EFL learning before using Quizizz application The finding shows that most of the students know how to develop learning strategies. Firstly, students need to clear that to make the learning process run effectively, they must know what will be achieved or the aims from the first place. It is to avoid the ignorance to own learning. Secondly, their emotional engagement towards the learning shows good, although, some of them still have a neutral attitude whether they enjoy learning in class or not. To the factor loaded from students' behavior towards learning, students listen the material carefully, especially about the new topic. But, they need to take more time to be accustomed with. Further, some students prefer to ask if they find difficulty in learning. Related to their efforts in learning, many of them are confused to express preferences and opinions about the lesson to the lecturer. Sometimes, some prefer to talk with friends rather than listen to the lecturer's explanation. Moreover, almost half of them agree or doubt that they still have difficulty understanding the lesson. Quizizz-based-test result The reliability test of overall students' score results through Quizizz application showed an indication of whether the tests are reliable or not and is acceptable for both upper and lower group level (Hair, 2017). The test results showed some fairly unreliable tests, for instance in the first, second and last meeting on individual tasks (classic mode). The highest reliable tests are shown in homework mode. We consider this phenomenon occurs for some reason which is the same as the factors affecting the level of reliability written by Hair (2017). In the first meeting, study time is limited by other lecture hours caused a very brief explanation of the subject and doing the exercise on Quizizz in a hurry. Some students have not finished working quizzes while others guessed which the right answer. There are some students did not answer or complete their quizzes on homework mode. They said that they lost internet connection while working so they could not continue their work. This we state the weakness of this application requiring a strong internet connection before losing any work. But also, they have a lot of time to do homework rather than doing the test live-on class. Students' behavior during the EFL class The finding identified that students tend to discuss/ask their friends than to the teacher. However, all the students agreed to listen to the teacher's explanation or take a friend's note before missing the lesson. We also took control of the class by influencing students through orders. This is as our effort constructively to influence and to keep the students' attention in the class. Command sentences are "Listen to what explain, once the Quizizz is running, you will have no time to look the book, you will race against the time!", and "Pay attention to what explain, you won't get this such lesson in any lecture". On the medium category finds some students did something irrelevant during the teaching and learning activities. This is still tolerated. Some sets of class where require students to look and work on different team randomly several times caused this reaction. It was also found that in-class students prefer to ask a friend when meet problems operating ENGLISH REVIEW: Journal of English Education p-ISSN 2301-7554, e-ISSN 2541-3643 Volume 10, Issue 1, December 2021 https://journal.uniku.ac.id/index.php/ERJEE 303 Quizizz application. Meanwhile, in the low category was connected to the good criteria where students seem to discuss/ask a friend than to the teacher. Also, students do not read the related lesson from any resources except to the teacher explanations. Student engagement and learning outcomes in EFL learning after using Quizizz application Once done statistical analysis with the Structural Equation Modelling, the results are as shown in the following figure. Besides looking at the overall impact of using Quizizz on students' engagement and learning outcomes, we also analyze several possible effects of students' engagement and learning outcomes. The influence of these two dependent variables might be even greater than the Quizizz on both. Table 2. The empirical model of the role of Quizizz on student engagement and learning outcomes (n=24) Based on the calculation of the Structural Equation Modelling for the modified model, the result of the Goodness-of-fit index is summarized in the following table. It concludes that the results of the model analysis show the criteria for Goodness- of-fit models that are unfit. Those are indicated by the SRMR value (standardized root mean squared residual for the between-person model) which is greater than 0,08 and NFI value (normed fit index) which is smaller than 0,90. Unfit SRMR and NFI indicate the model cannot be accepted (Browne & Cudeck, 1993). The chi-square value (397,319) is much greater than the critical value 36,42 which according to Joreskog & Sorbom (1993) to assess the fit model, it is expected that the chi-square value is not significant (p-value > 0,05). This is because the result indicates that there is no difference between the model and the data. Table 3. Fit summary Saturated Model Estimated Model Critical Value Evaluation Model SRMR 0.154 0.187 ≥ 0,08 Unfit Chi-Square 367.732 397.319 ≥ 36,42 Fit NFI 0.238 0.176 ≤ 0,90 Unfit Table 4. Path coefficients Ahmad Munawir & Nurul Pratiwi Hasbi The effect of using quizizz to EFL students’ engagement and learning outcome 304 The roles of students' engagement and their learning outcomes turn out to be more indirect where through variable using Quizizz in-class and out-class. The tables show that those findings cannot be read as a single result that is sufficient to represent the true effect of using Quizizz because they stand equally as strong as evidence. Student cognitive engagement and cognitive outcomes after using Quizizz application The influence of Quizizz test scores has little impact or not related to the students' cognitive engagement. Student activities involve to learning strategies i.e. relating learning to what have already known and/or making own example to ease to understand the material while learning and activities involve to meta-cognition i.e. thinking about what to get done before begin to study, focusing on certain things that she/he can understand apparently cannot be measured only by direct questioning. This is because actually, cognitive involvement activity occurs when the learning process is ongoing. There are still quite a lot of respondents who chose to be undecided about the statement of cognitive engagement factors. We conclude students could not figure out the expression towards cognitive engagement factors. It is like the statement "I make up my own example so it is easy to understand the important material" dominates undecided responses. This needs further study when Reeve & Tseng (2011) use this kind factor loaded on engagement, whether this can be used to other research or only addressed to their research purposes. Based on the findings, the only reliable result used as an indicator for students' cognitive engagement activities during class is the students' scores result on Quizizz itself which improves in each section/meeting. Increased reliability is one proof that the level of difficulty of the test on Quizizz that we made fit to students who are in the upper group and lower group. To the cognitive outcomes results still seem better than the influence on student engagement above. It is as seen on the path coefficient chart, students' cognitive outcomes related to Quizizz around 28% and r square chat, Quizizz gives an effective contribution 23% to cognitive outcomes. Learning outcomes factors involve intellectual aspects such as knowledge, understanding and thinking skill. In fact, more respondents choose a neutral attitude towards the statements of learning outcomes factors that make the contribution or effect of Quizizz on learning outcomes look very small. The results are students operate Quizizz application well and at the same time, they get used to submitting the assignment on Quizizz. They also will recommend using this app for lectures. Moreover, some students can post quizzes they made to be played to friends. Students' emotional engagement and affective outcomes after using Quizizz application To the effect of using Quizizz to the student emotional engagement and affective outcomes seems to be improved compared to the first. It is shown by the path coefficient test where the effect of using Quizizz to the students' emotional engagement and affective outcomes is 0,35. Likewise, the emotional engagement survey findings look satisfying. Students enjoyed and felt interested in learning through Quizizz in the class. They felt curious about the learning and it made learning fun. ENGLISH REVIEW: Journal of English Education p-ISSN 2301-7554, e-ISSN 2541-3643 Volume 10, Issue 1, December 2021 https://journal.uniku.ac.id/index.php/ERJEE 305 However, a review of the overall score results on Quizizz seems not so. The score results show that many students got low scores and some of them did not accomplish answering all the quizzes. The fact happened in class that we observed shows the reason. Several students admitted somewhat 'depressed' working on Quizizz. This is due to the time demand. This is true, that the score on Quizizz is not only determined by the correct number of answers but also the speed in answering each question. So, even though some students have the same number of correct answers, it does not mean they have the same scores due to different duration answering each question. This why there are still among students guessed the answer, it is because of suggestion being the earlier finished. It is found that the model inequality between Quizizz and students' engagement on the structural equation modeling test is a different data figure. This phenomenon also found on the effect on students' affective outcomes. The data collected reveal that Quizizz was successful in capturing students' affective outcomes. The survey of this factor showed high mean scores instead of the Quizizz score results which mean scores tend to be low despite increasing in each section. This is what makes the path coefficient test model is not so effective or influential due to the different data figure we stated. However, the result of students' affective outcomes found that almost all students paid attention to what we taught. They enjoyed learning Sentence Structure and assignments both individual and group through Quizizz application. They also became not hesitate to get involved in group working and received feedback from group members. They were interested in learning English even harder. Students' behavioral engagement and psychomotor outcomes after using Quizizz application The first thing to know is that the study established that English as a foreign language is not only perceived as an important subject for the broader majority of participating students. But, it is how to bring English to be enjoyable learning in the EFL classroom. Furthermore, the findings revealed that using Quizizz changes students' behavioral engagement in-class regarding students' on-task attention, lesson involvements and students' efforts on learning English. The survey on students' behavioral engagement found that students listen to the material more carefully, tried to understand the material while learning and more interestingly, they got paying more attention to what the teacher talked about a new topic. Once you start playing on Quizizz, you have a little or even no chance to see your textbook due to time running. Especially, for the effect of using Quizizz to the students' psychomotor outcomes viewed by the r- square chart found that Quizizz has an impact or greatest significant contribution among others. This indicates that the most significant factor being influenced by the use of Quizizz is on the students' psychomotor outcomes. The varied respondents' answers tend to be the cause. However, all students agree that the use of Quizizz is effective in enhancing students' engagement in the class. Also, it found an interesting fact that because of the requirement, students should accomplish all quizzes. It makes students inevitably have to listen to the lesson in the first place. A fact also majority students agreed that doing in-class exercise through Quizizz is helpful than doing exercise on paper besides there are still at least four students who did not agree on it so. CONCLUSION Students vary in the way they react to the learning activities that teacher provides, as some students work harder with greater joy and prioritizing strategies in learning. These behaviors, emotional and cognitive are important in predicting students learning and their achievement in the future. But students are more varied in how much or little they are 'depressedly' learning because of the demands by the teacher as requirement in-class or school rules in general. With this students' engagement test as a way for how they can be enriched, personalized, or enhanced in their learning especially in English as a foreign language. The use of Quizizz found that students are more interested in the lesson, and being encouraged to be more ambitious for success. Similarly, the use of learning platform devices proved to have a positive impact on student motivation. Students responded that collaboration is very important during learning which supported Quizizz's application and make them help each other in learning. The students who play and learn on this application stated that the Reward and Leaderboard system increases their Ahmad Munawir & Nurul Pratiwi Hasbi The effect of using quizizz to EFL students’ engagement and learning outcome 306 motivation. The students can see the status of their progress on projection displayed and on the smartphone. They can evaluate which sub material that they are deficient. Additionally, a combination of a gamification approach to the learning method help students better understands the lesson. Badges make students feel important and students are easier to remember information when presented in a gamified manner. Competition helps them to improve their response to time management and allows them learning topics that difficult to understand. ACKNOWLEDMENT Alhamdulillahi Rabbil Alamin, the writers say greatest thanks to the Almighty God Allah SWT, bestowed his blessing, mercy, and guidance which Him alone we worship and ask for help. Peace be upon to the last messenger of Allah, the Prophet Muhammad SAW who guided us from the darkness to the lightness. This article has been finished under the assistance, direction, and guidance of some people. Therefore, the writers would like to express greatest gratitude and appreciation to all my colleagues at English Eduation Department of University of Sulawesi Barat. The writers hope this article can give positive contribution esepecially in Englsih language Teaching. REFERENCES Anderson, L.W., Krathwohl, D.R. (Eds.) (2001). A Taxonomy For Learning, Teaching, And Assessing: A Revision Of Bloom’s Taxonomy Of Educational Objectives, Complete Ed. Ed. Longman, New York. Andrew, C., (2008). Bloom’s Digital Taxonomy, revised. ed. Kristin School. Appleton, J.J., Christenson, S.L., Furlong, M.J. (2008). Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools, 45, 369– 386. https://doi.org/10.1002/pits.20303 Aşıksoy, G., Sorakin, Y. (2018). The effects of clicker- aided flipped classroom model on learning achievement, physics anxiety and students’ perceptions. Battin-Pearson, S., Newcomb, M.D., Abbott, R.D., Hill, K.G., Catalano, R.F., Hawkins, J.D. (2000). Predictors of early high school dropout: A test of five theories. Journal of Educational Psychology 92, 568–582. https://doi.org/10.1037/0022- 0663.92.3.568 Beer, C., Clark, K., Jones, D., n.d. Indicators of Engagement 11. Bicen, H., Kocakoyun, S. (2018). Perceptions of students for gamification approach: kahoot as a case study. International Journal of Emerging Technologies in Learning (iJET) 13, 72-. https://doi.org/10.3991/ijet.v13i02.7467 Biggs, J. (2003). Aligning Teaching for Constructing Learning. The Higher Education Academy. Bloom’s Digital Taxonomy. (2015). URL https://www.commonsense.org/education/video/bl ooms-digital-taxonomy Bloom’s Verb Digital Taxonomy. (2019). BINUS University. Broughton, G. (1980). Teaching English as a Foreign Language. Routledge education books. Routledge and Kegan Paul, London [etc.]. Bungin, B. (2005). Metodologi Penelitian Kuantitatif. Kencana. Butcher, C., Davies, C., Highton, M. (2006). Designing learning: from module outline to effective teaching, Key guides for effective teaching in higher education series. Routledge, New York, NY. Carsono, N. (2014). Penyusunan Learning Outcomes Mata Kuliah. Crano, W.D., Prislin, R. (2006). Attitudes and Persuasion. Annual Review of Psychology 57, 345–374. https://doi.org/10.1146/annurev.psych.57.102904. 190034 Creswell, J.W. (2014). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 4th ed. ed. SAGE Publications, Thousand Oaks. da Rocha Seixas, L., Gomes, A.S., de Melo Filho, I.J., (2016). Effectiveness of gamification in the engagement of students. Computers in Human Behavior 58, 48–63. https://doi.org/10.1016/j.chb.2015.11.021 Evolution, S.L., n.d. Pembelajaran Bahasa Inggris Abad 21. Semesta Learning Evolution. URL https://sle.co.id/news/pembelajaran-bahasa- inggris-abad-21/ (accessed 9.18.19). Explanatory Research, (2008). The SAGE Encyclopedia of Qualitative Research Methods. SAGE Publications, Inc., 2455 Teller Road, Thousand Oaks California 91320 United States. https://doi.org/10.4135/9781412963909.n164 Figueroa, J.F. (2015). Using Gamification to Enhance Second Language Learning, Digital Education Review. Universidad del Este, Puerto Rico. Frankfort-Nachmias, C., Nachmias, D. (1996). Research Methods in The Social Sciences, 5th ed. ed. St. Martin’s Press, New York. Fredricks, J.A., Blumenfeld, P.C., Paris, A.H. (2004). School engagement: Potential of the concept, state ENGLISH REVIEW: Journal of English Education p-ISSN 2301-7554, e-ISSN 2541-3643 Volume 10, Issue 1, December 2021 https://journal.uniku.ac.id/index.php/ERJEE 307 of the evidence. Review of Educational Research 74, 59–109. https://doi.org/10.3102/00346543074001059 Gadiyar, A.R. (2014). Gamification 3.0: The power of personalization. Gonyea, R.M., Kuh, G.D. (2009). NSSE, organizational intelligence, and the institutional researcher. New Directions for Institutional Research, 107–113. https://doi.org/10.1002/ir.290 Hair, J.F. (Ed.), 2017. A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM), Second edition. ed. Sage, Los Angeles. Hamari, J., Koivisto, J., Sarsa, H. (2014). Does gamification work? -- a literature review of empirical studies on gamification, in: 2014 47th Hawaii International Conference on System Sciences. Presented at the 2014 47th Hawaii International Conference on System Sciences (HICSS), IEEE, Waikoloa, HI, pp. 3025–3034. https://doi.org/10.1109/HICSS.2014.377 Kapp, K.M. (2012). The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education. Pfeiffer, San Francisco, CA. KKNI Team. (2015). Paradigma Capaian Pembelajaran. Koenigs, S.S., Fiedler, M.L., Decharms, R., 1977. Teacher beliefs, classroom interaction and personal causation1. Journal of Applied Social Psychology 7, 95–114. https://doi.org/10.1111/j.1559- 1816.1977.tb01332.x Kuh, G.D. (2009). What student affairs professionals need to know about student engagement. Journal of College Student Development 50, 683–706. https://doi.org/10.1353/csd.0.0099 Leaning, M. (2015). A study of the use of games and gamification to enhance student engagement, experience and achievement on a theory-based course of an undergraduate media degree. Journal of Media Practice 16, 155–170. https://doi.org/10.1080/14682753.2015.1041807 Mac Namara, D., Murphy, L. (2017). Online Versus Offline Perspectives on Gamified Learning. Maloshonok, N. (2014). Vygotsky’s theory: Lessons for Student engagement research. 13. Michelini, M., Groupe International de Recherche sur l’Enseignement de la Physique (Eds.), (2004). Quality development in teacher education and training: second international GIREP seminar 2003 ; selected contributions ; [1 - 6 September 2003, Udine, Italy]. Forum, Ed. Univ. Udinese, Udine. Muntean, C.I. (2011). Raising engagement in e-learning through gamification. University of Bucharest and “Babeş-Bolyai” University 324 of Cluj-Napoca, The 6th International Conference on Virtual Learning ICVL 2011 323–329. NAIS. (2016) NAIS Report on the High School Survey of Student Engagement [WWW Document], n.d. URL https://www.nais.org/articles/pages/research/2016 -nais-report-on-the-high-school-survey-of- student-engagement/ (accessed 9.10.19). Oermann, M.H. (1990). Psychomotor skill development. J Contin Educ Nurs 21, 202–204. Pace, C.R. (1984). Measuring the Quality of College Student Experiences. An Account of the Development and Use of the College Student Experiences Questionnaire. Higher Education Research Institute, Graduate School of Education, University of California, Los Angeles, CA 90024 ($7. Papalia, D.E., Olds, S.W., Feldman, R.D. (2009). Human Development. McGraw-Hill. Perry, V., Albeg, L., Tung, C. (2012. Meta-analysis of single-case design research on self-regulatory interventions for academic performance. J Behav Educ 21, 217–229. https://doi.org/10.1007/s10864-012-9156-y Porcaro, P.A., Jackson, D.E., McLaughlin, P.M., O’Malley, C.J. (2016). Curriculum design of a flipped classroom to enhance haematology learning. Journal of Science Education and Technology, 25, 345–357. https://doi.org/10.1007/s10956-015-9599-8 Poth, R.D. (2018). 12 digital tools to try in 2018. Getting Smart. URL https://www.gettingsmart.com/2018/01/12-tools- to-try-in-2018/ (accessed 3.26.19). Quizizz. (n.d) Quizzez raises $3 million and surpasses 10 million users [WWW Document], n.d. URL https://venturebeat.com/2018/08/09/Quizizz- raises-3-million-and-surpasses-10-million-users/ (accessed 3.27.19). Ramsden, P. (1992). Learning to Teach in Higher Education. Routledge, London ; New York. Reeve, J., Tseng, C.-M. (2011). Agency as a fourth aspect of students’ engagement during learning activities. Contemporary Educational Psychology 36, 257–267. https://doi.org/10.1016/j.cedpsych.2011.05.002 Rienecker, L., Jørgensen, P.S., Skov, S. (2015). The good paper: a handbook for writing papers in higher education. Robbins, S.P., Judge, T. (2007). Organizational Behavior, 12th ed. ed. Pearson/Prentice Hall, Upper Saddle River, N.J. Ssentence structure - Dictionary Definition [WWW Document], n.d. . Vocabulary.com. URL https://www.vocabulary.com/dictionary/sentence %20structure (accessed 9.30.19). Ahmad Munawir & Nurul Pratiwi Hasbi The effect of using quizizz to EFL students’ engagement and learning outcome 308 Williamson, S. D., Halverson, R., Squire, K. R. (2005). Video games and the future of learning. WCER Working Paper 87, 104–111. Sullivan, P., Mornane, A., Prain, V., Campbell, C., Deed, C., Drane, S., Faulkner, M., McDonough, A., Smith, C. (2009). Junior secondary students’ perceptions of influences on their engagement with schooling. Australian Journal of Education 53, 176–191. https://doi.org/10.1177/000494410905300206 Tobi, H., Kampen, J.K. (2018). Research design: the methodology for interdisciplinary research framework. Qual Quant, 52, 1209–1225. https://doi.org/10.1007/s11135-017-0513-8 Tyas, N. (2018). Pemanfaatan aplikasi digital learning untuk pembelajaran pengayaan di sekolah menengah kota Surabaya. Departemen Sistem Informasi, Fakultas Teknologi Informasi dan Komunikasi, Institut Teknologi Sepuluh Nopember. van Rosmalen, P., Westera, W. (2014). Introducing serious games with Wikis: empowering the teacher with simple technologies. Interactive Learning Environments 22, 564–577. https://doi.org/10.1080/10494820.2012.707128 Werbach, K., Hunter, D. (2012). For The Win: How Game Thinking Can Revolutionize Your Business. Wharton Digital Press, Philadelphia. What is Quizizz? [WWW Document], n.d. . Help Center. URL http://Quizizz.zendesk.com/hc/en- us/articles/203610052-What-is-Quizizz- (accessed 9.19.19). Williams, M., Burden, R.L. (1997). Psychology for Language Teachers: A Social Constructivist Approach, Cambridge language teaching library. Cambridge University Press, Cambridge ; New York. Zhao, F. (2019). Using quizizz to integrate fun multiplayer activity in the accounting classroom. International Journal of Higher Education 8, 37. https://doi.org/10.5430/ijhe.v8n1p37 Žukauskas, P., Vveinhardt, J., Andriukaitienė, R. (2018). Exploratory research, in: Žukauskas, P., Vveinhardt, J., Andriukaitien?, R. (Eds.), Management Culture and Corporate Social Responsibility. InTech. https://doi.org/10.5772/intechopen.70631