Using Digital Formative Assessment to Evaluate EFL Learners’ English Speaking Skills1 Uso de la evaluación formativa digital para evaluar las habilidades de habla inglesa de los estudiantes de inglés como lengua extranjera Zeynep Çetin Köroğlu2* Aksaray University, Turkey 1 Received: October 18th, 2020/ Accepted: May 14th 2021 2 zeynepcetin86@gmail.com Gist Education and Learning Research Journal. ISSN 1692-5777. No. 22 (January - June, 2021). pp. 103-123. 104 No. 22 Abstract As it is known formative assessment focuses on both learning process and learner’s performance. In this study, digital formative assessment and traditional speaking tests were utilized comparatively to evaluate 52 upper-intermediate EFL learners’ English language speaking skills. The study was designed as a mixed-method. The quantitative data were collected via achievement tests which had been administered both in traditional speaking tests and digital formative tests. The qualitative findings were collected with students’ interviews which consisted of four open-ended questions. The results of the study showed that participants outperformed in digital formative tests in comparison to traditional speaking tests. Another significant finding of the study is that participants are satisfied with the digital formative assessments in terms of peer collaboration during tests, enriched test materials, and preparation time for the speaking test. Although they have positive views on digital formative assessment, participants are dissatisfied with it in terms of technical problems that they encountered during the administration of digital formative tests. Keywords: Digital formative assessment, foreign language speaking skills, formative assessment, language testing. Resumen Como se conoce, la evaluación formativa se enfoca tanto en el proceso de aprendizaje como en el desempeño del estudiante. En este estudio, la evaluación formativa digital y las pruebas tradicionales orales se usaron comparativamente para evaluar a 52 estudiantes de inglés en nivel intermedio superior en la habilidad de habla. El estudio se diseñó como un método mixto. Los datos cuantitativos se recolectaron por medio de pruebas de logros las cuales se habían administrado tanto en pruebas tradicionales orales como en pruebas formativas digitales. Los datos cualitativos se recolectaron por medio de entrevistas a los estudiantes que consistían en cuatro preguntas abiertas. Los resultados del estudio mostraron que los participantes superaron las pruebas formativas digitales en comparación con las pruebas tradicionales. Otro resultado significativo es que los participantes estaban satisfechos con las evaluaciones formativas digitales en términos de colaboración entre compañeros durante las pruebas, materiales de prueba enriquecidos y tiempo de preparación para las pruebas orales. Aunque han tenido opiniones positivas sobre la evaluación formativa digital, los participantes están insatisfechos con dicha prueba en términos de los problemas técnicos que se han encontrado durante la administración de estas. Palabras clave: evaluación formativa digital; habilidades para hablar un idioma extranjero; evaluación formativa; prueba de idiomas. Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 105 No. 22 Digital Formative Assessment In Speaking Skills Çetin-Köroğlu Resumo Como se conhece, a avaliação formativa se enfoca tanto no processo de aprendizagem como no desempenho do estudante. Neste estudo, a avaliação formativa digital e as provas tradicionais orais se usaram comparativamente para avaliar a 52 estudantes de inglês em nível intermédio superior na habilidade de fala. O estudo se desenhou como um método misto. Os dados quantitativos se recolheram por meio de provas de aproveitamentos as quais se tinham administrado tanto em provas tradicionais orais como em provas formativas digitais. Os dados qualitativos se coletaram por meio de entrevistas aos estudantes que consistiam em quatro perguntas abertas. Os resultados do estudo mostraram que os participantes superaram as provas formativas digitais em comparação com as provas tradicionais. Outro resultado significativo é que os participantes estavam satisfeitos com as avaliações formativas digitais em termos de colaboração entre colegas durante as provas, materiais de prova enriquecidos e tempo de preparação para as provas orais. Mesmo que tiveram opiniões positivas sobre a avaliação formativa digital, os participantes estão insatisfeitos com mencionada prova em termos dos problemas técnicos que se encontraram durante a administração destas. Palavras chave: avaliação formativa digital; habilidades para falar um idioma estrangeiro; avaliação formativa; prova de idiomas. 106 No. 22 Introduction A ssessing the oral production of foreign language learners has become one of the main concerns of scholars in the 20th century (Celce-Murcia, 2013). Language skills assessment plays a very crucial role in the learning process but it turns out that changing or reforming assessment is quite difficult. However, in our changing and developing world cultural, theoretical, technological development and changes make reform a necessity in assessment. Thus, these changes affect both teaching and learning as well. Foreign language speaking skills assessment is perceived as an active process that demonstrates whether language learners have learned what had been taught, and if not, what can be done to solve the problem (William, 2007). There is a direct link between assessment and learning that has been realized by scholars (Boz & Boz, 2005). Assessment studies have begun to shift away from investigating restricted test types to other techniques such as formative assessment, dynamic assessment, digital formative assessment, classroom assessment, etc. Thus, the former test types generally make little contribution to language learners’ learning during the assessment and include strict components that cause stress and anxiety during the speaking skills assessment (Knight, 1992). However, alternative assessment types such as formative assessment or dynamic assessment consist of scaffolding, peer collaboration, teacher’s mediation, and a more relaxing atmosphere for language learners to perform their skills. In this respect, assessment and learning are highly related to each other, and such assessment techniques should be used to make both learners and teachers benefit from these techniques (Boz & Boz, 2005). Changing and transforming educational technologies make innovative assessment techniques a must in our developing world. For this reason, the present research seeks to unveil different aspects related to the assessment of foreign language speaking skills. It tries to find out whether the digital formative assessment is more effective than traditional speaking tests to evaluate and improve language learners’ foreign language speaking skills. Additionally, the current research tries to figure out upper- intermediate language learners’ perspectives towards digital formative assessment type which is quite new for them. The present research study is guided by the following research questions: 1. Is there any statistically significant difference between test scores of upper- intermediate EFL learners’ speaking skills in traditional speaking tests and digital formative tests? 2. What conceptions do the upper-intermediate EFL learners enrolled at a state university, Turkey have on digital formative assessment of speaking skills? Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 107 No. 22 Related Literature As it was mentioned before, assessment has a big impact on students’ learning, and generally, students spend their time on the materials that are covered in the assessment part. In other words, assessment directly affects their learning (Baleni, 2015; Leung, 2005; Leung & Mohan, 2004; Poehner & Lantolf, 2005). The important point to be considered is whether the assessment should be for learning or assessment of learning. The distinction between formative or summative assessment is that summative assessment heavily focuses on what students have learned, in other words outputs of the learning process but formative assessment emphasizes the learning process rather than the learning product by providing feedback (Dixson & Worrell, 2016). The distinction between these two different assessment types is perceived as two different poles of a continuum (Capel et al., 1995) that summative assessment serves the educational system’s needs and it has a bureaucratic purpose. On the other hand, the formative assessment supports learners’ future learning by providing meaningful and constructive feedback. The present research paper focuses mainly on formative assessment and its more recent version of digital formative assessment. Another distinction between formative assessment and summative assessment is that purpose and timing differ in these two assessment types. Formative assessment is used to assess learners’ positive achievements during the learning process and by giving feedback learners are prepared for a further step of learning. On the other hand, summative assessment is used to record the overall achievement or performance of learners in a systematic way (TGAT, 1988). One of the main differences is that formative assessment provides more chances to compensate for learning deficiencies during the learning process. Formative assessment is defined as an assessment type that includes quizzes, homework, questions, etc. during instruction to provide direction for future learning of learners (O’Connor, 2002). Main concerns of formative assessment; it serves to maximize students’ future learning possibilities, develops students’ motivation and learning opportunities, and uses assessment as an actual part of the learning process (Boz & Boz, 2005; McMillan, 2014). Teachers should follow some main approaches to achieve the goals of formative assessment. It is clear that formative assessment includes highly effective steps to reach the goals. Questioning provides critical thinking and active construction of knowledge if students try to find out answers by researching and asking further questions. Providing feedback makes students understand their mistakes and shows the correct information. As the third step, peer- and self-assessment serve as reflective parts of formative assessment. In the last step, formative use of summative tests provides a continuous assessment cycle and by doing so both students and teachers can see the missing pieces of information in the learning process. Therefore, formative assessment should be cyclical and continuous to make learners more active during the learning process (Wood, 2010). Formative assessment should include an assessment of students’ behaviour and learning process and feedback which are quite effective to recover deficiencies in learning. Wood (2010) mentions that learners are active in learning process and learning environment is Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 108 No. 22 challenging for both learners and teachers. However, the formative assessment fosters permanent learning and contributes to the future learning of learners. In this respect, its contribution to the learning process and effectiveness is quite obvious. Particularly, with the help of formative assessment teachers can check learners’ understanding and help them to make up for lacking knowledge (Dirksen, 2011). Formative assessment can be planned or interactive; the planned formative assessment provides and evidence with a diagnostic purpose but interactive formative assessment is more spontaneous (Bell & Cowie, 2001). It is possible to see that both summative assessment and formative assessment are used to test foreign language learners’ language skills and their development in various skills such as reading, writing, speaking, and listening. However, related research indicates that generally summative assessment is used to measure students’ overall achievement in language skills (Stanca, et al. 2015). However, with the advent of technologies and students’ increasing demands and readiness towards changes in educational and assessment tools make new assessment types a must. Foreign language speaking skills assessment involves multiple activities and various tasks (Luoma, 2004) and speaking skills assessment should measure language ability or the use of language rather than students’ knowledge about the topic if the language test does not have a specific purpose (Huang, Hung & Plakans, 2018). Speaking skills as a part of pragmatic knowledge represent communication ability, knowledge related to language use in practice, and appropriate use of target language in an appropriate context such as functional and sociolinguistic knowledge (Luoma, 2004). In such an assessment environment, students, assessor(s), tools, or tasks for assessment and rubric exist in an interrelated way. It was discussed previously that formative assessment provides many opportunities to students in many ways and its usage to test foreign language speaking skills contributes respectively more than summative assessment. Thus, foreign language speaking assessment and tests are perceived as stressful and anxiety increasing moments by foreign language learners (Çetin Köroğlu, 2019). However, if the formative assessment is administered to test foreign language learners, both their speaking skills are tested and their improvement in foreign language speaking skills is enhanced. The digital formative assessment is quite a new term for speaking assessment and there is very limited research on this term (Faber, Luyten & Visscher, 2017). There is only one research about it which is carried out by Fabet et. al. (2017). The results of their study show that the digital formative assessment tool has positive effects on student achievement and motivation. Additionally, students’ use measurements support students’ achievement and motivation. One more important finding of the study is that achievement effects were higher for high-performing students. In this respect, there is a big gap both in the definition, application, and research area of digital formative assessment. The term can be defined as the use of formative assessment through digital platforms and steps to foster students’ learning by assessment. Various tools can be Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 109 No. 22 used to carry out the digital formative assessment, specific to the current research; speaking skills can be assessed through digital chats, recordings of students’ speaking performance, or video scripts commentary which represent authentic language use in a context (Jamieson, 2005). Additionally, using such tasks provides teachers and researchers the opportunity to better understanding students’ speaking skills development and trace test-takers (Dhalhoub-Deville, 2001). Moreover, such tasks can be used effectively nearly at all levels of language learners because children are called as digital natives nowadays (Dingli, & Seychell, 2015). Some researchers discuss that using e-portfolios is an effective way to assess language learners’ speaking skills development (Cepik & Yastibas, 2013). Gray (2008) states that using e-portfolios provides evidence, reflections, and feedback about learners’ abilities. In addition to these, useful websites can be used to carry out digital formative assessments such as flipgrid. The website presents a video discussion program and teachers can insert any video which they want to the website. They can ask questions and students respond to the questions. Seesaw and voicethread can be used as well because these websites have similar features with flipgrid. Performance tests and authentic assessments allow students to speak and perform in a real-life context that is crucial in the communicative language (Bachman & Palmer, 2010). Similarly, digital formative assessment provides authentic tasks and rich content during assessment phases and divides it into steps that decrease students’ speaking anxiety and stress. Moreover, digital formative assessment supports instruction by focusing on students’ performances and learning rather than their overall achievement. Even though developing technologies offer a range of digital assessment tools, only a little portion of teachers are eager to use them. This can be a result of difficulties and hardship when teachers use digital assessment tools (Levy & Stockwell, 2006). However, utilizing these tools modify and evolve foreign language speaking skills by chipping in and enhancing students’ learning while they are assessed. Additionally, the digital formative assessment draws students’ attention to different contexts and tools instead of text anxiety. Methodology The current research study aims to provide both qualitative and quantitative examination of the upper- intermediate EFL learners who study at a state university, Turkey. The study was designed as a mixed-method study to investigate participants’ speaking skills performance. To this end, it presents digital formative tests’ impact on EFL learners’ speaking skills development in comparison to traditional speaking tests and participants’ perceptions towards digital formative assessment and its usage in a speaking skills evaluation. Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 110 No. 22 Research Participants The participants for this research were enrolled in foreign language teaching department of a state university, Turkey. The participants were selected based on convenience sampling. Their age varied between 18 to 21 years old. The total number of participants included in this study is fifty-two. Participants consisted of 16 male and 36 female students. The participants were first-graders of the department and they are student teachers of English. Participants nearly have a similar speaking skills performance which is presumed due to their speaking skills test scores of preparatory classes of the department. The research was carried out in Oral Communication Skills I and II during an academic year, 2019- 2020. Data Collection Instrument The present study relied on two traditional speaking assessment tests and two digital formative assessments which were evaluated through speaking skills assessment rubrics and a written structured interview. The rubric included five domains which were fluency, accuracy, grammatical structure, pronunciation, and vocabulary. The rubric was used to grade participants’ speaking skills performance in both test types. The written structured interview consisted of four open-ended questions that aimed to find out participants’ perceptions towards digital formative test administration in terms of its effectiveness. Data Collection Procedure Before the administration of both traditional speaking tests and digital formative tests, students’ speaking skills achievement scores of the preparatory class were examined and concluded that they have nearly the same level of speaking skills in English. As the first test phase, the traditional speaking test was administered to all participants. In traditional tests, ten different topics such as globalization, nature, friendship, hobbies, etc. were selected and these themes were written on a piece of paper and each topic was put in an envelope. Students were called one by one to test the office and they selected one of the envelopes and thought for five minutes about the theme. Then each student was expected to speak about the theme nearly for five to ten minutes. Two test administrators applied the traditional speaking test and graded each student’s speaking performance. The second test phase was carried out with the same procedure as the traditional speaking test after three months. The third test phase was carried out as a digital formative assessment which was quite different than the traditional speaking test. The researcher of the study utilized Edmodo as a learning management system to administer the test. Before the test was administered, four Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 111 No. 22 videos were uploaded to Edmodo. Vocabulary charts were prepared and uploaded for each video too. Students had two hours to prepare their speech about the videos. They were expected to select one of them and to record a video of their own while they were talking on the material. Various tasks were introduced to students before the test administration such as role-play, interview, group discussion, etc. Additionally, students were free to talk individually or in a group. After they recorded their videos, they were expected to deliver their videos to the researcher through Edmodo, Google drive, and Whatsapp. The fourth test phase was administered as the same as the third test phase. Students were free to select videos from the used LMS, in current research which was Edmodo. Additionally, they were free to select a task and perform individually or in a group. They recorded their speaking performance and then sent to the researcher. At the end of the fourth phase, the written structured interview was administered to the participants of the study. Participants participated in the study voluntarily and their names were kept anonymous for confidential reasons. Test Types Test 1 Traditional speaking skills test Test 2 Traditional speaking skills test Test 3 Digital formative speaking skills test Test 4 Digital formative speaking skills test Data Analysis Method Students’ test scores were calculated and the mean score was presented for each test. The quantitative data helped in the interpretation and discussion of the two different speaking tests’ effectiveness in terms of student achievement. The written structured interview questions were analysed through content analysis. For each question, various themes and codes were created and discussed within the light of related literature. Thematic analysis was adopted to extract the theme from the written interviews. That is, the themes were emerged from the data of the research instead of imposing predetermined themes on the data, so the inductive approach was adopted (Charmaz, 2006). Findings Quantitative Findings Quantitative findings of the four test phases are presented through the table which is placed below. Analysis of findings reveals that participants have similar test scores in test 1 and test 2. Their average score in test one is 61.25 and in test 2 score is 63.50. As it was mentioned in test three and test four digital, formative tests were administered with a different procedure than the traditional speaking tests. The average score for Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 112 No. 22 test three is 84.50 which is quite higher than the test two. Additionally, in test four participants’ average test score is higher than test three, which shows participants experienced a novel speaking skills assessment type and succeeded in terms of academic achievement. Table 1. The average score of traditional speaking skills tests and digital formative speaking tests. Participants’ number Test 1 traditional speaking skills test Test 2 traditional speaking skills test Test 3 Digital formative speaking skills test Test 4 Digital formative speaking skills test Average test score of participants 60 61.25 63.50 84.50 86.50 Participants speaking performances were graded through an analytic rubric which included fluency, accuracy, grammatical structure, pronunciation, and vocabulary dimensions. Hence, analytic rubrics offer detailed, focused, and precise assessment by covering various aspects of oral performance (Mertler, 2001). Participants’ average score for each dimension is provided through the table below. Table 2. Participants’ average test scores due to sub-dimensions. Test 1 traditional speaking skills test Test 2 traditional speaking skills test Test 3 Digital formative speaking skills test Test 4 Digital formative speaking skills test Fluency 11,25 11,7 19,11 21,6 Accuracy 13,25 13,9 15,7 17,3 Grammatical Structure 11,50 10,9 14,5 12,3 Pronunciation 12,50 13,2 13,3 13,3 Vocabulary 13,25 13,8 19,9 21,0 Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 113 No. 22 It is indicated in table 2 that participants nearly had the same result in fluency dimension in test 1 and test 2 phases. Their average score is 11.25 % and 11.7 %. The result of fluency dimension relatively increased in test 3 and test 4 as 19.11 % and 21.6 % in digital formative tests, which can be interpreted participants spoke more fluently in digital formative tests. When the accuracy dimension’s test scores are examined, a gradual increase can be seen in table 2. As it is known traditional speaking test doesn’t provide material during the speaking performance but digital formative test provides students with prepared materials, background information, necessary vocabularies, and preparation time. In this respect, students may have a chance to prepare their speech for the test. As it is presented in table 2, participants got the highest accuracy- test score in test 4 which was administered in the form of a digital formative test. There is no such development in grammatical structure and pronunciation dimension of four test phases. Participants’ test scores are nearly similar. On the other hand, they experienced significant improvements in terms of vocabulary dimension. Despite their vocabulary, average scores are 13.25 and 13.8 in test 1 and test 2, participants’ vocabulary test scores are 19.9 and 21.0 in test 3 and test 4. The results may indicate that participants used various vocabularies during speaking skills tests. Qualitative Findings Qualitative findings of the study were collected through a written structured interview with questions. Themes and codes were created for each question due to participants’ responses. Some students’ responses were also presented. The first question was asked to question students’ perspectives towards both traditional speaking tests and digital formative speaking tests. Students’ responses lead the researcher to create seven themes for the first question. The first question, themes and these themes’ repetitions rates are presented below; Q.1. Could you compare the traditional speaking exam and the new speaking exam type Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 114 No. 22 Table 3. Themes and repetitions rates for question 1. Theme Repetitions Interaction with teacher 8 Collaboration & İnteraction with peers 8 Negative views on traditional speaking skills assessment 15 Positive views on traditional speaking skills assessment 16 Positive views on digital formative assessment of speaking skills 29 Negative views on digital formative assessment of speaking skills 12 Topic diversity 2 The findings of the first question show that participants have positive views on digital formative speaking tests because the codes for this theme were repeated 29 times in the interview. On the other hand, participants have both negative and positive views on the traditional speaking test. They criticized and got dissatisfied with some parts of traditional speaking test such as teachers’ existence during the test, lack of supportive materials before the test and during the test, being alone during the test and academic topics for speaking test. Another important finding of the first question is that participants are satisfied with students’ collaboration and peer interaction features of digital formative assessment. Some students have negative views about the new speaking test and prefer interaction with the teacher during the speaking test which is absent in digital formative speaking tests. Samples from participants’ responses are provided below; S’s R: ‘I think that the new speaking exam type is so nice because we are relaxing and we can speak fluently. The traditional one makes us nervous so we cannot express our feelings. I think that the new speaking exam type is so useful for students. Even if students can speak fluently the traditional speaking exam doesn’t let students express their ideas openly because of its stress and pressure. Additionally, it causes stress’. S’s R: ‘I think traditional speaking exam causes stress and nervousness in person but the new speaking exam provides relaxation. You can be comfortable during the exam. The traditional speaking exam is harder than the new one. On the other hand, the new speaking exam is a group study, so it is hard to evaluate each students’ speaking skills’. S’s R: ‘I was less excited in new speaking exam and it had more fun than the traditional one. Because there wasn’t a teacher during the exam and we were together with our friends which decreased our stress and anxiety. In traditional Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 115 No. 22 speaking exam, there are at least 2 teachers and you can’t make a mistake if you want a good grade from them. In new speaking exam, we had a chance to rerecord us video and time for preparation to our speech. Although new type’s all advantages, I would prefer the traditional speaking exam. Because, we tried to upload our video to website and ıt took 40 minutes which caused lots of stresses. Participants’ responses to the second question lead the researcher to create six themes and various codes for these themes. The themes and their repetitions rates are presented below in table 4. Q. 2. What are the positive sides of digital formative assessment? Table 4. Themes and repetitions rates for question 2. Themes Repetitions Students’ collaboration during exam 14 Stress and anxiety 13 New learning opportunities 8 Preparation 18 Fluency & accuracy 15 Technology 3 Table 4 indicates that students are satisfied with the digital formative speaking test in terms of preparation before the speaking exam, peer collaboration before, and during the speaking exam. Besides, they think that digital formative speaking tests decrease their stress and speaking anxiety because of the relaxing test atmosphere. Hence, participants had the opportunity to perform their speech wherever they want. It is seen from students’ performance videos that they were relaxing and they rested and relaxed at their homes and dormitories during the test. Some of the participants think that the digital formative speaking test provides new learning opportunities and it is useful to integrate technology into speaking tests. Another important result is that they think this test type makes them more fluent and accurate in their English speaking performance. Samples from participants’ responses are provided below; S’s R: ‘We studied as a group and discussed our ideas with each other. We learned many new information through videos which had been provided by teacher to choose a topic. We used technology during our speaking exam and it was fun. By talking to camera, I think we can improve our role-play skills’. Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 116 No. 22 S’s R: ‘We are less excited because if we make mistake we can record the video again. We had fun because we made role plays and we improvised. We could search a Google about the topic so we can learn some information. It was beneficial for our imagination for example we behaved as if we were someone else (Felix Baumgartner). It was beneficial for us to collaborate with my friends we discussed and shared our ideas. It was exactly a teamwork’. S’s R: ‘New exam type provides us a relax environment. It has flexible test environment. You can research whatever you want in dormitory or your home. You can choose the best topic which suits with your ideas and new exam type includes using technology which one knows to use it well’. S’s R: ‘The positive sides of new exam type are much more than the negative sides the first one is self-confidence. When I make practice I feel myself comfortable. Speaking with my friends gives me more energy than teachers during the exam. The second one is happiness. Normally, I like speaking English, but when I am anxious I don’t speak clearly in English. Our new exam type provided this relax atmosphere to me. Thus I was happy yesterday’. S’s R: ‘The new exam type enables to explain your ideas or feelings. Thanks to the new type students can make new style about speaking topic and the new type makes them free about expressing their ideas. Maybe students can create atmosphere that give them comfort. Students share their idea with their friends to create good dialogues and good interview’. Participants’ responses to the third question were categorized under various categories and four themes were created to present the data. The themes for the third questions are ‘individual assessment and peer work problem’, ‘ time and time management problems’, ‘ data size and video uploading problems’, and ‘ application problems’. Q. 3. What are the negative sides of new exam type? Did you experience and technological difficulties in the new exam? Table 5. Themes and repetitions rates for question 3. Themes Repetitions Individual assessment of student & peer work problem 8 Time & time management problems 16 Data size & video uploading problems 46 Application problems 6 Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 117 No. 22 Participants stated that they had faced technological problems in digital formative speaking tests. Thus, they wrote that after they recorded their videos, they were unable to upload those videos to digital platforms and applications such as Edmodo, Google drive, and Whatsapp. The ‘data size and video uploading problems’ have the highest repetition rate with 46 repetitions among all themes of qualitative data. Another negative aspect of this test arising from the participants is that they had time limitations and couldn’t manage their time efficiently. The theme for this issue got 16 relatively high repetitions. According to participants’ responses, another problematic side of the digital formative speaking test is peer-based problems and application’s complicated features. Samples from participants’ responses are provided below; S’s R: ‘Actually, when we filmed it we didn’t come across with any difficulty but when we send it to teacher it is a bit difficult eventually we did it’. S’s R: ‘I think there are some negative sides of new exam type. One of them there must be a person who manages the conversation in developed a device conversation can proceed smoothly and that gives anxiety to students’. S’s R: ‘I think one of the negative sides of new exam type of definitely a shortage of time because we recorded our video again and again and this put me in stress. I thought that I will miss the leading time of our exams and also I had trouble while I am sending video’. S’s R: ‘Yes unfortunately we had trouble when we were trying to send the video it is size was really huge and we had problems with Internet’. S’s R: ‘The negative side maybe being shy in front of the camera for some students and Technology side must be easy because we had got time problem. We nearly couldn’t have caught the other exam. We couldn’t look at some terms about the topic that we forgot details about the topic’. S’s R: ‘No I had any problem except for internet speed some of my friends cannot upload a video to the Edmodo’. Question four had been asked to find out students’ suggestions and recommendations to improve digital formative speaking test in terms of its usefulness and effectiveness. Various themes were created for analysing students’ responses to the fourth question as ‘alternatives for video uploading’, ‘internet speed’, ‘topic diversity’ and ‘more time’. The results for the question four are presented through table 6 below; Q.4. what are your suggestions and solution to these problems? Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 118 No. 22 Table 6. Themes and repetitions rates for question 4. Themes Repetitions Alternatives for video uploading 17 Internet speed 2 Topic diversity 3 More time 9 The results show that students are dissatisfied with the video uploading channels and prefer alternatives such as USB. Besides, they think that more time should be allotted to the digital formative speaking test. Some of the participants who mentioned about internet speed should be more improved and also topic diversity is required. Samples from participants’ responses are provided below; S’s R: ‘I think this new type exam has no problems may be the problem of application should be solved. Instead of these apps we can find a new app or we can download the videos to the USB’. S’s R: ‘Actually if he has more time to make video it will be better for us. Before the exam, we can make more practice to shoot a video smoothly. if we make more video assignments in class, I think it improve speaking skills and we don’t have lack of time’. S’s R: ‘I believe including technology into the exams are not very proper. Paper- based but creative exams or face-to-face communication will take away these problems and won’t make students me at least a bit more comfortable’. S’s R: ‘I think students need more time and more subjects about talking it will be really cool if they could talk about their hobbies or occupations’. Discussion The findings of the current research indicate that digital formative tests contribute to students’ speaking skills development. Especially, students’ fluency and accuracy skills have been developed with the administration of these test types. Another significant finding of the research is that students enriched their vocabulary knowledge through the test content. As was discussed in the review of the literature part of the research, the main difference between formative and summative assessment is that formative Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 119 No. 22 assessment is used for both learning and assessment and it fosters instruction during testing. The results of the current article support the idea with its findings. One more significant finding of the current research is that nearly all of the participants thought that digital formative assessment is an innovative testing type and highly effective for assessment of foreign language speaking skills. Additionally, participants stated that text content and preparation time are useful features of these test types. Another significant finding of the study is that some of the participants had negative perceptions towards digital formative tests because of technical problems such as weak internet connection, deficiencies in their technological devices, etc. In parallel with the findings of Black and William’s research (1998) in which they reviewed over 250 research articles about formative assessment, formative assessment leads to highly important learning gains to students. Moreover, they found out that formative assessment enables students to focus on self-assessment, corrective feedback, and learning goals rather than performance goals by testing them more frequently than traditional testing (Black & William, 1998). The current research study reveals that foreign language speaking skills which are one of the most challenging skills for language learners require innovative and up to date assessment type and digital formative assessment is effective for academic achievement. Moreover, participants are satisfied with the digital formative assessment. Students’ speaking skills such as pronunciation, accuracy, fluency, vocabulary development are improved with corrective feedbacks during digital formative assessment. Thus, formative assessment is a continuous process that aims to define learning deficiencies and develop learning process during assessment (Kincal & Ozan, 2018). Digital formative assessment is compatible with the constructivist approach which has been implemented since 2005 in Turkey (Boz & Boz, 2005). Another significant contribution of digital formative assessment to speaking skills’ assessment process is that the researcher could compare the performances of the participants throughout the process. Unlike traditional summative assessment, digital formative assessment provides chances to teachers to compare student’s performances. Despite its various positive sides, the digital formative has also drawbacks for students and teacher. First of all, it is somehow more challenging for students because they are exposed to more tests which actually make them practice more. As it is known, practice is crucial for productive skills (Golkova & Hubackova, 2014). The other negative aspect of this assessment type is that some students do not have advanced technological devices and they may have an internet connection problem. One another negative aspect is that its burdensome assessment type for teachers. Thus, each assessment cycle requires a preparation phase before the test, the test application will be time-consuming and giving feedback means extra burden for teachers. Despite these negative sides of digital formative assessment, it is highly effective and fruitful to develop language learners’ speaking skills as the findings of the current research presented. Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 120 No. 22 Conclusion The present research aimed to find out whether the digital formative assessment is more effective than a summative assessment to test foreign language learners’ speaking skills development. The findings of the research indicate that digital formative assessment fosters participants’ fluency and accuracy skills. Besides, participants’ vocabulary knowledge has been developed through digital formative assessment. Additionally, the current research shows that nearly all of the participants have a positive attitude towards this assessment type and they prefer to have it in the future assessment process with some modifications. Participants suggest that more time should be devoted to students’ preparing for tests. Participants complain about the data size of videos, which they recorded their speaking performance. It causes uploading problems. The digital formative assessment occurs regularly and allows feedback. When classroom activities are implemented to assess language learners’ oral performance, these tasks draw participants’ attention, increase their motivation, support their metacognitive skills, and critical thinking (Stiggins, 2002; Facione, 2011). Digital formative assessment uses classroom-like activities on a digital platform for assessment purposes. Due to its continuous nature, language learners stick to the learning process. Jandris (2001, p. 4) “the heart of assessment is a continuing process in which the teacher, in collaboration with the student, uses the information to guide the next steps in learning” (p. 4). The digital formative assessment made language teachers and participants active during the assessment phase of the current research. The results show that it is quite effective to develop foreign language learners’ speaking skills in many aspects. Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 121 No. 22 References Bachman, L. F., Palmer, A. S. (2010). Language assessment in practice: Developing language assessments and justifying their use in the real world. Oxford: OUP. Baleni Z. (2015). Online formative assessment in higher education: Its pros and cons, The Electronic Journal of e-Learning,13(4), 228-236. Bell, B., & Cowie, B. (2001). Formative assessment and science education. Dordrecht, The Netherlands: Kluwer Academic. Black, P., & Wiliam, D. (1998a). Assessment and classroom learning. Assessment in Education, 5(1),7-74. Brown, G. T. L. (2008). Conceptions of assessment: Understanding what assessment means to teachers and students. New York, NY: Nova Science Publishers Boz, N. & Boz, Y. (2005). Investigating Formative Assessment. Education and Science, 30(138), 63-69. Celce-Murcia, M. (2013). Teaching English in the context of world Englishes. In M. CelceMurcia, D. M. Brinton,& M. A. Snow (Eds.), Teaching English as a Second or Foreign Language (4th ed, pp. 2-14).Boston, MA: National Geographic Learning/ Cengage Learning. Cepik, S., & Yastibas, A. E. (2013). The use of e-portfolio to improve English speaking skill of Turkish EFL learners. Anthropologist, 16(1-2), 307-317. Çetin Köroglu, Z. (2019). Interventionist Dynamic Assessment›s Effects on Speaking Skills Testing: Case of Elt Teacher Candidates. Advances in Language and Literary Studies, 10(3), 23-31. Dingli, A., & Seychell, D. (2015). Who Are the Digital Natives?. In The New Digital Natives (pp. 9-22). Springer, Berlin, Heidelberg. Dirksen D. (2011). Hitting the reset button. Phi Delta Kappan 92(7), 26-31. Dixson, D. D., & Worrell, F. C. (2016). Formative and summative assessment in the classroom. Theory Into Practice, 55(2), 153-159. Faber, M. J., Luyten, H. & Visscher, J. A. (2017). The effects of a digital formative assessment tool on mathematics achievement and student motivation: Results of a randomized experiment, Computers& Education, 106, 83-96. Facione, P. A. (2011). Critical thinking: what it is and why it counts. Millbrae: Measured Reasons and the California Academic Press. Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 122 No. 22 Golkova, D., & Hubackova, S. (2014). Productive skills in second language learning. Procedia- Social and Behavioral Sciences, 143, 477-481. Huang, D. H., Hung, A. S., & Plakans, L. (2018). Topical knowledge in L2 speaking assessment: Comparing independent and integrated speaking test tasks. Language Testing, 35(1), 27–49. Jamieson, J. (2005). Trends in computer-based second language assessment. Annual Review of Applied Linguistics, 25, 228-242. Kincal, R.Y. & Ozan, C. (2018). Effects of Formative Assessment on Prospective Teachers’ Achievement, Attitude and Self-Regulation Skills. International Journal of Progressive Education, 14(2), 77-92. Knight, B. (1992). Assessing speaking skills: A workshop for teacher development, ELT Journal, 46 (3), 294-302. Leung, C. (2005). Classroom teacher assessment of second language development: Construct as practice. In E. Hinkel (Ed.), Handbook of research in second language teaching and learning. Mahwah, NJ: Erlbaum. Leung, C, &Mohan, B. (2004). Teacher formative assessment and talk in classroom contexts: Assessment as discourse and assessment of discourse. Language Testing, 21, 335- 359. Levy, M., & Stockwell, G. (2006). CALL dimensions: Options and issues in computer assisted language learning. Mahwah, NJ: Lawrence Erlbaum Associates. Luoma, S. (2004). Assessing speaking. Cambridge: Cambridge University. McMillan, J. H. (2014). Classroom assessment: Principles and practice for effective standards- based ınstruction (5th ed.). Essex: Pearson. O’Connor, K. (2002). How to grade for learning. Arlington Heights, IL: Skylight. Ounis, A. (2017). The Assessment of Speaking Skills at the Tertiary Level. International Journal of English Linguistics, 7(4), 95-112. Poehner, M. E., & Lantolf, J. P. (2005). Dynamic assessment in the language classroom. Language Teaching Research, 9 (1), 1-33. Stanca, C., Georgescu, S., Mina, S. & Olteanu, A. (2015). Measures of transforming the summative assessment in formative assessment in students activities evaluation at Constanta Maritime University. Karabük University Journal ıf Institute of Social Sciences, 5(1), 32-47. Digital Formative Assessment In Speaking Skills Çetin-Köroğlu 123 No. 22 Stiggins, R. J. (2002). Where is our assessment future and how can we get there from here? In R. W. Lissitz & W. D. Schafer (Eds.), Assessment in Educational Reform: Both Means and Ends (pp. 18-48). Boston: Allyn & Bacon. Wood, D. F. (2010). Formative assessment. In T. Swanwick (Ed.), Understanding medical education: Evidence, theory and practice (pp. 259-270). Hoboken: Wiley- Blackwell. Author * Zeynep Çetin Köroğlu has been working as an Assistant Professor Doctor at English Language Teaching Department of Aksaray University, Turkey. Her academic background mainly consists of English Language Teaching. She graduated from Gazi University and got her MA degree. Then, she started her PhD at the same university and completed it in 2015. She especially interested in intercultural communication, language teachers’ education, ICT implementations in language pedagogy and language assessment. She published numerous book chapters and articles related with her research interests. ORCID: https://orcid.org/0000-0002-9456-8910 Digital Formative Assessment In Speaking Skills Çetin-Köroğlu How to reference this article: Çetin-Köroğlu, Z. (2021). Using Digital Formative Assessment to Evaluate EFL Learners’ English Speaking Skills. GIST – Education and Learning Research Journal, 22(1), 103-123. https://doi.org/10.26817/16925777.1001