










































English Language Teaching Educational Journal   ISSN 2621-6485 

Vol. 5, No. 1, 2022, pp. 48-59  

  10.12928/eltej.v5i1.5779                                  http://journal2.uad.ac.id/index.php/eltej/index        eltej@pbi.uad.ac.id 

Comparing the effectiveness of explicit EAL feedback through 

slideshow (text+audio) and captioned video 

Jonathan Harrison 

Nihon University, CST, 7-24-1 Narashinodai, Funabashi, Chiba 274-8501, Japan 
jon.harrison@nihon-u.ac.jp  

 

A R T I C L E  I N F O 

 

ABST RACT  

 

 

Article history 

Received 20 February 2022 

Revised 23 April 2022 

Accepted 27 April 2022 

 Research seems to show that captions and subtitles are generally 
beneficial to learners of English as an Additional Language (EAL), but 
some research does contradict this. Research on slideware and slide 
design seems to focus more on attractiveness of slides and less on 
educational effectiveness. However, research on slide design and specific 
approaches to slide design continue to become more detailed. This study 
compared comprehension of explicit feedback received through either 
slideshow (text+audio) video or captioned video on an EAL writing task 
in an on-demand university setting. Results (n=163) indicated that 
approximately 50% of learners clicked the feedback video to advance to 
the quiz without viewing it completely. Of the learners with at least one 
full viewing (n=86), slideshow video seems to have engaged students for 
a longer duration than captioned video. The quiz items were easier for the 
slideshow video groups, and the quiz items performed better for these 
groups. The slideshow video groups had slightly higher means, but a 
significant difference between the effectiveness of slideshow video and 
captioned video to transmit feedback to students was not found.  

This is an open access article under the CC–BY-SA license. 

    

 

 
Keywords 

captioned video 

English as an Additional Language 

explicit feedback 

group feedback 

slideshow video 

 

 

 

 

How to Cite: Harrison, J. (2022). Comparing the effectiveness of explicit EAL feedback through slideshow 
(text+audio) and captioned video. English Language Teaching Educational Journal, 5(1), pp. 48-59. 

1. Introduction 

During the 2020 and 2021 academic years, the number of educational institutions engaged in online 
education substantially increased due to the coronavirus pandemic. Video is an important component 
of many asynchronous, on-demand, lessons both for instructional and feedback purposes, and how 
instructors use video varies by course content, purpose, activity flow, access to technology, instructor 
technological skills and digital literacy, etc. For face-to-face instruction, slideware, such as Microsoft 
PowerPoint, has been popular for decades, and instructors use this for synchronous and asynchronous 
online instruction. Also, for synchronous and asynchronous online instruction, Zoom or other video 
conferencing software with recording functionality have become popular recently. With asynchronous 
online English as an Additional Language (EAL) instruction, learners on occasion request captions or 
subtitles for videos. In this report the term EAL will be used when discussing English as a Foreign 
Language (EFL) and English as a second language (ESL). Although there are studies comparing the 
effectiveness of captions and subtitles in various educational settings, comparisons specifically 
focusing on slideshow video (text+audio) and captioned video could not be located. Research on the 
effectiveness of captions and subtitles will be discussed further in the background sections as will the 
history of PowerPoint, some PowerPoint related research, and slide design research. In order to 
identify which practice is better when using video in the classroom, this report compares student 
performance on a quiz after students watched either a slideshow presentation with in-screen video of 

https://doi.org/10.12928/eltej.v5i1.5779
http://journal2.uad.ac.id/index.php/eltej/index
mailto:jon.harrison@nihon-u.ac.jp
http://creativecommons.org/licenses/by-sa/4.0/
http://creativecommons.org/licenses/by-sa/4.0/
http://journal2.uad.ac.id/index.php/eltej/article/view/5779


ISSN 2621-6485 English Language Teaching Educational Journal 49 
 Vol. 5, No. 1, 2022, pp. 48-59 

 Harrison, J. (Comparing the effectiveness of explicit EAL feedback ….) 

the teacher speaking or a captioned slideware video of the teacher giving general feedback on a task 
regarding 10 common language mistakes on a previous task. The research questions were: 

1. Do EAL learners engage longer with slideshow video or captioned video?  

2. Does either format garner more views from EAL learners?  

3. Which format is more effective for transmitting feedback content demonstrated through quiz 
scores?  

4. If EAL learners are divided by quiz performance level, do all groups under both conditions 
perform similarly? 

1.1 Background on Captioned Video 

We will begin with research regarding research on captions and native speakers, in this case of 
English. First, a differentiation between captions and subtitles must be made. Captions are in the same 
language as the audio, and subtitles are in a different language than the audio. Captions used in native-
speaking education seem to be taboo or possibly have negative effects. Ritzhaupt et al. (2015), studied 
the effects of captions on native speakers using time-compressed video at 3 speeds of accelerated 
playback. A negative significant difference was found, and a significant difference was found toward 
normal video speed. These results indicate that learners may be able to accelerate video speeds to 1.5 
times the normal speed, but will probably be less satisfied with the experience. Mayer (2021) 
suggested not adding captions that repeat spoken words. His article is on how to design effective 
instructional videos using evidence-based principles that are grounded in cognitive theories of learning 
and instruction. These principles include multimedia (present words and graphics), coherence (avoid 
extraneous material in slides and script), signaling (highlight key material), redundancy (do not add 
captions that repeat the spoken words), spatial contiguity (place printed text next to corresponding 
part of graphic), temporal contiguity (present corresponding visual and verbal material at the same 
time), segmenting (break a complex slide into progressively presented parts), pre-training (provide 
pre-training in the names and characteristics of key concepts), modality (present words as spoken 
text), personalization (use conversational language), voice (use appealing human voice), image (do 
not display static image of instructor's face), embodiment (display gesturing instructor), and 
generative activity (add prompts for generative learning activity). 

However, Liu (2018, 2019) responded to Mayer’s earlier work arguing that because Mayer’s 
theory and principles were developed based on empirical studies of native English-speaking students, 
they may not be applicable to EAL students. Specifically, Liu found issues with the modality and 
redundancy principles that involve text and audio saying that they could become compromised in the 
EAL context as learners have difficulty fully comprehending the English text and audio. Liu’s 2018 
study sought to identify EAL learners' optimum input mode (graphics and audio, graphics and text, or 
graphics, audio and text) and to test whether the modality and redundancy principles also applied to 
their multimedia learning. Results indicated that when it came to knowledge retention, no statistically 
significant differences were found in EAL students' learning outcomes from the three input modes. 
These findings demonstrated that Mayer's modality and redundancy principles did not apply to content 
knowledge and vocabulary learning for EAL students when certain multimedia learning principles 
were followed. 

Some research supports that captioned video increases EAL learner performance. Hayati and 
Mohmedi (2011) studied the effect of English captions, Persian subtitles, and no captions or subtitles 
on listening comprehension of 90 EAL learners. Groups of learners watched 6 5-minute episodes of a 
DVD under 1 of these conditions, and after each viewing session, tests were administered to examine 
listening comprehension rates. The English caption group outperformed the Persian subtitles group at 
a “considerably higher level,” and the Persian subtitles group outperformed the no subtitle group at a 
“substantially higher level.” Arndt (2014, 2018) in research comparing vocabulary acquisition via 
blog post reading and watching YouTube video blogs reported that the caption group significantly 
outperformed the non-caption group. Ashcroft, Garner, and Hadingham (2018) suggest that subtitles 
lead to incidental vocabulary learning, demonstrated through recall, after a single viewing of a movie. 
Ketabi and Sadeghi (2020) performed research with captioned and non-captioned video in regard to 
the comprehension of idiomatic expressions. In this research both groups saw the video twice, and the 
group that viewed video with captions performed significantly higher than the group without captions. 
Zhang and Zou (2021) performed a literature review of 41 Social Sciences Citation Index journal 



50 English Language Teaching Educational Journal   ISSN 2621-6485 

 Vol. 5, No. 1, 2022, pp. 48-59 

 Harrison, J. (Comparing the effectiveness of explicit EAL feedback ….) 

articles regarding multimedia input, including text images, audio, animation, and captions/subtitles in 
EAL learning and  concluded that the effectiveness of three features were reported most frequently: 
audio-plus-animation-plus-captions/subtitles in developing vocabulary and grammar knowledge; 
audio-plus-animation in promoting listening comprehension; and text-plus-audio in facilitating 
reading comprehension and called for more research in these areas. These reports seem to demonstrate 
that for language learning captioned video leads to better comprehension of listening comprehension, 
idiomatic expressions, incidental vocabulary learning and vocabulary acquisition. 

In French and Spanish as an additional language, similar results have been put forth by a number 
of researchers. Fievez et. al (2020) studied incidental vocabulary learning with captions and subtitles 
with French learners, and results indicated that both subtitle and caption groups outperformed the 
control group in meaning recognition group but only the caption group outperformed the control group 
on meaning recall tests. Camacho Velez and Pozo Estévez (2021) in their research on vocabulary 
development concluded that audiovisual materials with captions have a positive effect on the 
development of second language vocabulary. Cintrón-Valentín and García-Amaya (2021) 
investigated the effect of captioned video on grammar and vocabulary through form-focused 
instruction. In this research a randomized control design was used to investigate the effect of captioned 
media on the learning of vocabulary and grammar. Through four data-collection sessions, participants 
(n = 369) were presented with a grammar-lesson video and a multimodal video with one of three 
captioning formats: textually enhanced target vocabulary, textually enhanced target grammar, or no 
captioning. Results showed strong immediate effects of captioning on target vocabulary and on some, 
but not all, of the target-grammar structures. According to the authors, learning of some grammatical 
structures is more conducive to captioning than others. These studies suggest that for language 
learning captioned video was more conducive to vocabulary development and learning some, but not 
all targeted grammar structures. 

Other research suggests that captioned video is better than non-captioned video, but results may 
vary by language of the caption and language proficiency level. Winke, Gass, and Sydorenko (2010) 
concluded via results from t-tests and two-way ANOVA, that captioned videos were more effective 
than non-captioned videos. However, they deduced that for performance on aural vocabulary tests 
captioning of the first showing of the videos was more effective. Yet, results varied by language of 
the captions, for Spanish and Russian, captioning of the first showing was generally more effective 
than captioning of the second, and for Arabic and Chinese, captioning the second showing seemed to 
be more effective. Learners revealed in interviews that they used captions to increase their attention, 
improve processing, reinforce previous knowledge, and analyze language. Some learners stated that 
they used captions as a crutch, for support, as they got through the videos. Further research by Gass 
et al. (2019) used eye-tracking to gather data and describe how captions help people learn during 
captioned-video watching.  This work confirmed previous work in which captions generally promoted 
second language comprehension, and it also discussed the possible role of working memory with 
results showing “The two working memory groups went in opposite directions: the high groups 
reduced their caption reading time, whereas the low-working memory groups increased their reading 
time” (p. 97). This corresponded with Desjarlais (2017) who summarized first language research on 
multimedia learning and suggested that individual differences would account for variance in 
information processing during multimedia learning. These differences include working memory. The 
above research suggests that different proficiency groups may use captions in different ways. 

Regarding the benefits of subtitles, only a single study was found. Pujadas and Muñoz (2020) 
investigated caption and subtitle use for TV dramas viewed by secondary school EAL learners, and 
came to the seemingly obvious conclusion that subtitles led to significantly higher content 
comprehension than captions. 

Regarding improvements in pronunciation, results were mixed. Wisniewska and Mora (2020) 
studied if extended exposure to captioned videos would affect adult second language pronunciation, 
and tests were run to determine effects on speech processing skills (segmentation, speed of lexical 
access, and sentence processing), phonological accuracy in perception (ABX discrimination), and 
production (accentedness ratings). All control and non-control groups showed benefits in speech 
segmentation and speech processing skills irrespective of viewing mode. For phonological accuracy 
in perception, no significant differences were detected. In production, a focus on phonetic form 
improved pronunciation only in the absence of captions, whereas captioned viewing led to 
pronunciation gains as long as there was no focus on phonetic form. Results, therefore, indicate that 



ISSN 2621-6485 English Language Teaching Educational Journal 51 
 Vol. 5, No. 1, 2022, pp. 48-59 

 Harrison, J. (Comparing the effectiveness of explicit EAL feedback ….) 

improvements in pronunciation can take place either with captions or without captions when learners’ 
attention is focused on pronunciation. As with some other studies, the authors believed cognitive 
overload may be the reason no benefits were obtained when attention was directed to pronunciation 
when using captions. 

There is research that suggests captions and subtitles have no significant effect. Gordon (2020) 
added captions to recorded lectures for EAL learners in an e-learning environment and found no 
significant effects. Korucu-Kis (2021), researching captioning as a scaffold for L2 listening to 
discover if the dual coding of aural and written stimuli may lead to more comprehensible input and 
in-depth processing, stated that the effectiveness of captioning does not have a significant impact of 
listening. This narrative literature review concluded that captions do not necessarily lead to improved 
comprehension, and caption effectiveness is influenced by learner, material, measurement, task, and 
L1/L2 characteristics. Kruger et al. (2014) in a study on eye movement found that in terms of attention 
distribution, subtitle language and comprehension, the language or presence/absence of subtitles did 
not have any significant impact on comprehension of a work discussed in a recorded lecture. However, 
the three groups in this study distributed their visual attention resources differently in tests that were 
indicators of short term and long term retention of knowledge respectively. The findings for one of 
the groups suggested that captions resulted in a higher retention of knowledge in the longer term. 

1.2 Background on Slideware and Slide Design 

Research specifically comparing slideshow video with captioned video was limited or not able to 
be found in searches perhaps because slideshows are multi-modal including text, images, audio, video, 
etc. Therefore, a brief, limited summary of some of the research in the field will be given in 
chronological order.  

From PowerPoints debut in 1987, slideshows, overwhelmingly PowerPoint slideshows, became 
standard for face-to-face classroom situations, conferences, and business meetings. It was, and 
perhaps still is, generally accepted that slideshows support lectures, but the support is more related to 
aesthetics, attractiveness and entertainment than to effective learning. Although dated, Bartsch and 
Cobern (2003) explored the effectiveness of PowerPoint versus overhead transparency use in lectures. 
Results indicated that during semester transparencies were preferred but by end of term preferred 
PowerPoint and that students performed worse on quizzes when PowerPoint presentations included 
non-text items such as pictures and sound effects. In a second study participants were shown 
PowerPoint slides that contained only text, contained text and a relevant picture, or contained text 
with a picture that was not relevant. Students performed worse on recall and recognition tasks and had 
greater dislike for slides with pictures that were not relevant. They concluded that PowerPoint could 
be beneficial, but irrelevant material could interfere with learning. Craig (2006), in a well-researched 
and witty article, decries the lack of studies between 1987 and 2006 (less than 20) on the effectiveness 
of PowerPoint and the mountain of articles touting PowerPoint’s catchiness and entertainment value. 
He states, “Generally, the available studies lack substance and internal and external validity and adopt 
rather constrained characterizations of the concept effectiveness” (p. 149). During this time, it appears 
that researchers wanted to say that a product, in this case PowerPoint slideware, could create a 
statistically significant difference in learning; however, slideshow software has many different aspects 
and uses. Therefore, educators have been using it in many different ways that are complex and difficult 
to quantify. However, some researchers and designers realized this and have done research and offered 
advice on how to design better slides. A couple of these authors were Duarte (2008) and Reynolds 
(2012), and an internet search will offer quite a selection of books on presentation and slide design. 
Even now, ten years later the research focuses on a mix of aesthetics, attention-holding, and 
educational aspects. LeFebvre (2022) investigated two different PowerPoint slide designs for 
multimedia learning, presentation and teleprompter. Differences in participant fixations were assessed 
via eye tracking measures. Participants demonstrated greater fixation counts for teleprompter slides, 
measures of aesthetic liking evidenced that slides incorporating imagery resulted in more pleasurable 
learning experiences, and visually-based slides influenced more reflective learning and greater 
activation of information processing. 

One case study in scientific presentation slide design from this era is that of the assertion-evidence 
approach which uses a single sentence and a highly relevant graphic on each slide to ensure attention 
is drawn to the most important points in the presentation. This case demonstrates how researchers 
realized that slideware needed a slide design approach that would lead to educational gains for the 



52 English Language Teaching Educational Journal   ISSN 2621-6485 

 Vol. 5, No. 1, 2022, pp. 48-59 

 Harrison, J. (Comparing the effectiveness of explicit EAL feedback ….) 

audience (and the presentation creator) and have tried to bring about change in the way presenters 
communicate information to audiences. Research on this approach seems to focus on audience 
comprehension and recall as well as slideshow presenter depth of content understanding. Garner et al. 
(2011) wrote about assertion-evidence slide design and how it may lead to better comprehension and 
recall of more complex topics within presentations. In this experiment two groups of roughly 55 
audience members each watched a 6-minute presentation with either a topic-subtopic or an assertion-
evidence slide structure, and they were tested immediately afterward and then again one week later. 
The authors reported that on comprehension items and items related to retention of more complex 
concepts, learners using assertion-evidence slides achieved higher scores than learners from topic-
subtopic slides. This reportedly occurred on both tests, and some items achieved statistical 
significance. The authors point out that although learners of topic-subtopic slides viewed more written 
information during the presentation, those learners did not understand and remember that material. 
The authors go one to reiterate the importance of that point, referring to “additional benefits” of the 
assertion-evidence approach, and citing themselves (Garner et al., 2009; Alley, 2003) for theoretical 
support that presenters creating a presentation with this approach will develop “a more focused and 
overall stronger presentation” than if they used the traditional topic-subtopic approach.  Garner and 
Alley (2016) compared “open” student-created slide presentations with assertion-evidence structured 
slide presentations, approximately equal groups, for 120 undergraduate engineers. They argue that 
students usually create topic-subtopic structure slides dictated by the default settings of PowerPoint 
or other slideware and support that indicating that over 80 percent of the “open” group created topic-
subtopic slides. An unannounced post-test of comprehension was given 1 day later, and results 
revealed a statistically significant advantage (p < 0.05) for participants who created assertion-evidence 
slides. It appears that the assertion-evidence structure slide creation led to a statistically significant 
increase in the presenter’s understanding of the content, which seems to support earlier research by 
these authors.  

Independent research on assertion-evidence slide design audience comprehension and recall 
indicated significant gains for both slideware formats but better retention of information. Root Kustritz 
(2014) reported on a study of third-year veterinary students in a required theriogenology diagnostics 
course who blindly self-selected to attend either a presentation with PowerPoint slides in a traditional 
format or one with PowerPoint slides in the assertion-evidence format. Students took a pre-test, a take-
home assignment, an online post-test, and another online post-test after one month to evaluate 
retention. Groups did not differ on pre-test, assignment, or post-test scores, and both groups showed 
significant gains from pre-test to post-test and from pre-test to retention test. However, the traditional 
group showed a significant decline from post-test to retention test, while the assertion-evidence group 
did not. 

Miraldi (2021) published a dissertation investigating the reasons that the assertion-evidence 
approach has not been more widely implemented. Miraldi writes, “Despite the theoretical guidance 
from cognitive psychology and multimedia learning, the common practice of plugging text into the 
default template of PowerPoint persists throughout educational and professional settings” (p. 14). 
Later, Gaskin (2012) is cited, but I will include more of the quote here. Gaskin, the co-inventor of 
PowerPoint, reflects on PowerPoint saying, “Since the defaults can easily be changed (any 
presentation made in PowerPoint can be set as the default style), and a single default can be augmented 
with unlimited libraries of templates constructed in any style desired, I’ve often wondered myself why 
users don’t change and replace the defaults more often.” (p. 417) Miraldi’s results indicated that two 
innovation attributes, compatibility and trialability, were significantly and positively associated with 
implementation. First, re-invention was a significant moderating variable, and, second, that workplace 
norms play a significant role in the diffusion of this innovation. In other words, it is hard to get people 
to rally around new slide design formats that are not defaults, and it is hard to change 
office/education/scientific community culture.   

In EAL research, some studies indicate that content comprehension is better when PowerPoint is 
used and that learners prefer PowerPoint with audio to other means of on-demand instruction. Gordani 
and Khajavi (2020) investigated the effect of PowerPoint-supported lectures on immediate 
comprehension and longer term retention of content by EAL university students. Participants (n=69) 
were randomly assigned into three groups: A. PowerPoint-supported lectures with slides’ contents 
being read out, B. PowerPoint-supported lectures with slides at propositional level, and C. oral 
presentation with no multimedia. During each of the twelve 1-hour sessions, the students were 



ISSN 2621-6485 English Language Teaching Educational Journal 53 
 Vol. 5, No. 1, 2022, pp. 48-59 

 Harrison, J. (Comparing the effectiveness of explicit EAL feedback ….) 

presented with a lesson followed by a posttest of immediate recall, and they sat for the 1-month 
delayed posttest after the last session. Results indicated that learners’ comprehension improved 
significantly both immediately and after a 1-month delay when they are provided with PowerPoint-
supported lectures with slides at propositional level. The authors offer a caveat stating that for 
efficiency, the content of the slides “must be selected and designed with great care so that key terms, 
issues and main ideas are covered. Copying course material into slides and including too much 
explanations and details will have an adverse effect on students’ retention and recall.” Oh (2021) in a 
report on blended learning writes that students preferred on-demand instruction (non-real-time online 
instruction) to real-time online instruction. Regarding content presentation, the students preferred 
class video containing a PowerPoint presentation and the instructor’s audio explanation to real-time 
instruction via Zoom, PowerPoint and instructor’s face, and PowerPoint only.  

This is by no means a complete review of the literature, but the gist of the research indicates there 
some researchers are vaguely researching PowerPoint while others are doing more detailed research 
on how aspects of slideware, slide design, and approaches to presentation making affect educational 
outcomes. The latter approach is where future researcher should concentrate their efforts. 

2. Methods 

2.1 Setting and Participants 

This study was conducted at a private university in Japan. Students (n=163) in 8 courses of English 
Communication II completed all three stages of an introduction task. Due to the coronavirus situation, 
all of the lessons concerned with this project were in an online, on-demand format on a Moodle LMS.    

2.2 Research Instruments 

A quantitative design was used with 2 videos acting as variables and a quiz to measure the results. 
Students watched either a video of a PowerPoint slideshow with text on the slides recorded in Zoom 
which showed the teacher speaking in the upper-right corner (without captions) or a video of the 
teacher speaking with captions (no slides). There was not a control group with only audio and no 
written aspect as from an ethical standpoint all students were to receive the same content. Therefore, 
both videos had the same content and the same length. The common mistakes detailed in the videos 
are given in Table 1, which also includes the time each point began and the total time spent on each 
point. The quiz on the common English errors discussed in the videos contained 2 items for each of 
the 10 common errors. Contact the author for details. 

Table 1.  Points of General Feedback and Duration 

Order Feedback Point Start time Duration 

 Opening comments 0:00 1:32 

1 Paragraph writing vs. sentence writing 1:33 1:40 

2 Use of spaces after punctuation 3:13 1:33 

3 Use of "in" for affiliation, e.g. I am in the x department. 4:46 1:23 

4 General capitalization: first word, abbreviations 6:09 2:24 

5 "What x do you like?" vs. "What do you like x?" 8:33 1:06 

6 Name order in English and Japanese 9:39 2:41 

7 Name capitalization and use of title and punctuation 12:20 1:23 

8 Comma use before conjunctions when joining sentences 13:43 1:24 

9 Use of "because" when joining sentences 15:07 1:01 

10 Plurals/Use of S 16:08 1:12 

 Closing comments 17:20 0:20 

Total  17:40 17:40 



54 English Language Teaching Educational Journal   ISSN 2621-6485 

 Vol. 5, No. 1, 2022, pp. 48-59 

 Harrison, J. (Comparing the effectiveness of explicit EAL feedback ….) 

 

2.3 Student Tasks, Data Collection Procedures, and Data Anaalysis 

The student task was to be completed in 3 stages (1 stage per lesson for 3 lessons). The first part 
of Lesson 1 focused on self-introductions, and meeting others, and asking questions, and the second 
part of the lesson focused on health and health during the pandemic. One of the final tasks in the lesson 
was for each student to post 1 new discussion in a forum. The directions were: “Introduce yourself to 
the other people in the course. Write or speak at least 5 good self-introduction sentences, and end with 
a question to the other students.” Scoring criteria and instructions on how to add a discussion, add 
audio and videos files, etc. were also given.    

The second stage of the task was in Lesson 2 where students were asked to reply to 2 other students’ 
questions with responses of at least 3 sentences each. Again, scoring criteria and instructions on how 
to reply were given. During this stage, the teacher edited student self-introductions from the first stage 
by putting asterisks where errors occurred and offering private feedback on posts that were lacking in 
amount of content or off-topic. 

The third stage of the project was the review and feedback stage. First, students were instructed to 
watch a 17-minute 40-second video of general feedback on common mistakes. Half (4 courses) of the 
students watched a video of a PowerPoint slideshow, and the other half of the students (4 courses) 
watched a video of the teacher speaking with captions. Second, students were asked to review their 
posted discussion and the replies from other students to their discussion questions and to notice the 
asterisked marked areas. Finally, students were asked to review the video again before taking a 1-
attempt online quiz which consisted of 20 items.  

Regarding data collection, data was downloaded at the beginning of the sixth week. Therefore, 
students who had not completed the tasks due to late registration, absences, etc. were excluded from 
this study. Data from the quiz was analyzed to check for differences between the slideshow group and 
the caption group. An item facility analysis and item discrimination analysis were also conducted for 
each group to compare how the items performed for both groups. 

3. Findings and Discussion 

3.1 Results for All Participants 

Results for all students, shown in Table 2, suggested that both the slideshow and caption groups 
were similar. The minimums and means (M) were slightly higher for the slideshow group. For 
example, out of 20 total points, the mean score was .4 higher for the slideshow group, but with 
rounding to the nearest whole number the quiz score means and standard deviations (SD), the mean 
quiz time, and the mean of actual views were the same. The most noticeable difference was the mean 
time on video was 114 seconds (approx. 2 minutes) longer for the slideshow group. The p-values were 
high and did not denote significance. Regarding distributions, the slideshow scores were 61% within 
1 SD and 100% within 2 SDs, and the caption scores were close to normal but with 75% within 1 SD. 

Table 2.  Comparison of Scores, Views and Time on Video by Slideshow and Caption Groups 

 Slideshow (n=76)  Captions (n=87)    

Variable Range M SD  Range M SD t df p 

Quiz Score 5-19 12.20 4.21  5-20 11.78 3.84 0.65 153 0.51 

Quiz Time (s) 296-1201 849 243  239-1201 831 259 0.45 160 0.65 

Views (clicks) 1-4 1.51 0.7  1-4 1.53 0.74 -0.14 160 0.089 

Actual Views 0-2 0.73 0.61  0-2 0.7 0.51 0.62 148 0.054 

Time on Video 
(s) 

26-3301 1010 973  928-3567 896 727 0.84 137 0.04 

 

3.2 Results for All Participants with at least 1 Complete View 

One issue with this analysis was that students clicked to view the video but many students clicked 
only as a means to open the quiz and expedite advancement through the lesson. Therefore, a second 



ISSN 2621-6485 English Language Teaching Educational Journal 55 
 Vol. 5, No. 1, 2022, pp. 48-59 

 Harrison, J. (Comparing the effectiveness of explicit EAL feedback ….) 

analysis was conducted after removal of participants with 0 actual views, a view ending before the 
first feedback point ended (1 min. 40 sec.). This removed exactly 45% of each group. Therefore, 42 
students from the slideshow group and 48 students from the caption group remained with .1 or more 
views, .1 representing the viewing of 1 feedback point from the video. When students with less than 
1 full view of all 10 feedback points were removed (views under the duration of the video). This 
removed another 5% of the slideshow group from the data set but the caption group remained the 
same. What remained was 50% (n=38) of the slideshow group and 55% (n=48) of the caption group 
who had viewed the full video at least once. 

The results, shown in Table 3, indicate that for the caption group the range is one point higher and 
the views are negligibly higher. However, for the slideshow group the mean quiz score is .15 points 
higher, the mean of actual views is .1 higher, and the mean time on video is higher by 605 seconds 
(approx. 10 mins). A glance at the raw data showed that most students in the slideshow group were 
taking 1-2 minutes longer for viewing and were watching the video longer the second time.  The 
distributions of the slideshow and caption groups were nearly identical with 81% and 79% within 1 
SD and 100% within 2 SDs.  

Table 3.  Comparison of Scores, Full Views and Time on Video by Slideshow and Caption Groups 

 Slideshow (n=38)  Captions (n=48)    

Variable Range M SD  Range M SD t df p 

Quiz Score 5-19 13.22 4.15  6-20 13.06 3.69 0.17 74 0.86 

Quiz Time (s) 326-1201 843 222  326-1201 859 259 -0.14 83 0.89 

Views (clicks) 1-3 1.49 0.68  1-4 1.56 0.77 -0.06 83 0.95 

Actual Views 1-2 1.18 0.4  1-2 1.10 0.26 1.39 61 0.17 

Time on Video 

(s) 
970-3301 1501 908  928-3567 896 535 1.13 57 0.26 

 

3.3 Item Facility and Item Discrimination 

The quiz had 20 items, 2 items for each common error addressed in the feedback video. Item 
facility (IF) analysis and item discrimination (ID) present a picture of what items were difficult for 
students and how high and low proficiency students perform compared to each other. The feedback 
video lead to a quiz which was meant to be a criterion-referenced test; however, the quiz will be 
evaluated as a as a norm-referenced test as pre-tests and post-tests were not given as part of this 
feedback exercise.  

For the analysis IF and ID values were calculated for the entire slide group (n=76) and caption 
group (n=87) as well as the full view slide group (n=38) and caption group (n=48), shown in Table 4. 
Considering many students did not have a full view, item facility can be used as if for a norm-reference 
test. The lower the IF, the more difficult the item was for students. When an IF of .6 is used as a cut 
off to mark “difficult” items, meaning less than 60% of students correctly answered the item, the full 
slide group had 10 difficult items, and the 1-view group had 6 difficult items. The caption groups had 
more difficult items as the full caption group had 12 and the 1-view caption group had 10 difficult 
items.  Therefore, the slide groups seem to have had less difficulty with the items. When the IFs for 
all 20 items are summed, the caption groups had lower IF totals (11.79, 13.05) than the slide groups 
(12.01, 13.22) suggesting that in total the slide group found the items easier. Regarding the feedback 
items and areas where these EFL students struggled even after the feedback video, looking at the IDs 
in Table 4, lower IDs represent items where the item both groups struggled with defining a paragraph 
(1a), capitalization of proper nouns (4ab, 7ab), comma conjunction (8a), connecting sentences with 
“because” (9ab) and plurals (10ab). 

The item description indicates the degree to which an item separates the students who performed 
well for those who did poorly on the test as a whole. An ID of .29 or lower indicates that the item 
performed poorly and may need adjustment (Brown, 2005). None of the items performed poorly for 
all four of the groups. However, the full slide group had 3 items and the full caption group had 7 items 
which performed badly. The 1-view slide group had 4 poorly performing items and the 1-view caption 
group had 5. When the IDs for all items are summed the full slide group and 1-view slide group total 
ID sums are higher (9.83, 9.76) than those of the caption groups (7.50, 8.47) suggesting that the items 



56 English Language Teaching Educational Journal   ISSN 2621-6485 

 Vol. 5, No. 1, 2022, pp. 48-59 

 Harrison, J. (Comparing the effectiveness of explicit EAL feedback ….) 

perform better with the slide groups. Also, the items performed better for the full slide group than the 
full caption group on 14 items, and they also performed better for the 1-view slide group on 13 items. 
Items 2b, 6a, 6b, and 9a performed better for both caption groups, but it is unclear why this is the case.  

Table 4.  Item Facility and Item Discrimination for Groups 

Item Feedback Point 

Slide 

IF 

(n=76) 

Cap. 

IF 

(n=87) 

Slide 

ID 

(n=76) 

Cap. 

ID 

(n=87) 

Slide 

IF 

(n=38) 

Cap. 

IF 

(n=48) 

Slide 

ID 

(n=38) 

Cap. 

ID 

(n=48) 

1a Paragraph 0.49 0.45 0.69 0.27 0.53 0.58 0.83 0.50 

1b Paragraphs 0.64 0.69 0.62 0.21 0.74 0.79 0.58 0.38 

2a 
Spaces after 
punctuation 

0.85 0.83 0.38 0.23 0.92 0.90 0.25 0.19 

2b 
Spaces after 

punctuation 
0.86 0.80 0.23 0.27 0.95 0.79 -0.08 0.31 

3a 
Statement "in" 
department 

0.72 0.77 0.65 0.35 0.84 0.85 0.42 0.19 

3b 
Interrogative "in" 

department 
0.68 0.74 0.73 0.39 0.79 0.75 0.50 0.31 

4a 
Capitalization of 
proper nouns 

0.42 0.32 0.77 0.57 0.53 0.50 1.00 0.63 

4b 
Capitalization of 

proper nouns 
0.51 0.45 0.77 0.78 0.61 0.54 0.67 0.81 

5a 
Imperitive: Tell me 
what x you like 

0.63 0.56 0.69 0.44 0.71 0.58 0.75 0.56 

5b 
Interrogative: What x 

do you like? 
0.72 0.67 0.38 0.39 0.76 0.71 0.58 0.44 

6a 
Capitalization of 
names, name order 

0.63 0.56 0.31 0.34 0.68 0.65 0.25 0.38 

6b 
Capitalization of 

names, name order 
0.79 0.78 0.15 0.32 0.82 0.83 0.25 0.38 

7a 
Capitalization of 
proper nouns 

0.41 0.45 0.31 0.24 0.37 0.44 0.50 0.25 

7b 
Name order, cap. of 

proper nouns 
0.54 0.59 0.42 0.40 0.66 0.67 0.68 0.38 

8a Comma conjunction 0.59 0.52 0.62 0.42 0.68 0.52 0.33 0.69 

8b Comma conjunction 0.78 0.77 0.46 0.28 0.87 0.81 0.33 0.44 

9a Connecting because 0.35 0.41 0.38 0.64 0.26 0.56 0.50 0.69 

9b Connecting because 0.41 0.45 0.62 0.38 0.42 0.54 0.67 0.56 

10a Plurals 0.41 0.43 0.42 0.32 0.45 0.46 0.42 0.13 

10b Plurals 0.58 0.55 0.23 0.26 0.63 0.58 0.33 0.25 

4. Discussion  

This research focused on how 2 groups of learners performed on a 20-item grammar quiz after 
watching a feedback video regarding an online writing task. To briefly discuss the results, significant 
differences were not found between slideware groups and caption groups. These results are similar to 
Liu (2018) who did not find statistically significant differences when trying to identify an optimum 
input mode for EAL learners. The findings in this report, therefore, add to the existing literature that 
captions have no significant effect (Kruger et al, 2014; Gordon, 2020; Mayer, 2021; Korucu-Kis, 
2021). That being said, there are caveats in the research; for example, findings for one of the groups 
in research by Kruger et al, (2014) suggested that captions resulted in a higher retention of knowledge 
in the long term.  

A caveat in the findings of this study is that although significant differences were not found 
between slideware and caption groups, EAL learners engaged longer with slideshow video, mean 
scores on quizzes were slightly higher for the slideshow groups, and quiz items performed better for 
the slideshow groups. These results also seem to support that learners prefer or are more comfortable 



ISSN 2621-6485 English Language Teaching Educational Journal 57 
 Vol. 5, No. 1, 2022, pp. 48-59 

 Harrison, J. (Comparing the effectiveness of explicit EAL feedback ….) 

with the slideshow format with audio, e.g. PowerPoint, to other means of on-demand instruction 
(Gordani & Khajavi, 2020), in this case captioned video. 

To consider this more deeply, one area requiring more inquiry is regarding the length of learner 
engagement with the video and the quiz results. The point of view discussed in the previous paragraph 
is that the learners engaged longer with the slideshow video and this is a positive factor as it may have 
led to slightly higher mean scores for that group. However, it must also be considered that as the 
differences in mean scores between sideshow and caption groups were not significantly different that 
the excess time spent viewing the slideshow video was perhaps inefficient as it did not lead to 
significantly better quiz scores.  

Returning to the opening point of the discussion, if captions do not hinder learners as significant 
differences were not found in this study or in studies by the above mentioned researchers, then 
educators in EAL settings should include captions as research by other researchers seems to 
demonstrate that for language learning captioned video leads to better comprehension of listening 
comprehension, idiomatic expressions, incidental vocabulary learning and vocabulary acquisition 
(Wink et al., 2010; Hayati & Mohmedi, 2011; Arndt, 2014, 2018; Ketabi & Sadeghi, 2020). For 
educators teaching pronunciation, captions do not hinder learners as long as there is not a focus on 
phonetic form (Wisniewska & Mora, 2020).  

Limitations of this study are numerous as it was a short-term study focused on feedback 
comprehension of a variety of grammar points and long tern retention was not considered in the 
research design. The study also merely focused on the comprehension of text and audio and did not 
include graphics or animation as in other studies. Thirdly, slideshow text and captions were compared; 
therefore, first language subtitles were not a variable in this study. Lastly, some research suggests that 
different proficiency groups use captions in different ways, but proficiency level as measured by a 
standardized test or other means was not a part of this study. Research (Desjarlais, 2017; Gass et al., 
2019; Wisniewska & Mora, 2020) does suggest that the proficiency levels of learners should be a 
consideration in future research regarding captions and working memory.    

The findings in this report add to the existing literature, but more specific, detailed studies are 
needed. As stated earlier, some researchers are vaguely researching PowerPoint while others are doing 
more detailed research on how aspects of slideware, slide design, and approaches to presentation 
making affect educational outcomes. The latter approach is where future researchers should 
concentrate their efforts as technology continues to be developed and integrated for educational 
purposes. 

5. Conclusion 

The goal of this research was to compare the effectiveness of explicit EAL feedback between 
slideshow video and captioned video. This research was carried out at a university in an on-demand 
(asynchronous) educational setting, and in this setting approximately 50% of the learners clicked the 
feedback video to advance to the quiz without watching it. To answer the research questions based on 
the learners who did partake of at least 1 full viewing of the video, Q1. EAL learners engage longer 
with a slideshow video (text+audio) than with a captioned video. Q2. Regarding which format 
garnered more views, views (clicks) and actual views were almost exactly the same for all groups. 
Q3. The format that was more effective for transmitting the feedback content demonstrated via quiz 
scores seems to be slideshow video as the mean scores are slightly higher than those of the captioned 
video group. However, a significance difference was not demonstrated. Q4. When EAL learners were 
divided by quiz performance level, the slideshow group seems to have benefited more. The results of 
the item facility analysis suggested that the slideware group found the items easier. The areas where 
these EAL students struggled even after watching the feedback video were defining a paragraph, 
capitalization of proper nouns, comma use before conjunctions separating sentences, connecting 
sentences with “because,” and use of plurals. Item discrimination analysis suggested that the items 
performed better with the slideshow video groups. To conclude, slideshow video seems to have 
engaged students for a longer duration than captioned video. The quiz items were easier for the 
slideshow video groups, and the items performed better for these groups. The slideshow video groups 
had slightly higher scores, but a significant difference between slideshow video and captioned video 
to transmit feedback to students was not found.  



58 English Language Teaching Educational Journal   ISSN 2621-6485 

 Vol. 5, No. 1, 2022, pp. 48-59 

 Harrison, J. (Comparing the effectiveness of explicit EAL feedback ….) 

REFERENCES 

Alley, M. (2003). The craft of scientific presentations. Springer-Verlag. 

Arndt, H. (2014). English vocabulary learning through watching YouTube video blogs and reading blog Posts. 

[Master’s thesis, University of Oxford]. 

Arndt, H. L., & Woore. R. (2018).Vocabulary learning from watching YouTube videos and reading blog posts. 

Language Learning & Technology, 22(1), 124–142. 

Ashcroft, R. J., Garner, J., & Hadingham, O. (2018). Incidental vocabulary learning through watching movies. 

Australian Journal of Applied Linguistics, 1(3), 135-147. https://doi.org/10.29140/ajal.v1n3.89  

Bartsch, R. A., & Cobern, K. M. (2003). Effectiveness of PowerPoint presentations in lectures, Computers & 

Education, 41(1),77-86. https://doi.org/10.1016/S0360-1315(03)00027-7  

Brown, J. D. (2005) Testing in language programs: A comprehensive guide to English language assessment. 

McGraw-Hill ESL/ELT.  

Camacho Velez, A. B. & Pozo Estévez, V. A. (2021). Effects of captioning on L2 vocabulary development. 

[Master’s thesis, University of Central Ecuador].  

Cintrón-Valentín, M., & García-Amaya, L. (2021). Investigating textual enhancement and captions in L2 

grammar and vocabulary: An experimental study. Studies in Second Language Acquisition, 1(26). 

https://doi.org/10.1017/S0272263120000492  

Craig, R.J., & Amernic, J.H. (2006). PowerPoint presentation technology and the dynamics of 

teaching. Innovation in Higher Education, 31, 147–160. https://doi.org/10.1007/s10755-006-9017-5  

Desjarlais, M. (2017). The use of eye-gaze to understand multimedia learning. In C. Was, F. Sansosti, & B. 

Morris (Eds.), Eye-tracking Technology Applications in Educational Research, 122–142. Hershey, PA: IGI 

Global. https://doi.org/10.4018/978-1-5225-1005-5.ch007  

Duarte, N. (2008). Slide:ology: The art and science of creating great presentations. O'Reilly Media. 

Fievez, I, Perez, M.M., Cornillie, F.,  & Desmet, P. (2020). Vocabulary learning through viewing captioned or 

subtitled videos and the role of learner- and Word-Related Factors, CALICO Journal, 37(3). 

https://doi.org/10.1558/cj.39370  

Garner, J., Alley, M., Gaudelli, A., & Zappe, S. (2009). Common use of PowerPoint versus assertion-evidence 

slide structure: a Cognitive Psychology Perspective. Technical Communication, 56(4), 331−345. 

Garner, J. K., Alley, M., Sawarynski, L. E., Wolfe, K. L., & Zappe, S.E. (2011). Assertion-evidence slides 

appear to lead to better comprehension and recall of more complex concepts. ASEE Annual Conference 

and Exposition, Conference Proceedings. 

Garner, J.K., & Alley, M.P. (2016). Slide structure can influence the presenter’s understanding of the 

presentation’s content. International Journal of Engineering Education, 32(1)(A), 39–54. 

Gaskins, R. (2012). Sweating bullets: Notes about inventing PowerPoint.  

https://www.robertgaskins.com/powerpoint-history/sweating-bullets/gaskins-sweating-bullets-webpdf-

isbn-9780985142414.pdf. 

Gass, S., Winke, P., Isbell, D. R., & Ahn, J. (2019). How captions help people learn languages: A working-

memory, eye-tracking study. Language Learning & Technology, 23(2), 84–104. 

https://doi.org/10125/44684  

Gordani, Y., & Khajavi, Y. (2020). The impacts of multi-modal PowerPoint presentation on the EFL students’ 

content knowledge attainment and retention over time. Education and Information Technologies 25, 403–

417. https://doi.org/10.1007/s10639-019-09979-z  

Gordon M. (2020). The effect of adding same-language subtitles to recorded lectures for non-native, English 

speakers in e-learning environments, Research in Learning Technology, 28. 

https://doi.org/10.25304/rlt.v28.2340 

https://doi.org/10.29140/ajal.v1n3.89
https://doi.org/10.1016/S0360-1315(03)00027-7
https://doi.org/10.1017/S0272263120000492
https://doi.org/10.1007/s10755-006-9017-5
https://doi.org/10.4018/978-1-5225-1005-5.ch007
https://doi.org/10.1558/cj.39370
https://www.robertgaskins.com/powerpoint-history/sweating-bullets/gaskins-sweating-bullets-webpdf-isbn-9780985142414.pdf.
https://www.robertgaskins.com/powerpoint-history/sweating-bullets/gaskins-sweating-bullets-webpdf-isbn-9780985142414.pdf.
https://doi.org/10125/44684
https://doi.org/10.1007/s10639-019-09979-z
https://doi.org/10.25304/rlt.v28.2340


ISSN 2621-6485 English Language Teaching Educational Journal 59 
 Vol. 5, No. 1, 2022, pp. 48-59 

 Harrison, J. (Comparing the effectiveness of explicit EAL feedback ….) 

Hayati, A., & Mohmedi, F. (2011). The effect of films with and without subtitles on listening comprehension 

of EFL learners. British Journal of Educational Technology, 42(1). https://doi.org/10.1111/j.1467-

8535.2009.01004.x  

Ketabi, S., & Sadeghi, A. (2020). The Effects of Captioned Slides on Learning of English Idiomatic 

Expressions among EFL Intermediate Learners. International Journal of Advanced Studies in Humanities 

and Social Science, 9(1), 1-20. https://doi.org/10.33945/SAMI/IJASHSS.2020.1.1 

Korucu-kiş, S. (2021). On the effectiveness and limitations of captioning in L2 listening. International Journal 

of Modern Education Studies, 5(2), 516-536.  

Kruger, J.-L., Hefer, E., & Matthew, G. (2014). Attention distribution and cognitive load in a subtitled 

academic lecture: L1 vs. L2. Journal of Eye Movement Research, 7(5). https://doi.org/10.16910/jemr.7.5.4  

LeFebvre, L., Parsons, M., Entwistle, C., Boyd, R., & Allen, M. (2022). Rethinking PowerPoint slide design 

for multimedia learning. Journal of Educational Technology Systems. 

https://doi.org/10.1177/00472395221077628 

Liu, Y., Jang, B. G, & Roy-Campbell, Z. (2018). Optimum input mode in the modality and redundancy 

principles for university ESL students' multimedia learning, Computers & Education, 127, 190-200. 

https://doi.org/10.1016/j.compedu.2018.08.025  

Liu, Y. (2019). Multimedia input modes, the Modality Principle, and the Redundancy Principle for university 

ESL students’ learning (Publication No. 27670850) [Doctoral dissertation, Syracuse University]. ProQuest 

Dissertations Publishing.  

Mayer, R. E., (2021). Evidence-based principles for how to design effective instructional videos, Journal of 

Applied Research in Memory and Cognition, 10(2), 229-240. https://doi.org/10.1016/j.jarmac.2021.03.007  

Miraldi, L.B. (2021). Using diffusion of innovations to understand factors related to the use of assertion-

evidence presentation slides in the engineering workplace (Publication No. 28929435) [Doctoral 

dissertation, Pennsylvania State University]. ProQuest Dissertations Publishing. 

Oh, E., (2021). Blended instruction in online learning environments, Asia-pacific Journal of Convergent 

Research Interchange, 7(1), 125-134. https://doi.org/10.47116/apjcri.2020.01.12 

Pujadas, G., & Muñoz, C. (2020). Examining adolescent EFL learners' TV viewing comprehension through 

captions and subtitles, Studies in Second Language Acquisition, 42(3), 551-575. 

https://doi.org/10.1017/S0272263120000042  

Reynolds, G. (2012). Presentation zen: Simple ideas on presentation design and delivery. New Riders. 

Ritzhaupt, A., Pastore, R., & Davis, Robert. (2015). Effects of captions and time-compressed video on learner 

performance and satisfaction, Computers in Human Behavior, 45. 

https://doi.org/10.1016/j.chb.2014.12.020  

Root Kustritz M. V. (2014). Effect of differing PowerPoint slide design on multiple-choice test scores for 

assessment of knowledge and retention in a theriogenology course. Journal of Veterinary Medical 

Education, 41(3), 311–317. https://doi.org/10.3138/jvme.0114-004R  

Wisniewska, N., & Mora, J. (2020). Can captioned video benefit second language pronunciation? Studies in 

Second Language Acquisition, 42(3), 599-624. https://doi.org/10.1017/S0272263120000029  

Winke, P., Gass, S. & Sydorenko, T. (2010). The effects of captioning videos used for foreign language 

listening activities, Language Learning & Technology, 14(1), 65-86.  

Zhang, R., & Zou, D. (2021). A state-of-the-art review of the modes and effectiveness of multimedia input for 

second and foreign language learning. Computer Assisted Language Learning, 1-27. 

https://doi.org/10.1080/09588221.2021.1896555  

 

https://doi.org/10.1111/j.1467-8535.2009.01004.x
https://doi.org/10.1111/j.1467-8535.2009.01004.x
file:///C:/Users/MuhammadWildan/Downloads/Ketabi,%20S.,%20&%20Sadeghi,%20A.%20The%20Effects%20of%20Captioned%20Slides%20on%20Learning%20of%20English%20Idiomatic%20Expressions%20among%20EFL%20Intermediate%20Learners.%20International%20Journal%20of%20Advanced%20Studies%20in%20Humanities%20and%20Social%20Science,%209(1),%201-20.%20https:/doi.org/10.33945/SAMI/IJASHSS.2020.1.1
https://doi.org/10.16910/jemr.7.5.4
https://doi.org/10.1016/j.compedu.2018.08.025
https://doi.org/10.1016/j.jarmac.2021.03.007
https://doi.org/10.1017/S0272263120000042
https://doi.org/10.1016/j.chb.2014.12.020
https://doi.org/10.3138/jvme.0114-004R
https://doi.org/10.1017/S0272263120000029
https://doi.org/10.1080/09588221.2021.1896555

