Journal of Teaching and Learning with Technology, Vol. Vol. 11, Special Issue, pp.57-61. doi: 10.14434/jotlt.v11i1.34352 Using Video Simulations for Assessing Clinical Skills in Speech- Language Pathology Students Abby Hemmerich University of Wisconsin-Eau Claire Jerry Hoepner University of Wisconsin-Eau Claire Abstract: A common challenge for clinical training programs is helping students apply academic knowledge to clinical settings. Authentic assessment using simulation offers a unique approach to bridging this gap. Miller’s pyramid provides a framework for competency-based education that integrates formative assessment and feedback at each stage of student learning. A multi-part assignment that builds from gathering data following a specific protocol (i.e., basic level), moves through interpretation of data (i.e., intermediate level), and then using that data to direct next steps (i.e., advanced level) scaffolds student learning toward clinical practice. Review of past student assignments indicated better performance on intermediate and advanced skills when using a video-based, multi- component assignment as compared to the original assignment design. Incorporating video components allows simulation of rare clinical populations, while also replicating current telepractice service provision. By simulating patient interactions, the instructor replicates real world challenges, allowing the students to demonstrate in-the-moment problem solving and clinical responsiveness. Keywords: competency-based education, video, videoconferencing, formative assessment Like other clinical disciplines, education in the field of speech-language pathology is increasingly shifting towards competency-based education practices (Hoepner & Hemmerich, 2020). Historically, students received knowledge-based education with descriptions of skills in their academic courses, often without practice implementing those skills. Students were expected to apply their knowledge and skills in clinical practicum. Clinical supervisors supported the application and implementation processes but did not always share the same approach or perspectives of the course instructors. Competency-based education (CBE) systematically and incrementally implements knowledge and skills training, ultimately measuring readiness for clinical practice (McAllister et al., 2011). Medical educators often use Miller’s pyramid as a framework for developing knowledge, skills, and preparedness for clinical practice (Lockyer et al., 2017; Miller, 1990). At the base of the pyramid, instruction focuses on building declarative knowledge (i.e., knowing). The next level, knowing how, prepares students to interpret and apply that knowledge through guided demonstrations and models. In level three, showing how, students demonstrate their knowledge and skills through formative and summative competencies. Finally, students are transitioned to clinical contexts, where performance is integrated into practice (i.e., doing). Of course, learning and refining one’s approach still takes place in that final level (i.e., doing), but they have reached entry-level clinical competence. Hoepner and Hemmerich (2020) modified this framework to encompass knowledge, skills, and professional dispositions necessary to competently enter clinical practice in speech-language pathology. Competency-based instruction and assessment in courses only addresses the bottom three levels of Miller’s pyramid, since doing occurs in clinical practice. Taking this one step further, Figure 1 depicts how iterative assignments can move students through all three course-based levels of the pyramid. Each level is addressed within each assignment (i.e., knowing, knowing how, and showing how); however, expectations for skills move from basic to advanced. Feedback for all levels is formative, Hemmerich and Hoepner Journal of Teaching and Learning with Technology, Vol. 11, Special Issue, jotlt.indiana.edu providing guidance for the levels that follow, as well as summative feedback in the form of grades on each assignment. Formative feedback at the final level is intended to carry over to clinical practice. Figure 1. Modified Miller’s pyramid mapped to three levels of course assignments. Mapping a field-specific example to Miller’s pyramid. Within the field of speech-language pathology, like other clinical fields, there are populations that are rare, making it difficult for students to have hands-on contact. In these cases, providing a simulated experience can be a valuable substitute. In speech-language pathology, children with cleft lip/palate are one example of a challenging population to access, as many of these children are seen in large medical centers where few graduate students get placements. The role of the speech-language pathologist (SLP) with children with cleft palate involves multiple steps. Initially, SLPs must evaluate the child and their family to identify areas of concern. This includes reviewing medical and educational documents, interviewing the family and other healthcare and educational professionals, and completing a hands-on examination of the child. Once a plan of treatment is determined, the SLP plays a role in feeding, swallowing, speech, and language development, as well as serving as a resource and counselor for the patient and family. Students require opportunities to practice all of these skills and receive formative feedback to hone their skills. Table 1 provides an example of a multi-part assignment to address these complex and interrelated levels required for student learning. Multi-part Assignment Components Following the protocol and gathering data As students tackle a new topic, they first need a chance to demonstrate prerequisite, foundational knowledge and basic skills for gathering relevant data. This can take multiple forms, such as completion of fact-based assignments or quizzes, but can also span higher levels of Miller’s pyramid, such as demonstrating a skill with a standard case. In assignment 1 (see Table 1), students first review materials and complete a protocol-based assignment by creating a plan for their oral mechanism exam and speech sound testing (i.e., planning data collection), and then implement that plan in a video- recorded submission where they complete the exam and testing on another person (i.e., actual data 58 Hemmerich and Hoepner Journal of Teaching and Learning with Technology, Vol. 11, Special Issue, jotlt.indiana.edu collection). This fits into the basic skills level for multiple reasons. First, the oral mechanism exam and speech testing students perform is relatively standardized; we are building skills in systematic data collection, ensuring the quality of their data for interpretation. Second, the video recording submission demonstrates that they know how to complete these skills and does not require a higher-level skill of interpretation. Multiple repetitions of this exam with individuals who have typical function provide a good baseline for assessing individuals who demonstrate deviations from that norm. Interpreting and reporting data Once students are comfortable with discipline-specific techniques for gathering information or data, they must learn what to do with that data (i.e., intermediate skill level). Interpretation of data requires a deeper level of knowledge and the ability to compare results to expectations. In the example in Table 1, this means applying their knowledge of normal oral motor function and typical speech to the results provided by the instructor. In this specific situation, the instructor provides a video recording with atypical findings because students do not have access to this clinical population. Thus, this becomes a simulation out of necessity and provides the option of viewing the video multiple times. Students review the video clinical exam and speech testing results, where they simulate the interpretation that would occur if this patient were present, distinguishing typical from atypical performance. Their interpretations lead them to final clinical decisions (i.e., conclusions), and they create a clinical report following disciplinary guidelines. Using data to direct next steps Once students have skills in data collection and interpretation, the next step is applying that knowledge to new or more complex situations. In some fields, this may entail designing a new experiment to test new hypotheses. In other fields, like speech-language pathology, this entails addressing their findings or remediating patient skills. Using role play to complete this step pushes students to a more advanced level, where they must carry out intervention and parent education, while responding in the moment to human variability enacted by the instructor (see Table 1). This spans nearly all levels of Miller’s pyramid, which include collecting data, interpreting data, and making adjustments based on that data in a live interaction. 59 Hemmerich and Hoepner Journal of Teaching and Learning with Technology, Vol. 11, Special Issue, jotlt.indiana.edu Table 1. Assignment components by levels of Miller’s pyramid Assignment Knows Knows How Shows How Assessment Basic: Following the protocol & gathering data (video submission) Structures of head & neck Oral mechanism exam plan Demonstrates oral mechanism exam on a partner Feedback on process of exam Speech sound characteristics Speech sound testing plan Implements speech sound testing on a partner (no scoring) Feedback on speech sounds used and techniques for eliciting sounds Intermediate:* Interpreting & reporting the data (video review, written submission) Parent input Interview plan Interprets findings from listening to parent input Feedback on summary of parent concerns Speech sound errors & patterns Speech sound testing plan Interprets findings from listening to child’s speech Feedback on speech sound summary – did you hear what you should have heard? Advanced:* Using data to direct next steps (role-play tele- session) Techniques for remediating speech sound errors Techniques & therapy plan Role play – teach instructor to make sounds 1) in-the-moment adjustments based on what instructor does 2) coaching by instructor on alternative approaches Parent education Parent education plan Verbalize parent education & respond to questions 1) in-the-moment responsiveness to instructor questions 2) coaching by instructor on alternative topics or ways to explain *Higher levels implicitly subsume prior levels Evaluating assignment approach Prior iterations of this course included a similar competency-based assignment compressed into a single live meeting with the instructor. Students received client information (i.e., case history and demographics) and planned a brief assessment to carry out in a role-play simulation with the instructor. Immediately following the assessment role-play, they interpreted their results and implemented a parent education and treatment simulation in the same meeting. This approach required some data collection skills but omitted a critical element—the oral mechanism examination—given time constraints. Compressing all elements into a single interaction put students under tremendous pressure to perform efficiently and did not always allow them to show their full skillset. Student performance on both iterations of this assignment were compared through a review of common errors (see Figure 2). The multi-part assignment provided more opportunities for formative feedback regarding clinical skills employed in assessment and intervention. This led to fewer errors in interpretation and parent education as compared to the time-constrained condition. The expanded treatment role-play allowed the instructor to identify more nuanced challenges, evidenced by more support required during the live interaction (Figure 2), and provide formative, in-the-moment feedback. 60 Hemmerich and Hoepner Journal of Teaching and Learning with Technology, Vol. 11, Special Issue, jotlt.indiana.edu Figure 2. Student performance summary on assignment iterations. Implications Video and video chat simulations are an innovative approach to implementing competency-based instruction. Students are engaged in video development, along with data collection and interpretation, which provides a realistic representation of clinical workplace contexts. The use of multi-component assignments allows instructors to divide content into manageable segments. These segments are linked to a single case, allowing learners to make connections across contexts. Using multiple segments provides repeated opportunities for formative assessment of knowledge and skills prior to the final portion of the assignment. Instructor-student interactions within simulations via video conferencing provide exposure to the teleservice context, which is integral to contemporary service provision. The competency-based framework ensures development and assessment of skills for entry-level clinical work. These skills are measured authentically in the context of a simulated clinical experience, allowing assessment of in-the-moment problem solving. References Hoepner, J.K. & Hemmerich, A.L. (2020). Using formative video competencies and summative in- person competencies to examine preparedness for entry-level professional practice. Seminars in Speech and Language, 41(04), 310-324. https://doi.org/10.1055/s-0040-1713782 Lockyer, J., Carraccio, C., Chan, M.K., Hart, D., Smee, S., Touchie, C., Holmboe, E.S., Frank, J.R., & on behalf of the ICBME Collaborators. (2017). Core principles of assessment in competency-based medical education. Medical teacher, 39(6), 609- 616. https://doi.org/10.1080/0142159X.2017.1315082 McAllister, S., Lincoln, M., Ferguson, A., & McAllister, L. (2011). A systematic program of research regarding the assessment of speech-language pathology competencies. International Journal of Speech-Language Pathology, 13(6), 469-479. https://doi.org/10.3109/17549507.2011.580782 Miller, G.E. (1990). The assessment of clinical skills/competence/performance. Academic medicine, 65(9), S63-7. https://doi.org/10.1097/00001888-199009000-00045 0 10 20 30 40 50 60 70 80 90 100 Error in interpretation or diagnosis Parent education errors Required support for treatment in live interaction Pe rc en ta ge o f s tu de nt s Single session Multi-part assignment 61