MANUSCRIPT TYPE EVIDENCE-TO-PRACTICE REVIEW 58 Copyright © by Indiana State University Clinical Practice in Athletic Training All rights reserved. ISSN Online 2577-8188 Volume 4 – Issue 2 – September 2021 Rater Reliability of the Functional Movement Screen: An Evidence-to- Practice Review Nate Orth, MS, SCAT, ATC; Adam Graham, MS, SCAT, ATC; Zachary K. Winkelmann, PhD, SCAT, ATC University of South Carolina, Columbia, SC ABSTRACT The purpose of this evidence-to-practice review is to summarize a systematic review on the inter- and intrarater reliability of the Functional Movement Screen (FMS). Reliability is crucial to the FMS, as clinicians may retest to view a patient’s changes and improvements in movement patterns. This review and analysis looked at 7 studies which showed that both inter- and intrarater reliability were good. Studies were only included if the primary focus was on inter- and intrarater reliability. Clinicians had varying levels of familiarity with the FMS process. Athletic training students with less than one year to no experience were found to have poor reliability. The findings also supported that clinicians who treated the same patient would have similar results about 80% of the time. The same clinician completing several screenings with the different patient would have reliable results about 85% of the time. Interrater reliability is an important aspect to a clinician’s ability to monitor progress or modifications that a patient may exhibit. Overall, the FMS has good inter- and intrarater reliability and can be a predictor of injury risk and mobility. For both inter- and intrarater reliability to improve, it is beneficial for clinicians to have clinical experience and practice using the test to aid in accurate scoring. Certification in FMS is a way to develop repetition training from a reliable source, but it is unclear from these studies if certification changes reliability. The FMS screening serves as a useful tool because it allows for unlimited testing, video recordings for patient education or additional clinician evaluation, and is reliable among clinicians. We suggest that components of the FMS be used clinically for injury prevention, such as pre-participation exams and return-to-play criteria for injuries if scored by a formally trained clinician with experience assessing patient functional movement. Key Phrases Preparticipation exams and screening, injury risk reduction, functional testing Correspondence Dr. Zachary Winkelmann, 1300 Wheat Street, Columbia, SC 29208. E-mail: winkelz@mailbox.sc.edu Twitter: @zachwinkelmann Full Citation Orth N, Graham A, Winkelmann ZK. Rater reliability of the functional movement screen: An evidence-to- practice review. Clin Pract Athl Train. 2021;4(2): 58- 63. https://doi.org/10.31622/2021/0004.2.9. Submitted April 20, 2020 Accepted: November 16, 2020. ORIGINAL REFERENCE Cuchna JW, Hoch MC, Hoch JM. The interrater and intrarater reliability of the functional movement screen: A systematic review with meta-analysis. Phys Ther Sport. 2016;19:57- 65. SUMMARY CLINICAL PROBLEM AND QUESTION The Functional Movement Screen (FMS) is a standardized movement screen for individuals using a score to predict risk of injury and mobility.1-3 It uses a series of 7 tests including the deep squat, hurdle step, in-line lunge, shoulder mobility, active straight leg raise, trunk stability push-up, and rotatory stability.1,3 Each test is scored 0-3 with a score of 3 meaning that there is no compensation, a score of 2 meaning there is compensation during the movement, a score of 1 meaning the movement is not fully completed as instructed, and a score of 0 meaning there is pain with the associated movement.1 The use of the FMS as an clinical assessment tool to identify limitations and restrictions in athletic movement has previously come under scrutiny due to rater reliability.2,4,5 Interrater reliability is the measure that assesses how different evaluators agree or disagree upon the grading, while intrarater reliability assesses how reliable grading is by the same evaluator at different points in time.6 In theory, high rates of reliability for the FMS are necessary to be able to identify score differences over time or among a team of clinicians in physical medicine (athletic training and physical therapy). If the interrater reliability is not adequate, the use of multiple clinicians can misrepresent intervention mailto:winkelz@mailbox.sc.edu https://doi.org/10.31622/2021/0004.2.9 Rater Reliability of the Functional Movement Screen: An Evidence-to-Practice Review 59 Copyright © by Indiana State University Clinical Practice in Athletic Training All rights reserved. ISSN Online 2577-8188 Volume 4 – Issue 2 – September 2021 outcome scores. If the intrarater reliability is not adequate, interventions over time may be misrepresented when the patient is re-tested. Due to the increased popularity of the movement screening in clinical practice,7 there is a need to review the evidence specific to inter- and intrarater reliability of the FMS. SUMMARY OF LITERATURE The focus of the guiding systematic review8 was to determine the inter- and intrarater reliability of the FMS in clinical practice. Articles that were selected included active populations (secondary school and collegiate athletes). The included studies had to use the original scoring system1,7 and incorporated all 7 tests that make up the standard movement screen for the FMS, with or without the three clearing tasks.3,7 The FMS clearing tasks (shoulder clearing, extension clearing, and flexion clearing) are pain provocation tests that take a joint through a full range of motion and meant to provide a clear stopping point when a medical referral is necessary. No studies involving injured athletes or that measured reliability as part of a larger study were included. The authors originally retrieved 110 articles and narrowed to 14 studies based off the purpose of the study. After author review, was narrowed to 7 studies that met specific inclusion criteria for the purpose of the study. SUMMARY OF OUTCOMES Of these 7 articles included, 6 studies evaluated interrater reliability and 6 studies evaluated intrarater reliability. All 7 of the studies went through rigorous risk bias using a Quality Appraisal of Reliability Studies (QAREL) checklist. The QAREL checklist further identified the quality of the studies included in the final grouping. After article selection and quality appraisal, the results from all included studies were meta-analyzed to identify FMS reliability. The sample size of clinicians in the included articles was also assessed (range=1-38), with 5 of the 7 articles reporting 1-5 clinicians. The studies also included a mixture of assessment procedures that used real-time assessments and video recordings. FINDINGS AND CLINICAL IMPLICATIONS The meta-analysis identified that the FMS has good inter- and intrarater reliability.8 The intraclass coefficient (ICC), or the measure of how a score closely matches up to another score, value for interrater reliability was 0.843. A value closer to 1 indicates similarity between different tests for a single rater. The 95% confidence interval for the interrater reliability ICC value was 0.797 to 0.882. The ICC value for intrarater reliability was 0.869 with a 95% confidence interval between 0.854 to 0.885. There was a moderate level of quality evidence showing good interrater reliability. There was only one study9 that found it to be fair or poor, but did not lower the overall average. From the one fair/poor study,9 the authors identified poor reliability in clinicians that were limited in their FMS experience, which ranged from self-taught to less than 1 year of experience. However, the study did not mention if the clinicians had been formally certified in FMS. The guiding systematic review findings8 indicated formal training paired with experience as the best way to achieve accurate results. It is unclear if certification is necessary, based on these studies, other than that a certification course provides the formal training and educational resources.3 The other aspect analyzed was clinical experience with the studies including healthcare providers and students from various fields including physical therapy,9-13 athletic training,9,14,15 strength & conditioning9,15 with certified and non-certified FMS testers of various experiences (0-4 years). Based off the data from the guiding systematic review, when the rater had greater experience with FMS scoring the interrater reliability subsequently increased.9,11 Rater Reliability of the Functional Movement Screen: An Evidence-to-Practice Review 60 Copyright © by Indiana State University Clinical Practice in Athletic Training All rights reserved. ISSN Online 2577-8188 Volume 4 – Issue 2 – September 2021 Intrarater reliability received similar results to interrater, which identified a moderate level of quality evidence for good reliability on the FMS. There was one study that was deemed fair, but the clinicians were athletic training students with no previous FMS experience.14 Based on the findings, it was determined that experience played a significant role in the intrarater reliability for many of the same reasons as interrater reliability. Several reiterations with the same patient would also be a major benefit for the clinician, as observing how different movements are completed could indicate which muscles are activated. Inter- and intrarater reliability were assessed with real time and video recorded scoring. Both scoring methods were deemed good. The finding suggests that scoring could be completed several times and by several clinicians if needed, or videos could be sent to a certified FMS tester for screening confirmation if additional input was warranted. Formal certification can be expensive with current Level 1 FMS training costing $400.3 Certification is a reasonable option for situations where analysis is needed, and the clinician will be primarily involved in correction strategies. However, sending videos to a more experienced FMS tester could also be an option for newly trained clinicians and athletic training students to obtain experience while receiving feedback on their scoring assessment. Video review may also be a viable option for FMS scorers who do not receive many repetitions of the test but still have a desire to implement the outcome measure. CLINICAL BOTTOM LINE One domain of athletic training practice is injury and illness prevention and wellness promotion.16 Through this domain, athletic trainers should seek to minimize the risk of injury, which has often been linked to mechanisms such as pre-participation exams, screening practices, and maintaining a safe environment for activities.17 One specific screening mechanism, the FMS, has been used as an established method to evaluate movement quality and subsequent insufficiencies.1,11 However, there is conflicting evidence whether the FMS tests can predict specific or overall injury risk using the scoring criteria.4,5,18 The issue is that the test is scored by a rater for movement quality at specific joints and through the kinetic chain on a graded scale which does not account for sport specific movements.7 Additionally, previous research has noted that there are conflated reports of injury prediction modeling leading to questions specific to its ability to screen for injury.2 In the guiding systematic review, the FMS test has shown good interrater and intrarater reliability which provides support to clinicians for long term outcome measures and evaluation.8 However, despite the data supporting the reliability, the external resources often call into question if the FMS test has the predictive capabilities or screening sensitivity necessary for that of directing prevention resources towards a targeted population.19 More specifically, the FMS has also been used as a tool to monitor and evaluate rehabilitation progress. As the FMS seems not to have the screening or preventative nature necessary for sports medicine, we suggest that raters incorporate the FMS as a baseline screening not for injury prevention planning but as a model for return-to-activity basis. The data supports the use of video recording for FMS testing,8 as well as during injury rehabilitation as a means to reduce the psychosocial impact of fear avoidance movements.20 We suggest that athletic trainers wishing to implement the FMS as a screening tool do so for all patients, regardless of injury, and video record their movements. After doing so, if the patient sustains an injury, the clinician could use the video recording of the FMS as a baseline for movement quality pre-injury and a goal setting technique for the rehabilitation phases. Rater Reliability of the Functional Movement Screen: An Evidence-to-Practice Review 61 Copyright © by Indiana State University Clinical Practice in Athletic Training All rights reserved. ISSN Online 2577-8188 Volume 4 – Issue 2 – September 2021 However, the issue still with the FMS is the subjective nature of the objective scoring.7 While, as an outcome measure, the FMS provides a specific and measurable goal for the patient to achieve following their injury. The FMS score alone leaves a lot to be desired, since there are many factors that lead to a specific score; thus, clinician interpretation is crucial. Factors such as range of motion across multiple joints and muscle recruitment are all integrated to create one numerical score, as well as observation from a singular vantage point through dynamic movement is of concern.15 The main takeaway of the FMS is to identify if there is an issue; however the unique biomechanical movements of specific athletes cannot be viewed in a score. The score also must be supplemented by commentary by a trained clinician who interprets the scoring to identify associated weaknesses and steps to strengthen primary muscle movers. The FMS does not guarantee the clinician will always notice the weakness either. For example, a clinician watching a deep squat might score a 2 for an athlete that has excessive forward lean as a compensatory movement. While this score would be correct, if another clinician received that score without any notes, it would be difficult to know where the incorrect movement occurred. A qualified individual scoring the FMS would be able to take that score and implement changes to ankle dorsiflexion to eliminate the compensatory movement, retest, and determine if further changes were necessary. Since the FMS has strong inter and intrarater reliability suggesting that it has a place in clinical practice, we suggest that athletic trainers wishing to incorporate it see the score as one piece of the data and that notes or qualitative explanations of the movement insufficiencies be explained in detail for clarity on the scoring, intervention planning, and later reevaluation of the test. The guiding systematic review suggested that for both inter- and intrarater reliability to improve, clinicians must have opportunities to use the FMS test with repetitions and directed feedback in their scoring.8 The use of feedback and deliberate practice requires time, which athletic training has continued to identify as a limitation in implementing evidence-based practice.21 However, we believe that the use of video to record the patient throughout the testing movements would allow multiple clinicians to score the patient outside of real time. The proposal would be best integrated during pre-season or a pre-participation exam. A clinician among the healthcare team, whether that be within the same college/university or hospital outreach team, or even a cohort of students within an athletic training program, could evaluate and score the videos from multiple angles (frontal and sagittal planes of the body) with specific feedback on the movement quality noted. Not only would this allow for a future clinician to get the needed practice with the current patient population, but it also removes the burden from a singular formally trained clinician with experience assessing patient functional movement. Overall, the data supports that there is reliability with the scoring within and between raters. However, clinicians have begun to adopt the FMS as the singular screening tool to predict injury, which is not supported by the literature. We must be creative in our pursuits of using objective outcome measures, such as the FMS, not as a number based criteria for injury prevention planning or ruling out risk, but as a means to have baseline data among the team of raters in case of an injury occurring for future return-to-activity therapeutic interventions. REFERENCES 1. Cook G, Burton L, Hoogenboom B, Voight M. Pre-participation screening: The use of fundamental movements as an assessment of Rater Reliability of the Functional Movement Screen: An Evidence-to-Practice Review 62 Copyright © by Indiana State University Clinical Practice in Athletic Training All rights reserved. ISSN Online 2577-8188 Volume 4 – Issue 2 – September 2021 function-part 1. N Am J Sports Phys Ther. 2006;1(2):62–72. 2. Warren M, Lininger MR, Chimera NJ, Smith CA. Utility of FMS to understand injury incidence in sports: current perspectives. Open Access J Sports Med. 2018;9:171. https://doi.org/10.2147/oajsm.s149139. 3. Functional Movement Systems. https://www.functionalmovement.com. Accessed March 31, 2020. 4. Bonazza NA, Smuin D, Onks CA, Silvis ML, Dhawan A. Reliability, validity, and injury predictive value of the Functional Movement Screen: A systematic review and meta- analysis. Am J Sports Med. 2016;45(3):725– 32. https://doi.org/10.1177/0363546516641 937 5. Dorrel B, Long T, Shaffer S, Myer GD. The Functional Movement Screen as a predictor of injury in national collegiate athletic association division II athletes. J Athl Train. 2018;53(1):29-34. https://doi.org/10.4085/1062-6050-528- 15. 6. Lange R. Inter-rater Reliability. Encyclopedia of Clinical Neuropsychology. 2011:1348. http://dx.doi.org/10.1007/978-0-387- 79948-3_1203. 7. Wright AA, Stern B, Hegedus EJ, Tarara DT, Taylor JB, Dischiavi SL. Potential limitations of the Functional Movement Screen: a clinical commentary. Br J Sports Med. 2016. https://doi.org/10.1136/bjsports-2015- 095796. 8. Cuchna JW, Hoch MC, Hoch JM. The interrater and intrarater reliability of the Functional Movement Screen: A systematic review with meta-analysis. Phys Ther Sport. 2016;19:57-65. https://doi.org/10.1016/j.ptsp.2015.12.00 2. 9. Shultz R, Anderson SC, Matheson GO, Marcello B, Besier T. Test-retest and interrater reliability of the Functional Movement Screen. J Athl Train. 2013;48(3):331-336. https://doi.org/10.4085/1062-6050- 48.2.11. 10. Gulgin H, Hoogenboom B. The Functional Movement Screening (FMS)™: An inter‐rater reliability study between raters of varied experience. Int J Sports Phys Ther. 2014;9(1):14. 11. Teyhen DS, Shaffer SW, Lorenson CL, et al. The Functional Movement Screen: A reliability study. J Ortho Sports Phys Ther. 2012;42(6):530-540. https://doi.org/10.2519/jospt.2012.3838. 12. Smith CA, Chimera NJ, Wright NJ, Warren M. Interrater and intrarater reliability of the functional movement screen. J Strength Cond Res. 2013;27(4):982-987. https://doi.org/10.1519/jsc.0b013e31826 06df2. 13. Parenteau-G E, Gaudreault N, Chambers S, et al. Functional movement screen test: A reliable screening test for young elite ice hockey players. J Phys Ther Sport. 2014;15(3):169-175. https://doi.org/10.1016/j.ptsp.2013.10.00 1. 14. Gribble PA, Brigle J, Pietrosimone BG, Pfile KR, Webster KA. Intrarater reliability of the Functional Movement Screen. J Strength Cond Res. 2013;27(4):978-981. https://doi.org/10.1519/jsc.0b013e31825 c32a8. 15. Onate JA, Dewey T, Kollock RO, et al. Real- time intersession and interrater reliability of the functional movement screen. J Strength Cond Res. 2012;26(2):408-415. https://doi.org/10.1519/jsc.0b013e31822 0e6fa. 16. Henderson J. The 2015 athletic trainer practice analysis study. Omaha, NE.: Board of Certification; 2015. https://bocatc.org/system/document_version s/versions/23/original/boc-comparison-of- https://doi.org/10.2147/oajsm.s149139 https://www.functionalmovement.com/ https://doi.org/10.1177/0363546516641937 https://doi.org/10.1177/0363546516641937 https://doi.org/10.4085/1062-6050-528-15 https://doi.org/10.4085/1062-6050-528-15 http://dx.doi.org/10.1007/978-0-387-79948-3_1203 http://dx.doi.org/10.1007/978-0-387-79948-3_1203 https://doi.org/10.1136/bjsports-2015-095796 https://doi.org/10.1136/bjsports-2015-095796 https://doi.org/10.1016/j.ptsp.2015.12.002 https://doi.org/10.1016/j.ptsp.2015.12.002 https://doi.org/10.4085/1062-6050-48.2.11 https://doi.org/10.4085/1062-6050-48.2.11 https://doi.org/10.2519/jospt.2012.3838 https://doi.org/10.1519/jsc.0b013e3182606df2 https://doi.org/10.1519/jsc.0b013e3182606df2 https://doi.org/10.1016/j.ptsp.2013.10.001 https://doi.org/10.1016/j.ptsp.2013.10.001 https://doi.org/10.1519/jsc.0b013e31825c32a8 https://doi.org/10.1519/jsc.0b013e31825c32a8 https://doi.org/10.1519/jsc.0b013e318220e6fa https://doi.org/10.1519/jsc.0b013e318220e6fa https://bocatc.org/system/document_versions/versions/23/original/boc-comparison-of-pa7-to-rd-pa6-20170612.pdf?1497278853 https://bocatc.org/system/document_versions/versions/23/original/boc-comparison-of-pa7-to-rd-pa6-20170612.pdf?1497278853 Rater Reliability of the Functional Movement Screen: An Evidence-to-Practice Review 63 Copyright © by Indiana State University Clinical Practice in Athletic Training All rights reserved. ISSN Online 2577-8188 Volume 4 – Issue 2 – September 2021 pa7-to-rd-pa6- 20170612.pdf?1497278853. 17. Guindon CC, Winkelmann ZK, Eberman LE, Games KE. Practice of and barriers to prevention by secondary school athletic trainers. Internet J Allied Health Sci Pract. 2018;16(4):4. 18. Moran RW, Schneiders AG, Mason J, Sullivan SJ. Do Functional Movement Screen (FMS) composite scores predict subsequent injury? A systematic review with meta-analysis. Br J Sports Med. 2017;51(23):1661-1669. https://doi.org/10.1136/bjsports-2016- 096938. 19. Bushman TT, Grier TL, Canham-Chervak MC, Anderson MK, North WJ, Jones BH. Pain on Functional Movement Screen tests and injury risk. J Strength Cond Res. 2015;29:S65-S70. https://doi.org/10.1519/jsc.00000000000 01040. 20. Podlog L, Dimmock J, Miller J. A review of return to sport concerns following injury rehabilitation: practitioner strategies for enhancing recovery outcomes. Phys Ther Sport. 2011;12(1):36-42. https://doi.org/10.1016/j.ptsp.2010.07.00 5. 21. Welch CE, Hankemeier DA, Wyant AL, Hays DG, Pitney WA, Van Lunen BL. Future directions of evidence-based practice in athletic training: perceived strategies to enhance the use of evidence-based practice. J Athl Train. 2014;49(2):234-244. https://dx.doi.org/10.4085%2F1062- 6050-49.2.15. https://bocatc.org/system/document_versions/versions/23/original/boc-comparison-of-pa7-to-rd-pa6-20170612.pdf?1497278853 https://bocatc.org/system/document_versions/versions/23/original/boc-comparison-of-pa7-to-rd-pa6-20170612.pdf?1497278853 https://doi.org/10.1136/bjsports-2016-096938 https://doi.org/10.1136/bjsports-2016-096938 https://doi.org/10.1519/jsc.0000000000001040 https://doi.org/10.1519/jsc.0000000000001040 https://doi.org/10.1016/j.ptsp.2010.07.005 https://doi.org/10.1016/j.ptsp.2010.07.005 https://dx.doi.org/10.4085%2F1062-6050-49.2.15 https://dx.doi.org/10.4085%2F1062-6050-49.2.15 ABSTRACT