MANUSCRIPT TYPE EVIDENCE-TO-PRACTICE REVIEW 50 Copyright © by Indiana State University Clinical Practice in Athletic Training All rights reserved. ISSN Online 2577-8188 Volume 6 – Issue 1 – April 2023 The Validity and Reliability of Assessments Through Telemedicine: An Evidence-to-Practice Review Connor L. Givens, MS, ATC*; Gabrielle A. Griffin, MS, ATC†; Matthew D. Johnson. MS, ATC‡; Zachary K. Winkelmann, PhD, ATC§ *Orthopaedic Medical Group of Tampa Bay, Tampa, FL; †East Carolina University, Greenville, NC; ‡Moore Orthopedics and Sports Medicine, Morehead City, NC; §University of South Carolina, Columbia, SC ABSTRACT The validity and reliability of telemedicine assessments has been explored through numerous studies. Telemedicine is a diagnostic tool that combines visual technology, audio technology, and tools for assessment. The assessment of musculoskeletal disorders is a key portion in the field of athletic training. The development of telemedicine assessment tools has improved over time but has not yet been perfected. Data was collected via four databases that included articles examining validity, inter- and intra- rater reliability, and telemedicine assessment for musculoskeletal tests. The reviews of the systematic review used Quality Assessment of Diagnostic Accuracy Studies (QUADAS) and quality appraisal tool for studies of diagnostic reliability (QAREL) to assess the quality of the selected articles. The results from the guiding systematic review revealed good concurrent validity and great reliability for range of motion (ROM), muscle strength, pain, and gait assessments. However, it revealed low to moderate concurrent validity for assessments such as orthopedic special tests and assessments of neurological disorders. Based on these findings, the recommendations for telemedicine include the need to create a standardized measure to ensure consistency between both inter- and intra- raters and a need for advancement in technology for the enhancement of diagnostic accuracy in patients with musculoskeletal disorders. It is feasible for athletic trainers to incorporate telemedicine into their clinical practices, while increasing their technological skills of assessment to continue improving patient-centered care. Content Focus: Health Information Technology Correspondence Dr. Zachary Winkelmann, 1300 Wheat Street, Columbia, SC 29208. E-mail: winkelz@mailbox.sc.edu Twitter: @zachwinkelmann Full Citation Givens CL, Griffin GA, Johnson MD, Winkelmann ZK. The validity and reliability of assessment through telemedicine: An evidence-to-practice review. Clin Pract Athl Train. 2023;6(1): 51-54. https://doi.org/10.31622/2022/0006.01.7. ORIGINAL REFERENCE Mani S, Sharma S, Omar B, Paungmali A, Joseph L. Validity and reliability of Internet-based physiotherapy assessment for musculoskeletal disorders: a systematic review. J Telemed Telecare. 2017;23(3):379-391. doi:10.1177/1357633X16642369 SUMMARY CLINICAL PROBLEM AND QUESTION Telemedicine is an emerging form of healthcare delivery provided from a distance using various forms of technology for prevention, evaluation, diagnosis, and treatment.1 Many clinicians are not able to successfully perform a musculoskeletal assessment due to the limited reliability and validity of the methods used.2 Other barriers for clinicians to the performance of telemedicine assessments include limited internet access, lack of adequate digital health literacy and skills, and socioeconomic status. Though many would assume that the majority of adults would have adequate computer skills, approximately 40% of healthcare workers are not specifically trained in computer skills, which implies a deficit in the ability to adopt telemedicine practices.3 In order to further progress into the use of telemedicine, more clinicians must have the adequate skills to perform not only reliable evaluations but also prescribe effective rehabilitation protocols. The ability to use specific software or online measurement tools may also improve the diagnostic accuracy of the assessment, and in turn improve the likelihood of adopting telemedicine practices.4 Teleassessment focused on the mailto:winkelz@mailbox.sc.edu https://doi.org/10.31622/2022/0006.01.7 The Validity and Reliability of Assessments Through Telemedicine: An Evidence-to-Practice Review 51 Copyright © by Indiana State University Clinical Practice in Athletic Training All rights reserved. ISSN Online 2577-8188 Volume 6 – Issue 1 – April 2023 different levels of validity and reliability when assessing the parts of the body. The assessments included low back, shoulder, elbow, ankle, and knee. The variation of reliability and validity is due to the limited nature of clinicians not being physically present during an exam. The adoption of telemedicine is crucial in reaching underserved populations with limited access to healthcare.5 The purpose of the reviewed study was to evaluate the validity and reliability of ROM, MMT, special tests, and postural assessments via telemedicine. SUMMARY OF LITERATURE The authors of the guiding systematic review completed an article search using PubMed, EMBASE, PsycINFO, Cochrane Library and CINAHL databases. The primary key terms for the search was, ‘telerehabilitation OR tele-rehabilitation OR teleassessment’.6 The search was then focused using individual search terms elaborating on the specific areas of the body (“...shoulder, elbow, wrist, neck pain etc.…”) and terms to identify the test/tester (“...inter tester, inter examiner, intra tester, intra examiner and test retest”).6 The same search terms were utilized in each of the electronic databases. The guiding systematic review’s authors also searched the reference lists of the main article for information pertaining to the topic. Literature that was included in the article was of the English language, measured the validity and/or reliability of musculoskeletal disorders assessment using telerehabilitation, and was published between from January 2000 and May 2015. The exclusion criteria for the guiding systematic review included unpublished manuscripts, letters, guidelines, conference proceedings, theses, other descriptive publications, and published literature on telerehabilitation assessments focused on neurological and other chronic conditions.6 After using the above criteria, the total number of identified articles was 898. After exclusion of duplicate articles and screening the articles for reliability of relevance to the topic using QUADAS and QAREL, 11 studies were included in the guiding systematic review. Of the 11 studies, two examined low back pain, one examined shoulder disorders, one examined lower limb musculoskeletal disorders, one examined elbow disorders, one examined total knee replacement, one examined ankle disorders, and examined four cases with no prior ailments.6 SUMMARY OF INTERVENTION Telemedicine is an area of healthcare that allows for remote access to healthcare using visual and audio technology. Technological access to healthcare not only increases care to remote geographical locations, but also increases care to populations in low socioeconomic situations.7 Telemedicine combines a variety of services used via technology to accomplish therapy in a setting that can be outside of the clinic that is conducive to patient needs. Telemedicine can be used for applications such as assessment of musculoskeletal conditions, and well as rehabilitation of stroke or neurological disorders.8 SUMMARY OF OUTCOMES The authors of the guiding systematic review used different types of statistical methods to determine the validity and reliability of a variety of telemedicine assessments. The data from five of the studies were converted. The data were used to calculate the association between telemedicine and face-to-face assessment methods. The data from all studies assessed validity and both intra- and inter-rater reliability for the telemedicine assessment. Assessments were determined to have either poor, moderate, good, or excellent validity and reliability. The studies did not include a range of scores that would give a grade for The Validity and Reliability of Assessments Through Telemedicine: An Evidence-to-Practice Review 52 Copyright © by Indiana State University Clinical Practice in Athletic Training All rights reserved. ISSN Online 2577-8188 Volume 6 – Issue 1 – April 2023 validity and reliability. For an assessment to be favored, either teleassessment or face-to-face, there must be a significant difference present between the two methods. Knee ROM assessed using a universal goniometer placed on the computer screen was found to have good concurrent validity (0.80).6 However, the internet goniometer used through a computer program was found to have good-to-excellent vailidity.6 When assessing straight leg raise (SLR), the results found good validity with endurance (0.80) and excellent validity with motor control (0.97) between face to face (FTF) assessments and teleassessments.6 Special orthopedic tests (SOT) were demonstrated, looking for the percentage of agreement (PA) found between FTF and teleassessments, the results showed 75% for elbow conditions, 76% for shoulder conditions, 82.9% for lower-limb musculoskeletal injuries and 99.3% for ankle injuries.6 Four studies looked at different techniques to assess strength via teleassessments, using static muscle resistance. These four studies showed good-to-excellent agreement in muscle strength assessment.6 FINDINGS AND CLINICAL IMPLICATIONS The guiding systematic review assessed the evidence that supports the use of telemedicine as a valid and reliable method to complete musculoskeletal assessments. The focus of the reviewed studies encompasses the elbow, ankle, knee joints in the musculoskeletal system, as well as various methods to assess musculoskeletal injuries: ROM, postural assessment, strength and endurance, special orthopaedic tests, etc. Measuring ROM has found to have high achievability due to the minimal differences in measurement by both universal goniometers on digital images and/or videos and using specific programs designed to measure the joint angles.9,10 Range of motion is one of the primary foundations for musculoskeletal assessment and can be thought of as setting the basis of the assessment, which makes it a critical aspect of the telemedicine assessment process. Performing a manual muscle test on a patient via telemedicine is difficult to assess muscle strength.6 However, the use of a telemedicine facilitator to carry-out the manual muscle test allows for the clinician to use a third party individual with the patient to perform an assessment which has yielded positive results.11 The limitation of poor bandwidth proves to be one of the most common issues faced when assessing the concurrent validity of telemedicine.11-14 A clear view of the patient is crucial in determining the abnormalities that may be present upon evaluation. If there is lack of clarity, there may be signs that are not logistically visible to the clinician. Having access to adequate video cameras is another limitation faced by clinicians when performing a telemedicine assessment.15 This would lead us to the assumption that more research needs to be conducted in order to further investigate the probability of being able to correctly assess posture via telemedicine. The evidence suggests that postural assessments are less likely to have good concurrent validity.6 Without the ability to connect to patients, there can be no assessments. Barriers such as this may impede the ability to further examine the validity and reliability of telemedicine assessments. The guiding systematic review identified a SLR to assess strength, endurance, and motor control of the low back pain population. The study demonstrated that there was good validity in endurance (α=0.80) and excellent validity in motor control (α=0.97), as well as good validity (α=0.85) in lower limb functional strength in total knee arthroplasty population.6 With the increased levels of validity and reliability, the authors showed that the transition from face-to-face to teleassessment would still be beneficial in doing the strength, endurance and motor control portions of the examination process. The straight leg raise test is easy to demonstrate and have a patient reproduce to observe discrepancies in a musculoskeletal assessment. The Validity and Reliability of Assessments Through Telemedicine: An Evidence-to-Practice Review 53 Copyright © by Indiana State University Clinical Practice in Athletic Training All rights reserved. ISSN Online 2577-8188 Volume 6 – Issue 1 – April 2023 The guiding systematic review also explored the relationship between teleassessments and face-to-face orthopedic selective tissue tests. Teleassessments include performing objective assessments in order to diagnose patients properly and effectively with a musculoskeletal disorder (MSD).9 Face-to-face orthopedic selective tissue tests are performed in person with the patient present, with the goal of diagnosing the patient with MSD. The authors identified that the reported percentage of agreement ranged from 75% (elbow and shoulder disorders) to 99% (ankle conditions).6 These findings demonstrate that while some selective tissue tests may be difficult to perform, technology-based orthopedic assessments do produce similar results as a face-to-face patient encounter. CLINICAL BOTTOM LINE The guiding systematic review suggests there is potential for the inclusion of telemedicine in physical medicine which includes athletic training. For example, ROM measures completed with the use of an internet goniometer displayed an excellent level of concurrent validity and inter- and intra-rater reliability. Assessment of posture displayed good results with intra- and inter-rater reliability but did not produce good results in the aspect of validity. Muscle strength demonstrated a good level of concurrent validity and reliability. Functional outcome measures displayed exceptional levels of both concurrent reliability and inter- and intra- rater reliability. Regarding both balance and gait, each assessment reported excellent levels of concurrent validity and inter- and intra- rater reliability. Although there were a lot of positives in most aspects of telemedicine assessment, there were some assessments that did not report clinically meaningful findings with respect to validity and reliability. Scar assessments, selective tissues tests, and neurodynamic tests all demonstrated moderate levels of concurrent validity, with no significant change to face-to-face assessment. Athletic trainers can apply the skills they already possess and use them in telemedicine to continue patient care beyond the clinic. The guiding systematic review delved into a wide variety of different assessments that are commonly used in telemedicine and face-to-face encounters. It can be determined that telemedicine may be used as an appropriate method of assessment, but only regarding functional outcomes and gait, balance, ROM, pain, swelling, and muscle strength of certain musculoskeletal disorders. In athletic training, these assessments are cornerstones to the orthopedic evaluation process. There are many pros and cons to telemedicine, but the systematic review was able to determine that telemedicine may be used to substitute or supplement face- to-face assessments when necessary. We suggest that athletic trainers seeking to implement telemedicine into their clinical practice use this review to substantiate the framework for their digital clinical practice alongside continual conversations with supervising physicians and state regulatory agencies. The key to integration could be from the innovative and creative mindset to perform selective tissue test in a non- traditional format. If athletic trainers can utilize telemedicine, it could drastically improve patient-centered care especially to populations that have limited access to healthcare services. REFERENCES 1. Scott Kruse C, Karem P, Shifflett K, Vegi L, Ravi K, Brooks M. Evaluating barriers to adopting telemedicine worldwide: A systematic review. J Telemed Telecare. 2018;24(1):4-12. https://doi.org/10.1177/1357633x1667408. 2. Kimberlin CL, Winterstein AG. Validity and reliability of measurement instruments used in research. Am J Health Syst Pharm. 2008;65(23):2276-2284. https://doi.org/10.2146/ajhp070364. https://doi.org/10.1177/1357633x1667408 https://doi.org/10.2146/ajhp070364 The Validity and Reliability of Assessments Through Telemedicine: An Evidence-to-Practice Review 54 Copyright © by Indiana State University Clinical Practice in Athletic Training All rights reserved. ISSN Online 2577-8188 Volume 6 – Issue 1 – April 2023 3. Cilliers L, & Flowerday, S. . Will computer literacy affect telemedicine acceptance among health care workers? . Research Network for e-Skills Conference, Premier Hotel Regent, East London, South Africa.; 2011. 4. Wochatz M, Tilgner N, Mueller S, et al. Reliability and validity of the Kinect V2 for the assessment of lower extremity rehabilitation exercises. Gait Posture. 2019;70:330-335. https://doi.org/10.1016/j.gaitpost.2019.03.020. 5. Grimmer K, Bowman P. Differences between metropolitan and country public hospital allied health services. Aust J Rural Health. 1998;6(4):181-188. https://doi.org/10.1111/j.1440- 1584.1998.tb00310.x. 6. Mani S, Sharma S, Omar B, Paungmali A, Joseph L. Validity and reliability of Internet-based physiotherapy assessment for musculoskeletal disorders: a systematic review. J Telemed Telecare. 2017;23(3):379-391. https://doi.org/10.1177/1357633x16642369. 7. Kahn JM. Virtual visits--confronting the challenges of telemedicine. N Engl J Med. 2015;372(18):1684- 1685. https://doi.org/10.1056/NEJMp1500533. 8. Amatya B, Galea MP, Kesselring J, Khan F. Effectiveness of telerehabilitation interventions in persons with multiple sclerosis: A systematic review. Mult Scler Relat Disord. 2015;4(4):358-369. https://doi.org/10.1016/j.msard.2015.06.011. 9. Russell T. Goniometry via the internet. Aust J Physiother. 2007;53(2):136. https://doi.org/10.1016/s0004-9514(07)70051-x. 10. Durfee WK, Savard L, Weinstein S. Technical feasibility of teleassessments for rehabilitation. IEEE Trans Neural Syst Rehabil Eng. 2007;15(1):23-29. https://doi.org/10.1109/tnsre.2007.891400. 11. Lade H, McKenzie S, Steele L, Russell TG. Validity and reliability of the assessment and diagnosis of musculoskeletal elbow disorders using telerehabilitation. J Telemed Telecare. 2012;18(7):413-418. https://doi.org/10.1258/jtt.2012.120501. 12. Russell T, Truter P, Blumke R, Richardson B. The diagnostic accuracy of telerehabilitation for nonarticular lower-limb musculoskeletal disorders. Telemed J E Health. 2010;16(5):585-594. https://doi.org/10.1089/tmj.2009.0163. 13. Russell TG, Blumke R, Richardson B, Truter P. Telerehabilitation mediated physiotherapy assessment of ankle disorders. Physiother Res Int. 2010;15(3):167-175. https://doi.org/10.1002/pri.471. 14. Lemaire ED, Boudrias Y, Greene G. Technical evaluation of a low-bandwidth, Internet-based system for teleconsultations. J Telemed Telecare. 2000;6(3):163-167. https://doi.org/10.1258/1357633001935266. 15. Cabana F, Boissy P, Tousignant M, Moffet H, Corriveau H, Dumais R. Interrater agreement between telerehabilitation and face-to-face clinical outcome measurements for total knee arthroplasty. Telemed J E Health. 2010;16(3):293-298. https://doi.org/10.1089/tmj.2009.0106. https://doi.org/10.1016/j.gaitpost.2019.03.020 https://doi.org/10.1111/j.1440-1584.1998.tb00310.x https://doi.org/10.1111/j.1440-1584.1998.tb00310.x https://doi.org/10.1177/1357633x16642369 https://doi.org/10.1056/NEJMp1500533 https://doi.org/10.1016/j.msard.2015.06.011 https://doi.org/10.1016/s0004-9514(07)70051-x https://doi.org/10.1109/tnsre.2007.891400 https://doi.org/10.1258/jtt.2012.120501 https://doi.org/10.1089/tmj.2009.0163 https://doi.org/10.1002/pri.471 https://doi.org/10.1258/1357633001935266 https://doi.org/10.1089/tmj.2009.0106 ABSTRACT