










































MANUSCRIPT TYPE


CLINCAL OUTCOMES RESEARCH  

 

24 
Copyright © by Indiana State University                                                                                Clinical Practice in Athletic Training  
All rights reserved. ISSN Online 2577-8188                                                                        Volume 4 – Issue 1 – February 2021 

 
 

Use of Response Shift to Improve Agreement between Patient-
Reported and Performance-Based Outcomes in Knee Patients 
Shelby Baez, PhD, ATC*; Johanna M. Hoch, PhD, ATC†; Carl Mattacola, PhD, ATC, FNATA‡; Philip Gribble, 
PhD, ATC, FNATA†, Jennifer S. Howard, PhD, ATC** 
*Michigan State University, East Lansing, MI; †University of Kentucky, Lexington, KY; ‡ University of North Carolina at Greensboro, 
Greensboro, NC; **Appalachian State University, Boone, NC 
  

 
ABSTRACT 
The purpose of this study was to determine whether the 
implementation of performance-based tasks (PBTs) prior to 
completion of patient-reported outcome measures (PROs) 
would create a change, or a response shift, in PROs in 
patients with knee injuries. A randomized controlled trial 
was implemented to examine the effectiveness of a 
response-shift based interventions to enhance the 
correlation between PBTs and PROs. Participants (n=20) 
were knee-injured patients who were removed from 
activity for a minimum of 1-week. Participants were 
randomly assigned to complete PBTs (intervention) or to 
watch videos detailing an injury prevention program 
(control). The International Knee Documentation Committee 
Subjective Form (IKDC) and the Knee Injury and 
Osteoarthritis Outcome Score Recreational and Sports 
Subscale (KOOSsports) were completed both pre- and 
post-testing. The independent variable was Group. 
Dependent variables included raw change scores and 
absolute change scores on the IKDC and KOOSsports. 
Mann-Whitney U tests were used to examine between 
Group differences. Changes in PRO scores were not 
considered statistically significant or clinically meaningful 
(IKDC Raw p = 0.14, Absolute p = 0.74; KOOSsports Raw 
p = 0.85 Absolute p = 0.32). Implementation of PBTs prior 
to PROs did not induce a response shift. PROs may better 
evaluate symptoms and/or confidence in a patient, while 
PBTs evaluate physical function. 
 
Key Phrases 
Patient-reported outcomes, clinician-rated outcomes, 
functional testing 

 
Correspondence 
Dr. Shelby Baez, Michigan State University, 
308 W Circle Drive 27P, East Lansing, MI 
48824. 
E-mail: baezshel@msu.edu  
 
Full Citation 
Baez S, Hoch JM, Mattacola C, Gribble P, Howard 
JS. Use of response shift to improve agreement 
between patient-reported and performance-based 
outcomes in knee patients. Clin Pract Athl Train. 
2021;4(1): 24-35. 
https://doi.org/10.31622/2021/0004.1.4.   
 
Submitted: July 30, 2020 Accepted: February 22, 2021. 

 
 
 

INTRODUCTION 
 

After an injury, patients often seek medical 

treatment to resolve their injury. Whether that 
treatment is therapeutic rehabilitation, surgical 
interventions, or even just the application of ice, it 
is important to evaluate the effectiveness of the 
method of treatment. The assessment of health 
outcomes is a fundamental component of a 
clinician’s duties, as it provides a basis to 
determine which treatments are more effective.1 
Assessment of health outcomes also provides a 
foundation to determine whether a patient can 
return to their pre-injury levels of activity.1 In 
athletics, most often the goal is to return patients 
to their pre-injury status and back to full 
participation as quickly and safely as possible. 
Different evaluation techniques have been 
employed to assess a patient’s health status and 
to determine whether a patient is able to return to 
sport. These different techniques often include 
disease-oriented, performance-based, and 
patient-oriented measures.2-4  

Disease-oriented measures assess outcomes 
associated with impairments at the body function 
and structure level.2 These measures typically 
include testing or evaluating the involved tissue, 
such as the Lachman’s maneuver or the KT-1000 
device, to test the integrity of the ACL.2 

Performance-based measures are typically 
closed-kinetic tasks that examine the patient’s 
ability to perform functional tasks that will likely 
stress the involved structure.3-5 Frequently used 
lower extremity performance-based measures 
include the single-leg hop for distance, crossover 

mailto:baezshel@msu.edu
https://doi.org/10.31622/2021/0004.1.4


Use of Response Shift to Improve Agreement between Patient-Reported and Performance-Based Outcomes in Knee Patients 
 

 

25 
Copyright © by Indiana State University                                                                                Clinical Practice in Athletic Training  
All rights reserved. ISSN Online 2577-8188                                                                        Volume 4 – Issue 1 – February 2021 
 

hop, 6-metered timed hop, shuttle run, and star 
excursion balance test.2,6-8 Lastly, patient-oriented 
outcomes are used to determine the perceived 
limitations and restrictions, most often through the 
use of patient reported outcome measures 
(PROs).9 While many reliable and valid measures 
exist for each type of outcome, the observed 
correlation between various types of outcome 
measures is often poor to moderate, at best.10,11  
Specifically, performance-based measures and 
PROs have frequently been observed to be only 
low to moderately correlated among knee-injured 
patients.10,11  

It has been theorized that failure to provide a 
frame of reference for patients to answer 
questions on PROs has led to the disagreement 
between PROs and performance-based 
measures.10 Patients may be asked to answer 
questions related to tasks that they have not 
performed since sustaining injury.12 Asking 
patients to complete questionnaires prior to 
completing functional testing may result in patients 
having an insufficient sample of experiences from 
which to self-evaluate their current function. As 
proposed by Logerstedt et al.,13 if patients can be 
provided with a relevant sample of experiences 
to be used to evaluate their current function, then 
their perceived function may be more in line with 
their true physical performance. Anecdotally, it 
has been observed that patients tended to either 
underestimate or overestimate on PROs if 
completed prior to performance-based testing.14 
Fitzgerald et al.14 hypothesized that completing 
performance-based tasks prior to the completion 
of PROs allowed the participant an opportunity to 
self-evaluate the status of their knee, which 
provided more accurate ratings of knee function 
on the self-report surveys.  

Inconsistencies between PROs and performance-
based measures may be due to the varying frame 
of reference patients use to complete PROs. As a 
result, PROs may be influenced by a response shift 
phenomenon. Although primarily studied in ill and 

chronic disease patients, response shift can also be 
detected in individuals suffering from an 
orthopaedic injury.15,16 Response shift is defined 
as a change in an individual’s internal standards, 
values, or conceptualization of a construct when 
evaluating their health related quality of life.17 A 
response shift results in a change in one’s self-
evaluation either through recalibration, 
reconceptualization, or reprioritization.18 
Recalibration refers to a person’s change in their 
internal standards of measurement; 
reconceptualization refers to a change in 
definition of the target construct; and 
reprioritization refers to a change in an 
individual’s internal values.18 If individuals are 
susceptible to these changes, then it may be 
possible to recalibrate and/or reconceptualize an 
individual’s self-perceived function through the 
implementation of a performance-based 
assessment intervention. While the response shift 
phenomenon has been reported in orthopaedic 
cases15,16, use of this theory to optimize PROs has 
not been previously evaluated. Therefore, the 
purpose of this study was to examine the effect of 
testing order on PRO scores to determine if 
completing a performance-based assessment with 
the intent to provide a frame of reference for the 
individual prior to the administration of PROs 
would alter PRO scores in knee patients. We 
hypothesized that completion of performance-
based assessments prior to PROs would lead to a 
greater change on PRO scores between pre-test 
and post-test when compared to a control group.  

PATIENTS 

Participants were between the ages of 14-40 and 
had been restricted due to a knee pathology from 
full participation in physical activity for at least 1-
week prior to testing and had recently been 
cleared to return to full activity. Participants could 
have no other injuries besides their current knee 
injury affecting their sports participation status, 
and they must have reported having a pre-injury 
activity level of 5 or greater on the Tegner 



Use of Response Shift to Improve Agreement between Patient-Reported and Performance-Based Outcomes in Knee Patients 
 

 

26 
Copyright © by Indiana State University                                                                                Clinical Practice in Athletic Training  
All rights reserved. ISSN Online 2577-8188                                                                        Volume 4 – Issue 1 – February 2021 
 

Physical Activity Assessment.20 Additionally, 
participants were excluded based on the 
following criteria: known balance disorders, not 
cleared to return to activity, not cleared to 
perform functional testing, scored below a 46 on 
the IKDC, failure to pass the pre-participation 
functional assessment screening, or cleared for 
activity > two weeks at time of recruitment.  As a 
precaution, participants who scored below a 46 
on their pre-intervention IKDC were not permitted 
to continue in the study. A cut-off of 46 was chosen 
because this value represented the mean pre-
operative value for surgical knee patients in the 
University of Kentucky Patient Registry at the time 
of study development which suggested that the 
patient may not be able to safely complete the 
functional tasks required of the study.  

INTERVENTION 

Design 

A randomized controlled trial was used for this 
study, with participants randomized into either a 
performance-based assessment intervention 
group or an educational control group (Figure 1). 
A block randomized design with random block 
sizes ranging from 2-6 participants was used to 
complete randomization. An independent third 
party generated the randomization scheme using 
a publicly available randomization generator 
(http://randomization.com/) and created blinded 
envelopes for subject allocation. Individuals in the 
performance-based assessment group completed 
a battery of functional tests, while those selected 
for the educational control group participated in 
a placebo intervention consisting of watching 
exercises from the FIFA 11+ program.19 The 
intervention session lasted approximately 40 
minutes for both groups. Both groups completed 
the International Knee Documentation Committee 
Subjective Knee Form (IKDC) and the Knee Injury 
and Osteoarthritis Outcome Score Sports and 
Recreation Subscale (KOOSsports) pre- and post-
intervention.  All participants reviewed and signed 

an informed consent approved by the University 
of Kentucky IRB prior to study participation. 

 

Figure 1. Study Design and Participant Allocation 

Procedures  

Participants completed a single testing session in 
a clinical laboratory lasting approximately 40 
minutes.  After informed consent and prior to 
randomization, all participants completed the 
IKDC and KOOSsports as pre-test assessments. 
Upon completion of pre-test PROs, the participant 
randomization envelope was opened, revealing 
group allocation.    

Educational Control Group 

Participants randomized into the educational 
control group were shown the FIFA11+19 
exercise videos. The FIFA11+ is a sports and 
injury performance program that has been used 
extensively in the soccer community and beyond 
and, when implemented with high compliance, has 
been shown to reduce injury risk among soccer 
players.19,24-27 It was chosen as an educational 
control for this study, as the information provided 
in the video may be of some benefit to study 
participants, but was not anticipated to alter PRO 



Use of Response Shift to Improve Agreement between Patient-Reported and Performance-Based Outcomes in Knee Patients 
 

 

27 
Copyright © by Indiana State University                                                                                Clinical Practice in Athletic Training  
All rights reserved. ISSN Online 2577-8188                                                                        Volume 4 – Issue 1 – February 2021 
 

scores. We chose to include a patient education 
component for the control group to reduce 
participant bias and provide some benefit to all 
participants. At the conclusion of the study, 
participants in both groups received a handout 
with links to the FIFA 11+19 website as additional 
information to take with them. Following conclusion 
of the videos, the participants were again asked 
to complete the IKDC and KOOSsports. 

Performance-Based Assessment Screening 

As a precaution, prior to the completion of the 
performance-based tests, participants 
randomized to the performance-based 
assessment group completed a screening to verify 
the patient’s eligibility/readiness to complete the 
performance-based tasks included in the study 
protocol.  This screening consisted of a series of 
single-leg squats, side-to side-hops, and vertical 
hops. All participants were required to complete 
these tasks without pain, or any other symptoms. 

Instrumentation 

Those participants randomized into the 
performance-based assessment intervention 
group completed a 5-minute jogging warm-up 
followed by a battery of performance-based 
assessments. To ensure there was minimal fatigue 
and muscle soreness throughout the study, the 
order of tests was standardized for all 
participants and a three-minute rest period was 
provided between each test. For all assessments, 
participants were given ample time to practice 
each task, and for unilateral assessments the 
uninvolved limb was tested first.  

Star Excursion Balance Test (SEBT) – Anterior 
Reach28 

The SEBT was used to assess dynamic balance. 
Participants maintained a single-leg stance on the 
involved limb while reaching for maximum 
distance in the anterior direction with the opposite 
limb. The participant’s first toe was placed at the 

0 point on the tape measure. Participants then 
reached as far anteriorly as possible and touched 
the tape measure while maintaining a single-leg 
stance. Participants were required to keep their 
hands on their hips and stance heel in contact with 
the ground. All participants were given the 
opportunity to complete 4 practice trials, followed 
by 3 test trials in the anterior direction only.  The 
test was performed bilaterally with the uninvolved 
limb first.  Reach distances were measured in 
centimeters and normalized by leg length. Leg 
length was measured from the anterior superior 
iliac spine to the base of the medial malleolus of 
the tibia. Mean reach distances were used to 
calculate limb symmetry index (LSI = (mean 
distance involved limb/mean distance uninvolved 
limb) x 100).  

Single-Leg Hop Tests29 

The Single-Leg Hop for Distance test is commonly 
used to measure power and confidence. The 
patient began standing on one leg and then 
jumped as far forward as possible landing on the 
same leg.  The total distance hopped forward was 
recorded (cm). The Crossover Hop for Distance 
also measures power and confidence in the tested 
leg. The patient began standing on one leg and 
then hopped as far as possible forward 3 times 
while alternating crossing over a 6-centimeters 
wide strip on the floor.  The total distance hopped 
forward was recorded (cm). Both hop tests were 
performed three times, with a 10 second recovery 
between trials.  A successful trial included landing 
stable on the test leg and maintaining balance for 
three seconds.  The trial was repeated if the 
participant landed with early touchdown of the 
contralateral leg, lost balance, touched the 
surrounding area, or included additional hops 
after landing. Mean hop distances were used to 
calculate limb symmetry index (LSI = [mean 
distance involved limb/mean distance uninvolved 
limb] x 100). The Single-Leg Hop for Distance and 
the Crossover Hop for Distance have previously 
been reported to be reliable and valid measures 



Use of Response Shift to Improve Agreement between Patient-Reported and Performance-Based Outcomes in Knee Patients 
 

 

28 
Copyright © by Indiana State University                                                                                Clinical Practice in Athletic Training  
All rights reserved. ISSN Online 2577-8188                                                                        Volume 4 – Issue 1 – February 2021 
 

of functional performance,30 and are routinely 
used clinically to evaluate recovery and readiness 
to return to sporting activity.  

30 Second Step-Down Test31 

Participants were asked to perform a step-down 
motion from a platform 8-inches high. Participants 
were instructed that contact with the floor should 
only be a brush and not be used to accelerate 
back onto the step, hands must remain on the hips, 
and to complete as many repetitions as possible 
within a single 30-second trial. Participants 
completed one 30-second test trial on each limb, 
with a 3-minute recovery time between test legs.31 
Limb symmetry index was calculated using the 
following equation: (LSI = [mean distance involved 
limb/mean distance uninvolved limb] x 100).  

Compact Agility Test (CAT) 

Participants completed an assessment of agility, 
quickness, and body control by completing a 
sprinting and shuffling task on a 4-meter path. The 
path was marked by tape and the individual 
performed the following tasks with their best 
effort.  The testing procedure followed the pattern 
presented in Figure 2. This resulted in a total 
distance traversed of 20 meters. Participants 
completed a walk-through of the task to gain 
familiarity and then were asked to complete a 
submaximal effort of the task to ensure fluidity of 
the task. Additional submaximal trials were 
permitted if necessary, for the participants to feel 
comfortable with the test. The participants were 
then asked to complete two test trials at the 
highest speed with which they were comfortable. 
A two-minute rest period was given between test 
trials. Verbal cuing occurred from the investigator 
throughout the entire testing session of the CAT. 
The faster of the two trials was used as the CAT 
result for analysis.  This test was designed to be 
completed in a small space, such as a doctor’s 
office, and to provide patients with a sampling of 
experiences that involved sprinting, 
cutting/planting, and lateral movements. 

OUTCOME MEASURES 

Following completion of the performance-based 
assessment battery, participants completed the 
post- IKDC and KOOSsports assessments.  To 
avoid biasing PRO scores, participants did not 
receive scores on performance-based measures 
until post-test PROs were completed. Both the 
IKDC and KOOS are valid and reliable PROs 
used for individuals suffering from knee 
pathology.21,22 The IKDC is a 20-item outcome 
measure used to determine patient perceived 
function.  The KOOSsports is a 5-question subscale 
of the KOOS used to evaluate patient perceived 
function for activities such as running, cutting, and 
jumping. All PROs were collected and managed 
using Research Electronic Data Capture (REDCap) 
tools hosted at the XXX. REDCap is a secure, web-
based application designed to support data 
capture for research studies.23   

Statistical Analysis 

Descriptive statistics were generated for all 
performance-based measures and PROs.  To 
ensure that the randomization resulted in 
comparable groups, independent t-tests or Mann-
Whitney U tests were used to compare height, 
weight, age, and time between clearance to RTS 
and the data collection session between groups. 
To determine if participation in performance-
based assessments resulted in a change in PRO 

Figure 2. Compact Agility Test (CAT). Completed on 4-
metered pathway. 



Use of Response Shift to Improve Agreement between Patient-Reported and Performance-Based Outcomes in Knee Patients 
 

 

29 
Copyright © by Indiana State University                                                                                Clinical Practice in Athletic Training  
All rights reserved. ISSN Online 2577-8188                                                                        Volume 4 – Issue 1 – February 2021 
 

scores, Mann-Whitney U tests were used to 
compare the raw change (post-test – pre-test) and 
absolute change (|post-test – pre-test|) between 
groups for both the IKDC and KOOSsports 
scores.15 Minimal detectable change (MDC) values 
were interpreted to identify clinically meaningful 
changes in the IKDC and KOOS sports scores.  

RESULTS 

A total of 20 participants were enrolled into the 
study (performance-based assessment n=10, 
educational control n=10). Participants’ 
demographic information are presented in Table 
1. Eleven out of 20 of the participants were 
cleared to return to activity following ACL-
reconstruction. Two other patients were post-
surgical (medial meniscal allograft transplantation 
and meniscectomy). The remaining nine 
participants suffered from IT band friction 
syndrome (n=1), Morel-Lavalle lesion (n=1), 
lateral collateral ligament sprain (n=2), articular 
cartilage damage (n=2), meniscal tear (n=2), or 
patellofemoral pain syndrome (n=1). Participants 
were tested a median of 1.5 days from physician 
and/or other healthcare professional’s clearance 
to return to sport. There were no differences in 
demographics between the two groups. Average 
pre-test, post-test, and change scores for the 
performance-based assessment group and 
educational control group are presented in Table 
2. Descriptive statistics for performance-based 
measures are presented in Table 3.  No significant 
differences were observed between groups when 
comparing raw change scores and absolute 
change scores for the IKDC and KOOSsports 
(IKDC Raw Change p = 0.14, Absolute Value p = 
0.74; KOOSsports Raw Change p = 0.85 
Absolute Value p = 0.32) (Table 2).  

DISCUSSION 

To our knowledge, this is the first study to use a 
response shift theory-based intervention to 
enhance the agreement between performance-
based measures and PROs. The aim of this study 

was to determine whether testing order would 
improve the agreement between performance-
based measures and scores on the IKDC and 
KOOSsports. However, our results did not support 
our hypothesis as we observed that providing a 
specific frame of reference from which patients 
could evaluate themselves did not result in 
systematic changes in PRO scores. These results 
suggest that PROs can be implemented before or 
after performance-based testing during the 
rehabilitation of knee patients and support the use 
of both PROs and performance-based testing in 
clinical practice.  

While no between group differences were 
identified, some participants included in this study 
did increase or decrease their score at the post-
test despite their ability to successfully complete 
performance-based tasks with high LSIs (Table 3). 
In accordance with Howard et al.,20 the present 
study also observed evidence that a response shift 
may occur on a patient-by-patient basis after 
knee injury. Fifty percent of the participants 
randomized into the performance-based 
assessment group reported lower scores on the 
IKDC at the post-test (Table 4). This suggests that 
participants overestimated their function prior to 
completion of functional testing. Concerns for 
overestimation on PROs in patients after ACL 
reconstruction has been previously postulated.14 
While evaluating decision-making criteria for 
return to sport after ACL reconstruction, Fitzgerald 
et al.14 altered their methods of administering 
PROs to their patients to accommodate anecdotal 
observations involving testing order.14 
Investigators reported observing that patients 
tended to overestimate or underestimate on self-
report scores if hop tests were performed after 
the completion of PROs. While not statistically or 
clinically meaningful between groups, the results 
from the present study further support this 
observation that individual patients may shift their 
responses on PROs due to their experience while 
engaging in the performance-based assessments.   



CLINCAL OUTCOMES RESEARCH  

 

30 
Copyright © by Indiana State University                                                                                Clinical Practice in Athletic Training  
All rights reserved. ISSN Online 2577-8188                                                                        Volume 4 – Issue 1 – February 2021 

 
 

Table 1. Participant Demographics 

Measure 
Performance 

Group  
(n=10) 

Educational 
Control (n=10) 

Total  
(n=20) P-Value 

 Mean (SD)  
Height (cm) 177.24 (9.43) 173.00 (11.27) 175.12 (10.32) 0.37* 

Weight (kg) 80.40 (18.70) 82.20 (17.22) 81.30 (17.52) 0.83* 

Tegner Score (Current 
Level) 

8.00 (1.70) 8.00 (1.42) 8.00 (1.52) 1.00* 

 Median (IQR)  
Age (years) 20.00 (3.00) 20.50 (4.00) 20.00 (3.00) 0.63# 

Time from Clearance 
(days) 
 

6.50 (7.00) 1.00 (3.00) 1.50 (7.00) 0.09# 

Tegner Score (Before 
Injury) 

9.00 (3.00) 9.00 (3.00) 9.00 (2.00) 0.39# 

*PBA = Performance-Based Assessment Group, EC = Educational Control Group 
*Independent T-Test, #Mann Whitney-U Test  

Table 2. Patient-Reported Outcome Measures Pre-Test, Post-Test, and Change Scores 
  Performance-

Based 
Assessment 

Median (IQR) 

Educational 
Control Median 

(IQR) 

Total 
Median  
(IQR) 

Mann-
Whitney U 

Test P-Value Measure  

IKDC Pre-Test 87.94 (11.78) 88.51 (22.13) 87.94 (13.21) 0.57 
IKDC Post-Test 87.36 (15.80) 89.66 (23.56) 87.94 (15.23) 0.35 
KOOSsports Pre-Test 90.00 (26.00) 92.50 (41.00) 90.00 (29.00) 1.00 
KOOSsports Post-Test 90.00 (26.00) 92.50 (41.00) 90.00 (29.00) 0.91 
Δ IKDC Raw Change -0.57 (3.16) 0.01 (2.59) 0.00 (3.16) 0.14 
Δ IKDC Absolute 
Value 

1.16 (2.87) 1.72 (2.30) 1.16 (2.30) 0.74 

Δ KOOSsports Raw 
Change 

0.00 (6.00) 0.00 (0.00) 0.00 (0.00) 0.85 

Δ KOOSsports Absolute 
Value 

2.50 (5.00) 0.00 (1.00) 0.00 (5.00) 0.32 

*IKDC = International Knee Documentation Committee, KOOSsports = Knee Injury and 
Osteoarthritis Outcome Score Sports and Recreational Subscale, PBA = Performance-Based 
Assessment Group, EC = Educational Control Group 

 

 



Use of Response Shift to Improve Agreement between Patient-Reported and Performance-Based Outcomes in Knee Patients 
 

 

31 
Copyright © by Indiana State University                                                                                Clinical Practice in Athletic Training  
All rights reserved. ISSN Online 2577-8188                                                                        Volume 4 – Issue 1 – February 2021 
 

 

 

 

 

  

Table 3. Outcomes of PROs change scores and performance-based testing 
demonstrating mean values and standard deviation (SD), and p-value 

Performance-Based Assessment Mean (SD) 

SEBT Anterior Reach 99.22 (4.90) 
SL Hop for Distance LSI 100.29 (11.12) 
Crossover Hop for Distance LSI 95.91 (13.66) 
30 Second Step Down Task LSI 96.50 (11.71) 
Compact Agility Test AVG 10.13 (1.57) 
*LSI = Limb Symmetry Index, SEBT = Start Excursion Balance Test, SL = Single Leg 

 
 
Table 4. Individual Changes in PRO Scores Between Groups 
 
 IKDC – PRE IKDC - POST KOOSsports - PRE KOOSsports - POST 

PBA 91.95 91.96 100 100 
PBA 89.66 89.66 90 90 
PBA 51.72 41.38* 25 35# 
PBA 80.46 79.32* 85 80* 
PBA 87.36 86.21* 75 70* 
PBA 79.31 77.02* 90 90 
PBA 78.16 73.57* 70 75# 
PBA 90.8 93.11# 100 95* 
PBA 90.8 91.96# 100 100 
PBA 88.51 88.51 100 100 
Control 57.47 55.18* 55 55 
Control 87.36 87.36 90 80* 
Control 83.91 82.76* 90 90 
Control 97.7 97.7 95 95 
Control 78.16 80.46# 60 60 
Control 94.25 97.71# 100 100 
Control 40.23 42.53# 35 40# 
Control 94.25 94.26 100 100 
Control 100 100 100 100 
Control 89.66 91.96# 100 100 
PBA = Performance-Based Assessment, IKDC = International Knee Documentation 
Committee, KOOSsports = Knee Injury and Osteoarthritis Outcome Score Sports and 
Recreational Subscale * = decreased from pre to post test, # = increased from 
pre to post test 



Use of Response Shift to Improve Agreement between Patient-Reported and Performance-Based Outcomes in Knee Patients 
 

 

32 
Copyright © by Indiana State University                                                                                Clinical Practice in Athletic Training  
All rights reserved. ISSN Online 2577-8188                                                                        Volume 4 – Issue 1 – February 2021 
 

These results also highlight the importance of 
integration of a holistic approach to health 
outcome evaluation. Performance-based 
measures completed in this study provided 
information about the patient’s strength, power, 
and agility. However, those tasks may only 
provide limited information about the patient’s 
pain or confidence levels, particularly when 
performed in a controlled laboratory 
environment.  Evaluation of a patient’s confidence 
levels, and perception of function are just as 
important as evaluating the patient’s strength, 
power, and agility. Providing a frame of 
reference did not alter median PRO scores, which 
suggests that these outcomes are measuring 
different aspects of health. This study adds to the 
growing body of literature supporting a 
multifaceted approach to outcome measurement 
to provide the most effective evaluation of patient 
care and progress. 

Interestingly, 4 out of 10 participants in the 
educational control group exhibited increased 
scores on the IKDC at the post-test (Table 4). We 
hypothesize that these results may have occurred 
because of the influence of modeling on 
confidence.32 It has been demonstrated that 
vicarious experiences, or knowledge gained 
through observing the experiences of others, can 
enhance confidence.32 Participants in the 
educational control group may have increased 
their confidence to complete functional tasks 
through observation of individuals completing 
functional tasks in the FIFA 11+19 videos. It is 
plausible that deficits in confidence were present 
in this sample at the time of testing as previous 
literature has demonstrated that lack of 
confidence is often reported as a barrier for 
return to sport after a sports-related knee injury.33 
These results further support the integration of 
PROs into clinical practice. Use of the IKDC and 
KOOSsports may also provide insight into a 
patient’s confidence levels prior to return to sport.  

Limitations 

This study is not without limitations. Due to the 
nature of subject recruitment, researchers were 
unable to test most participants on the day of 
clearance. Participants, particularly those 
randomized into the performance-based 
assessment group, could have already been 
exposed to different stimuli that would represent 
a sample of experiences from which to answer 
PRO questions.  Another limitation of this study is 
the small sample size (n = 20). However, despite 
the small sample size, given that only 1 participant 
demonstrated a change value exceeding minimal 
detectable change values for either PRO (IKDC = 
6.7-20.5, KOOS sports = 12.2-7), it is very clear 
that neither statistical nor clinical significance were 
likely to be reached for PROs even if more 
subjects had been enrolled.  

CLINICAL APPLICATION 

Evaluating health outcomes following 
rehabilitation is a fundamental component of a 
clinician’s duties.  This study demonstrated that 
providing a frame of reference for a highly 
trained athletic population did not significantly 
alter median PRO scores. Therefore, among knee 
patients it is acceptable to complete PROs either 
before or after completing performance-based 
measures.  Furthermore, these outcome measures 
should not be used in substitute and both should 
be incorporated into clinical practice to provide a 
holistic approach to rehabilitation after knee 
injury. Patient reported outcome measures may be 
better suited at the evaluation of symptoms 
and/or confidence in a patient, while 
performance-based measures evaluate physical 
function in a controlled setting. Thus, it is important 
to utilize all forms of health outcome evaluation 
techniques to provide the best healthcare for our 
patients. 

Acknowledgements 

This publication was supported by the National 
Athletic Trainers’ Association Research and 
Education Foundation Grant Number 15MGP002. 



Use of Response Shift to Improve Agreement between Patient-Reported and Performance-Based Outcomes in Knee Patients 
 

 

33 
Copyright © by Indiana State University                                                                                Clinical Practice in Athletic Training  
All rights reserved. ISSN Online 2577-8188                                                                        Volume 4 – Issue 1 – February 2021 
 

Its contents are the authors’ sole responsibility and 
do not necessarily represent official National 
Athletic Trainers’ Association Research and 
Education Foundation views. 

The project described was supported by the 
National Center for Research Resources and the 
National Center for Advancing Translational 
Sciences, National Institutes of Health, through 
Grant 8UL1TR000117-02. The content is solely 
the responsibility of the authors and does not 
necessarily represent the official views of the NIH. 

REFERENCES 

1. Kaplan S. Outcome Measurement & 
Management. Philadelphia, PA: F.A. Davis 
Company; 2007. 

2. Neeb TB, Aufdemkampe G, Wagener JHD, 
Mastenbroek L. Assessing anterior cruciate 
ligament injuries: the association and 
differential value of questionnaires, clinical 
tests, and functional tests. J Orthop Sports 
Phys Ther. 1997;26(6):324-331. 
https://www.jospt.org/doi/10.2519/jospt.1
997.26.6.324.  

3. Aalund PK, Larsen K, Hansen TB, Bandholm T. 
Normalized knee-extension strength or leg-
press power after fast-track total knee 
arthroplasty: which measure is most closely 
associated with performance-based and 
self-reported function? Arch Phys Med 
Rehabil. 2013;94(2):384-390. 
https://doi.org/10.1016/j.apmr.2012.09.0
31.  

4. Sernert N, Kartus J, Koehler K, et al. Analysis 
of subjective, objective, and functional 
examination tests after anterior cruciate 
ligament reconstruction: a follow-up of 527 
patients. Knee Surg Sports Traumatol 
Arthrosc. 1999;7(3):160-165. 
https://doi.org/10.1007/s001670050141.  

5. Risberg MA, Ekeland A. Assessment of 
functional tests after anterior cruciate 
ligament surgery. J Orthop Sports Phys Ther. 

1994;19(4):212-217. 
https://doi.org/10.2519/jospt.1994.19.4.2
12.  

6. Reiman M, Manske R. Functional Testing in 
Human Performance. Human Kinetics; 2009. 

7. Stratford PW, Kennedy DM. Performance 
measures were necessary to obtain a 
complete picture of osteoarthritic patients. J 
Clin Epidemiol. 2006;59(2):160-167. 
https://doi.org/10.1016/j.jclinepi.2005.07.
012.  

8. Logerstedt D, Grindem H, Lynch A, et al. 
Single-legged hop tests as predictors of self-
reported knee function after anterior 
cruciate ligament reconstruction: the 
Delaware-Oslo ACL cohort study. Am J 
Sports Med. 2012;40(10):2348-2356. 
https://doi.org/10.1177/0363546512457
551.  

9. Meadows KA. Patient-reported outcome 
measures: an overview. Br J Community Nurs. 
2011;16(3):146-151. 
https://doi.org/10.12968/bjcn.2011.16.3.1
46.  

10. Maly MR, Costigan PA, Olney SJ. 
Determinants of self-report outcome 
measures in people with knee osteoarthritis. 
Arch Phys Med Rehabil. 2006;87(1):96-104. 
https://doi.org/10.1016/j.apmr.2005.08.1
10.  

11. Mizner RL, Petterson SC, Clements KE, Zeni 
JA, Jr., Irrgang JJ, Snyder-Mackler L. 
Measuring functional improvement after total 
knee arthroplasty requires both 
performance-based and patient-report 
assessments: a longitudinal analysis of 
outcomes. J Arthroplasty. 2011;26(5):728-
737. 
https://dx.doi.org/10.1016%2Fj.arth.2010.
06.004.  

12. Noyes FR, Barber SD, Mooar LA. A rationale 
for assessing sports activity levels and 
limitations in knee disorders. Clini Orthop 
Relat Res. 1989(246):238-249. 

https://www.jospt.org/doi/10.2519/jospt.1997.26.6.324
https://www.jospt.org/doi/10.2519/jospt.1997.26.6.324
https://doi.org/10.1016/j.apmr.2012.09.031
https://doi.org/10.1016/j.apmr.2012.09.031
https://doi.org/10.1007/s001670050141
https://doi.org/10.2519/jospt.1994.19.4.212
https://doi.org/10.2519/jospt.1994.19.4.212
https://doi.org/10.1016/j.jclinepi.2005.07.012
https://doi.org/10.1016/j.jclinepi.2005.07.012
https://doi.org/10.1177/0363546512457551
https://doi.org/10.1177/0363546512457551
https://doi.org/10.12968/bjcn.2011.16.3.146
https://doi.org/10.12968/bjcn.2011.16.3.146
https://doi.org/10.1016/j.apmr.2005.08.110
https://doi.org/10.1016/j.apmr.2005.08.110
https://dx.doi.org/10.1016%2Fj.arth.2010.06.004
https://dx.doi.org/10.1016%2Fj.arth.2010.06.004


Use of Response Shift to Improve Agreement between Patient-Reported and Performance-Based Outcomes in Knee Patients 
 

 

34 
Copyright © by Indiana State University                                                                                Clinical Practice in Athletic Training  
All rights reserved. ISSN Online 2577-8188                                                                        Volume 4 – Issue 1 – February 2021 
 

13. Logerstedt D, Di Stasi S, Grindem H, et al. 
Self-reported knee function can identify 
athletes who fail return-to-activity criteria up 
to 1 year after anterior cruciate ligament 
reconstruction: A Delaware-Oslo ACL cohort 
study. J Orthop Sports Phys Ther. 
2014;44(12):914-923. 
https://doi.org/10.2519/jospt.2014.4852.  

14. Fitzgerald GK, Axe MJ, Snyder-Mackler L. A 
decision-making scheme for returning 
patients to high-level activity with 
nonoperative treatment after anterior 
cruciate ligament rupture. Knee Surg Sports 
Traumatol Arthrosc. 2000;8(2):76-82. 
https://doi.org/10.1007/s001670050190.  

15. Howard JS, Mattacola CG, Mullineaux DR, 
English RA, Lattermann C. Influence of 
response shift on early patient-reported 
outcomes following autologous chondrocyte 
implantation. Knee Surg Sports Traumatol 
Arthrosc. 2014;22(9):2163-2171. 
https://doi.org/10.1007/s00167-013-
2654-1.  

16. Balain B, Ennis O, Kanes G, et al. Response 
shift in self-reported functional scores after 
knee microfracture for full thickness cartilage 
lesions. Osteoarthritis Cartilage. 
2009;17(8):1009-1013. 
https://doi.org/10.1016/j.joca.2009.02.00
7.  

17. Schwartz CE, Sprangers MA. Methodological 
approaches for assessing response shift in 
longitudinal health-related quality-of-life 
research. Social Sci Med. (1982). 
1999;48(11):1531-1548. 
https://doi.org/10.1016/s0277-
9536(99)00047-7.  

18. Sprangers MA, Schwartz CE. Integrating 
response shift into health-related quality of 
life research: a theoretical model. Social Sci 
Med. (1982). 1999;48(11):1507-1515. 
https://doi.org/10.1016/s0277-
9536(99)00045-3.  

19. Soligard T, Myklebust G, Steffen K, et al. 
Comprehensive warm-up programme to 

prevent injuries in young female footballers: 
cluster randomised controlled trial. BMJ. 
2008;337:a2469. 
https://doi.org/10.1136/bmj.a2469.  

20. Tegner Y, Lysholm J. Rating systems in the 
evaluation of knee ligament injuries. Clin 
Orthop Relat Res. 1985;198:43-49.  

21. Irrgang JJ, Anderson AF, Boland AL, et al. 
Development and validation of the 
International Knee Documentation Committee 
Subjective Knee Form. Am J Sports Med. 
2001;29(5):600-613. 
https://doi.org/10.1177/0363546501029
0051301.  

22. Roos EM, Roos HP, Lohmander LS, Ekdahl C, 
Beynnon BD. Knee injury and osteoarthritis 
outcome score (KOOS)-development of a 
self-administered outcome measure. J Orthop 
Sports Phys Ther. 1998;28(2):88-96. 
https://doi.org/10.2519/jospt.1998.28.2.8
8.  

23. Harris PA, Taylor R, Thielke R, Payne J, 
Gonzalez N, Conde JG. Research electronic 
data capture (REDCap)—a metadata-driven 
methodology and workflow process for 
providing translational research informatics 
support. J Biomed Inform. 2009;42(2):377-
381. 
https://doi.org/10.1016/j.jbi.2008.08.010.  

24. Owoeye OBA, Akinbo SRA, Tella BA, 
Olawale OA. Efficacy of the FIFA 11+ 
warm-up programme in male youth football: 
a cluster randomised controlled trial. J Sports 
Sci Med.  2014;13(2):321-328. 

25. Longo UG, Loppini M, Berton A, Marinozzi A, 
Maffulli N, Denaro V. The FIFA 11+ program 
is effective in preventing injuries in elite male 
basketball players: a cluster randomized 
controlled trial. Am J Sports Med. 
2012;40(5):996-1005. 
https://doi.org/10.1177/0363546512438
761.  

26. Steffen K, Emery CA, Romiti M, et al. High 
adherence to a neuromuscular injury 
prevention programme (FIFA 11+) improves 

https://doi.org/10.2519/jospt.2014.4852
https://doi.org/10.1007/s001670050190
https://doi.org/10.1007/s00167-013-2654-1
https://doi.org/10.1007/s00167-013-2654-1
https://doi.org/10.1016/j.joca.2009.02.007
https://doi.org/10.1016/j.joca.2009.02.007
https://doi.org/10.1016/s0277-9536(99)00047-7
https://doi.org/10.1016/s0277-9536(99)00047-7
https://doi.org/10.1016/s0277-9536(99)00045-3
https://doi.org/10.1016/s0277-9536(99)00045-3
https://doi.org/10.1136/bmj.a2469
https://doi.org/10.1177/03635465010290051301
https://doi.org/10.1177/03635465010290051301
https://doi.org/10.2519/jospt.1998.28.2.88
https://doi.org/10.2519/jospt.1998.28.2.88
https://doi.org/10.1016/j.jbi.2008.08.010
https://doi.org/10.1177/0363546512438761
https://doi.org/10.1177/0363546512438761


Use of Response Shift to Improve Agreement between Patient-Reported and Performance-Based Outcomes in Knee Patients 
 

 

35 
Copyright © by Indiana State University                                                                                Clinical Practice in Athletic Training  
All rights reserved. ISSN Online 2577-8188                                                                        Volume 4 – Issue 1 – February 2021 
 

functional balance and reduces injury risk in 
Canadian youth female football players: a 
cluster randomised trial. Br J Sports Med. 
2013;47(12):794-802. 
https://doi.org/10.1136/bjsports-2012-
091886.  

27. Grooms DR, Palmer T, Onate JA, Myer GD, 
Grindstaff T. Soccer-specific warm-up and 
lower extremity injury rates in collegiate 
male soccer players. J Athl Train. 
2013;48(6):782-789. 
https://doi.org/10.4085/1062-6050-
48.4.08.  

28. Gribble PA, Hertel J, Plisky P. Using the Star 
Excursion Balance Test to assess dynamic 
postural-control deficits and outcomes in 
lower extremity injury: a literature and 
systematic review. J Athl Train. 
2012;47(3):339-357. 
https://doi.org/10.4085/1062-6050-
47.3.08.  

29. Noyes FR, Barber SD, Mangine RE. Abnormal 
lower limb symmetry determined by function 
hop tests after anterior cruciate ligament 
rupture. Am J Sports med. 1991;19(5):513-
518. 
https://doi.org/10.1177/0363546591019
00518.  

30. Munro AG, Herrington LC. Between-session 
reliability of four hop tests and the agility T-
test. J Strength Cond Res. 2011;25(5):1470-
1477. 
https://doi.org/10.1519/jsc.0b013e3181d
83335.  

31. Loudon JK, Wiesner D, Goist-Foley HL, Asjes 
C, Loudon KL. Intrarater reliability of 
functional performance tests for subjects with 
patellofemoral pain syndrome. J Athl Train. 
2002;37(3):256-261. 

32. Bandura A. Self-efficacy: toward a unifying 
theory of behavioral change. Psychol Rev. 
1977;84(2):191. 
https://psycnet.apa.org/doi/10.1037/003
3-295X.84.2.191.  

33. Burland JP, Toonstra J, Werner JL, Mattacola 
CG, Howell DM, Howard JS. Decision to 
return to sport after anterior cruciate 
ligament reconstruction, part I: a qualitative 
investigation of psychosocial factors. J Athl 
Train. 2018;53(5):452-463. 
https://doi.org/10.4085/1062-6050-313-
16. 

 

 

https://doi.org/10.1136/bjsports-2012-091886
https://doi.org/10.1136/bjsports-2012-091886
https://doi.org/10.4085/1062-6050-48.4.08
https://doi.org/10.4085/1062-6050-48.4.08
https://doi.org/10.4085/1062-6050-47.3.08
https://doi.org/10.4085/1062-6050-47.3.08
https://doi.org/10.1177/036354659101900518
https://doi.org/10.1177/036354659101900518
https://doi.org/10.1519/jsc.0b013e3181d83335
https://doi.org/10.1519/jsc.0b013e3181d83335
https://psycnet.apa.org/doi/10.1037/0033-295X.84.2.191
https://psycnet.apa.org/doi/10.1037/0033-295X.84.2.191
https://doi.org/10.4085/1062-6050-313-16
https://doi.org/10.4085/1062-6050-313-16

	ABSTRACT
	Table 2. Patient-Reported Outcome Measures Pre-Test, Post-Test, and Change Scores
	Table 3. Outcomes of PROs change scores and performance-based testing demonstrating mean values and standard deviation (SD), and p-value

