If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Excellent Correlation of the Patient-Reported Outcomes Measurement Information System Upper Extremity Score With Legacy Outcome Scores Preoperatively and at 1 Year After Arthroscopic Rotator Cuff Repair
Address correspondence to Mandeep S. Virk, M.D., Division of Shoulder and Elbow Surgery, Department of Orthopedic Surgery, NYU Langone Orthopedic Hospital, 246 E 20th St, New York, NY 10003, U.S.A.
To assess the preoperative and postoperative performance of the Patient-Reported Outcomes Measurement Information System Upper Extremity (PROMIS-UE, version 2.0) outcome score in comparison to the American Shoulder and Elbow Surgeons (ASES) and Western Ontario Rotator Cuff Index (WORC) instruments in patients undergoing rotator cuff repair.
Methods
This prospective longitudinal study included 91 patients undergoing rotator cuff repair. Patients completed the PROMIS-UE, ASES, and WORC instruments preoperatively and postoperatively at 2 weeks, 6 weeks, 3 months, and 12 months. The Pearson correlation coefficient (r) between these tools was calculated at each time point. Correlations were graded as excellent (>0.7), excellent-good (0.61-0.7), good (0.4-0.6), or poor (<0.4). Responsiveness to change was assessed using the effect size and the standardized response mean. Floor and ceiling effects for each instrument were also assessed.
Results
The PROMIS-UE instrument showed good to excellent correlation with the legacy instruments at all time points. There were variations in the measured effect sizes of the various instruments, with the PROMIS-UE instrument showing responsiveness to change at 3 and 12 months but the ASES and WORC instruments showing responsiveness at 6 weeks, 3 months, and 12 months. Both PROMIS-UE and ASES scores displayed ceiling effects at 12 months.
Conclusions
The PROMIS-UE instrument shows excellent correlation with the ASES instrument and a rotator cuff–specific outcome instrument—the WORC instrument—preoperatively and at 1 year after arthroscopic rotator cuff repair. Variations in the measured effect sizes at different postoperative time points and high ceiling effects of the PROMIS-UE instrument at the 1-year time point may limit its utility in the early postoperative phase and at long-term follow-up after rotator cuff repair.
Clinical Relevance
The performance of the PROMIS-UE outcome measure after arthroscopic rotator cuff repair was investigated.
Patient-reported outcome measures (PROMs) have become an increasingly important tool in orthopaedic clinical practice and research. Because the main outcome measure in orthopaedics is quality of life, incorporation of patient input is an essential component in the evaluation of treatment outcomes.
In addition to the use of PROMs as tools in clinical care delivery, their widespread adoption has enabled researchers to better compare treatments against one another and to perform cost-effectiveness analysis.
Within the field of shoulder and elbow surgery alone, there are more than a dozen commonly used PROMs, ranging from general shoulder assessments to disease-specific instruments.
In an effort to create a more standardized approach to patient-centered outcome measurement, the National Institutes of Health has developed a set of tools known as the Patient-Reported Outcomes Measurement Information System (PROMIS). These tools measure patient health across 5 subdomains: Physical Function, Social Function, Pain, Fatigue, and Emotional Distress.
Through the use of computer adaptive testing, PROMIS instruments utilize a patient’s responses to adaptively select questions from a large item bank. Each subsequent question is selected based on the patient’s answer to the previous item and can vary from one patient to another. This adaptive ability allows PROMIS instruments to avoid redundancy in subsequent questions and deliver accurate outcome measurements while minimizing both response burden and troublesome floor and ceiling effects.
The PROMIS Upper Extremity (PROMIS-UE) instrument (version 2.0), which was created from a subset of the questions in the Physical Function subdomain, is of particular importance to the field of shoulder and elbow surgery. Because many legacy PROMs used in shoulder and elbow surgery combine a variety of different domains of health measurement into one tool, their ability to precisely detect changes in patients’ physical function is diminished.
By focusing solely on upper-extremity physical function, the PROMIS-UE instrument can enable clinicians and researchers to more effectively measure outcomes and make treatment decisions. For this reason, there has been much research comparing the PROMIS-UE instrument with previously validated legacy instruments. Although a number of cross-sectional studies have shown that the PROMIS-UE instrument correlates well with legacy instruments,
PROMIS Upper Extremity underperforms psychometrically relative to American Shoulder and Elbow Surgeons score in patients undergoing primary rotator cuff repair.
Correlation of Patient-Reported Outcome Measurement Information System Physical Function Upper Extremity computer adaptive testing, with the American Shoulder and Elbow Surgeons shoulder assessment form and Simple Shoulder Test in patients with shoulder pain.
Correlation of PROMIS Physical Function Upper Extremity computer adaptive test with American Shoulder and Elbow Surgeons shoulder assessment form and Simple Shoulder Test in patients with shoulder arthritis.
Correlation of Patient-Reported Outcomes Measurement Information System (PROMIS) scores with legacy patient-reported outcome scores in patients undergoing rotator cuff repair.
Correlation of Patient Reported Outcome Measurement Information System (PROMIS) with American Shoulder and Elbow Surgeon (ASES), and Constant (CS) scores in idiopathic adhesive capsulitis.
Sabharwal S, Skolasky RL, Souza JM, Potter BK, Forsberg JA. Concurrent validity of PROMIS with DASH and DVPRS in transhumeral amputees [published online January 26, 2022]. Hand (N Y). https://doi.org/10.1177/15589447211073833.
A comparison of nerve-specific, condition-specific, and upper extremity-specific patient-reported outcome measures in patients with carpal and cubital tunnel syndrome.
Relationship between the Patient-Reported Outcomes Measurement Information System (PROMIS) computer adaptive testing and legacy instruments in patients undergoing isolated biceps tenodesis.
Correlation of Patient-Reported Outcomes Measurement Information System questionnaires with the Brief Michigan Hand Questionnaire in patients with 5 common hand conditions.
Preoperative psychometric properties of Patient-Reported Outcomes Measurement Information System Upper Extremity, Pain Interference, and Depression in Bankart repair and rotator cuff repair.
Correlation of Patient Reported Outcome Measurement Information System (PROMIS) with American Shoulder and Elbow Surgeon (ASES), and Constant (CS) scores in idiopathic adhesive capsulitis.
Performance of Patient-Reported Outcomes Measurement Information System (PROMIS) scores compared with legacy metrics in evaluating outcomes after surgical treatment for osteochondritis dissecans of the humeral capitellum.
The goal of this prospective longitudinal study was to assess the preoperative and postoperative performance of the PROMIS-UE outcome score in comparison to the American Shoulder and Elbow Surgeons (ASES) and Western Ontario Rotator Cuff Index (WORC) instruments in patients undergoing rotator cuff repair (RCR). We hypothesized that the PROMIS-UE instrument would correlate well with these legacy instruments in patients undergoing RCR and that it would be able to precisely capture changes in patient function while reducing response burden, avoiding any floor or ceiling effects.
Methods
This study was approved by the NYU Langone Health Institutional Review Board (No. s18-01216). All study patients provided informed consent prior to enrollment.
Study Design
This prospective, observational cohort study recruited 105 patients scheduled to undergo RCR surgery, of whom 14 were lost to follow-up. Study subjects were recruited from the outpatient orthopaedic surgery clinic at NYU Langone Health between March 2019 and December 2020. Potential subjects were identified by monitoring the upcoming surgery schedule. Subjects were considered for inclusion if they were scheduled to undergo RCR, were 18 years or older, had no other trauma to the arm or shoulder (eg, fracture or dislocation), and were willing and able to provide informed consent. Subjects were excluded if they were unable to communicate in English, if they did not complete the required preoperative PROMs, and if they did not complete either the 3- or 12-month follow-up surveys. Vulnerable subjects, such as prisoners or those with a mental handicap preventing their ability to provide informed consent, were not considered for enrollment. Consent for the study was obtained from all subjects in person during the preoperative office visit or, for those who did not have a preoperative clinic appointment, preoperatively on the day of surgery.
Data Collection
Subjects were asked to complete 3 PROMs preoperatively and postoperatively at 2 weeks, 6 weeks, 3 months, and 12 months. The outcome measures used in this study were the PROMIS-UE instrument (version 2.0), ASES instrument, and WORC instrument. Surveys were not checked for completeness until after study conclusion. Study data were collected online and managed using REDCap electronic data capture tools hosted at NYU Langone Health.
Links to complete the surveys were automatically sent to study subjects at the preoperative and postoperative time points using REDCap’s built-in scheduling functionality. If a subject did not complete a survey on the day it was sent, reminder emails with a link to the survey were automatically sent daily for up to 5 days.
Outcome Measures
The PROMIS-UE instrument (version 2.0) is a subset of the PROMIS Physical Function computer adaptive test. It was designed to reliably measure upper-extremity physical function while eliminating floor and ceiling effects and reducing response burden through the use of computer adaptive testing.
The instrument is designed so that it has a mean score of 50 and standard deviation of 10 in the general population, with higher scores indicating better physical function.
The ASES instrument is a general shoulder function assessment that includes 10 questions about function and 1 question about pain. Scores range from 0 to 100, with 50 points coming from the function section and 50 points coming from the pain section. The minimal clinically important difference for this instrument in patients with rotator cuff problems is reported to be 12 to 17 points.
The WORC instrument is a rotator cuff–specific PROM. It includes 21 questions: 10 about physical symptoms, 4 about sports and recreation, 4 about lifestyle function, and 3 about emotional function. Each question is graded from 0 to 100, and the subscores are summed to yield a total score out of 2,100. On this scale, higher function is indicated by a lower overall score.
For purposes of comparison, this score can be converted into an index (or percentage) score (with a higher score indicating higher function) via the following equation: % Score = (2,100 – Total WORC score)/21. The minimal clinically important difference for the WORC score in patients with rotator cuff tears is estimated to be –282.6.
Establishing minimally important differences for the American Shoulder and Elbow Surgeons score and the Western Ontario Rotator Cuff Index in patients with full-thickness rotator cuff tears.
RStudio software (version 3.6.2; RStudio, Boston, MA) was used for all statistical analysis. The Pearson correlation coefficient was calculated between the PROMIS-UE scores and the scores on both legacy instruments at all time points. In addition, the correlation between the PROMIS-UE instrument and the Function subsection of the ASES instrument (ASES-Function) was calculated. The strength of correlation was graded using the following ranges previously defined for PROMs: excellent, r > 0.7; excellent-good, r > 0.6 to r ≤ 0.7; good, r > 0.3 to r ≤ 0.6; and poor, r > 0.2 to r ≤ 0.3.
Correlation of Patient-Reported Outcome Measurement Information System Physical Function Upper Extremity computer adaptive testing, with the American Shoulder and Elbow Surgeons shoulder assessment form and Simple Shoulder Test in patients with shoulder pain.
Correlation of PROMIS Physical Function Upper Extremity computer adaptive test with American Shoulder and Elbow Surgeons shoulder assessment form and Simple Shoulder Test in patients with shoulder arthritis.
Change in mean outcome score versus baseline was assessed for each instrument, as well as the ASES-Function score, using a paired t test, and instrument responsiveness to change was measured using both the effect size (Cohen d) and the standardized response mean. Confidence intervals for the effect size and standardized response mean were estimated using bootstrap resampling (1,000 iterations). Values were defined as small (0.2), medium (0.5), or large (0.8).
For the ASES and WORC instruments, floor and ceiling effects were noted if more than 15% of subjects achieved scores at the minimum and maximum of the instrument’s range, respectively.
Owing to the administration of the PROMIS-UE instrument as a computer adaptive test, its floor or ceiling effects were defined as more than 15% of subjects choosing the lowest or highest response value for each completed item.
Response burden for the PROMIS-UE instrument was calculated as the mean number of questions answered for all PROMIS-UE surveys across all time points. Response burden for the ASES and WORC instruments was assumed to be the total number of questions available for each instrument (11 and 21, respectively). P < .05 was considered statistically significant, with a Bonferroni-corrected P value of .0063 used to account for multiple comparisons.
An a priori power analysis was conducted to determine minimum sample size requirements. With α set at <.05 (based on the Bonferroni-corrected P value) and 80% power, a minimum of 74 subjects would be required to detect a correlation of 0.4 (moderate).
Results
Demographic Characteristics
A total of 91 subjects (61 men and 30 women) with a mean age of 60.4 years were included in the final analysis of this study (86.7% follow-up). Demographic data are summarized in Table 1.
Table 1Demographic Characteristics
Characteristic
Data (N = 91)
Age, mean (SD), yr
60.9 (10.4)
Sex
Female
31 (34.1)
Male
60 (65.9)
Race
American Indian or Alaska Native
0 (0)
Asian
2 (2.2)
Black or African American
9 (9.9)
White
77 (84.6)
>1 Race
2 (2.2)
Unknown or not reported
1 (1.1)
Ethnicity
Hispanic or Latino
7 (7.7)
Not Hispanic or Latino
84 (92.3)
Unknown or not reported
0 (0)
BMI
Mean (SD)
29.1 (6.01)
Median (minimum, maximum)
27.4 (20.4, 54.0)
Smoking status
Never
59 (64.8)
Current
3 (3.3)
Former
29 (31.9)
Activity level
Not active
5 (5.5)
Somewhat active
41 (45.1)
Very active
45 (49.5)
Hand dominance
Right
81 (89.0)
Left
9 (9.9)
Ambidextrous
1 (1.1)
Tear size
Small (<1 cm)
19 (20.9)
Medium (1-3 cm)
50 (54.9)
Large (>3 cm)
22 (24.2)
NOTE. Data are presented as number (percentage) unless otherwise indicated.
Mean scores for each PROM are shown in Table 2, effect sizes are shown in Table 3, and box plots for each PROM are presented in Figures 1, 2, 3, and 4. For the PROMIS-UE instrument, there was a statistically significant change in scores relative to baseline at 3 and 12 months (P < .001) but not at 6 weeks (P = .20). The effect size was small at 6 weeks (0.16), medium to large at 3 months (0.71), and large at 12 months (1.98).
Table 2Mean PROM Scores
Preoperative (N = 91)
Postoperative
6 wk (n = 65)
3 mo (n = 86)
12 mo (n = 67)
PROMIS-UE score
Mean (SD)
31.8 (7.41)
32.0 (6.36)
36.9 (6.55)
47.6 (9.35)
Median (minimum, maximum)
31.1 (14.7, 54.7)
32.4 (17.8, 45.0)
35.8 (23.0, 55.9)
47.9 (26.8, 61.0)
ASES score
Mean (SD)
45.3 (18.8)
55.0 (18.4)
69.2 (17.0)
86.0 (17.4)
Median (minimum, maximum)
46.7 (5.00, 81.7)
58.3 (3.33, 90.0)
70.8 (25.0, 98.3)
91.7 (16.7, 100)
ASES-F score
Mean (SD)
25.5 (9.52)
22.8 (9.98)
32.1 (9.80)
43.3 (7.63)
Median (minimum, maximum)
26.7 (5.00, 45.0)
21.7 (3.33, 45.0)
32.5 (10.0, 48.3)
46.7 (16.7, 50.0)
WORC score
Mean (SD)
34.9 (19.3)
44.2 (17.5)
58.7 (21.5)
80.5 (20.6)
Median (minimum, maximum)
31.9 (3.76, 88.6)
45.3 (10.1, 91.7)
56.6 (15.4, 96.7)
85.8 (16.0, 100)
ASES, American Shoulder and Elbow Surgeons; ASES-F, American Shoulder and Elbow Surgeons Function subdomain; PROM, patient-reported outcome measure; PROMIS-UE, Patient-Reported Outcomes Measurement Information System Upper Extremity; SD, standard deviation; WORC, Western Ontario Rotator Cuff Index.
Table 3Effect Sizes of PROMs at Each Time Point Compared With Preoperative Baseline
Effect Size (95% CI)
6 wk Postoperatively
3 mo Postoperatively
12 mo Postoperatively
PROMIS-UE score
0.16 (–0.08, 0.41)
0.71 (0.47, 0.95)
1.98 (1.57, 2.41)
ASES score
0.60 (0.33, 0.87)
1.23 (0.95, 1.51)
2.09 (1.67, 2.53)
ASES-F score
–0.20 (–0.45, 0.04)
0.62 (0.39, 0.86)
1.93 (1.53, 2.35)
WORC score
0.59 (0.33, 0.86)
1.12 (0.85, 1.39)
2.26 (1.82, 2.73)
ASES, American Shoulder and Elbow Surgeons; ASES-F, American Shoulder and Elbow Surgeons Function subdomain; CI, confidence interval; PROM, patient-reported outcome measure; PROMIS-UE, Patient-Reported Outcomes Measurement Information System Upper Extremity; WORC, Western Ontario Rotator Cuff Index.
Fig 1Box-and-whisker plot of Patient-Reported Outcomes Measurement Information System Upper Extremity (PROMIS-UE) scores at each time point. The box spans the range between the 25th and 75th percentile scores, and the black center line denotes the median. The upper and lower whiskers are bounded by the maximum and minimum values that are within 1.5 times the interquartile range from the 75th and 25th percentiles. Individual scores are displayed for each time point, with random left-right scattering introduced to aid readability. (Preop, Preoperative.)
Fig 2Box-and-whisker plot of American Shoulder and Elbow Surgeons (ASES) scores at each time point. The box spans the range between the 25th and 75th percentile scores, and the black center line denotes the median. The upper and lower whiskers are bounded by the maximum and minimum values that are within 1.5 times the interquartile range from the 75th and 25th percentiles. Individual scores are displayed for each time point, with random left-right scattering introduced to aid readability. (Preop, Preoperative.)
Fig 3Box-and-whisker plot of Western Ontario Rotator Cuff Index (WORC) scores at each time point. The box spans the range between the 25th and 75th percentile scores, and the black center line denotes the median. The upper and lower whiskers are bounded by the maximum and minimum values that are within 1.5 times the interquartile range from the 75th and 25th percentiles. Individual scores are displayed for each time point, with random left-right scattering introduced to aid readability. (Preop, Preoperative.)
Fig 4Box-and-whisker plot of American Shoulder and Elbow Surgeons (ASES)–Function subdomain scores at each time point. The box spans the range between the 25th and 75th percentile scores, and the black center line denotes the median. The upper and lower whiskers are bounded by the maximum and minimum values that are within 1.5 times the interquartile range from the 75th and 25th percentiles. Individual scores are displayed for each time point, with random left-right scattering introduced to aid readability. (Preop, Preoperative.)
Both the ASES and WORC instruments showed statistically significant changes in scores relative to baseline at 6 weeks, 3 months, and 12 months (P < .001). Both instruments showed medium to large effect sizes at 6 weeks (0.60 and 0.59, respectively) and large effect sizes at 3 and 12 months (1.23 and 2.09, respectively, for the ASES instrument and 1.12 and 2.26, respectively, for the WORC instrument).
Correlation of PROMIS Scores With ASES and WORC Scores
Correlations between the PROMIS-UE instrument and the legacy PROMs are presented in Table 4. The correlations between the PROMIS-UE scores and the ASES, ASES-Function, and WORC scores were excellent at all time points, with the one exception of an excellent-good (0.68) correlation between the ASES and PROMIS-UE scores at 3 months. All correlations measured were statistically significant using a Bonferroni-corrected P value of .0063.
Table 4Pearson Correlation Coefficients Between PROMIS-UE Instrument and Legacy PROMs
Preoperative
Postoperative
6 wk
3 mo
12 mo
ASES score
0.71 (P < .001)
0.72 (P < .001)
0.68 (P < .001)
0.73 (P < .001)
ASES-F score
0.73 (P < .001)
0.77 (P < .001)
0.80 (P < .001)
0.81 (P < .001)
WORC score
0.71 (P < .001)
0.71 (P < .001)
0.79 (P < .001)
0.77 (P < .001)
ASES, American Shoulder and Elbow Surgeons; ASES-F, American Shoulder and Elbow Surgeons Function subdomain; PROM, patient-reported outcome measure; PROMIS-UE, Patient-Reported Outcomes Measurement Information System Upper Extremity; WORC, Western Ontario Rotator Cuff Index.
Floor and ceiling effect calculations are shown in Table 5. None of the instruments exhibited floor effects at any time point. However, both the PROMIS-UE and ASES instruments were found to exhibit ceiling effects at 12 months (16.42% and 20.90%, respectively). No ceiling effects were noted for the WORC instrument at any time point.
Table 5Floor and Ceiling Effects of PROMIS-UE CAT, ASES, ASES-F and WORC Scores
Preoperative
Postoperative
6 wk
3 mo
12 mo
Floor
Ceiling
Floor
Ceiling
Floor
Ceiling
Floor
Ceiling
PROMIS-UE score
1 of 91 (1.10)
0 of 91 (0.00)
0 of 65 (0.0)
0 of 65 (0.00)
0 of 60 (0.0)
0 of 86 (0.0)
0 of 86 (0.0)
11 of 86 (16.42)
ASES score
0 of 91 (0.0)
0 of 91 (0.0)
0 of 65 (0.0)
0 of 65 (0.0)
0 of 60 (0.0)
0 of 86 (0.0)
0 of 86 (0.0)
14 of 86 (20.90)
ASES-F score
0 of 91 (0.0)
0 of 91 (0.0)
0 of 65 (0.0)
0 of 65 (0.0)
0 of 60 (0.0)
0 of 86 (0.0)
0 of 86 (0.0)
15 of 86 (22.39)
WORC score
0 of 91 (0.00)
0 of 91 (0.0)
0 of 65 (0.0)
0 of 65 (0.0)
0 of 60 (0.0)
0 of 86 (0.0)
0 of 86 (0.0)
3 of 86 (4.48)
NOTE. Data are presented as number (percentage).
ASES, American Shoulder and Elbow Surgeons; ASES-F, American Shoulder and Elbow Surgeons Function subdomain; CAT, computer adaptive test; PROMIS-UE, Patient-Reported Outcomes Measurement Information System Upper Extremity; WORC, Western Ontario Rotator Cuff Index.
The mean number of questions answered for each PROMIS survey completed was 5.62 (95% confidence interval, 5.31-5.92), a significant reduction in comparison to the fixed-length ASES (11 questions) and WORC (21 questions) surveys.
Discussion
Our longitudinal study results show that the PROMIS-UE instrument has an excellent-good to excellent correlation with the ASES and WORC instruments both preoperatively and postoperatively while reducing patient response burden. These results are in line with the findings of cross-sectional studies comparing the PROMIS-UE instrument with legacy instruments in patients with rotator cuff pathology,
PROMIS Upper Extremity underperforms psychometrically relative to American Shoulder and Elbow Surgeons score in patients undergoing primary rotator cuff repair.
Correlation of Patient-Reported Outcomes Measurement Information System (PROMIS) scores with legacy patient-reported outcome scores in patients undergoing rotator cuff repair.
Preoperative psychometric properties of Patient-Reported Outcomes Measurement Information System Upper Extremity, Pain Interference, and Depression in Bankart repair and rotator cuff repair.
Correlation of Patient-Reported Outcome Measurement Information System Physical Function Upper Extremity computer adaptive testing, with the American Shoulder and Elbow Surgeons shoulder assessment form and Simple Shoulder Test in patients with shoulder pain.
Correlation of Patient Reported Outcome Measurement Information System (PROMIS) with American Shoulder and Elbow Surgeon (ASES), and Constant (CS) scores in idiopathic adhesive capsulitis.
Relationship between the Patient-Reported Outcomes Measurement Information System (PROMIS) computer adaptive testing and legacy instruments in patients undergoing isolated biceps tenodesis.
Sabharwal S, Skolasky RL, Souza JM, Potter BK, Forsberg JA. Concurrent validity of PROMIS with DASH and DVPRS in transhumeral amputees [published online January 26, 2022]. Hand (N Y). https://doi.org/10.1177/15589447211073833.
A comparison of nerve-specific, condition-specific, and upper extremity-specific patient-reported outcome measures in patients with carpal and cubital tunnel syndrome.
Correlation of Patient-Reported Outcomes Measurement Information System questionnaires with the Brief Michigan Hand Questionnaire in patients with 5 common hand conditions.
The few longitudinal studies previously published have similarly shown that the PROMIS-UE instrument correlates with legacy instruments in patients with upper-extremity fractures,
Correlation of Patient Reported Outcome Measurement Information System (PROMIS) with American Shoulder and Elbow Surgeon (ASES), and Constant (CS) scores in idiopathic adhesive capsulitis.
Performance of Patient-Reported Outcomes Measurement Information System (PROMIS) scores compared with legacy metrics in evaluating outcomes after surgical treatment for osteochondritis dissecans of the humeral capitellum.
analyzed a similar population of patients with rotator cuff disorders as a subset in their study but compared the PROMIS Global-10 instrument with legacy outcome scores.
Our study showed that both the PROMIS-UE and ASES instruments exhibit ceiling effects at 12 months (16.42% and 20.90%, respectively) whereas the WORC instrument does not exhibit ceiling effects at any time point. Cross-sectional studies have reported differing ceiling effects, ranging from 0% for the PROMIS-UE and ASES instruments preoperatively in patients undergoing RCR to 10.82% for the PROMIS-UE instrument in the general hand patient population.
The PROMIS-UE score, which is a pure physical function score, starts showing increasing ceiling effects at 12 months, which correlates with the plateau of physical function after RCR. In contrast, the WORC score includes a mix of physical, lifestyle, and emotional functions, which may continue to show improvement beyond the 1-year mark without showing ceiling effects. Broughton et al.
Performance of Patient-Reported Outcomes Measurement Information System (PROMIS) scores compared with legacy metrics in evaluating outcomes after surgical treatment for osteochondritis dissecans of the humeral capitellum.
reported findings similar to our study findings with respect to high ceiling effects at long-term follow-up time points. These results indicate that the PROMIS-UE instrument can accurately measure physical function in patients with rotator cuff injury; however, owing to the high ceiling effects seen in patients with longer follow-up (≥1 year), the WORC instrument may be a more sensitive outcome instrument in these situations. Regarding floor effects, 6 of the 7 previously cited longitudinal studies found no significant floor effects for the PROMIS-UE instrument, which is in line with our findings.
Correlation of Patient Reported Outcome Measurement Information System (PROMIS) with American Shoulder and Elbow Surgeon (ASES), and Constant (CS) scores in idiopathic adhesive capsulitis.
Our results also indicate that although the PROMIS-UE instrument correlates well with the ASES and WORC instruments postoperatively, there are some key differences in the longitudinal responsiveness of these instruments. The PROMIS-UE instrument showed a small effect size at 6 weeks, medium to large effect size at 3 months, and large effect size at 12 months. The ASES and WORC instruments, however, exhibited medium to large effect sizes at 6 weeks and large effect sizes at both 3 and 12 months. The reason for this difference is evident when we examine the mean scores in Table 2 and the box plots in Figures 1, 2, and 3. Unlike both the ASES and WORC scores, the PROMIS-UE score did not show a statistically significant change at 6 weeks. This finding is similar to the results for the ASES-Function subdomain, which showed no significant change at 6 weeks (P = .106) but showed statistically significant increases at 3 months and 12 months (P < .001). The difference in longitudinal responsiveness between the PROMIS-UE instrument and both the ASES and WORC instruments observed in this study highlights 2 important issues that make direct comparison between these PROMs difficult: First, each of these instruments measures a different combination of domains of patient health, and second, each differs in the weight it places on each domain when the overall score is calculated. Whereas the PROMIS-UE instrument exclusively measures upper-extremity physical function, the ASES and WORC instruments cover additional domains related to pain, emotion, work, recreation, and/or lifestyle. To complicate matters further, 50% of the total ASES score is determined by a single pain question on a visual analog scale, with the other 50% determined by a 10-question function section that also includes questions about work, leisure, and sleep. The WORC instrument, on the other hand, assigns approximately 48% of its score to pain/symptoms, 19% to sports/recreation/work, 19% to lifestyle, and 14% to emotion, with questions devoted solely to physical function dispersed throughout each subsection. Improvements in 1 domain of patient health, such as pain or range of motion, will therefore affect each score in unique ways. To highlight this difference, we also calculated the correlation between the PROMIS-UE instrument and the Function subdomain of the ASES instrument (ASES-Function). As shown in Table 3, the correlation between these 2 instruments was stronger at all time points than the correlation between the PROMIS-UE score and the total ASES score. Table 4 shows that the effect sizes calculated for both the PROMIS-UE and ASES-Function scores followed a similar pattern across the 3 postoperative time points, which can be visually confirmed in the score box plots in Figures 1 and 4, respectively.
Overall, the variation in the content of these particular PROMs—and of PROMs in general—makes it difficult for researchers to compare studies that use different PROMs with one another, and it makes it difficult for clinicians to use any single PROM score to obtain an accurate picture of how a patient is faring across all domains of health. This issue is in fact one of the main motivations for the development of the PROMIS tools by the National Institutes of Health, as well as why each PROMIS instrument is devoted to a single subdomain of patient health. However, it is not certain whether additional PROMIS instruments (pain interference or pain intensity, mobility, and global) are necessary to improve the responsiveness of the PROMIS-UE instrument compared with the ASES or WORC instrument for rotator cuff disorder or other musculoskeletal disorders. More research is needed to determine the efficacy of the other available PROMIS instruments in the RCR patient population, the best combination of instruments to administer to accurately capture patient health while minimizing response burden, and the minimal clinically important difference for each PROMIS instrument to better understand the clinical importance of changes in patient scores.
Limitations
There are a few limitations of our study. First, we did not compare the PROMIS score with every available shoulder outcome score, such as the Constant score or Simple Shoulder Test score. Although other outcome scores could have been added to the study, doing so could have increased the dropout rate and possibly skewed the results owing to responder fatigue. We selected the general shoulder ASES instrument and the rotator cuff–specific WORC instrument based on their well-established excellent validity, reliability, and responsiveness, as well as their ability to be entirely completed by subjects without the input of a physician (as would be required by the physical examination section of the Constant score, for example).
Second, like all survey-based studies, this study has the potential to suffer from self-selection and/or response bias. Subjects were required to have access to a working email account (and a computer or phone) to participate in this study, so socioeconomic factors and advanced age could have limited participation and confounded our results. Although use of electronic administration is not generally thought to be a barrier to the use of PROMs, there is limited research in this area.
Third, the overall number of subjects included in this study is not large (N = 91). However, the power analysis showed that a sample size of 74 would be sufficient to detect an effect size of 0.4 between outcome scores. Fourth, we did not randomize the order in which subjects completed surveys, so responder fatigue can potentially skew the results. However, a previous study investigating PROMIS performance found no difference in results when randomizing the order of outcome instruments during survey responses.
Finally, in addition to undergoing RCR, some subjects underwent procedures such as biceps tenodesis and subacromial decompression, depending on the pathology present and surgeon preference. Although subgroup analysis of patients with and without additional procedures may show important differences, this comparison was not the intent of our study and therefore was not undertaken.
Conclusions
The PROMIS-UE instrument shows excellent correlation with the ASES instrument and a rotator cuff–specific outcome instrument—the WORC instrument—preoperatively and at 1 year after arthroscopic RCR. Variations in the measured effect sizes at different postoperative time points and high ceiling effects of the PROMIS-UE instrument at the 1-year time point may limit its utility in the early postoperative phase and at long-term follow-up after RCR.
PROMIS Upper Extremity underperforms psychometrically relative to American Shoulder and Elbow Surgeons score in patients undergoing primary rotator cuff repair.
Correlation of Patient-Reported Outcome Measurement Information System Physical Function Upper Extremity computer adaptive testing, with the American Shoulder and Elbow Surgeons shoulder assessment form and Simple Shoulder Test in patients with shoulder pain.
Correlation of PROMIS Physical Function Upper Extremity computer adaptive test with American Shoulder and Elbow Surgeons shoulder assessment form and Simple Shoulder Test in patients with shoulder arthritis.
Correlation of Patient-Reported Outcomes Measurement Information System (PROMIS) scores with legacy patient-reported outcome scores in patients undergoing rotator cuff repair.
Correlation of Patient Reported Outcome Measurement Information System (PROMIS) with American Shoulder and Elbow Surgeon (ASES), and Constant (CS) scores in idiopathic adhesive capsulitis.
Sabharwal S, Skolasky RL, Souza JM, Potter BK, Forsberg JA. Concurrent validity of PROMIS with DASH and DVPRS in transhumeral amputees [published online January 26, 2022]. Hand (N Y). https://doi.org/10.1177/15589447211073833.
A comparison of nerve-specific, condition-specific, and upper extremity-specific patient-reported outcome measures in patients with carpal and cubital tunnel syndrome.
Relationship between the Patient-Reported Outcomes Measurement Information System (PROMIS) computer adaptive testing and legacy instruments in patients undergoing isolated biceps tenodesis.
Correlation of Patient-Reported Outcomes Measurement Information System questionnaires with the Brief Michigan Hand Questionnaire in patients with 5 common hand conditions.
Preoperative psychometric properties of Patient-Reported Outcomes Measurement Information System Upper Extremity, Pain Interference, and Depression in Bankart repair and rotator cuff repair.
Performance of Patient-Reported Outcomes Measurement Information System (PROMIS) scores compared with legacy metrics in evaluating outcomes after surgical treatment for osteochondritis dissecans of the humeral capitellum.
Establishing minimally important differences for the American Shoulder and Elbow Surgeons score and the Western Ontario Rotator Cuff Index in patients with full-thickness rotator cuff tears.
The authors report the following potential conflicts of interest or sources of funding: This study was funded by NYU Langone Health; no outside funding was received. The authors certify that they have no affiliations with or involvement in any organization or entity with any financial or nonfinancial interests in the subject matter or materials discussed in this article. Y.W.K. is a paid consultant for DJO Surgical. J.D.Z. receives publishing royalties from SLACK, Wolters Kluwer Health, and Thieme; is a design surgeon for Exactech; receives royalties from Exactech; is a board member of Musculoskeletal Transplant Foundation; and owns stock in Apos Therapy and Hip Innovation Technology. M.S.V. is a paid consultant for Exactech. Full ICMJE author disclosure forms are available for this article online, as supplementary material.
00191-2&id=gr1.jpg){kind=link}
00191-2&id=gr2.jpg){kind=link}
00191-2&id=gr3.jpg){kind=link}
00191-2&id=gr4.jpg){kind=link}