Advertisement
Original Article| Volume 4, ISSUE 4, e1429-e1436, August 2022

Download started.

Ok

Reporting of Patient-Reported Outcome Measures in Randomized Controlled Trials on Shoulder Rotator Cuff Injuries Is Suboptimal and Requires Standardization

Open AccessPublished:July 14, 2022DOI:https://doi.org/10.1016/j.asmr.2022.04.032

      Purpose

      To evaluate the completeness of patient-reported outcomes (PROs) reporting using Consolidated Standards of Reporting Trials Patient-Reported Outcome (CONSORT-PRO) in randomized controlled trials (RCTs) involving rotator cuff injuries.

      Methods

      We performed a comprehensive search of MEDLINE, Embase, and Cochrane Central Register of Controlled Trials for published RCTs focused on rotator cuff injuries that included at least one PRO measure. We included RCTs published from 2006 to 2020. Investigators extracted data from RCTs using the CONSORT-PRO and evaluated each RCT using the Cochrane Risk of Bias 2.0 tool. Our primary objective was to evaluate the mean completion percentage of CONSORT-PRO. Our secondary objective used bivariate regression analyses to explore the relationship between trial characteristics and completeness of reporting.

      Results

      The initial search returned 467 results, with 33 published RCTs meeting the prespecified inclusion criteria. The mean CONSORT-PRO completeness across all included RCTs was 49.7% (standard deviation 15.43). An increase in sample size was associated with an increase in mean completeness of reporting (t = 2.31; P = .028). The Risk of Bias assessment found 29 (of 33, 87.88%) RCTs had “some concerns” for bias. We did not find any additional significant associations between completeness of reporting and trial characteristics.

      Conclusions

      Randomized controlled trials involving rotator cuff injuries frequently use PRO measures as primary outcomes. Reporting of these PRO measures is suboptimal and may benefit from rigorous standardization.

      Clinical Relevance

      PRO measures are increasingly incorporated as primary or secondary outcomes of RCTs. Appropriate reporting and use of state-of-the-art PRO measures may improve the dissemination of clinical knowledge from RCTs to guide treatment and determine intervention effectiveness. With increased adoption of Patient-Reported Outcome Measure Information System and adherence to CONSORT-PRO, orthopaedic literature may improve PRO reporting to optimize the interpretability of PROs and facilitate patient-centered care.
      Rotator cuff injuries are the most common type of shoulder disability,
      • Weber S.
      • Chahal J.
      Management of rotator cuff injuries.
      leading to approximately 4.5 million physician visits each year.
      • Oh L.S.
      • Wolf B.R.
      • Hall M.P.
      • Levy B.A.
      • Marx R.G.
      Indications for rotator cuff repair: A systematic review.
      While rotator cuff injuries can be linked to trauma, they also can be considered a part of normal degenerative changes associated with aging.
      • Tashjian R.Z.
      Epidemiology, natural history, and indications for treatment of rotator cuff tears.
      ,
      • Sher J.S.
      • Uribe J.W.
      • Posada A.
      • Murphy B.J.
      • Zlatkin M.B.
      Abnormal findings on magnetic resonance images of asymptomatic shoulders.
      It is estimated that approximately 25% of adults in their 60s and 50% of adults in their 80s have a full thickness rotator cuff tear.
      • Tashjian R.Z.
      Epidemiology, natural history, and indications for treatment of rotator cuff tears.
      ,
      • Sher J.S.
      • Uribe J.W.
      • Posada A.
      • Murphy B.J.
      • Zlatkin M.B.
      Abnormal findings on magnetic resonance images of asymptomatic shoulders.
      Symptoms of rotator cuff injuries often include pain, decreased strength, and range of motion, which can significantly limit the ability to complete activities of daily living.
      • Oh L.S.
      • Wolf B.R.
      • Hall M.P.
      • Levy B.A.
      • Marx R.G.
      Indications for rotator cuff repair: A systematic review.
      In addition to direct cost to individuals for treatment, the loss of function can lead to the inability to work, loss of income, and depression.
      • Mather 3rd, R.C.
      • Koenig L.
      • Acevedo D.
      • et al.
      The societal and economic value of rotator cuff repair.
      ,
      • Cho C.-H.
      • Seo H.-J.
      • Bae K.-C.
      • Lee K.-J.
      • Hwang I.
      • Warner J.J.P.
      The impact of depression and anxiety on self-assessed pain, disability, and quality of life in patients scheduled for rotator cuff repair.
      Furthermore, injury recurrence rates after surgical repair have been estimated at 57.8%,
      • Shin Y.K.
      • Ryu K.N.
      • Park J.S.
      • Jin W.
      • Park S.Y.
      • Yoon Y.C.
      Predictive factors of retear in patients with repaired rotator cuff tear on shoulder MRI.
      and several studies have failed to determine a significant difference in efficacy between surgical and nonsurgical management.
      • Ryösä A.
      • Laimi K.
      • Äärimaa V.
      • Lehtimäki K.
      • Kukkonen J.
      • Saltychev M.
      Surgery or conservative treatment for rotator cuff tear: a meta-analysis.
      • Kukkonen J.
      • Joukainen A.
      • Lehtinen J.
      • et al.
      Treatment of nontraumatic rotator cuff tears: A randomized controlled trial with two years of clinical and imaging follow-up.
      • Longo U.G.
      • Risi Ambrogioni L.
      • Candela V.
      • et al.
      Conservative versus surgical management for patients with rotator cuff tears: a systematic review and META-analysis.
      Taken together, the prevalence, symptomatology, and rate of reinjury highlight the need to consider patient perspectives when evaluating management strategies for rotator cuff injuries.
      An increased focus on patient-centered care has led to the desire to gather patient perspectives on treatment effects, which can be assessed through patient-reported outcomes (PROs).
      • Snyder C.F.
      • Jensen R.E.
      • Segal J.B.
      • Wu A.W.
      Patient-reported outcomes (PROs): Putting the patient perspective in patient-centered outcomes research.
      The Cochrane Collaboration defines PROs as a report directly from the patient pertaining to their treatment, health, and functional status without the interpretation by health care professionals.
      • HIGGINS J.P.
      Cochrane handbook for systematic reviews of interventions.
      PROs play a vital role in diseases, such as cancer,
      • Chen J.
      • Ou L.
      • Hollis S.J.
      A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting.
      and have been deemed useful in identifying a patient’s quality of life as they experience health deterioration, emotional stress, and the economic burden of treatment.
      • Deshpande P.
      • Sudeepthi B.
      • Rajan S.
      • Nazir C.P.A.
      Patient-reported outcomes: A new era in clinical research.
      In a study assessing physiotherapy for rotator cuff tears, the authors used the Oxford Shoulder Disability Questionnaire and showed that while patients’ pain and overall function improved, their outlook on general health had declined.
      • Ainsworth R.
      Physiotherapy rehabilitation in patients with massive, irreparable rotator cuff tears.
      Owing to the influence that a shoulder injury has on a person’s overall health, it is important that randomized controlled trials (RCTs)—which evaluate the efficacy of treatments—include PROs to assess quality of life following treatment.
      • Osoba D.
      Translating the science of patient-reported outcomes assessment into clinical practice.
      The use of PROs in orthopaedics has been increasing in recent years
      • Gagnier J.J.
      Patient reported outcomes in orthopaedics.
      ; within shoulder literature, Mosher et al.
      • Mosher Z.A.
      • Ewing M.A.
      • Collins C.S.
      • et al.
      Usage trends of patient-reported outcome measures in shoulder literature.
      found that the overall use of PRO measures had increased by 18% from 2007 to 2017. Considering this increase in patient-reported measures, the American Academy of Orthopaedic Surgeons (AAOS) has endorsed the Patient-Reported Outcome Measure Information System (PROMIS) to improve the convenience, validity, and high-quality reporting of PROs.
      • Patel A.A.
      Patient-reported outcome measures: The promise of PROMIS.
      To further enhance the reporting of PROs, the Consolidated Standards of Reporting Trials Patient-Reported Outcome (CONSORT-PRO) provides additional guidance for the promotion of high-quality reporting.
      • Calvert M.
      • Blazeby J.
      • Altman D.G.
      • et al.
      Reporting of patient-reported outcomes in randomized trials: The CONSORT PRO extension.
      The purpose of this study was to evaluate the completeness of PROs reporting using CONSORT-PRO in RCTs involving rotator cuff injuries. We hypothesized that RCTs with PROs as a primary outcome would more completely report PRO measures compared with studies with PROs as a secondary outcome.

      Methods

      Study Design

      We conducted a meta-epidemiologic study of published RCTs regarding the management of rotator cuff injuries. Data were collected by extracting information from published RCTs. This study did not involve humans and, thus, did not meet the regulatory definition for human subjects research per the U.S. Code of Federal Regulations and was not subject to institutional review board oversight. Our study adhered to the reporting guidelines for meta-epidemiological studies.
      • Murad M.H.
      • Wang Z.
      Guidelines for reporting meta-epidemiological methodology research.

      Search Strategy

      One researcher (R.O.), with the consultation of a medical research librarian, used Ovid interface to search MEDLINE, Embase, and Cochrane Central Register of Controlled Trials for published RCTs focused on the management of rotator cuff injuries. To identify RCTs, we used the Cochrane search strategy, a validated filter for OVID interfaces.
      • Lefebvre C Manheimer.
      • E Glanville.
      Searching for studies.
      The search string was uploaded to Open Science Framework.
      • Shepard S.
      • Wise A.
      • Heigle B.
      • et al.
      Rotator cuff injuries (July 2021). Open Science Framework.

      Eligibility

      RCTs that focused on the management of rotator cuff injuries with a PRO as a primary or secondary outcome were included. Studies published between 2006 and 2020 and written in English were included. We excluded the following: clinical trial protocols, clinical efficacy trials that did not have a PRO measure, observational studies, letters to the editor, animal studies, cost-effectiveness studies, secondary analyses, case reports, meta-analyses, systematic reviews, and other reviews.

      Selection Process

      The returns from our literature search were then combined and uploaded to a systematic review screening platform, Rayyan (https://rayyan.qcri.org/). Duplicates were removed and 2 authors (L.B and B.H.) screened the title and abstract in a masked, duplicate fashion. A third author (S.S.) was available to resolve disagreements but was not needed.

      Data-Collection Process

      Two authors (B.H. and T.M.) received training on how to appraise reporting completeness by applying CONSORT-PRO used in previous methodology to evaluate CONSORT-PRO adherence.
      • Calvert M.
      • Blazeby J.
      • Altman D.G.
      • et al.
      Reporting of patient-reported outcomes in randomized trials: The CONSORT PRO extension.
      ,
      • Mercieca-Bebber R.
      • Rouette J.
      • Calvert M.
      • et al.
      Preliminary evidence on the uptake, use and benefits of the CONSORT-PRO extension.
      Following training, 3 RCTs not included in our sample were extracted in a masked, duplicate fashion until consensus was achieved in an effort to reduce discrepancies. Authors (L.B. and D.N.) were trained for Risk of Bias (RoB) appraisal using Cochrane training modules.
      RoB 2.0: A revised tool to assess risk of bias in randomized trials [webinar].
      Both CONSORT-PRO and RoB were performed in a masked, duplicate fashion using pilot-tested Google forms. Upon completion of data extraction and the RoB assessment, the authors resolved all differences; a third author (S.S.) was available to resolve disagreements, but was not needed.

      Data Items

      We reported our primary objective as the mean percent completion (see scoring of CONSORT-PRO).
      • Mercieca-Bebber R.
      • Rouette J.
      • Calvert M.
      • et al.
      Preliminary evidence on the uptake, use and benefits of the CONSORT-PRO extension.
      Our secondary objective assessed relationships between the mean completeness of PRO reporting and trial characteristics. The characteristics analyzed were (1) year of publication (before or after 2014, a year following the publication of CONSORT-PRO); (2) intervention of RCT (e.g., drug or surgical technique); (3) conflict of interest statement; (4) journal endorsement of CONSORT-PRO; (5) citation of the CONSORT guideline within the publication; (6) whether an RCT used a PRO as a primary or secondary outcome; (7) RoB assessed by the Cochrane RoB 2.0 tool (see RoB assessment); (8) the length of PRO follow-up time; (9) sample size of the trial; and (10) whether the trial cited the use of PROMIS measures.
      The publishing journal’s endorsement of CONSORT guidelines were coded as follows: not mentioned, recommended, or required. This data item was evaluated by screening the instructions to authors' pages for mention of EQUATOR, CONSORT, or CONSORT-PRO guidelines.
      The following bias domains were evaluated: (1) bias arising from the randomization process, (2) bias due to deviations from intended interventions, (3) bias due to missing endpoint data, (4) bias in measurement of the endpoint, (5) bias in selection of the reported result, and (6) overall RoB assessment.

      Scoring CONSORT-PRO

      Our scoring framework follows published methodology from Mercieca-Bebber et al. 2017.
      • Mercieca-Bebber R.
      • Rouette J.
      • Calvert M.
      • et al.
      Preliminary evidence on the uptake, use and benefits of the CONSORT-PRO extension.
      Of note, items not part of the PRO extension or elaborations were not scored to better evaluate PRO-related reporting. Item 4a of CONSORT-PRO (the use of PROs in eligibility or stratification) was removed from scoring in our study as there is difficulty in the verification of this criteria. Instead, we recorded whether a study describes adherence to this item as a “yes” or “no.” We allocated a maximum value of 0.5 or 1 when information for an item was present. Items that received the maximum value (1 or 0.5 if the item is double-barreled) were considered “complete,” while items that failed to reach maximum value are reported as “not complete.” Item P1b was scored partially complete if an RCT reported the PRO measure used in the study, but failed to identify whether the PRO was a primary or secondary outcome. Therefore, Item P1b could be scored as 0, 0.5, or 1 based on the information available. Item 7a was dependent on the PRO measure being reported as a primary outcome. Because of this dependency, RCTs with a primary PRO had a maximum score of 15 versus RCTs with a secondary outcome had a maximum score of 14. A percent completeness of the checklist per RCT was calculated by adding the items and dividing by the total of possible items.

      RoB Assessment

      We used a decision algorithm developed by the Cochrane Collaboration to score RoB. If investigators had partially divergent assessments on bias domains (e.g., one investigator answered “yes” and another investigator answered “partial yes”), the overall RoB judgment for the trial outcome was not altered. The overall risk of bias domain was determined per the Excel tool provided by Cochrane as “high,” “some concerns,” or “low’ risk.”

      Data Analysis

      First, we report our search return followed by the frequencies and percentages of trial characteristics among the RCTs in our sample. Additionally, we report the frequency of RCTs that mention PROMIS measures. We then report the mean completion percentage of CONSORT-PRO across all RCTs in our sample and by outcome category (i.e., RCTs with PROs as either primary or secondary outcomes). Next, we reported on the frequency and percentage of individual items on CONSORT-PRO for all RCTs and by outcome category. Finally, to address our secondary outcome, we used bivariate regression models to determine the association between mean completion percentage of CONSORT-PRO and the trial characteristics listed in Data Items. All analyses were performed using Stata 16.1 (StataCorp, LLC, College Station, TX).

      Reproducibility

      The study protocol, data sheets, analysis scripts, a data dictionary, and extraction forms were uploaded on Open Science Framework, to ensure the transparency and reproducibility of our study. This investigation was conducted in tandem with other studies addressing completeness of reporting in other fields of medicine using similar methodology.

      Results

      General Characteristics

      Our literature search returned 467 records. After deduplication, 312 records remained and then were screened by title and abstract. After title and abstract screening, 86 RCTs were included for full-text screening and 33 published RCTs were included in our final sample. Rationales for exclusion can be found in Figure 1.
      The majority of RCTs were published after 2014 (24/33, 72.73%). The most common of the interventions studied were therapies as defined by our data dictionary (11/33, 33.33%). Fifteen (of 33, 45.45%) RCTs were published in journals that required CONSORT reporting. Among RCTs that included a conflict-of-interest statement, 18 (of 30, 60%) reported no conflicts. Thirty-one (of 33, 93.94%) RCTs did not mention the CONSORT guideline in the manuscript. When evaluating RCTs for risk of bias, we found that 1 RCT was rated as “high” (of 33, 3.03%), 29 RCTs were rated as “some concerns” (of 33, 87.88%), and 3 were rated as “low” (of 33, 9.09%). Twelve (of 33, 36.36%) RCTs had a follow-up time of 3 months or less. Of the 33 RCTs in our sample, no RCTs reported using PROMIS. Additional characteristics of RCTs are found in Table 1.
      Table 1Characteristics of Randomized Controlled Trails and Bivariate Associations With CONSORT-PRO Completion
      CharacteristicTotal 33 (100)Coef. (SE)tP
      Year of publication, No. (%)
       <20149 (27.27)1 (Ref)
       ≥201424 (72.73)–3.97, (6.08)–0.65.519
      Intervention of RCT, No. (%)
       Combination4 (12.12)1 (Ref)
       Device1 (3.03)–10.42, (17.38)–0.6.554
       Drug6 (18.18)–18.47, (10.03)–1.84.077
       Other2 (6.06)–13.75, (13.46)–1.02.316
       Surgical1 (3.03)–33.75, (17.38)–1.94.063
       Surgical technique8 (24.24)–15.83, (9.52)–1.66.108
       Therapy11 (33.33)–14.03, (9.08)–1.55.134
      Includes COI statement, No. (%)
       No statement3 (9.09)1 (Ref)
       Reports COI12 (36.36)–1.09, (10.14)–0.11.915
       Reports No COI18 (54.55)–5.93, (9.8)–0.6.55
      Journal requirement of reporting guidelines, No. (%)
       Not mentioned8 (24.24)1 (Ref)
       Recommended10 (30.3)0.14, (7.53)0.02.986
       Required15 (45.45)2.86, (6.95)0.41.684
      Mention of CONSORT or CONSORT-PRO within RCT, No. (%)
       No31 (93.94)1 (Ref)
       Yes2 (6.06)9.19, (11.32)0.81.423
      PRO as a primary or secondary outcome, No. (%)
       Primary30 (90.91)1 (Ref)
       Secondary3 (9.09)1.63, (9.49)0.17.864
      Overall ROB, No. (%)
       High1 (3.03)1 (Ref)
       Some concern29 (87.88)–11.14, (16)–0.7.492
       Low3 (9.09)–5.56, (18.16)–0.31.762
      Length of PRO follow-up
       3 months or less12 (36.36)1 (Ref)
       3+ to 6 months2 (6.06)–21.94, (11.41)–1.92.064
       6+ months to 1 year8 (24.24)4.31, (6.82)0.63.533
       1 years +11 (33.33)1.64, (6.24)0.26.795
      Sample size
       Mean (SD)85.27 (56.98)0.1, (0.04)2.3.028
      Reported PROMIS measures
       Not reported33 (100.0)
       Reported0 (0.0)
      COI, conflict of interest; CONSORT, Consolidated Standards of Reporting Trials; CONSORT-PROs; PROMIS, Patient-Reported Outcomes Measurement Information System; RCT, randomized controlled trial; SD, standard deviation.
      A P-value less than or equal to 0.05 was considered statistically significant indicated in bold.

      Completeness of Reporting According to CONSORT-PRO

      The mean CONSORT-PRO complete percentage of included RCTs was 49.70% (standard deviation [SD] 15.42). RCTs with a PRO as a primary outcome had a mean completion percentage of 49.55% (SD 14.79) and RCTs with a PRO as a secondary outcome had a mean completion percentage of 51.19% (SD 25.08).
      Throughout all the RCTs in our sample, we found that item 17aii—results include an estimate of precision—was the most completely reported item (31/33, 93.94%). Additionally, no RCTs completely reported Item 2a—rationale for PRO endpoint. Regarding RCTs with a primary PRO, the most completely reported items included: PRO identified as RCT endpoint in abstract (P1b), number of questionnaires at baseline (13ai), and results include an estimate of precision (17aii) (28/33, 93.33%). Item 2a—rationale for PRO endpoint—was not reported by any RCT with a primary PRO. All RCTs with a secondary PRO (3/3, 100%) completely reported the following items: number of questionnaires at baseline and at time of analysis (13ai and 13aii), results include an estimate of precision (17aii), and PRO study limitations (P20). No RCTs with a secondary PRO completely reported the following items: rationale for PRO endpoint (2a), PRO domains specified in the hypothesis (P2bi and P2bii), and completion of PRO questionnaire (P6aiii). We did not assess Item 7a—sample size determination—for RCTs with secondary PRO. Additional information related to completion of CONSORT PRO may be found in Table 2.
      Table 2Completion of CONSORT-PRO by Primary and Secondary Objective Designation
      Primary 30 (90.91)Secondary Endpoint 3 (9.09)Total 33 (100)
      CompleteNot CompleteCompleteNot CompleteCompleteNot Complete
      CONSORT-PRO itemn (%)n (%)n (%)n (%)n (%)n (%)
      Introduction
       P1b. Abstract—PRO as primary/secondary endpoint28 (93.33)2 (6.67)2 (66.67)1 (33.33)30 (90.91)3 (9.09)
       2a. Rationale for including PRO endpoint0 (0)30 (100)0 (0)3 (100)0 (0)33 (100)
       P2bi. PRO hypothesis present6 (20)24 (80)0 (0)3 (100)6 (18.18)27 (81.82)
       P2bii. PRO domains in hypothesis1 (3.33)29 (96.67)0 (0)3 (100)1 (3.03)32 (96.97)
      Methods
       P6ai. Evidence of PRO instrument validity26 (86.67)4 (13.33)2 (66.67)1 (33.33)28 (84.85)5 (15.15)
       P6aii. Statement of the person completing the questionnaire15 (50)15 (50)2 (66.67)1 (33.33)17 (51.52)16 (48.48)
       P6aiii. Mode of administration (paper, e-PRO)3 (10)27 (90)0 (0)3 (100)3 (9.09)30 (90.91)
       P7a. How sample size was determined (not required unless PRO is a primary endpoint)
      Item P7a only applies to PROs identified as primary outcomes.
      25 (83.33)5 (16.67)25 (83.33)5 (16.67)
       P12a. Statistical approach for dealing with missing data (imputation, exclusion, other)6 (20)24 (80)2 (66.67)1 (33.33)8 (24.24)25 (75.76)
      Results
       13ai. Report no. questionnaires submitted/available for analysis at baseline28 (93.33)2 (6.67)3 (100)0 (0)31 (93.94)2 (6.06)
       13aii. Report no. questionnaires submitted/available for analysis principal time point for analysis27 (90)3 (10)3 (100)0 (0)30 (90.91)3 (9.09)
       15. Demographics table includes baseline PRO12 (40)18 (60)1 (33.33)2 (66.67)13 (39.39)20 (60.61)
       16. Number of pts (denominator) included in each PRO analysis14 (46.67)16 (53.33)2 (66.67)1 (33.33)16 (48.48)17 (51.52)
       17ai. PRO results reported for the hypothesized domains and time point specified in the hypothesis—OR—reported for each domain of the PRO questionnaire if no PRO hypothesis provided6 (20)24 (80)2 (66.67)1 (33.33)8 (24.24)25 (75.76)
       17aii. Results include confidence interval, effect size or some other estimate of precision28 (93.33)2 (6.67)3 (100)0 (0)31 (93.94)2 (6.06)
       18. Results of any subgroup/adjusted/exploratory analyses10 (33.33)20 (66.67)1 (33.33)2 (66.67)11 (33.33)22 (66.67)
      Discussion
       P20. PRO study limitations27 (90)3 (10)3 (100)0 (0)30 (90.91)3 (9.09)
       P21. Implications of PRO results for generalizability, clinical practice6 (20)24 (80)1 (33.33)2 (66.67)7 (21.21)26 (78.79)
       22. PROs interpreted in relation to clinical outcomes12 (40)18 (60)1 (33.33)2 (66.67)13 (39.39)20 (60.61)
      CONSORT-PRO, Consolidated Standards of Reporting Trials Patient-Reported Outcomes.
      Item P7a only applies to PROs identified as primary outcomes.

      Associations Between Trial Characteristics and Completeness of Reporting

      Our bivariate regression analyses revealed that an increase in trial sample size was significantly associated with increased completeness of reporting (t1 = 2.3; P = .028). We found no other significant associations between trial characteristics and completeness of reporting. Relationships are demonstrated in Table 1.

      Discussion

      Our study identified that PRO reporting was incomplete within RCTs focused on rotator cuff injuries, with an overall mean completeness of less than 50%. Studies with PROs as secondary outcomes were found to be associated with a greater level of completeness; this finding is ambiguous due to the contrast in number of studies with PROs as primary outcomes versus secondary. One commonly underreported item in CONSORT-PRO checklist was the description of statistical methods for handling missing data. Conversely, a well reported item within RCTs was the documentation of instrument validity. Additionally, our analysis revealed that none of the included RCTs mentioned the use of PROMIS. In the following discussion, we will elaborate on our findings within the context of existing literature, address the implications of the aforementioned checklist items, and recommend the implementation of PROMIS to improve PRO reporting within orthopaedics.
      In accordance with our findings, studies have found that CONSORT-PRO completeness was suboptimal.
      • Bylicki O.
      • Gan H.K.
      • Joly F.
      • Maillet D.
      • You B.
      • Péron J.
      Poor patient-reported outcomes reporting according to CONSORT guidelines in randomized clinical trials evaluating systemic cancer therapy.
      ,
      • Coleman R.L.
      • Beck J.T.
      • Baranda J.C.
      • et al.
      The use of patient-reported outcome measures in phase I oncology clinical trials.
      Additionally, a systematic review assessing PRO reporting in multiple myeloma found similar findings with commonly omitted checklist items including the statistical plan for handling missing data (23.0% reported) and implications for generalizability and clinical practice (26.7% reported).
      • LeBlanc M.R.
      • Hirschey R.
      • Leak Bryant A.
      • LeBlanc T.W.
      • Smith S.K.
      How are patient-reported outcomes and symptoms being measured in adults with relapsed/refractory multiple myeloma? A systematic review.
      In a study looking at PROs in 23 publications related to osteoporosis, the authors found that the mean CONSORT-PRO completeness was approximately two-thirds.
      • Mack D.E.
      • Wilson P.M.
      • Santos E.
      • Brooks K.
      Standards of reporting: the use of CONSORT PRO and CERT in individuals living with osteoporosis.
      Taken together, these findings suggest reporting deficits, which can be addressed through the application of the CONSORT-PRO checklist.
      An inconsistently reported item on the CONSORT-PRO checklist was a lack of reporting of an approach to handling missing data. Nearly three-fourths of the RCTs in our sample failed to report this item. This finding is not unique to our study. An investigation of the Journal of the American Medical Association, The New England Journal of Medicine, the Lancet, and The BMJ found that 77% of RCTs published between 2013 and 2014 failed to report on appropriate handling of missing data within their publication.
      • Fielding S.
      • Ogbuagu A.
      • Sivasubramaniam S.
      • MacLennan G.
      • Ramsay C.R.
      Reporting and dealing with missing quality of life data in RCTs: Has the picture changed in the last decade?.
      Missing data can introduce bias and decrease the power of the study when researchers fail to report their statistical approach.
      • Little R.J.
      • D’Agostino R.
      • Cohen M.L.
      • et al.
      The prevention and treatment of missing data in clinical trials.
      These deficits in reporting could impair interpretation of the quality of data collection and analysis.
      • Devereaux P.J.
      • Manns B.J.
      • Ghali W.A.
      • Quan H.
      • Guyatt G.H.
      The reporting of methodological factors in randomized controlled trials and the association with a journal policy to promote adherence to the Consolidated Standards of Reporting Trials (CONSORT) checklist.
      While deficiencies in reporting were common in our sample, information regarding PRO instrument validity was well documented at 84.85% across rotator cuff injury RCTs. This finding contrasts with previous literature, suggesting that medical research may not report necessary information regarding instrument selection. In our previous example by LeBlanc et al.,
      • LeBlanc M.R.
      • Hirschey R.
      • Leak Bryant A.
      • LeBlanc T.W.
      • Smith S.K.
      How are patient-reported outcomes and symptoms being measured in adults with relapsed/refractory multiple myeloma? A systematic review.
      only 50% of studies in their systematic review of multiple myeloma literature cited validation studies for the included PROs. Additionally, a study analyzing PRO quality across head and neck cancer RCTs found that 34% of these RCTs reported evidence of PRO instrument validity.
      • Mercieca-Bebber R.L.
      • Perreca A.
      • King M.
      • et al.
      Patient-reported outcomes in head and neck and thyroid cancer randomised controlled trials: A systematic review of completeness of reporting and impact on interpretation.
      Without the use of a validated PRO instrument, clinicians are unable to assess the relevance of RCTs to patients and whether or not the PROs are appropriate for the trial design.
      • Mercieca-Bebber R.L.
      • Perreca A.
      • King M.
      • et al.
      Patient-reported outcomes in head and neck and thyroid cancer randomised controlled trials: A systematic review of completeness of reporting and impact on interpretation.
      Despite the strengths in reporting that were found in orthopaedic literature, opportunities to further improve PRO reporting exist. The endorsement of PROMIS by the AAOS offers the opportunity to overcome historical barriers associated with PRO reporting within clinical trials.
      • Brodke D.J.
      • Saltzman C.L.
      • Brodke D.S.
      PROMIS for Orthopaedic Outcomes Measurement.
      In a study assessing the use of PROMIS in upper extremity injuries—including rotator cuff injuries—PROMIS physical function computerized adaptive testing was found to be more efficient and have a strong correlation with American Shoulder and Elbow Surgeons score and the Simple Shoulder Test scores.
      • Cheung E.C.
      • Moore L.K.
      • Flores S.E.
      • Lansdown D.A.
      • Feeley B.T.
      • Zhang A.L.
      Correlation of PROMIS with Orthopaedic Patient-Reported Outcome Measures.
      Several elements which make PROMIS superior to legacy systems are: (1) computerized adaptive testing is a form of computer based questioning which is built on item-response theory to adapt to the ability level of the patient; (2) the ability to report baseline functional T-score using a standard set of outcomes across differing subspecialties in orthopaedics
      • Horn M.E.
      • Reinke E.K.
      • Yan X.
      • et al.
      Use of Patient-Reported Outcomes Measurement Information System (PROMIS) measures to characterise health status for patients seeking care from an orthopaedic provider: a retrospective cohort study.
      ; (3) the multidomain capability which PROMIS offers with its physical function, pain, emotional distress, and fatigue domains would normally require multiple scoring systems to be used.
      • Wong L.H.
      • Meeker J.E.
      The promise of computer adaptive testing in collection of orthopaedic outcomes: an evaluation of PROMIS utilization.
      Eighteen RCTs in our sample were published after the AAOS endorsement of PROMIS in 2016,
      • Patel A.A.
      Patient-reported outcome measures: The promise of PROMIS.
      and no trials in our sample incorporated PROMIS measures. Our findings are supported by a cross-sectional analysis among orthopaedic trauma literature from 2014 to 2018 looking at 319 RCTs.
      • O’Hara N.N.
      • Richards J.T.
      • Overmann A.
      • Slobogean G.P.
      • Klazinga N.S.
      Is PROMIS the new standard for patient-reported outcomes measures in orthopaedic trauma research?.
      They found only 7 trials used PROMIS measures.
      • O’Hara N.N.
      • Richards J.T.
      • Overmann A.
      • Slobogean G.P.
      • Klazinga N.S.
      Is PROMIS the new standard for patient-reported outcomes measures in orthopaedic trauma research?.
      Given these findings, there is still opportunity for the field of orthopaedics to consistently adopt PROMIS and become a leader in reporting patient outcomes.

      Recommendations

      To increase high-quality reporting in orthopaedics, we agree with Hussain et al.
      • Hussain J.A.
      • Bland M.
      • Langan D.
      • Johnson M.J.
      • Currow D.C.
      • White I.R.
      Quality of missing data reporting and handling in palliative care trials demonstrates that further development of the CONSORT statement is required: A systematic review.
      in their recommendation of emphasizing the use of the CONSORT checklist. To further emphasize the importance of RCTs abiding by CONSORT-PRO, Mercieca-Bebber et al.
      • Patel A.A.
      Patient-reported outcome measures: The promise of PROMIS.
      found that greater rates of complete reporting correlated with RCTs that cited CONSORT-PRO in their study.
      • Mercieca-Bebber R.
      • King M.T.
      • Calvert M.J.
      • Stockler M.R.
      • Friedlander M.
      The importance of patient-reported outcomes in clinical trials and strategies for future optimization.
      Despite the adoption of PROMIS by the AAOS in 2016, https://paperpile.com/c/WAvgO0/oThM use of this tool is lacking in recent studies. Following the guidance of the AAOS, we recommend the use of PROMIS to increase the quality in which PRO data is captured.
      • Patel A.A.
      Patient-reported outcome measures: The promise of PROMIS.
      Additionally, reporting standardization of PROs will allow for incorporation of convenient, value-based assessments in clinical practice.
      • Makhni E.C.
      • Hamamoto J.T.
      • Higgins J.D.
      • et al.
      How comprehensive and efficient are patient-reported outcomes for rotator cuff tears?.
      Finally, further investigation of the relationship between sample size and completeness of reporting is encouraged. Factors influencing the sample size of RCT s are numerous and may have influenced our results. These factors may include funding source, amount of funding, recruitment methods, and study design.

      Limitations

      This study is not without limitations. First, three RCTs in our sample used a PRO as a secondary outcome. This limited the assessment of our hypothesis. Second, although MEDLINE, Embase, and Cochrane Central Register of Controlled Trials database searches were performed, it is possible that all relevant RCTs were not included. Lastly, the results of our cross-sectional study should not be generalized to other qualifying studies, other journals, or studies published from other years.

      Conclusions

      Randomized controlled trials involving rotator cuff injuries frequently use PRO measures as primary outcomes. Reporting of these PRO measures is suboptimal and may benefit from rigorous standardization.

      Acknowledgments

      We are grateful to April Schweikhard, who assisted in the development of our search strategy and to the OSU medical library for their procurement of relevant literature.

      .Supplementary Material

      References

        • Weber S.
        • Chahal J.
        Management of rotator cuff injuries.
        J Am Acad Orthop Surg. 2020; 28: e193-e201
        • Oh L.S.
        • Wolf B.R.
        • Hall M.P.
        • Levy B.A.
        • Marx R.G.
        Indications for rotator cuff repair: A systematic review.
        Clin Orthop Relat Res. 2007; 455: 52-63
        • Tashjian R.Z.
        Epidemiology, natural history, and indications for treatment of rotator cuff tears.
        Clin Sports Med. 2012; 31: 589-604
        • Sher J.S.
        • Uribe J.W.
        • Posada A.
        • Murphy B.J.
        • Zlatkin M.B.
        Abnormal findings on magnetic resonance images of asymptomatic shoulders.
        J Bone Joint Surg Am. 1995; 77: 10-15
        • Mather 3rd, R.C.
        • Koenig L.
        • Acevedo D.
        • et al.
        The societal and economic value of rotator cuff repair.
        J Bone Joint Surg Am. 2013; 95: 1993-2000
        • Cho C.-H.
        • Seo H.-J.
        • Bae K.-C.
        • Lee K.-J.
        • Hwang I.
        • Warner J.J.P.
        The impact of depression and anxiety on self-assessed pain, disability, and quality of life in patients scheduled for rotator cuff repair.
        J Shoulder Elbow Surg. 2013; 22: 1160-1166
        • Shin Y.K.
        • Ryu K.N.
        • Park J.S.
        • Jin W.
        • Park S.Y.
        • Yoon Y.C.
        Predictive factors of retear in patients with repaired rotator cuff tear on shoulder MRI.
        AJR Am J Roentgenol. 2018; 210: 134-141
        • Ryösä A.
        • Laimi K.
        • Äärimaa V.
        • Lehtimäki K.
        • Kukkonen J.
        • Saltychev M.
        Surgery or conservative treatment for rotator cuff tear: a meta-analysis.
        Disabil Rehabil. 2017; 39: 1357-1363
        • Kukkonen J.
        • Joukainen A.
        • Lehtinen J.
        • et al.
        Treatment of nontraumatic rotator cuff tears: A randomized controlled trial with two years of clinical and imaging follow-up.
        J Bone Joint Surg Am. 2015; 97: 1729-1737
        • Longo U.G.
        • Risi Ambrogioni L.
        • Candela V.
        • et al.
        Conservative versus surgical management for patients with rotator cuff tears: a systematic review and META-analysis.
        BMC Musculoskelet Disord. 2021; 22: 50
        • Snyder C.F.
        • Jensen R.E.
        • Segal J.B.
        • Wu A.W.
        Patient-reported outcomes (PROs): Putting the patient perspective in patient-centered outcomes research.
        Med Care. 2013; 51: S73-S79
        • HIGGINS J.P.
        Cochrane handbook for systematic reviews of interventions.
        (Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. www.cochrane-handbook.org. Published online 2011.)
        https://ci.nii.ac.jp/naid/20000796633/
        Date accessed: June 29, 2021
        • Chen J.
        • Ou L.
        • Hollis S.J.
        A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting.
        BMC Health Services Res. 2013; 13
        • Deshpande P.
        • Sudeepthi B.
        • Rajan S.
        • Nazir C.P.A.
        Patient-reported outcomes: A new era in clinical research.
        Perspect Clin Res. 2011; 2: 137
        • Ainsworth R.
        Physiotherapy rehabilitation in patients with massive, irreparable rotator cuff tears.
        Musculoskelet Care. 2006; 4: 140-151
        • Osoba D.
        Translating the science of patient-reported outcomes assessment into clinical practice.
        J Natl Cancer Inst Monogr. 2007; : 5-11
        • Gagnier J.J.
        Patient reported outcomes in orthopaedics.
        J Orthop Res. 2017; 35: 2098-2108
        • Mosher Z.A.
        • Ewing M.A.
        • Collins C.S.
        • et al.
        Usage trends of patient-reported outcome measures in shoulder literature.
        J Am Acad Orthop Surg. 2020; 28: e774-e781
        • Patel A.A.
        Patient-reported outcome measures: The promise of PROMIS.
        J Am Acad Orthop Surg. 2016; 24: 743
        • Calvert M.
        • Blazeby J.
        • Altman D.G.
        • et al.
        Reporting of patient-reported outcomes in randomized trials: The CONSORT PRO extension.
        JAMA. 2013; 309: 814-822
        • Murad M.H.
        • Wang Z.
        Guidelines for reporting meta-epidemiological methodology research.
        Evid Based Med. 2017; 22: 139-142
        • Lefebvre C Manheimer.
        • E Glanville.
        Searching for studies.
        in: Higgins J.P.T. Green S. Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0. The Cochrane Collaboration, 2011 (Accessed June 30, 2021)
        • Shepard S.
        • Wise A.
        • Heigle B.
        • et al.
        Rotator cuff injuries (July 2021). Open Science Framework.
        (Published online June 15, 2021. Accessed July 21, 2021.)
        • Mercieca-Bebber R.
        • Rouette J.
        • Calvert M.
        • et al.
        Preliminary evidence on the uptake, use and benefits of the CONSORT-PRO extension.
        Qual Life Res. 2017; 26: 1427-1437
      1. RoB 2.0: A revised tool to assess risk of bias in randomized trials [webinar].
        (Accessed July 21, 2021.)
      2. Risk of bias tools—Current version of RoB 2.
      3. Risk of bias tools—RoB 2 for crossover trials.
      4. Risk of bias tools—RoB 2 for cluster-randomized trials.
        • Bylicki O.
        • Gan H.K.
        • Joly F.
        • Maillet D.
        • You B.
        • Péron J.
        Poor patient-reported outcomes reporting according to CONSORT guidelines in randomized clinical trials evaluating systemic cancer therapy.
        Ann Oncol. 2015; 26: 231-237
        • Coleman R.L.
        • Beck J.T.
        • Baranda J.C.
        • et al.
        The use of patient-reported outcome measures in phase I oncology clinical trials.
        Oncology. 2021; 99: 444-453
        • LeBlanc M.R.
        • Hirschey R.
        • Leak Bryant A.
        • LeBlanc T.W.
        • Smith S.K.
        How are patient-reported outcomes and symptoms being measured in adults with relapsed/refractory multiple myeloma? A systematic review.
        Qual Life Res. 2020; 29: 1419-1431
        • Mack D.E.
        • Wilson P.M.
        • Santos E.
        • Brooks K.
        Standards of reporting: the use of CONSORT PRO and CERT in individuals living with osteoporosis.
        Osteoporos Int. 2018; 29: 305-313
        • Fielding S.
        • Ogbuagu A.
        • Sivasubramaniam S.
        • MacLennan G.
        • Ramsay C.R.
        Reporting and dealing with missing quality of life data in RCTs: Has the picture changed in the last decade?.
        Qual Life Res. 2016; 25: 2977-2983
        • Little R.J.
        • D’Agostino R.
        • Cohen M.L.
        • et al.
        The prevention and treatment of missing data in clinical trials.
        N Engl J Med. 2012; 367: 1355-1360
        • Devereaux P.J.
        • Manns B.J.
        • Ghali W.A.
        • Quan H.
        • Guyatt G.H.
        The reporting of methodological factors in randomized controlled trials and the association with a journal policy to promote adherence to the Consolidated Standards of Reporting Trials (CONSORT) checklist.
        Control Clin Trials. 2002; 23: 380-388
        • Mercieca-Bebber R.L.
        • Perreca A.
        • King M.
        • et al.
        Patient-reported outcomes in head and neck and thyroid cancer randomised controlled trials: A systematic review of completeness of reporting and impact on interpretation.
        Eur J Cancer. 2016; 56: 144-161
        • Brodke D.J.
        • Saltzman C.L.
        • Brodke D.S.
        PROMIS for Orthopaedic Outcomes Measurement.
        J Am Acad Orthop Surg. 2016; 24: 744-749
        • Cheung E.C.
        • Moore L.K.
        • Flores S.E.
        • Lansdown D.A.
        • Feeley B.T.
        • Zhang A.L.
        Correlation of PROMIS with Orthopaedic Patient-Reported Outcome Measures.
        JBJS Rev. 2019; 7: e9
        • Horn M.E.
        • Reinke E.K.
        • Yan X.
        • et al.
        Use of Patient-Reported Outcomes Measurement Information System (PROMIS) measures to characterise health status for patients seeking care from an orthopaedic provider: a retrospective cohort study.
        BMJ Open. 2021; 11e047156
        • Wong L.H.
        • Meeker J.E.
        The promise of computer adaptive testing in collection of orthopaedic outcomes: an evaluation of PROMIS utilization.
        J Patient Rep Outcomes. 2022; 6: 2
        • O’Hara N.N.
        • Richards J.T.
        • Overmann A.
        • Slobogean G.P.
        • Klazinga N.S.
        Is PROMIS the new standard for patient-reported outcomes measures in orthopaedic trauma research?.
        Injury. 2020; 51: S43-S50
        • Hussain J.A.
        • Bland M.
        • Langan D.
        • Johnson M.J.
        • Currow D.C.
        • White I.R.
        Quality of missing data reporting and handling in palliative care trials demonstrates that further development of the CONSORT statement is required: A systematic review.
        J Clin Epidemiol. 2017; 88: 81-91
        • Mercieca-Bebber R.
        • King M.T.
        • Calvert M.J.
        • Stockler M.R.
        • Friedlander M.
        The importance of patient-reported outcomes in clinical trials and strategies for future optimization.
        Patient Relat Outcome Meas. 2018; 9: 353-367
        • Makhni E.C.
        • Hamamoto J.T.
        • Higgins J.D.
        • et al.
        How comprehensive and efficient are patient-reported outcomes for rotator cuff tears?.
        Orthop J Sports Med. 2017; 52325967117693223