If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
University of the Incarnate Word, School of Osteopathic Medicine, San Antonio, Texas, U.S.A.Burkhart Research Institute for Orthopaedics, San Antonio, Texas, U.S.A.
University of the Incarnate Word, School of Osteopathic Medicine, San Antonio, Texas, U.S.A.Burkhart Research Institute for Orthopaedics, San Antonio, Texas, U.S.A.
UT Health San Antonio, Department of Orthopaedic Surgery, San Antonio, Texas, U.S.A.Burkhart Research Institute for Orthopaedics, San Antonio, Texas, U.S.A.
To characterize the quality of YouTube total shoulder arthroplasty videos as a source of patient information using the DISCERN instrument.
Methods
An analysis of the YouTube video library was performed, using a string of 6 search terms related to "total shoulder replacement" and "total shoulder arthroplasty" in the YouTube search engine. The first 20 videos from each search (n = 120) were selected. The top 25 most viewed videos were compiled, screened, and evaluated with the DISCERN score in the final analysis. Pearson’s correlation coefficients were used to assess the correlation of DISCERN scores and video characteristics. Inter-rater reliability was calculated with the conger kappa score for multiple raters.
Results
Twenty-five videos met inclusion criteria, 13 (52%) were produced by academic institutions, 7 (28%) by physicians, and 5 (20%) by commercial entities. Median total DISCERN score was 33 out of 80 (IQR: 28-44). The overall total DISCERN scores, showed no correlation with video likes or views and was negatively correlated with video power index (r = −0.75, P = .001). No association between total shoulder arthroscopy video source and DISCERN score could be demonstrated. All videos analyzed scored poorly by the DISCERN instrument.
Conclusions
The current most popular shoulder replacement videos on YouTube are low-quality patient education resources. Furthermore, our study found no correlation between video popularity, as measured by the number of views and the DISCERN score.
Clinical Relevance
Successful outcomes following total shoulder arthroplasty may be influenced by the quality of information patients receive.
Introduction
Modern-day medicine faces many challenges in providing adequate, understandable, and high-quality online patient education. This is mostly due to the increased time and effort required to regulate online educational resources, the variability of content, and low health literacy in the United States.
With more than 80% of patients obtaining health-related information from publicly available online resources, the Internet has become the dominant source for patients looking to learn about their medical conditions.
With patient education becoming more accessible on an individual level, understanding the material is vital for the patient to fully comprehend the necessary information to make informed decisions.
Patients and physicians should familiarize themselves with the quality of the resources due to the variety of options available. High-quality preoperative patient education has been shown to reduce patient anxiety, improve pain management, and associated expectations, prevent harmful misunderstandings, and identify barriers to treatment that may be addressed before surgery.
YouTube is the second-largest internet search engine behind Google and is one of the most popular and accessible informational resources on the Internet. However, because of YouTube’s free and unregulated structure, the quality and content of publicly available health-related YouTube videos can be questionable.
Despite YouTube’s popularity as a source of patient education, there is minimal information on the quality of online video content for total shoulder replacement.
The purpose of this study was to characterize the quality of YouTube total shoulder arthroplasty videos as a source of patient information using the DISCERN instrument. We hypothesized that the most popular videos on YouTube would not be of the highest quality and, therefore, lack essential information.
Methods
Search Method
An analysis of the YouTube video library was performed on June 14, 2022, using a string of 6 search terms related to “total shoulder replacement” and “total shoulder arthroplasty” (Fig 1) in the YouTube search engine. The first 20 videos from each search (n = 120) were selected. The search was conducted on the Chrome Browser using Incognito mode for cache clearance to minimize biased results for individual reviewers.
Videos shorter than 60 seconds, longer than 20 minutes, audio-only, not about total shoulder arthroscopy (TSA), and in languages other than English were excluded from this study.
Source of Videos
On the basis of their source, the videos were categorized as follows: 1) physician or physician group, 2) healthcare organization/hospital, and 3) commercial entity. Physician videos were uploaded by physicians or groups of physicians in private practices. Video uploads associated with medical centers, hospitals, and professional medical societies were defined as healthcare organization/hospital videos. Commercial videos were defined as those uploaded by a company with the intention of promoting and/or selling a product.
Video Characteristics
The following descriptive data were analyzed and documented for each video: 1) video length, 2) number of views, 3) number of days since upload, 4) view ratio (defined as views/day), 5) number of likes, 6) number of dislikes, 7) like ratio, (the number of likes × 100)/(the number of likes + number of dislikes), and 8) video power index. VPI is an index used to assess a video’s popularity based on the number of views and likes, and it has been investigated by prior research.
Online videos provide poor information quality, reliability, and accuracy regarding rehabilitation and return to sport after anterior cruciate ligament reconstruction.
The VPI is calculated by dividing (like ratio × view ratio) by 100. Since YouTube does not have a system for rating video popularity, the VPI has been used as an alternative.
A fellowship-trained orthopaedic shoulder surgeon and an orthopaedic surgery research resident used the DISCERN instrument to evaluate video quality and analysis. The DISCERN score is a validated tool that evaluates the quality of consumer information using a 16-question survey to provide a standard approach for evaluating the quality of written consumer health information on treatment options.
The 16 questions were categorized in 3 sections, with questions that evaluated 1) credibility, 2) quality and content, and 3) overall general score. These sections were scored from 1 (poor reliability) to 5 (highest reliability) for a total possible score of 80 points, yielding grades of “very poor” (16 to 28), “poor” (29 to 41), “fair” (42 to 54), “good” (55 to 67), and “excellent” (68 to 80). The reviewers (R.H. and J.G.) were blinded to the number of views per video, as they evaluated them in a randomized order. The reviewers then scored the videos independently, and the scores were averaged.
Statistical Analysis
Statistical analyses were performed in R version 4.2.0 using the RStudio integrated development environment version 2022.07.2 (Build 576).
R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/. Accessed November 10, 2022.
Continuous variables are reported as medians and interquartile ranges, whereas categorical and ordinal variables are reported as frequencies and percentages of the total cohort. The corrr package version 0.4.4 was used to calculate Pearson’s correlation coefficients among DISCERN scores and video characteristics.
Inter-rater reliability estimates for the DISCERN scores, and subscales were calculated using Conger’s quadratic-weighted kappa for 2 raters using the irrCAC package version 1.0.
One-hundred twenty videos were screened for inclusion and 45 met initial inclusion criteria. Secondary to the constraint of author time and availability, the top 25 most viewed videos out of the pool of 45 were used. Thirteen of the top 25 videos were categorized as academic, 7 as physician, and 5 as commercial. The median number of views per TSA video was 34,891, with a high of 76,906 and a low of 14,791 (Table 1). The medium length of a video was 317 seconds (IQR: 225-641). The days since video upload was 1,693 (IQR: 1,245-2,693). Table 1 describes the remaining video properties.
Commercial sources (n = 5) reported the highest median number of views per video with 97,156 (IQR: 29,404-237,515), whereas physician-produced videos (n = 7) had reported the lowest number of views, at 9,605 (IQR: 6,854-45,280) (Table 2). Physician videos had 2,872 days (IQR: 1,832-3084) on YouTube after video posting, the most of any source, while academic sources had the fewest at 1,513 (IQR: 1,090-1,693). Commercial videos had the longest median video length at 543 seconds (IQR: 225-641), followed by academic videos with a duration of 415 seconds (IQR: 231-840), while physician videos had the shortest video length at 242 seconds (IQR: 190-258). Table 2 includes additional information on videos by sources.
The overall median DISCERN score for YouTube TSA videos was 33 (IQR: 28-44) of 80 possible points, as shown in Table 3. No videos scored a 5, which is the maximum points within the individual general score of the videos, while only 2 videos scored a 4/5. The overall total DISCERN scores, showed no correlation with video likes, or views and was negatively correlated with VPI (r = −0.75; P = .001). In fact, the DISCERN general score was negatively correlated with VPI, videos likes or views (r = −.012; P = .001). When categorized by video sources, the mean total DISCERN scores for YouTube TSA videos were low scoring and showed no significant differences between sources: physician 34 (29, 42), academic 33 (30, 45), and commercial videos 32 (25, 32) (Table 2).
DISCERN scoring demonstrated substantial agreement with inter-rater reliability of 0.67 and 0.73 for the overall and general DISCERN scores, respectively.
Discussion
In this study, we found that the most commonly viewed videos about shoulder arthroplasty on YouTube are of low-quality for patient education. Furthermore, there was no correlation between popularity measured by the number of views and the DISCERN score. With more patients turning to the Internet for health information and the prevalence of TSA surgeries rising at a faster rate than the most common orthopaedic procedures, such as total knee and total hip arthroplasty
Previous research has highlighted the low quality and consistency of information accessible when examining educational health material on YouTube. Celik et al. examined the educational content on YouTube regarding rotator cuff repair (RCR).
Their evaluation of 67 videos using DISCERN found consistently poor scores. The average DISCERN score was 30.5, indicating that rotator cuff repair videos had essential and potentially serious shortcomings. In contrast, Ng et al. found that YouTube videos on total knee and hip arthroplasty videos scored an average of 54.7 on the DISCERN, deeming them of “fair-good” quality.
In 2025, 1,300,000 primary TKA and 650,000 primary THA procedures are projected to be performed in the United States versus 175,000-350,000 predicted TSA.
Thus, the differences in quality between TKA/THA versus TSA could be potentially attributed to more video producers for the former with an incentive to create high-quality content. It is evident that there is a need for improved TSA educational resources.
An assessment of the quality of information regarding the return to sports following hip arthroscopy showed that physician uploads scored the highest in quality of content.
The overall quality of these videos demonstrates the potential importance and areas of improvement for video creators. Our investigation did not find a significant difference in the DISCERN score among video sources. Although, educational videos produced by physicians comprised 28% of all the TSA videos, they were not superior to the other sources. Interestingly, our study reported a video by Stanford Health Care ranked first with a 55 out of 80 (69%) score. Orthopaedic shoulder surgeons, either individually or through societies or other group efforts, should develop improved patient-focused videos published on YouTube that adhere to a standard of accuracy and quality to address many of the deficiencies identified in this study.
Prior research has attempted to assess YouTube videos based on their VPI. In a study examining the quality of 122 YouTube videos on arteriovenous malformations (AVM), the videos with the greatest VPI had the lowest DISCERN score, indicating that the most popular videos on AVMs are not of the highest educational value.
Additionally, an evaluation of gastrectomy sleeve videos discovered that those produced by surgeons had a higher DISCERN score and a significantly lower VPI.
Evaluation of the reliability, utility, and quality of the information in sleeve gastrectomy videos shared on open access video sharing platform YouTube.
These findings support our study, as we discovered a negative correlation between VPI and total DISCERN score, and no correlation between the number of views and DISCERN score. The physician sources in our study had the lowest VPI score of the sources (11) but produced the highest quality of educational content. In contrast, commercial videos reported the highest VPI but the lowest DISCERN score. These findings indicate that VPI may be more suitable for entertainment than as a meaningful instrument for medical education.
This study has a few limitations. First, videos are frequently developed with various objectives, such as encouraging symptomatic individuals to seek medical care or providing insight on preoperative, intraoperative, and postoperative measures. Thus, the quality of the content may vary based on the target audience, making it a challenge to compare the overall quality of videos.
The study was limited by only analyzing the top 25 most viewed videos on YouTube. However, it has been reported that Internet users use only the top 10 search results.
How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews.
Furthermore, the quality of the content of other orthopaedic-related videos is consistent with our findings. Lastly, some researchers propose establishing a more reliable methodology for evaluating YouTube video content quality.
The current most popular shoulder replacement videos on YouTube are low-quality, patient education resources. Furthermore, our study found no correlation between video popularity, as measured by the number of views and the DISCERN score.
Online videos provide poor information quality, reliability, and accuracy regarding rehabilitation and return to sport after anterior cruciate ligament reconstruction.
R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/. Accessed November 10, 2022.
Evaluation of the reliability, utility, and quality of the information in sleeve gastrectomy videos shared on open access video sharing platform YouTube.
How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews.
The authors report the following potential conflicts of interest or sources of funding: R.U.H. reports board or committee membership in American Shoulder and Elbow Surgeons and Arthroscopy Association of North America; editorial or governing board membership in Arthroscopy; consulting fees from Stryker; publishing royalties and financial or material support from Wolters Kluwer Health – Lippincott Williams & Wilkins, outside the submitted work. Full ICMJE author disclosure forms are available for this article online, as supplementary material.
00040-8&id=gr1.jpg){kind=link}