If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Address correspondence to Mary K. Mulcahey, M.D., F.A.A.O.S., F.A.O.A., Department of Orthopaedic Surgery, Tulane University School of Medicine, 1430 Tulane Ave., New Orleans, LA 70112, U.S.A.
To obtain a quantifiable measure of the frequency with which a shoulder instability article is discussed online and the association with its corresponding bibliometric impact, based on the Scopus Cite Score (SCS) or Web of Science Impact Factor (WSIF).
Methods
The top 100 most-mentioned articles on shoulder instability based on Altmetric Attention Score (AAS) were extracted from the Altmetric Database. Mentions within blogs, news articles and outlets, public policy, and social media platforms, such as Twitter and Facebook, were included. Study impact was assessed using SCS or WSIF. The degree of association between AAS and impact was determined using Spearman correlation, logarithmic regression, and multivariate regression.
Results
The most common study designs were “Clinical Trial,” with 52 articles (49.5%), “Systematic Review” with 16 articles (15.2%), and “Review” with 10 articles (9.5%). Twitter provided more online mentions than other platforms, with the average article being discussed 27.7 times (range 0-220 times). A significant positive effect (estimate = 2.616, P = .0075) was observed between the AAS and WSIF, based on the logarithmic regression. Multivariate regression revealed that blogs help raise both WSIF and SCS (estimate = 7.272, P < .05).
Conclusions
Social media and other online platforms are a strong way to disseminate information to patients. A positive association was observed between overall online attention and the bibliometric impact of an article related to shoulder instability. Clinical trials related to shoulder instability that receive online mentions, especially discussion in blogs, are more likely to be cited in the future than their counterparts.
Clinical Relevance
The results of our study can guide authors as they aim to disseminate their articles. Twitter may be used as a tool to reach patients who may not venture into academic journals with current peer-reviewed articles. Further, blogs may be used to reach academic audiences and raise bibliometric impact broadly.
Online platforms, including social media and blogs, are among the most effective methods for disseminating health care information.
Rapidly transmitting health care information, while keeping it accessible via improved user interaction, helps patients more efficiently gain the knowledge necessary to take initiative in contributing to their treatment plans.
demonstrated that patients who receive their preferred, personalized treatment have greater satisfaction and improved health care outcomes. Based on the role and benefits of online media in patient care, it is important to take into account the social media impact and online presence of articles, as those with the greatest scores are most likely to be identified and reviewed by patients.
The established standard for evaluating an article’s impact is bibliometric analysis, which quantifies citations an article receives since the date of publication.
There are several measures that can be used to determine the impact of an article based on citations, including the Scopus Cite Score (SCS), used by the Scopus database, and Web of Science Impact Factor (WSIF), used by the Web of Science database.
They are not exact proxies for an article’s impact and are more representative of the impact of a journal’s average article. Because the SCS is calculated by Scopus and the WSIF is calculated by Web of Science, including both allows citations not included in one database, and therefore not factored into one of the scores, to be accounted for in the other.
Altmetric Attention Score (AAS) takes into account the number of times other articles cite the article in question and is an effective combined measure of an article’s online attention and impact. However, there may not always be a direct correlation between number of times cited, which is the basis of the SCS and WSIF, and AAS. For example, Celik et al. compared academic attention via citations of general oncology articles to their online attention and found no correlation between number of times cited and AAS.
Even so, weak correlations between AAS and number of citations have been observed in the fields of urology, endodontology, radiology, and spine surgery, strengthening the possibility of an association between the two.
The online availability of medical articles can influence patient understanding, including raising patient comprehension of the etiology of their disease process, rehabilitation and management, and prognosis after an intervention.
However, the association between online mentions of medical articles and their overall impact remains unclear in orthopaedic surgery. The purpose of this study was to obtain a quantifiable measure of the frequency with which a shoulder instability article is discussed online and the association with its corresponding bibliometric impact, based on the SCS or WSIF. We hypothesized that overall online attention likely would be associated with bibliometric impact of an article related to shoulder instability.
Methods
Search Strategy
The top 100 articles by AAS related to shoulder instability from October 2011 to January 2022 were identified by searching the Altmetric database for the term “shoulder instability” on January 2022.
Data Collection
The AAS is a weighted measure based on the number and type of online mentions an article has received. Sources of online attention tracked by the Altmetric database include peer-reviewed journal articles, clinical trials, dissertations, reports, conference proceedings, online data sets, manually updated online news outlets, public policy documents, and books.
Online mentions tracked by the Altmetric database are incorporated into the AAS, which adjusts for factors that could artificially inflate an article’s calculated worth (duplicate tweets and multiple mentions).
In addition to the AAS, the total number of online mentions in the news, online blogs, public policy, and social media were collected from the Altmetric database for each article. Table 1 lists the social media outlets that were included along with their Altmetric Attention Score. The SCS and the WSIF were collected from Scopus and Web of Science, respectively. To effectively use these scores, each article included was assumed to perform at or near its native journal’s average. Other collected data included publication date, journal, article type, and topic for each article.
R, an open-source statistical programming language and data analysis tool, was used to perform the quantitative aspects of the study. Spearman correlation, logarithmic regression, and multivariate regression were used to determine the degree of association between each type of online mention and an article’s SCS and WSIF. In addition, the AAS was also compared with SCS and WSIF using the tests described previously. Correlation strength was determined using previously established categories: weak correlations fall between 0.1 and 0.3, moderate correlations fall between 0.3 and 0.5, and strong correlations are greater than 0.5.
The statistical significance cut-off was set at P < .05.
Results
Search Results
During the initial search, 15 articles were tied for the 92nd position within the top 100 articles related to shoulder instability. As a result, this study included 107 articles; however, one article could not be located during the review process, leaving 106 articles for analysis.
Study Designs
Among the top 100 articles related to shoulder instability, clinical trials, not including observational studies, were the most common study design (49.5%, N = 53), followed by systematic reviews (15.2%, N = 16), and review articles (9.5%, N = 10) (Fig 1 A and B).
Fig 1(A-B) Study design for the top 100 articles related to shoulder instability.
The average AAS was 21 (range 10-104). Twitter appears to be the major contributor to an article’s AAS within the top 100 articles on shoulder instability (Table 1). Other online media platforms mentioned shoulder instability articles much less frequently compared with Twitter and Facebook (Table 2).
Table 2Average Number of Online Mentions of a Top 100 Article Related to Shoulder Instability
To assess the association between an article’s online mentions and the impact of the average article within the journal in which it was published, Spearman correlation, logarithmic regression, and multivariate regression were performed.
Spearman Correlation
The correlation coefficients between online mentions and impact of an article within the journal in which it was published are included in Table 3.
Table 3Correlation Coefficients Between Media Outlets and Impact Measures
A logarithmic transformation was applied to each of the included media outlets and the AAS. Linear regression was performed using the log-transformed variables and the impact measured by both the SCS and the WSIF (Table 4). The only significant effect was observed between the AAS and WSIF. Based on the results of the logarithmic regression, for every 1% increase in AAS, there is a 0.0262-point increase in WSIF (P = .0075). In addition, the model generated via logarithmic regression only explains about 8.1% of the variance seen in WSIF (R2 = 0.0811).
Table 4Multivariate Regression: Web of Science Impact Factor
Multivariate regression was performed to evaluate the effect of each online media outlet on bibliometric impact (WSIF and SCS), while accounting for the effects of the other online media outlets. The multivariate regression model demonstrated that online mentions in blogs were the only type of online mention that significantly affected both WSIF and SCS (Tables 4 and 5).
Being mentioned in blogs appears to lead to a 9.911-point increase in the WSIF. While this effect size is large, only 5 of the articles included here (4.7%) were mentioned in blogs. In addition, the multivariate regression model only explains 26.6% of the variance seen in the WSIF (R2 = 0.2657).
Table 5 lists the estimated effects of each type of online mention on SCS and their statistical significance based on the multivariate regression model. According to the model, one online blog mention leads to a 7.272-point increase in the SCS. However, this model only explains about 20.1% of the variance seen in the SCS (R2 = 0.2011). Similar to the results relating to WSIF, the percentage of variance explained by the model also implies that factors other than online mentions of articles influence a journal's impact based on the SCS. Therefore, like the WSIF, it is unclear whether blogs alone impact SCS despite the statistically significant result.
Discussion
Clinical trials made up the majority of the top 100 articles related to shoulder instability. An association was revealed between AAS and both bibliometric impact measures (WSIF and SCS). Blogs appear to have a significant positive effect on both bibliometric impact measures.
Clinical trials were the most common study design within the top 100 shoulder instability articles, making up 49.5% of the total. This indicates that clinical trials receive more online attention than other article types. The observed increased attention received by clinical trials is consistent with other publications in orthopaedics. For instance, Parrish et al.
determined that clinical trials were the most common article type within the top 100 articles by AAS in spinal surgery. The prominence of clinical trials observed here for articles related to shoulder instability is also consistent with the results observed in a study by Barbic et al.,
in which the authors analyzed the Altmetric performance of the 200 most-cited emergency medicine articles in the top 10 emergency medicine journals according to the 2011 Journal citation report. However, there are some specialties with differing results. For example, an endodontologic Altmetric study by Kolahi et al.
determined that meta-analyses and systematic reviews received the most online attention. The origin of the discrepancies within article type popularity between specialties is currently unknown and holds potential for future research.
The most common online mention outlet within the top 100 was Twitter. Parrish et al.
also determined that Twitter is the most common outlet within the top 100 spinal surgery articles by online attention. The prevalence of Twitter in disseminating research articles is not limited to articles within orthopaedic surgery. A recent study by Wang et al. evaluated the top 100 neurosurgery articles and found that Twitter was the most prominent source of online attention.
Overall, Twitter appears to be a ubiquitous source of online media attention for medical articles. Future research may investigate patient preferences of how to access online information and their understanding of the articles shared.
A weak correlation was observed between AAS and both measures of bibliometric impact used in this study. Similarly, weak correlations between AAS and the bibliometric impact measures were observed in Altmetric papers within urology, endodontology, radiology, and spine surgery.
Because the discovered association is weak, it was difficult to determine definitively if there was an association between AAS and bibliometric impact. Therefore, this association was further explored via logarithmic regression. When considering the aforementioned studies’ results, it is important to note that these studies did not assess the correlation between AAS, WSIF, and SCS scores. Instead, citation count was the primary focus. Although citation count is a different metric, WSIF and SCS are calculated based on citation count, making these results analogous, although not ideal one-to-one comparisons.
To definitively assess the weak correlation between the log-transformed variables, AAS scores, and bibliometric impact using WSIF and SCS, a linear regression was performed. The results disclosed a significant effect between AAS and WSIF (for every 10% increase in AAS, there is a 0.262-point increase in WSIF). The study by Parrish et al.
evaluating the top 100 spinal surgery articles by AAS and the social media outlets used for their dissemination did not observe a similar statistically significant relationship between AAS and WSIF, making the relationship between AAS and WSIF potentially unique to shoulder instability. Furthermore, the same effect has not been observed in the previously discussed studies focused on the AAS of articles in other medical fields (e.g., urology, endodontology, and radiology).
The models generated by the multivariate regression revealed that blogs were the only type of online mention that had a significant effect on both WSIF and SCS. However, blogs were relatively rare when compared with the other online media types. Therefore, it is possible that these results occurred due to other factors. In addition, there is evidence in the literature that supports the positive effects of blogs on the citation of articles. For instance, a multidisciplinary study of scientific articles conducted by Costas et al.
on the relationship between Altmetrics and citations determined that blogs play a crucial role in identifying highly cited publications. More specifically, the study determined that online attention in blogs had a stronger correlation with citations than in other media outlets.
Limitations
There are several limitations to this study. First, there is a temporal association between online attention and the date of a manuscript’s publication. More recent articles will receive more attention. Second, the AAS is susceptible to change, and an article’s attention can decline over time.
However, older articles are more likely to be cited than recent publications and will carry more impact than recent articles. Therefore, the long-term bibliometric impact of recently published articles is unclear and is susceptible to change. Some online media outlets, such as Instagram, were not included in the calculation of AAS by Altmetric.
As a result, AAS is not a perfectly comprehensive measure of online attention. Finally, factors beyond online attention may play a role in defining an article’s bibliometric impact. For instance, the notoriety and the nationality of the first author can play a role in how often an article is cited.
Therefore, although the findings offered by this study propose possible advantages of specific online outlets, such as blogs and Twitter, in promoting bibliometric impact, their effectiveness may be mediated by other factors.
Conclusions
Social media and other online platforms are a strong way to disseminate information to patients. A positive association was observed between overall online attention and the bibliometric impact of an article related to shoulder instability. Clinical trials related to shoulder instability that receive online mentions, especially discussion in blogs, are more likely to be cited in the future than their counterparts.
The authors report the following potential conflicts of interest or sources of funding: M.K.M. reports board or committee member, AAOS; editorial or governing board, American Journal of Sports Medicine Electronic Media; board or committee member, American Orthopaedic Association and American Orthopaedic Society for Sports Medicine; paid consultant, Arthrex; editorial or governing board, Arthroscopy; board or committee member, Arthroscopy Association of North America; editorial or governing board, Ortho Info; and board or committee member, Ruth Jackson Orthopaedic Society and The Forum. Full ICMJE author disclosure forms are available for this article online, as supplementary material.
Study performed at the Tulane University School of Medicine, New Orleans, Louisiana, U.S.A.
00065-7&id=gr1.jpg){kind=link}