Advertisement
Original Article|Articles in Press

Comparison of Sports Medicine Questions on the Orthopaedic In-Training Examination Between 2009 and 2012 and 2017 and 2020 Reveals an Increasing Number of References

Open AccessPublished:February 28, 2023DOI:https://doi.org/10.1016/j.asmr.2023.01.018

      Purpose

      To provide an updated analysis of the sports medicine section of the Orthopedic In-Training Examination (OITE).

      Methods

      A cross-sectional review of OITE sports medicine questions from 2009 to 2012 and 2017-2020 was performed. Subtopics, taxonomy, references, and use of imaging modalities were recorded and changes between the time periods were analyzed.

      Results

      The most tested sports medicine subtopics included ACL (12.6%), rotator cuff (10.5%), and throwing injuries to the shoulder (7.4%) in the early subset, while ACL (10%), rotator cuff (6.25%), shoulder instability (6.25%), and throwing injuries to the elbow (6.25%) were the most common in the later subset. The American Journal of Sports Medicine (28.3%) was the most cited journal referenced from 2009 to 2012, while The Journal of the American Academy of Orthopaedic Surgeons (17.5%) was most referenced in questions from 2017 to 2020. The number of references per question increased from the early to the late subset (P < .001). There was a trend toward an increased taxonomy type one questions (P = .114), while type 2 questions had a decreased trend (P = .263) when comparing the new subset to the early group.

      Conclusion

      When comparing sports medicine OITE questions from 2009 to 2012 and 2017 to 2020, there was an increase in the number of references per question. Subtopics, taxonomy, lag time, and use of imaging modalities did not show statistically significant changes.

      Clinical Relevance

      This study provides a detailed analysis of the sports medicine section of the OITE, which can be used by residents and program directors to direct their preparation for the annual examination. The results of this study may help examining boards align their examinations and provide a benchmark for future studies.

      Introduction

      The Orthopaedic In-Training Examination (OITE) was developed by the American Academy of Orthopaedic Surgeons (AAOS) in 1963 to help determine whether each residency program was maintaining minimum standards for orthopedic education and achieving educational goals.
      • Luginbuhl J.C.
      • Sobel A.D.
      • Mulcahey M.K.
      Analysis of the sports medicine section of the Orthopaedic In-Training Examination: Improvements in levels of evidence and question taxonomy over a 12-year period.
      The OITE focused on addressing two of the six core competencies of residency training, as recommended by the Accreditation Council for Graduate Medical Education (ACGME) and American Board of Medical Specialties (ABMS): patient care and medical knowledge.
      • Batalden P.
      • Leach D.
      • Swing S.
      • Dreyfus H.
      • Dreyfus S.
      General competencies and accreditation in graduate medical education.
      ,
      • Ganesh Kumar N.
      • Benvenuti M.A.
      • Drolet B.C.
      Aligning in-service training examinations in plastic surgery and orthopaedic surgery with competency-based education.
      At present, the exam consists of 275 questions evaluating 11 different domains, including basic science, foot and ankle, hand, hip and knee, oncology, pediatrics, shoulder and elbow, spine, sports medicine, trauma, and practice management. The test is currently administered in computer format to over 4,000 residents across 20 countries.
      • DePasse J.M.
      • Haglin J.
      • Eltorai A.E.M.
      • Mulcahey M.K.
      • Eberson C.P.
      • Daniels A.H.
      Orthopedic in-training examination question metrics and resident test performance.
      At the conclusion of residency, graduating orthopaedic residents take the ABOS (American Board of Orthopaedic Surgery) Board Certification exams. Although several studies have found a correlation between OITE performance and passing the ABOS Board Certification examinations, these two examinations have historically been written by different organizations without a common blueprint.
      • Swanson D.
      • Marsh J.L.
      • Hurwitz S.
      • et al.
      Utility of AAOS OITE scores in predicting ABOS Part I outcomes: AAOS exhibit selection.
      • Herndon J.H.
      • Allan B.J.
      • Dyer G.
      • Jawa A.
      • Zurakowski D.
      Predictors of success on the American Board of Orthopaedic Surgery examination.
      • Klein G.R.
      • Austin M.S.
      • Randolph S.
      • Sharkey P.F.
      • Hilibrand A.S.
      Passing the Boards: Can USMLE and Orthopaedic in-Training Examination scores predict passage of the ABOS Part-I examination?.
      • Dougherty P.J.
      • Walter N.
      • Schilling P.
      • Najibi S.
      • Herkowitz H.
      Do scores of the USMLE Step 1 and OITE correlate with the ABOS Part I certifying examination? A multicenter study.
      • Fritz E.
      • Bednar M.
      • Harrast J.
      • et al.
      Do orthopaedic in-training examination scores predict the likelihood of passing the American Board of Orthopaedic Surgery part I examination? An update with 2014 to 2018 data.
      One multicenter study found that 57% of residents who scored at or below the 27th percentile on their OITE failed their Part I ABOS Certifying Examination.
      • Dougherty P.J.
      • Walter N.
      • Schilling P.
      • Najibi S.
      • Herkowitz H.
      Do scores of the USMLE Step 1 and OITE correlate with the ABOS Part I certifying examination? A multicenter study.
      Most recently, a review of scores on the OITE and Part I Certifying Examination from 2014 to 2018 found an increasing correlation between examination performance with increased residency training, noting a correlation of 0.504 for senior residents (PGY4 and PGY5s). However, a minimum OITE score correlated with passing the Part I Certifying exam was not identified.
      • Fritz E.
      • Bednar M.
      • Harrast J.
      • et al.
      Do orthopaedic in-training examination scores predict the likelihood of passing the American Board of Orthopaedic Surgery part I examination? An update with 2014 to 2018 data.
      The value of the OITE has been questioned because of its lack of alignment with the Part I Certifying Exam. Recently, the ABOS and AAOS have begun to collaborate “to identify the score on the AAOS OITE that approximately corresponds to the minimum passing performance level on the ABOS Part I Certifying Examination” by developing questions included on both examinations. A linking study found that on the 2021 AAOS OITE, the minimum OITE score that corresponded to passing the Part I Certifying Exam was 69.2%, but this was noted to be applicable to the 2021 OITE examination only. In 2021, the AAOS adopted a new examination blueprint developed by the ABOS for the purpose of test development to further align the OITE with the Part I Certifying examination.
      Therefore, it has become more important for orthopaedic residency programs to guide resident learning in preparation for the annual OITE examination. Our study asked 1) Have the most common subsections of sports medicine questions changed? 2) How has question taxonomy changed? 3) Has there been a change to the most commonly referenced sources and the number of references utilized to support each question? 4) Has there been a change in lag time of reference publication to examination date? 5) Has there been a change in frequency of imaging modalities used in question stems? The purpose of this study was to provide an updated analysis of the sports medicine section of the OITE. Our team hypothesized that there would be no change in subsections, taxonomy, reference sources, lag time, or use of imaging modalities between the two question subsets.

      Methods

      A cross-sectional review of all OITEs from 2009 to 2012 and from 2017 to 2020 were performed by the authors. Examinations from 2013 to 2016 were previously reviewed by Synovec et al., and analysis of trends of these years was completed using their data.
      • Synovec J.
      • Plumblee L.
      • Barfield W.
      • Slone H.
      Orthopedic in-training examination: An analysis of the sports medicine section—An update.
      Questions from 2009 to 2012 were obtained from past examinations as distributed by the AAOS. These questions were not designated for subsection by the test writers, and thus, two independent reviewers (BK & ML) determined which questions would be designated as sports medicine. Any discrepancies were reviewed by a third author (PBW) to determine appropriateness for the sports medicine section. Overall, 8.6% (95/1100) of these questions were determined by the reviewers to be part of the sports medicine section. Questions from years 2017 to 2020 were obtained directly from the AAOS website via the “ResStudy” online portal.
      American Academy of Orthopaedic Surgeons
      Orthopaedic in-Training Examination (OITE). Orthopaedic In-Training Examination (OITE).
      The authors reviewed the OITE examinations from these years, which were designated by category, including sports medicine. All questions not included in this sports medicine question were also reviewed in the event that one of these questions was classified into a different category by AAOS. Overall, 7.5% of these questions were designated by AAOS as part of the sports medicine section.
      American Academy of Orthopaedic Surgeons
      Orthopaedic in-Training Examination (OITE). Orthopaedic In-Training Examination (OITE).
      All OITE questions from the years 2009 to 2012, and from 2017 to 2020, were further broken down into specific subsections. The specific subsections used in our study were based off Synovec et al., who previously analyzed the most tested sports medicine questions on the OITE.
      • Synovec J.
      • Plumblee L.
      • Barfield W.
      • Slone H.
      Orthopedic in-training examination: An analysis of the sports medicine section—An update.
      These subsections included anterior cruciate ligament (ACL), shoulder instability, and rotator cuff (RTC), among others.
      Question taxonomy was determined for all questions using the system developed by Buckwalter et al.
      • Buckwalter J.A.
      • Schumacher R.
      • Albright J.P.
      • Cooper R.R.
      Use of an educational taxonomy for evaluation of cognitive performance.
      Taxonomic I (T1) questions tested isolated recognition and recall of specific information; taxonomic II (T2) was assigned to those testing diagnostics and interpretation of imaging and translated it into another concept in the question stem; taxonomic III (T3) was assigned to those testing application of knowledge or interpreting information to solve a problem. The questions were categorized by two authors (B.K. and M.L.), and any disputes among taxonomy classification were determined by a third author (P.B.W.).
      The references cited for each question were analyzed. The authors noted the most referenced journals and total references cited per question. The authors also calculated the length of time from article publication to appearance on the OITE exam, defined as lag time. For example, if a reference was published 2003 and was used on the 2010 OITE examination, the lag time would be calculated as 7 years. The authors made note of which questions included imaging and which type of imaging modality (i.e., computed tomography [CT] scan, radiograph, magnetic resonance imaging [MRI]) were utilized.
      Data including subtopic, question taxonomy, number of references, most common references, lag year of references, and use of imaging modalities were analyzed and reported. Statistical analysis was performed using Excel and Graphpad. Continuous data between two groups were compared with independent 2-tailed t-tests and 2-sample Z test for proportions. Categorical data for groups were compared via chi square tests. Descriptive statistics are described as means with standard deviations and proportions.

      Results

      In review of the 8 years of Orthopedic In-Training Examination included in this study (2009-2012; 2017-2020), there was a total of 175 sports medicine questions. Our analysis reveals that between 2009 and 2012, the average percentage of sports medicine questions per year was 8.6% (range: 8%-9.45%), compared to 7.5% between 2017 and 2020, (range: 6.18%-9.36%) (P = .136). When recent years were compared to 2013-2016 data compiled by Synovec et al. (average 7.73%), there was no significant difference found (P = .689). With inclusion of this previously published data, there was no statistical difference in sports medicine questions from 2009 through 2020 (r = .453, P = .139) (Fig 1).
      Figure thumbnail gr1
      Fig 1The percentage of sports medicine questions on the Orthopaedic In-Training Examination varies each year. Trends in the number of sports medicine questions on each exam can influence residents’ degree of focus on this section. Data from 2013 to 2017 were obtained from literature previously published by Synovec et al.
      • Synovec J.
      • Plumblee L.
      • Barfield W.
      • Slone H.
      Orthopedic in-training examination: An analysis of the sports medicine section—An update.
      Specific subtopics of individual sports medicine questions were further analyzed. From 2009 to 2012, the most tested subjects were ACL (12.6%), rotator cuff (10.5%), and throwing injury to shoulder (7.4%). From 2017 to 2020, the most tested subjects were ACL (10.0%), rotator cuff (6.25%), shoulder instability (6.25%), and throwing injury to elbow (6.25%) (Table 1). Changes in the proportion of these common subtopics between time periods was not found to be significant for ACL (P = .741), rotator cuff (P = .312), throwing injury to the shoulder (P = 0.711), throwing injury to the elbow (P = .542), and shoulder instability (P = .542). The review by Synovec et al. found that from 2013 to 2016, the most tested subtopic was related to the ACL (16.47%), followed by patient safety (5.88%), rotator cuff (4.71%), shoulder instability (4.71%), and stress fractures (4.71%). When the data from Synovec et al. were compared to our recent exam group, there was no significant difference in the proportion of any of the aforementioned subtopics; ACL (P = .222), rotator cuff (P = .660), throwing injuries to the shoulder (P = .153), throwing injuries to the elbow (P = .215), or shoulder instability (P = .660). Differences in proportions of questions related to patient safety and stress fractures were unable to be evaluated, as these questions were categorized differently in this study.
      Table 1Sports Medicine Question Subtopics
      Subtopic BreakdownTotalTotalTotal
      Knee2009-20122013-2016∗2017-2020
      ACL11178
      Meniscus repair0N/A2
      Patellofemoral pain131
      Tibial tubercle fracture0N/A1
      Patellofemoral instability132
      Meniscus pathology242
      Osteochondral defect2N/A2
      MPFL2N/A2
      Quad contusion1N/A1
      Multi-ligament repair412
      HTO0N/A1
      PCL2N/A0
      Osteochondritis Dissecans210
      Hamstring pathology120
      Patellar tendon rupture110
      LCL injury1N/A0
      MCL anatomy1N/A0
      OtherN/A6N/A
      Shoulder
      Throwing injury624
      Distal clavicle osteolysis0N/A1
      Rotator cuff1055
      Shoulder instability465
      Shoulder dislocation612
      Multidirectional instability2N/A0
      AC joint sprain1N/A0
      Spinoglenoid cyst120
      OtherN/A2N/A
      Elbow
      Throwing injury425
      Biceps anatomy1N/A0
      Biceps pathology111
      LUCL injury121
      Osteophyte resection100
      Lateral epicondylitis011
      OtherN/A3N/A
      Foot/Ankle
      Achilles tendon1N/A0
      Lateral ankle instability1N/A0
      Anterior ankle impingement0N/A1
      Navicular bone stress fracture0N/A1
      Turf toe0N/A1
      OtherN/A1N/A
      General Medical Condition/other
      Hypertrophic cardiomyopathy110
      Female athlete triad1N/A0
      Sports hernia2N/A0
      Stats1N/A0
      Occult fracture in runner1N/A0
      Gamekeeper thumb1N/A0
      Exercise science3N/A0
      Overuse injury in young athlete114
      Surgical time-out0N/A2
      Tooth avulsion0N/A1
      Mallet finger0N/A1
      Physeal injury0N/A2
      Infection212
      OtherN/A1N/A
      Hip
      AIIS avulsion fracture1N/A1
      Hip arthroscopy0N/A1
      Femoral neck fracture0N/A1
      Femoro-acetabular Impingment0N/A4
      Neuro
      Stinger1N/A1
      Suprascapular nerve injury1N/A0
      Axillary nerve injury220
      Concussion133
      Spinal cord injury0N/A2
      Spine
      Spondylolysis1N/A0
      Spondylolisthesis0N/A1
      OtherN/A1N/A
      This table represents a comprehensive list of topics tested on the OITE from 2009 to 2020. Data included from 2013 to 2016 was presented from previously published data.
      • Synovec J.
      • Plumblee L.
      • Barfield W.
      • Slone H.
      Orthopedic in-training examination: An analysis of the sports medicine section—An update.
      Subtopics that were not included in the Synovec et al. review were designated by “N/A”. All subtopics presented by Synovec et al. that were not included in our categorization were designated as “Other”. Although questions relating to the ACL are consistently on the OITE, the incidence of questions relating to hip, foot and ankle, and spine pathology is more variable, and were not even categorized in the previous review by Synovec et al.
      The taxonomy grading of questions was analyzed. The authors determined that 46 of the 175 (26.2%) total questions were T1 questions, 55 (31.4%) were T2 questions, and 74 (42.2%) were T3 questions. Upon further analysis, between the years 2009 and 2012, 22.1% of questions were T1 compared to 31.2% in 2017-2020; 34.7% of questions were T2 questions between 2009 and 2012 compared to 27.5% between 2017 and 2020; and lastly, 43.2% of questions from 2009 and 2012 were T3 questions compared to 41.2% between 2017 and 2020. Overall, between 2009 and 2012, the majority of questions were T3, while the majority of questions between 2017 and 2020 were T2. There was no significant change in distribution among the two sets of years (P = .250). When comparing taxonomy of questions from recent years (2017-2020) to data presented by Synovec et al., there was a significant decrease in T1 questions (P = .0001) and increase in T2 questions (P = .0005), with no difference in T3 questions (P = .512) (Fig 2).
      Figure thumbnail gr2
      Fig 2Taxonomy of questions was determined using the classification system set forth and verified by Buckwalter et al. in 1981.
      • Buckwalter J.A.
      • Schumacher R.
      • Albright J.P.
      • Cooper R.R.
      Use of an educational taxonomy for evaluation of cognitive performance.
      Increased question taxonomy is associated with higher level of thinking compared to lower question taxonomy associated with straight recollection of information. Question taxonomy was compared to published data from 2012 to 2016.
      • Synovec J.
      • Plumblee L.
      • Barfield W.
      • Slone H.
      Orthopedic in-training examination: An analysis of the sports medicine section—An update.
      The authors noted there to be a total of 194 references from the questions between 2009 and 2012, and a total of 188 references from the questions between 2017 and 2020, for a total of 382 references. The number of references used for sports medicine questions on examinations from 2009 to 2012 ranged between 44 and 53, with a minimum of 2 references per question, and an average of 2.04 references per question. The number of citations referenced for 2017-2020 examinations ranged from 42 to 58, with a minimum of 2 references per question, and an average of 2.32 references per question. This increase in number of references per question was found to be statistically significant (P = .0001). The increased references in 2017-2020 was also found to be higher than that which was previously published by Synovec et al.; however, statistical comparison was unable to be performed. The most referenced sources from 2009 to 2012 were the American Journal of Sports Medicine (28.3%), Journal of Bone and Joint Surgery (13.4%), and Arthroscopy (7.2%). The most referenced sources from 2017 to 2020 were the Journal of the American Academy of Orthopedic Surgery (17.5%), Journal of Bone and Joint Surgery (9.5%), and the American Journal of Sports Medicine (9.5%). The number of unique journal references from years 2009 to 2012 was 49, while the number of unique references in 2017- 2020 was 64 (Fig 3). Synovec et al. identified 41 journals and 12 textbooks referenced by questions between 2012 and 2016, with the American Journal of Sports Medicine as the most popular citation (n = 50, 21.3%). The decrease in the proportion of references from the American Journal of Sports Medicine in this previous study to our review was found to be significant (P = .001); however, there was no change in the proportion of citations from the Journal of Bone and Joint Surgery (P = .303) or the Journal of the American Academy of Orthopaedic Surgery (P = .529).
      Figure thumbnail gr3
      Fig 3Many academic resources are used by the American Academy of Orthopaedic Surgeons in the development of the Orthopaedic In-Training Examination (OITE). The American Journal of Sport Medicine, Journal of the American Academy of Orthopaedic Surgeons, and the Journal of Bone and Joint Surgery are the most common resources referenced by OITE questions. The figure includes only those journals that were referenced multiple times and were included in both the 2009-2012 and 2017-2020 subsets. Data from Synovec et al. were included for comparison.
      • Synovec J.
      • Plumblee L.
      • Barfield W.
      • Slone H.
      Orthopedic in-training examination: An analysis of the sports medicine section—An update.
      The lag time for the references supporting each question was calculated for both the early and late examination years. The average lag time for examinations from 2009 to 2012 was found to be 6.77 years, while the average lag time for 2017-2020 was found to be 6.69 years (Fig 4). This decrease in lag time was not found to be significant (P = .536).
      Figure thumbnail gr4
      Fig 4Lag time, defined as the amount of time between article publication and reference for the Orthopaedic In-Training Examination, represents how up-to-date the tested information is.
      The authors further categorized the questions into those where imaging was provided and those that did not include imaging. In total, 40.6% of all sports medicine questions provided imaging. Between 2009 and 2012, 37.9% of questions included some form of imaging (XR, CT, MRI, or Arthroscopic Photo/Clinical Photo/Video), while 43.75% of questions between 2017 and 2020 included some form of imaging modality (Fig 5). This increase in imaging was not found to be significant (P = .484). Synovec et al. found 37% of questions from 2012 to 2016 to include imaging, with an insignificant difference when compared to questions from 2017 to 2020 exams (P = .779). The most common imaging modalities between 2009 and 2012 were MRI (37.2%), radiograph (34.8%), and clinical photograph/video (16.3%). Between 2017 and 2020, the most common imaging modalities were MRI (54.3%), radiograph (48.6%), and clinical photograph/video (14.3%). These changes in proportions were not found to be significant for MRI (P = .477), radiograph (P = .653), or clinical photograph/video (P = .596). The evaluation of 2012-2016 examinations by Synovec et al. found XR imaging to be the most common presented imaging modality (48%), with no significant change when compared to our data from 2017 to 2020 (P = .332) (Fig 6).
      Figure thumbnail gr5
      Fig 5Imaging has a critical role in the diagnosis and management of patients in orthopaedic surgery. Therefore, it is no surprise that imaging provides an important role in the Orthopaedic In-Training Examination.
      Figure thumbnail gr6
      Fig 6With the transition of the Orthopaedic In-Training Examination to an electronic format, a variety of imaging modalities can now be included on the OITE. Radiographs and MRI imaging were the most provided imaging modalities. Residents should be familiar with reviewing each of these modalities.

      Discussion

      In this study, we found that there was an increase in the number of references per question when comparing sports medicine OITE questions from 2009 to 2012 and 2017 to 2020. Subtopics, taxonomy, lag time, and use of imaging modalities did not show statistically significant changes. With the recent collaboration between ABOS and AAOS to align the OITE with the Part I Certifying Exam, success on the OITE will only become more important in resident education. This study will serve as a benchmark for future comparison of this section, before and after the implementation of the new examination blueprint in 2021. While prior studies have examined sports medicine questions, this review included the lag time of references from publication date to appearance on the OITE.
      • Luginbuhl J.C.
      • Sobel A.D.
      • Mulcahey M.K.
      Analysis of the sports medicine section of the Orthopaedic In-Training Examination: Improvements in levels of evidence and question taxonomy over a 12-year period.
      ,
      • Synovec J.
      • Plumblee L.
      • Barfield W.
      • Slone H.
      Orthopedic in-training examination: An analysis of the sports medicine section—An update.
      ,
      • Srinivasan R.C.
      • Seybold J.D.
      • Salata M.J.
      • Miller B.S.
      An analysis of the orthopaedic in-training examination sports section: The importance of reviewing the current orthopaedic subspecialty literature.
      ,
      • Osbahr D.C.
      • Cross M.B.
      • Bedi A.
      • et al.
      Orthopaedic in-training examination: An analysis of the sports medicine section.
      Lag time represents the novelty of research and is important to assess if residents are being tested on present or historical concepts. In addition to reference lag time, this study assessed changes in commonly tested subtopics, question taxonomy, reference sources, and use of imaging modalities.
      In our review of the OITE examinations from 2009 to 2012 and 2017 to 2020, 175 sports medicine questions were analyzed from a total of 2180 questions. In the early subset, sports medicine questions comprised an average of 8.63% of the examination, while the more recent subset included an average of 7.5%. These numbers fall closely in line with OITE objectives, as the committee targets 7.5% of questions to cover sports medicine topics.
      • Marsh J.L.
      OITE Test Specifications Survey.
      The most tested OITE subtopics within sports medicine had little change between the two blocks of time. In 2009-2012, the most common subtopics were ACL (12.6%), rotator cuff (10.5%), and throwing injury to the shoulder (7.4%). In 2017-2020, the most common subtopics were ACL (10%), rotator cuff (6.25%), and throwing injury to the elbow (6.25%). The ACL has remained the most tested sports medicine subtopic. In a review of questions from 2005 to 2009, questions on the ACL composed 23.5% of sports medicine questions.
      • Srinivasan R.C.
      • Seybold J.D.
      • Salata M.J.
      • Miller B.S.
      An analysis of the orthopaedic in-training examination sports section: The importance of reviewing the current orthopaedic subspecialty literature.
      In a review of 2012-2016 OITE questions, Synovec et al. found that ACL questions constituted 19.3% of sports medicine questions, while the rotator cuff, throwing injuries to shoulder, and throwing injuries to elbow made up 5.7%, 2.3%, and 2.3% respectively.
      • Synovec J.
      • Plumblee L.
      • Barfield W.
      • Slone H.
      Orthopedic in-training examination: An analysis of the sports medicine section—An update.
      While the questions relating to the ACL remain one of the most important focuses of the sports medicine section, the decrease of questions in this subtopic may be attributed to the increase in questions related to hip pathology,
      • Klein G.R.
      • Austin M.S.
      • Randolph S.
      • Sharkey P.F.
      • Hilibrand A.S.
      Passing the Boards: Can USMLE and Orthopaedic in-Training Examination scores predict passage of the ABOS Part-I examination?.
      which encompassed several subtopics in this review, as there was only one question on hip pathology from 2009 to 2012, and there were no hip questions reported by Synovec et al.
      • Synovec J.
      • Plumblee L.
      • Barfield W.
      • Slone H.
      Orthopedic in-training examination: An analysis of the sports medicine section—An update.
      When comparing questions from 2009 to 2012 with 2017 to 2020, T1 questions increased from 22.1% to 31.2%, while T2 questions had a corresponding decrease from 34.7% to 27.5%. T3 questions remained relatively stable between the two question blocks. Synovec et al. compared question taxonomy for the periods 2004 to 2008 and 2012 to 2016 and found that the proportion of T3 questions increased from 25% to 45%, while T1 questions decreased from 67% to 49%.
      • Synovec J.
      • Plumblee L.
      • Barfield W.
      • Slone H.
      Orthopedic in-training examination: An analysis of the sports medicine section—An update.
      This indicates an increase in taxonomy toward T3 questions after 2008, which has remained constant since that time. The increase in T1 questions relative to T2 questions may be due to a broader set of sports medicine topics covered by the OITE and is an area for further investigation.
      A close analysis of question references was performed to evaluate for the most common reference source used, average number of references per question, and lag time between reference publication year and OITE examination year. The most referenced journal from 2009 to 2012 was the American Journal of Sports Medicine (55 citations), while the most referenced journal from 2017 to 2020 was the Journal of the American Academy of Orthopaedic Surgeons (33 citations). Synovec et al. found that the American Journal of Sports Medicine was the most referenced journal from 2012 to 2016, followed by The Journal of Bone and Joint Surgery, and the Journal of the American Academy of Orthopaedic Surgeons.
      • Synovec J.
      • Plumblee L.
      • Barfield W.
      • Slone H.
      Orthopedic in-training examination: An analysis of the sports medicine section—An update.
      While this may represent an increased emphasis on references from the Journal of the American Academy of Orthopaedic Surgeons in recent years, all three of these journals have remained commonly cited.
      This study demonstrated a lack of lag time reduction in recent examination years, which presents concern that tested literature may be outdated. Previous studies examining OITE questions demonstrated the use of more recent literature, but this was not demonstrated in our review.
      • Luginbuhl J.C.
      • Sobel A.D.
      • Mulcahey M.K.
      Analysis of the sports medicine section of the Orthopaedic In-Training Examination: Improvements in levels of evidence and question taxonomy over a 12-year period.
      ,
      • Stone 3rd, N.E.
      • Kang K.K.
      • Eisemon E.O.
      • Choueka J.
      The most-cited sources on the orthopaedic in-training examination: question yield, reading efficiency, and time lag from publication.
      Although an increase in the number of references per question may suggest evidence-based management, an increasing reliance on secondary review sources, such as the Journal of the American Academy of Orthopaedic Surgeons, rather than primary manuscripts counteracts true evidence-based practice. With an increased emphasis on secondary review sources, residents may not adequately learn how to interpret primary literature, which may negatively impact their ability to incorporate evidence-based management into their future practice.
      There was a slight increase in the proportion of questions that included imaging (radiograph, CT, MRI, arthroscopic photo, clinical photo/video). From 2009 to 2012, 37.9% of questions included imaging, which increased to 43.75% of questions from 2017 to 2020 (P = .484). This is similar to the results of Synovec et al., who compared 2004-2008 (20%) with 2012-2016 (37%).
      • Synovec J.
      • Plumblee L.
      • Barfield W.
      • Slone H.
      Orthopedic in-training examination: An analysis of the sports medicine section—An update.
      As imaging is a vital resource in orthopaedic management, it is no surprise that this has continued to increase.
      While not a primary outcome measure of our study, resident performance on sports medicine questions was obtained for examinations from 2017 to 2020. Residents correctly answered 75.9% of these questions (72.8% in 2017, 80.1% in 2018, 82% in 2019, 68.6% in 2020). Osbahr et al. evaluated resident performance on sports medicine questions from the 2005-2009 examination years and found an overall success rate of 67.1%.
      • Osbahr D.C.
      • Cross M.B.
      • Bedi A.
      • et al.
      Orthopaedic in-training examination: An analysis of the sports medicine section.
      Although this study evaluated trends between sports medicine questions on the OITE, its utility in resident preparation for the ABOS Certification Exams remains unclear. Subsequent studies evaluating trends in OITE questions after the collaboration between the AAOS and ABOS to align the examinations are warranted to further guide resident preparation.

      Limitations

      Our review had several limitations. Questions from 2017 to 2020 were obtained through the AAOS website. The test writers divided the questions into sections making sports medicine questions easily identifiable. However, there could be overlap in questions that included sports medicine topics but were categorized into a different section. While all questions of different categories were reviewed for the inclusion of sports medicine questions, there may have been questions relating to sports medicine that were not included in our study. Questions from 2009 to 2012 were obtained from past examinations outside of the AAOS website. These questions were not designated by subsection, and thus, reviewer judgment was used to determine which questions would be designated as sports medicine. Although careful review was performed, there may have been questions that were included in this study that would have been designated an alternate subsection by the AAOS. Likewise, there may have been sports medicine questions that the reviewers considered to be of another subsection and, thus, were not included in our review. As several authors have recently reviewed OITE questions of the shoulder and elbow (SE) and foot and ankle (FA) exam domains, an analysis of overlapping questions between independently performed reviews was performed. There was found to be a large degree of overlap of questions included on the previous shoulder elbow study and this current sports medicine review (2009-2012: 26.3%; 2017-2020: 11.3%). However, there was much less overlap when comparing questions included in this review to that of the previous foot and ankle study (2009-2012: 0.0%; 2017-2020: 1.3%), demonstrating the similarities between the sports medicine and SE exam sections. Assigning question taxonomy is subjective and could vary between reviewers. However, classification of question taxonomy was verified by two residents with discrepancies further reviewed by third resident. Buckwalter et al. found 85% agreement between test makers and residents of varying levels of training when determining taxonomy.
      • Buckwalter J.A.
      • Schumacher R.
      • Albright J.P.
      • Cooper R.R.
      Use of an educational taxonomy for evaluation of cognitive performance.
      Resident performance evaluation was limited, as only questions from the AAOS website (2017-2020) provided the correct question percentage, so this information from 2009 to 2012 was not able to be obtained for comparison.

      Conclusion

      When comparing sports medicine OITE questions from 2009 to 2012 and 2017 to 2020, there was an increase in the number of references per question. Subtopics, taxonomy, lag time, and use of imaging modalities did not show statistically significant changes.

      Supplementary Data

      References

        • Luginbuhl J.C.
        • Sobel A.D.
        • Mulcahey M.K.
        Analysis of the sports medicine section of the Orthopaedic In-Training Examination: Improvements in levels of evidence and question taxonomy over a 12-year period.
        Orthopedics. 2020; 43: e460-e464
        • Batalden P.
        • Leach D.
        • Swing S.
        • Dreyfus H.
        • Dreyfus S.
        General competencies and accreditation in graduate medical education.
        Health Aff (Millwood). 2002; 21: 103-111
        • Ganesh Kumar N.
        • Benvenuti M.A.
        • Drolet B.C.
        Aligning in-service training examinations in plastic surgery and orthopaedic surgery with competency-based education.
        J Grad Med Educ. 2017; 9: 650-653
        • DePasse J.M.
        • Haglin J.
        • Eltorai A.E.M.
        • Mulcahey M.K.
        • Eberson C.P.
        • Daniels A.H.
        Orthopedic in-training examination question metrics and resident test performance.
        Orthop Rev (Pavia). 2017; 9: 7006
        • Swanson D.
        • Marsh J.L.
        • Hurwitz S.
        • et al.
        Utility of AAOS OITE scores in predicting ABOS Part I outcomes: AAOS exhibit selection.
        J Bone Joint Surg Am. 2013; 95: e84
        • Herndon J.H.
        • Allan B.J.
        • Dyer G.
        • Jawa A.
        • Zurakowski D.
        Predictors of success on the American Board of Orthopaedic Surgery examination.
        Clin Orthop Relat Res. 2009; 467: 2436-2445
        • Klein G.R.
        • Austin M.S.
        • Randolph S.
        • Sharkey P.F.
        • Hilibrand A.S.
        Passing the Boards: Can USMLE and Orthopaedic in-Training Examination scores predict passage of the ABOS Part-I examination?.
        J Bone Joint Surg Am. 2004; 86: 1092-1095
        • Dougherty P.J.
        • Walter N.
        • Schilling P.
        • Najibi S.
        • Herkowitz H.
        Do scores of the USMLE Step 1 and OITE correlate with the ABOS Part I certifying examination? A multicenter study.
        Clin Orthop Relat Res. 2010; 468: 2797-2802
        • Fritz E.
        • Bednar M.
        • Harrast J.
        • et al.
        Do orthopaedic in-training examination scores predict the likelihood of passing the American Board of Orthopaedic Surgery part I examination? An update with 2014 to 2018 data.
        J Am Acad Orthop Surg. 2021; 29: e1370-e1377
      1. Orthopaedic In-Training Examination (OITE).
        (Access October 15, 2022)
        • Synovec J.
        • Plumblee L.
        • Barfield W.
        • Slone H.
        Orthopedic in-training examination: An analysis of the sports medicine section—An update.
        J Surg Educ. 2019; 76: 286-293
        • American Academy of Orthopaedic Surgeons
        Orthopaedic in-Training Examination (OITE). Orthopaedic In-Training Examination (OITE).
        (Access October 15, 2022)
        • Buckwalter J.A.
        • Schumacher R.
        • Albright J.P.
        • Cooper R.R.
        Use of an educational taxonomy for evaluation of cognitive performance.
        J Med Educ. 1981; 56: 115-121
        • Srinivasan R.C.
        • Seybold J.D.
        • Salata M.J.
        • Miller B.S.
        An analysis of the orthopaedic in-training examination sports section: The importance of reviewing the current orthopaedic subspecialty literature.
        J Bone Joint Surg Am. 2010; 92: 778-782
        • Osbahr D.C.
        • Cross M.B.
        • Bedi A.
        • et al.
        Orthopaedic in-training examination: An analysis of the sports medicine section.
        Am J Sports Med. 2011; 39: 532-537
        • Marsh J.L.
        OITE Test Specifications Survey.
        American Academy of Orthopaedic Surgeons, Chicago, IL2007
        • Stone 3rd, N.E.
        • Kang K.K.
        • Eisemon E.O.
        • Choueka J.
        The most-cited sources on the orthopaedic in-training examination: question yield, reading efficiency, and time lag from publication.
        Bull Hosp Jt Dis. 2013; 71: 245-252