Become a member and receive career-enhancing benefits
Our top priority is providing value to members. Your Member Services team is here to ensure you maximize your ACS member benefits, participate in College activities, and engage with your ACS colleagues. It's all here.
Become a member and receive career-enhancing benefits
Our top priority is providing value to members. Your Member Services team is here to ensure you maximize your ACS member benefits, participate in College activities, and engage with your ACS colleagues. It's all here.
Reimagining the General Surgery Residency Match Process: 2021 Update
Jennifer LaFemina, MD, FACS; Cary B. Aarons, MD, MSEd, FACS; Kareem Abdelfattah, MD, FACS; Jennifer Choi, MD, FACS; Karole Collier, MD; Jamie Hillas, MD; Kari Rosenkranz, MD, FACS; Jennifer Serfin, MD, FACS; Vance Sohn, MD, FACS; Kyla Terhune, MD, FACS; Paul Wise, MD, FACS; and Doug Smink, MD, FACS
November 1, 2021
15 MinPrintShare
Bookmark
Introduction
COVID-19 has advanced the conversation regarding challenges in the residency selection process impacting both applicants and programs. To address these challenges—guided by the principles of equity and safety and by available data—the Association of Program Directors in Surgery (APDS) provided recommendations to programs for the 2021–2022 residency recruitment cycle.1 This article summarizes the recommendations and reviews long-term considerations.
Applications
While available data such as Association of Medical Colleges (AAMC) diminishing returns (which correlates USMLE Step 1 for U.S. MD graduates to application numbers and Match rate)2 inform the number of recommended applications for a given candidate, the number or programs applied to, per applicant, far exceed these values. In the past cycle, self-reported data reveal that the median number of applications ranged from 56 to 84.3 AAMC ERAS® 2020 data reflect similar trends (i.e., an average of 60 programs applied to per surgery applicant4), dramatically in excess of diminishing return estimates and representing a 17 percent increase from two years earlier.
These increases translate to a similar substantial burden on programs. More received applications result in more time needed to review applications, compromising holistic review and forcing programs to adopt convenient screening metrics resulting in artificial filtration.5 It is well established that use of such metrics disadvantage applicants who are underrepresented in medicine as well as perpetuates gender bias.6-8 The average number of applications per program rose another staggering 18 percent in the last year,9 translating to programs receiving approximately 150 applications to fill one categorical position.10 All of this to yield the same match rate at the expense of time, stress, money, and, ultimately, an unclear understanding of alignment of program-applicant values.
Several potential solutions were reviewed, some of which were recommended by the APDS. If regulated application caps as a solution to over-application would reduce application numbers, though potentially disadvantaging applicants who need to apply more broadly and those who are undecided on region or culture. Additionally, this may carry legal ramifications (for example, restriction of trade). However, evidence exists to advise students in a data-driven manner and with consideration of unique applicant circumstances and characteristics (for example, geographic preferences, applicant type, scope of achievements). It is important to note that the data on diminishing returns will be less relevant after the conversion of USMLE Step 1 to Pass/Fail. Ongoing data sources are summarized in Table 1.
Table 1. Data Sources to Guide Application Numbers
NRMP Interactive Charting Outcomes in the Match21
NRMP Charting Outcomes in the Match13
Residency Explorer22
APDS Recommendations
Given the lack of available data and the potential negative consequences to the applicants, we do not recommend application caps.
USMLE Step 1 and COMLEX Level 1 should be de-emphasized in application review.
Programs should transparently provide information that best informs applicant choices, potentially limiting applications to programs in which a candidate is unlikely to match. Examples include, but are not limited to, standard testing minimums and requirements, visa sponsorship, and other program requirements.
Interviews
Mitigating interview inflation is a point of interest and might also help address the “prisoner’s dilemma” faced by applicants.11,12 Data suggest that the number of interviews exceeds the number needed to yield a successful match for most candidates. Data from the National Resident Matching Program (NRMP) guides that matched candidates have a mean of 13.2 “contiguous ranks” (number of programs ranked in the preferred specialty before a program in a different specialty appears on the rank order list) with a 90 percent match probability with 11 contiguous ranks.13 Interview-rich applicants (IRAs) receive and complete a large number of available interviews. In 2021, the median number of interviews received was 19 and attended was 16.3 Further, Lee et al demonstrated that 6 percent of IRA in general surgery attended 28 percent of the interviews.14 This inequity calls to light the interview inflation, or so called “hoarding” of interviews. Driven by a fear of nonmatching and limited data on optimal interview numbers, interview inflation may be beneficial to some while negatively impacting the many who are subsequently left with an insufficient number of interviews, among other subsequent domino effects on the recruitment process. Unfortunately, there is not a centralized mechanism to collect and analyze interview-match data, which is imperative to guide future innovation. Given the lack of such data in general surgery, the optimal number of interviews was not recommended by the APDS. Interview caps, if regulated, may offer a more equitable distribution of interviews paired with cost savings, fewer cancellations, and ultimately a “signal” (by way of attending the interview) of true interest by the applicant. Other solutions such as early acceptance programs have been discussed; potential opportunities and challenges are present, and data will guide its role in future cycles.15
Standardized interview invitation release periods are another potential solution. Adopted by specialties including Obstetrics and Gynecology, standardized interview invitation offer dates by programs reduce excess interviewing by applicants. While the downstream impact on cost savings, hoarding, education optimization (as applicants can focus on their rotations rather than being distracted with invitations at all hours), and stress from missing an opportunity are unknown, we hypothesize a positive impact with this practice and recognize the need to evaluate the subsequent impact on programs and applicants.
APDS Recommendations
Programs should consider a first-round, single-interview invitation release period, aligned to the historical peak interview invitation release period (in 2021, this week is October 27-November 2, 2021).
Programs should adhere to a code of conduct that includes:
Extending invitations to, and not exceeding, the number of available interview slots (i.e., not “over-inviting”).
Allowing a period of at least 48 hours for applicant response before inviting the next candidate.
Additional Considerations
Holistic Review
Essential to the alignment of program-applicant values and achieving a culturally rich learning and working environment, holistic review (i.e., reviewing a candidate’s unique attributes and experiences in addition to one’s academic performance) is a central focus of the current (and future) application season. Resources on implementation are summarized in Table 2.
Table 2. Resources to Facilitate Holistic Review
AAMC Holistic Review23
APDS Diversity and Inclusion Toolkit24
Recruiting a Diverse Resident Population. Everyone Wins by Cary Aarons, MD25*
*Free to APDS Members
Supplemental Application
For the 2021–2022 cycle, AAMC/ERAS offered optional participation in the Supplemental Application for General Surgery programs and applicants as part of a pilot program. While aspects such as geographic preference are under study and not visible to surgical programs this cycle, work-life experience and preference signaling are available to participants. The pilot study will help determine whether these components facilitate effective holistic review and help to align program-applicant fit.
Virtual Interviews
The APDS, in concordance with the Coalition for Physician Accountability17 and the NRMP18, recommends virtual interviews in the 2021–2022 cycle. This recommendation promotes equity in the current selection process and safety in a time of uncertainty regarding regional COVID-19 surges. While there are no data on the impact of virtual interviews, data from the NRMP suggests that factors important to applicants are those that may be best served by in-person interviews.3 Hybrid interviews may arise in the future. To ensure equity for applicants who may have travel limitations due to geography, financial strain, clinical responsibilities, or otherwise, the certifications of the rank order lists could be decoupled between applicants and programs. Decoupling occurs if a program certifies a rank order list prior to hosting an on-site visit (which would be before the applicant rank list certification). With decoupling, the campus visits become truly voluntary. Candidates who wish to consider “goodness of fit” and “culture” in a non-evaluative way, in person, may do so without disadvantaging those who do not.
Signaling
Signaling was originally implemented by the American Economic Association and utilizes a centralized, credible system through which applicants can send up to two signals to employers to indicate interest in a firm.19 In the last match cycle (2020–2021), signaling was introduced in otolaryngology. Given the aforementioned challenges, the formal “signal” may substitute for some more traditional signals, like attending a live interview or direct contact to the program. While data is forthcoming, early data suggest that the addition of preference signaling was not harmful to applicants and might address some of the concerns about interview inflation: matched applicants interviewed at a rate lower than the prior match cycle applicants, with less cost, no signs of geographic clustering, and without an impact on matched candidate characteristics.20
This year in general surgery, through the collaboration with the AAMC and the ERAS Supplemental Application pilot, general surgery applicants had the option to signal up to five programs (if they elected participation). While the AAMC offered guidelines for using signals, it is unclear what a signal from an individual applicant means, how applicants will or should best utilize them (for example, send them to “reach” programs, “safety” programs), and how programs should interpret them. Signals are considered to be high-value items in their scarcity since each applicant was only able to send five. The lack of a signal should not be interpreted as a lack of interest, but rather the culmination of insufficient signals to show interest since applicants will use them differently, if at all. The number of optimal signals is unknown and is being studied. Without these data, programs should not overvalue a signal. By overvaluing a signal, programs may overlook highly qualified or otherwise interested applicants. Signals, at this point, should not be used for program decisions about their rank list as applicants will likely change their minds throughout the recruitment process, and programs should not require a signal to offer an applicant an interview.
Over time we will understand the importance of a signal and how it is best employed by applicants and by programs. We hypothesize that the pre-interview signal is the first step of many future changes. The future could hold options for post-interview/pre-certification signals, which might better reflect an applicant’s changing interest in a program and better align to the desire to match. Further, “bidirectional signals” (from applicant to program and program to applicant) might facilitate the much needed and desired “mutual interest,” if done within the regulations of the Match agreement.
APDS Recommendations
Holistic review of applicants is essential and should be considered.
Virtual interviews are recommended in 2021–2022.
Participation in the ERAS Supplemental Application Pilot Study allows for signaling and other applicant information to be made available to programs.
Conclusion
COVID-19 has brought graduate medical education and the residency recruitment process to a crossroads. It has forced us to look at equity differently (and broadly) and has catapulted forward the importance of innovation in this process. The APDS recommendations for the 2021–2022 cycle are the first step in this innovation. These steps require analysis, renegotiation, and revision to continue to improve and advance the recruitment process for future cycles. These recommendations will ideally herald in a future that better achieves the mutual goals of programs and applicants.
Carmody JB, Rosman IS, Carlson JC. Application Fever: Reviewing the Causes, Costs, and Cures for Residency Application Inflation. Cureus. 2021;13(3):e13804.
Edmond MB, Deschenes JL, Eckler M, Wenzel RP. Racial bias in using USMLE step 1 scores to grant internal medicine residency interviews. Acad Med. 2001;76(12):1253-1256.
Jarman BT, Kallies KJ, Joshi ART, et al. Underrepresented Minorities are Underrepresented Among General Surgery Applicants Selected to Interview. J Surg Educ. 2019;76(6):e15-e23.
Rubright JD, Jodoin M, Barone MA. Examining Demographics, Prior Academic Performance, and United States Medical Licensing Examination Scores. Acad Med. 2019;94(3):364-370.
Gardner AK, Smink DS, Scott BG, Korndorffer JR, Jr., Harrington D, Ritter EM. How Much Are We Spending on Resident Selection? J Surg Educ. 2018;75(6):e85-e90.
Weissbart SJ, Hall SJ, Fultz BR, Stock JA. The urology match as a prisoner's dilemma: a game theory perspective. Urology. 2013;82(4):791-797.
Santos-Parker KS, Morgan HK, Katz NT, et al. Can Standardized Dates for Interview Offers Mitigate Excessive Interviewing? J Surg Educ. 2021;78(4):1091-1096.
Lee AH, Young P, Liao R, Yi PH, Reh D, Best SR. I dream of Gini: Quantifying inequality in otolaryngology residency interviews. Laryngoscope. 2019;129(3):627-633.
Winkel AF, Morgan HK, Akingbola O, et al. Perspectives of Stakeholders About an Early Result Acceptance Program to Complement the Residency Match in Obstetrics and Gynecology. JAMA Netw Open. 2021;4(10):e2124158.
The Coalition for Physician Accountability. The Coalition for Physician Accountability’s Undergraduate Medical Education-Graduate Medical Education Review Committee (UGRC): Recommendations for Comprehensive Improvement of the UME-GME Transition. https://physicianaccountability.org/wp-content/uploads/2021/08/UGRC-Coalition-Report-FINAL.pdf. Published 2021. Accessed October 27, 2021.
Coles P KA, Niederle M. Preference signaling in matching markets. American economic journal: Microeconomics. 2013;5(2):99-134.
Lenze NR, Mihalic AP, Kovatch KJ, Thorne MC, Kupfer RA. Impact of the COVID-19 Pandemic on the 2021 Otolaryngology Residency Match: Analysis of the Texas STAR Database. Laryngoscope. 2021.
