RAS-ACS Symposium: Competency-based training: A gateway to efficiency or a hurried sprint to the finish line

Each year at the American College of Surgeons (ACS) Clinical Congress, the Resident and Associate Society (RAS) Advocacy and Issues Committee hosts the RAS Symposium. This is a highly anticipated event for RAS members in which a panel of speakers discusses a salient topic that affects surgical residents and surgeons. Participants in the 2021 RAS Symposium will discuss the topic of competency-based training and whether this shift in surgical training serves to improve the quality and efficiency of surgical education or jeopardizes true objectivity in trainee evaluation. The RAS Symposium will take place Sunday, October 24, as a part of the virtual ACS Clinical Congress 2021.

Introduction

Surgical residency training has evolved greatly since its inception, but one factor that has remained consistent over the last century is a time-based model of training. At present, surgical trainees must complete five years of clinical training in general surgery to be eligible for board certification from the American Board of Surgery (ABS). However, in recent years, the value of this time-based requirement has been called into question, as studies indicate that newer graduates are less prepared to enter surgical practice.¹ Factors contributing to this include work-hour restrictions,² increased number and complexity of interventions, and less autonomy in the operating room (OR).³ In a survey of fellowship program directors, one-third of the respondents noted that new fellows were unable to operate for 30 minutes unsupervised, and many respondents attributed this shortcoming to a lack of autonomy during residency training.¹

The measure of competency in the present paradigm is largely based on defined category minimums as outlined by the Accreditation Council for Graduate Medical Education. Graduates are required to complete at least 850 total major cases, with 200 major cases completed in the final year.⁴ Completion of these requirements is based on self-reported participation from surgical trainees, tracked by case logs. In the last few years, surgical educators have proposed ways to improve the training paradigm by focusing more on establishing competency, rather than strictly time-based training.

Competency-based training may appear to be a novel concept; however, outcome-based education has been used in professional training since the 1970s.⁵ Surgery is hardly the first medical profession to redesign the educational process; this type of educational structure has been incorporated into training of social workers, pharmacists, and chiropractors, among others, for decades.⁵

In 2004, the American Surgical Association Blue Ribbon Committee published a report on surgical education.⁶ This committee was a collaboration between the American Surgical Association, the ACS, the ABS, and the Resident Review Committee for Surgery (now the Review Committee for Surgery [RC-S]) to examine the health care environment and offer recommendations on improving surgical education. Included in RC-S recommendations was the development of a standard curriculum focused on demonstrating competency in predetermined subjects. The authors note that “education research is pushing toward competency-based advancement, replacing time-in-service.”⁶ A separate article expands on these assertions and proposes a new system in which the duration of training is more relevant to the long-term goals of the trainee. This new system would develop objective methods of evaluation in order to implement competency-based advancement.⁷

The topic of competency-based training has been debated for many years. Based on the ongoing discussions by governing bodies including the ABS, it is clear that we are approaching a paradigm shift toward competency-based training in surgical education. Transforming surgical education is a complex endeavor, and the upcoming RAS Symposium debate seeks to highlight the potential advantages and pitfalls that may result from these changes.

The evolution of surgical training

In 1904, William S. Halsted, MD, FACS, described his pyramid model of surgical training based on the principles of a strong basic science knowledge, the pursuit of medical research, and graduated responsibility.⁸ Trainees were accepted into the program and underwent an indefinite period of rigorous training during which only the most competent physicians were chosen for promotion and, ultimately, graduation. Over the course of 32 years, Dr. Halsted trained 17 chief residents and 55 “assistant” residents at Johns Hopkins Hospital, Baltimore, MD.⁸ This paradigm of surgical training, while producing remarkable academic surgeons and seven departmental chairs, did not ensure a generalizable and standardized method of training young surgeons.

Recognizing this deficiency in the Halsted method, Edward D. Churchill, MD, FACS, of Massachusetts General Hospital, Boston, introduced the “rectangular” model of surgical education, which limited training to five years and eliminated some of the intense inter-resident competition found in the Halsted pyramidal system. Further standardization occurred after the ABS implemented the first board examination to test resident knowledge in 1937. The next major change occurred in 1965 following the introduction of Medicare and Medicaid, in which the days of graduated responsibility espoused by Halsted were slowly eroded by the requirement of an attending surgeon’s presence for all procedural billing.⁸

General surgery training has undergone major changes in the last several decades, particularly with the emergence of subspecialization and the increased frequency of fellowship training. The overall length of training, compounded with rising medical school debt, has led to the emergence of integrated residency programs for surgical subspecialities, such as vascular surgery, cardiothoracic surgery, urology, and plastic surgery, which allow for a shortened training period by negating the need for a general surgery residency, followed by several more years of fellowship specialty training. The ABS, noting the popularity of these integrated programs, approved the “flexibility in surgical training” initiative, which allows for early tracking into subspecialties through customization of the fourth and fifth general surgery training years at certain institutions.⁹

Beyond subspecialization, the sheer diversity of training programs—from urban academic centers to rural community hospitals—creates a lack of uniformity in exposure to various surgical issues. Based on where surgical residents complete their training, they may have drastically different opportunities to operate and gain technical experience. The question of how to optimally train general surgery residents in an efficient and effective manner looms over the current generation.

A gateway to efficiency

In 2016, the ABS issued a Statement Regarding Residency Redesign, in which the board outlined the ongoing efforts toward redesigning surgical residency with a focus on improving competency and efficiency in surgical training.¹⁰ The framework for this redesign is centered on a core-plus general surgery curriculum, along with a modular framework that uses competency-based assessment tools for resident advancement. After completing the “core” curriculum, residents would advance to “plus” years, which would be tailored to their future career. In the case of surgical subspecialization, the “plus” years would take place in the form of fellowship training. Unique to this design, however, is the requirement of “plus” training in general surgery for residents who are not interested in pursuing a surgical subspecialty. Similar to fellowship, the goal is for residents to receive more focused experience relevant to their future career.

A common criticism of the traditional time-based model is that surgical residents only become technically competent in the final years of training,11 which calls into question the efficiency of the early years of training. Junior residents often are consumed with extraneous tasks that may be important for patient care but do not directly align with the core competencies of surgical training. A barrier to overcoming this hurdle is the cost of hiring additional providers to alleviate a portion of these extra tasks.¹¹

Competency-based programs may allow junior residents to focus on the development of technical skills in addition to patient care, which, in turn, will help to promote training efficiency within general surgery programs. As a result, the length of surgical training may be tailored to trainees’ career aspirations, potentially leading to reduced training cost. Surgical training is an expensive endeavor and improved efficiency may reduce costs for the hospital as well. In Canada, the cost per year to train a resident is approximately $100,000.¹¹ In the U.S., the annual cost of resident training has been estimated at $180,000−$244,000.¹² The reduction of even one year of surgical training may significantly decrease cost to the health care system. In addition, surgeons may enter the workforce more expeditiously, leading to increased hospital profit. Moreover, surgical trainees may begin to earn wages suitable for repaying the amassed student loan debt.

The shift toward competency-based education has been implemented in some areas of surgery. The orthopaedic surgery residency at the University of Toronto, ON, took on the challenge of transforming its curriculum to include a competency-based track to evaluate whether any appreciable differences were identifiable between it and the established system.¹¹ The framework was restructured to be modular-based rather than service-specific, allowing for focused learning of one general topic at a time. The results comparing the two systems were reported after a three-year pilot test. There was an increase in the time commitment initially for faculty development of the curriculum, but after the program was established, it declined considerably.¹¹ The residents who participated in the competency-based training demonstrated significant improvement in technical skills when matched with their traditional counterparts, and these differences continued even several months into training.¹¹ Furthermore, junior-level residents in the competency-based program had technical proficiencies equal to the traditional pathway of postgraduate year-five residents.¹¹ With the evaluation tools in place, residents having difficulty advancing through modules could be identified and early strategies enacted to intervene and help them. Notably, two of the 14 residents finished their training in four years rather than the standard five.¹¹

With the implementation of the 80-hour workweek and the increased complexity and specialization of the surgical profession, it is important to consider the efficiency of the training process. Competency-based training is an opportunity to refocus surgical education so that residents are achieving clinical milestones in a more direct and purposeful manner. Although advancement based on competency may mean some residents can complete training in fewer than five years, it is not the only measure of efficiency. By determining which core milestones must be achieved and developing objective methods of evaluation, residents will have a clear path to determining how time is spent in residency to achieve these goals.

A sprint to the finish line

Although a competency-based training paradigm has several advantages over the time-based model, it also has several potential pitfalls. The complexity of engineering a competency-based system requires agreement on what competency means and how it can be measured reliably. This model potentially translates to an increased burden on faculty to provide objective evaluations of resident performance. In addition, achieving competency in a set area of clinical subjects has to fit into a system that is highly dependent on reliable resident coverage across a range of services for a set amount of time.

Some surgeons argue that the art of practicing surgery is more than simply completing a list of clinical scenarios and “checking the box.” In the time-based model, residents often will see a given clinical scenario multiple times in different clinical settings, each not only solidifying the clinical pattern, but also highlighting the nuanced clinical decision-making that comes with seeing different types of patients in different clinical settings. In his book Outliers, Malcolm Gladwell proposes that mastery of a complex skill or subject takes 10,000 hours of practice.¹³ Although it is hard to pinpoint the exact amount of time it takes to develop an appropriate level of competence as a surgical resident, there is reason to believe that experience develops over time and through a complex synthesis of countless clinical exposures. This aspect of training may be lost if we shift to simply completing a set of modules as the basis for graduating residency. The time-variable component of this system of training may potentiate and, without adequate safeguards, improperly incentivize residents to “race” through competency-based training in an attempt to sprint to the finish line of training. The adverse manifestations of a checkbox approach to surgical education could be disastrous for both patient outcomes and public confidence in surgical training mechanisms.

Another challenge of competency-based training is how it will fit into the health care system as a whole. Academic institutions across the U.S. are heavily dependent on the role of trainees in covering various surgical services for a set period of time. A competency-based model would change this precedent, as it would require variability in trainee schedules based on time required to advance through required modules. Without reliable resident coverage for defined periods of time, the health care system will likely require additional support from advanced practice providers to share in the day-to-day workload.

Furthermore, some residents will require more than five years to attain competency, which poses the issue of the number of years an institution will be able to provide funding for a trainee and also would affect the number of new positions available annually. There are economic implications for the trainee as well in this scenario, as prolonged training delays entry into the workforce and loan repayment. Although prioritizing resident education is important, it is imperative to recognize the economic factors that would need to be overcome to achieve this goal.

One of the biggest hurdles in a competency-based training model is the objective evaluation of trainee competence. The establishment of required competencies will necessitate an evidence-based consensus on how they should be objectively measured. Beyond this complication is the risk that implicit bias poses to achieving truly objective evaluations. A study in 2019 identified potential gender bias among surgical trainees, with males more commonly receiving positive evaluations with standout terms such as “exceptional,” “outstanding,” and “superb.”¹⁴ Racial bias is another threat to the evaluation process. Not only are nonwhite trainees more likely to report experiencing discrimination of any kind, but Black residents, in particular, more commonly report experiencing differences in standards of evaluation.¹⁵ Another study on letters of recommendation for surgery residency applicants identified racial bias in terms used to describe Black and Hispanic applicants versus their white counterparts.¹⁶ If we continue to demonstrate the presence of implicit bias throughout the medical field, it is reasonable to assume that such biases would also be an obstacle to achieving objective assessment of competency.

Ultimately, the validity of a competency-based training framework will depend on practical, scalable systems that can reliably reflect trainees’ mastery of skills, as well as identify not only those residents who have deficiencies but also pinpoint specific competencies to which residents should devote additional time.¹¹ Absent these evaluative techniques, both trainees and programs may be tempted to adopt a “minimum necessary” approach that neither reflects competency nor satisfies the tenets of time-variable training.

Conclusion

How will we identify the limitations of our current systems of trainee assessment and competency evaluation? A natural history experiment already may be under way. The coronavirus pandemic has thrust upon the surgical education community the opportunity to examine how and when to graduate residents whose case volumes and learning experiences have been disrupted.¹⁷ The American Board of Medical Specialties has continued to support each training program’s clinical competency committee’s ability to determine trainees’ suitability for graduation and eligibility for board certification. In addition to reviewing the validity of existing assessments like examinations and summative feedback, surgeon-educators faced with holding residents back based on case logs alone may need to explore novel methods of determining competency, such as technical simulation and crowdsourced assessment of trainee performance.

The landscape of surgical training has changed greatly since the time of Dr. Halsted. Just as the specialty of surgery continues to innovate and adapt to the ever-changing environment, surgical education will have to transform to meet the new needs of the future trainee. This evolution will require significant buy-in from stakeholders and a thoughtful approach in order to strike a balance between more effective, efficient training and maintaining appropriate standards for graduating well-trained surgeons.

References

1. Mattar SG, Alseidi AA, Jones DB, et al. General surgery residency inadequately prepares trainees for fellowship: Results of a survey of fellowship program directors. Ann Surg. 2013;258(3):440-449.
2. Ahmed N, Devitt KS, Keshet T, et al. A systematic review of the effect of resident duty hour restrictions in surgery: Impact on resident wellness, training, and patient outcomes. Ann Surg. 2014;259(6):1041-1053.
3. Damewood RB, Blair PG, Park YS, Lupi LK, Newman RW, Sachdea AK. Taking training to the next level: The American College of Surgeons Committee on Resident Training Survey. J Surg Educ. 2017;74(6):e95-e105.
4. The American Board of Surgery training and certification. Training requirements. Available at: www.absurgery.org/default.jsp?certgsqe_training. Accessed July 15, 2021.
5. Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: Theory to practice. Med Teach. 2010;32(8):638-645.
6. Debas HT, Bass BL, Brennan MF, et al. American Surgical Association Blue Ribbon Committee Report on Surgical Education: 2004. Ann Surg. 2005;241(1):1-8.
7. Pellegrini CA, Warshaw AL, Debas HT, et al. Residency training in surgery in the 21st century: A new paradigm. Surgery. 2004;136(5):953-965.
8. Sealy WC. Halsted is dead: Time for change in graduate surgical education. Curr Surg. 1999;56(1,2):34-39.
9. Eberlein TJ. A new paradigm in surgical training. J Am Coll Surg. 2014;218(4):511-518.
10. American Board of Surgery. ABS statement regarding residency redesign. April 16, 2016. Available at: www.absurgery.org/default.jsp?news_resredesign0416. Accessed July 15, 2021.
11. Ferguson PC, Kraemer W, Nousiainen M, et al. Three-year experience with an innovative, modular competency-based curriculum for orthopaedic training. J Bone Joint Surg Am. 2013;95(e166):1-6.
12. Ben-Ari R, Robbins RJ, Pindiprolu S, Goldman A, Parsons PE. The costs of training internal medicine residents in the United States. Am J Med. 2014;127(10):1017-1023.
13. Gladwell M. Outliers: The Story of Success. New York, NY: Back Bay Books; 2001.
14. Gerull KM, Loe M, Seiler K, McAllister J, Salles A. Assessing gender bias in qualitative evaluations of surgical residents. Am J Surg. 2019;217(2):306-313.
15. Yuce TK, Turner PL, Glass C. National evaluation of racial/ethnic discrimination in U.S. surgical residency programs. JAMA Surg. 2020;155(6):526-528.
16. Polanco-Santana JC, Storino A, Souza-Mota L, Gangadharan SP, Kent TS. Ethnic/racial bias in medical school performance evaluation of general surgery residency applicants. J Surg Educ. February 23, 2021 [Epub ahead of print].
17. Coleman J. COVID-19 pandemic and the lived experience of surgical residents, fellows, and early-career surgeons in the American College of Surgeons. J Am Coll Surg. 2021;232(2):119-135.