Simulation-based education has been shown to be an effective teaching and assessment method for technical skill-based professions such as tactical military, aviation, disaster preparedness agencies, and, more recently, medicine. The use of simulation-based assessment for high-stakes purposes such as licensing, board certification, or professional remediation has been proposed but not clearly defined or implemented in surgery. In this article we aim to review the role of simulation for the certification and remediation of surgeons.
Currently, simulation is used mainly for the training of surgeons. The certifying body in surgery, the American Board of Surgery (ABS), requires a minimum number of predefined operative cases to be completed before residents can graduate. The assumption is that resident exposure to this predefined number of cases will lead to technical proficiency. However, the literature suggests that this assumption may not always be true as the number of cases is a proxy for skill and does not ensure technical competence.1,2 A number of studies have demonstrated that even surgical residents in their final year of training commit many errors or inefficiencies in commonly performed operations. Simulation-based assessment can identify such performance deficits through the use of objective metrics.3-7
Recognizing this attribute, the ABS has incorporated simulation-based modules in its certification process, including the Fundamentals of Laparoscopic Surgery, Fundamentals of Endoscopic Surgery, Advanced Trauma Life Support, and Advanced Cardiovascular Life Support.8 Similar to laparoscopic surgery, there are discussions about the use of simulation for the credentialling of robotic-assisted surgery; the Fundamentals of Robotic Surgery relies on simulation for surgical skill acquisition and credentialing.9 In 2012, the American Board of Colon and Rectal Surgery (ABCRS) and the American Society of Colon and Rectal Surgeons collaborated to develop the Colorectal Objective Structured Assessment of Technical Skill (COSATS), a performance-based technical skill examination specific to the practice of colorectal surgery.10 It was the first and only technical skill examination used for national surgical board certification. After a pilot study demonstrated feasibility and initial validity, in 2014 the COSATS was made a mandatory component of the ABCRS certification exam but it was discontinued after the first trial year. All of the individuals who failed COSATS (10%) had passed the oral (certifying) examination, confirming that technical proficiency is not measured by the current examination modalities.10 The initial experience with this exam raised logistical concerns related to exam administration as well as ethical concerns related to repercussions for practicing surgeons who fail the technical examination. In the end, it was felt that the administration of this exam would be a potentially insurmountable challenge both financially and operationally and, therefore, it was not continued (personal communication from David J. Schoetz, Jr., MD, former Executive Director, ABCRS). Along similar lines, the American College of Surgeons recently developed the Objective Assessments of Skills in Surgery, a formative assessment of junior residents using simulated stations of multiple core procedures.11 All these assessment modules have strong supporting validity evidence; are standardized, affordable, and reproducible; and can be proctored by nonsurgeons, which makes test taking more convenient and easier to schedule. Thus, it should be used as a model for further simulation-based assessment in high-stakes exams.
Simulation also provides an effective assessment tool for maintenance of certification.12 Competence is not an endstate but encompasses lifelong learning and maintenance.13 Currently, surgeons maintain and improve their skills by passing the written ABS maintenance of certification exam periodically, and by obtaining a required number of continuing medical education credits by attending conferences/courses, a paradigm that is rooted in self-assessment. Unfortunately, this paradigm does not include assessment of technical skills and self-assessment has several weaknesses.14 In contrast, for many other very technical and high-performance, demanding professions such as pilots, simulation is a vital part of both licensing and license renewal. More specifically, every six months, pilots must demonstrate proficiency in normal and emergency situations in a simulator; failure to do so, results in remediation and further testing to prove adequacy, while repeated unsuccessful attempts lead to contract termination.1,15 In the authors’ opinion, implementation of similar pathways for the maintenance of surgeon certification is likely to benefit them and the patients they care for.
Remediation has been defined in the literature as “any additional training, practice, or instruction provided due to identified deficiencies in an area of core competency.” Remediation has been instituted using simulation in anesthesia, internal medicine, nursing, and emergency medicine, among others. Simulation can be used to both diagnose needs for remediation and facilitate remediation until competence is met. Remediation programs usually incorporate several strategies, including teaching, coaching or counseling, and hands-on experiential learning using simulation.16 Emergency medicine literature recommends that all remediation programs need to be clearly delineated and completely transparent to all parties involved. It is also recommended that any remediation related to high-stakes decisions such as promotion or probation be thoroughly documented.16 Simulation has historically been a “safe environment” where clinical performances—specifically errors—are not necessarily discussed outside of the simulated environment.17 However, learners must be made aware that in a simulation-based remediation program, their performance will be discussed with those responsible for their performance and decisions based on promotion or probation may be made based on their simulated performance.16
Remediation via simulation is not commonly included in surgical curricula. A recently published study at Indiana University has demonstrated that performance of surgical procedures in porcine models can reliably identify critical technical deficiencies of chief residents that, when combined with a remediation program, can ensure competency prior to graduation.16 It is also important to highlight the importance of simulation for the assessment and remediation of nontechnical skills. Simulation for nontechnical skills often involves the utilization of standardized patients recreating scenarios that will trigger desirable and undesirable behaviors. Prior studies have used simulation to teach behavioral strategies that reduce unprofessional behavior and have demonstrated excellent long-term remediation with no recurrence of reported unprofessional behavior four years later.18-20
All simulation-based remediation plans need to be individualized and based on the specific learner’s needs. Such plans should incorporate clear performance goals and promote learner-deliberate practice to master the required skills and behaviors. Further, skill acquisition should be closely monitored and training adjusted as necessary until the required level of performance is reached. It is also recommended that all faculty engaged in simulation-based remediation have extensive experience and expertise with simulation.21
Since its introduction for surgical skill acquisition, simulation has come a long way and has become part of the fabric of training surgery residents. The accumulated experience with simulation in surgery has clearly demonstrated its value for training and assessment of skills. Given that existing certification pathways in surgery are outdated, simulation provides a great opportunity to add objectivity to the initial certification and its maintenance. The extensive network of ACS Accredited Education Institutes can serve this purpose. Further, simulation can be a very effective tool for remediation of skills. In this article, the authors provided examples of how simulation has been applied to certification and remediation and propose that additional work is necessary to fully incorporate simulation into these processes and realize its full potential in enhancing and retaining surgeon skills at many points throughout a career.
