How to develop a practical and effective American Board of Surgery In-Service Training Examination remediation program

David A. Rogers, MD, MHPE

Imagine a surgery resident who is technically proficient, exquisitely honest, and relates well to patients and staff but who consistently performs poorly on the American Board of Surgery In-Service Training Examination (ABSITE). Most surgical faculty members would not have much problem recalling a resident who met this profile. Surgical residents are goal oriented and highly competitive, so their first reaction to doing poorly on an examination is one of embarrassment and frustration. This scenario is particularly true if they have performed poorly after attempting to study harder after a previous unacceptable performance on the ABSITE. The program director or designee is then given the task of developing a remediation plan for the individual resident. The purpose of this report is to describe such a program that requires little in the way of additional resources.

The initial step in designing the program was to discuss with the residents why they thought they had performed poorly on the ABSITE. While each resident had some unique issues, two general themes developed. First, residents tended to underestimate the amount of time that it takes to review the mate-rial that was covered by this examination. A typical scenario: residents would intend to begin studying after receiving their ABSITE scores; however, residents would then procrastinate until late fall, at which time they would realize that they could not possibly read, much less memorize, all of the material that needs to be covered. Another common theme was confusion as to what material should be mastered in order to prepare for the ABSITE. Specifically, residents believed that only reading about their patients would adequately prepare them for the examination.

Based on published research, several assumptions were made in developing the remediation program:

1. The ABSITE is an examination of general surgical knowledge¹ including basic sciences relevant to the practice of surgery.
2. While multiple different types of thinking are tested in this examination,² it is not possible for the resident to analyze or infer unless they are able to recall the basic facts.
3. A self-study program is the most effective preparation for the examination.^3-5
4. A self-study program is as effective as faculty coaching for remediation of this type of examination.⁶

The reading program was developed by selecting a general surgical textbook and a textbook devoted to surgically relevant basic sciences. The portion of the general surgical textbook devoted to basic sciences was subtracted from the reading program as this material was covered in more detail in the surgical basic science text. The total number of pages devoted to clinical surgical conditions in the general surgical textbook was added to the number of pages in the surgical basic sciences textbook to calculate the number of textbook pages that needed to be read. The number of textbook pages to be read weekly was calculated by dividing the total number of pages by the number of weeks in the study period (middle of spring to the end of the calendar year) and was approximately 70 pages a week. The reading schedule was created by selecting 70 page segments of either textbook. In some cases, readings from both textbooks were used if the information covered was related. The resulting reading schedule was given to the resident at the beginning of the program (see Table).

Date Due	General Surgery Textbook	Basic Science Textbook
June 14		531-596
June 21	513-532	637-680
June 28	1755-1832
July 5	533-600

Table. A section of the weekly reading schedule.

A weekly examination was constructed by surveying the assigned reading and selecting key concepts. Open-ended questions were constructed to promote both the memorization of important facts and the understanding of how these facts related to the management of specific diseases. The page number of the textbook where the key concept could be found was listed with the question (see Figure). Eight to 12 questions were created for each 70 page segment and combined to create the weekly examination. Residents were instructed to read the material and then take the test without referring to the textbook. They were then told to correct their own answers using the textbook. Finally, they were instructed to make a copy of the completed work and submit the original at the end of the week. These instructions were included in the weekly quiz (Figure below.) The residents were advised that, in the month before the examination, they should review all of the quizzes and their own corrected answers. A commitment was made to each resident that his or her adherence to the program would be described in resident progress decision faculty meetings. In this way, adherence to the program became a measure of resident motivation. Residents who were delinquent in submitting their work were contacted initially by e-mail. The program director was contacted if a resident was chronically delinquent in submitting the completed quizzes. In some cases, participating in the program was a condition of probation and so the program director could issue the appropriate warnings.

ABSITE Reading Program Questions November 8, 2004 Instructions: Answer the questions without the textbook. Look up the correct answer in the textbook and make any corrections to your answers. Return the completed document to me after making a copy for yourself. ( )-Page Number in General Surgery Textbook What are the treatment options for lung abscess? (739) What are the indications for surgical therapy of bronchiectasis? (740) What are the indications for surgical therapy in the treatment of histoplasmosis of the lung? (747) Describe the surgical management of patients with non-small cell lung carcinoma. (755-758) What is the most common paraneoplastic syndrome related to lung cancer? (752) Outline the appropriate approach for a solitary pulmonary nodule. (762)

Figure. Portion of a weekly reading examination.

All of the residents who participated in the program demonstrated substantial improvement in their ABSITE scores. There were a number of activities that residents added to the program that may also be a part of their success. Most of the residents also used MCQ type review manuals and at least one took an ABSITE review course. All struggled on occasion to adhere to the schedule but the majority responded to e-mail prompts. In some cases, residents failed to repeat the program in subsequent years and have struggled to obtain acceptable scores on the ABSITE. Some of the residents subsequently developed their own study program and have continued to perform at acceptable levels on the ABSITE and have passed the American Board of Surgery Qualifying Examination. The major resource required for the program was the time it took to initially develop the questions. Faculty time expenditure was kept to a minimum as there were no formal meetings to discuss the readings. Instead, the completed exami-nations were reviewed to assure that the assignments were being taken seriously but were not formally corrected and returned to the resident. The faculty time invested has been worthwhile as the program allowed motivated residents to address a deficiency that caused them frustration. The degree of resident adherence to the program added unique information in the resident progress meetings. In some cases, the program prompted residents to adopt behaviors that will enhance their test preparation throughout their careers.

References:

 

Online March 30, 2005