Strategies Tied to Your Program's Vision can Enhance the Resident Selection System

Kimberly D. Anderson, PhD
Professor of Surgery
University of Texas-Houston

Much has been written about the factors attracting and detracting students from the field of surgery. But once the applicants hit the door, what can be done to ensure that you sustain their interest in the program? And furthermore, what can you do to select those candidates who will have staying power in your program?

The single most important concept that needs to be shared and understood by faculty and residents alike is the program's vision. It is essential that everyone participating in the recruitment and selection process be oriented to the goals and expectations of the program and the process that will be followed. It is not enough to know what your program uniquely has to offer, it is about conveying your mission in a consistent and tangible manner.¹ If your goal is to train academic surgeons who are facile in research, then ensure that you screen for those candidates with a research background; make presentations by departmental researchers part of your orientation to the program; give tours of labs; and promote research productivity of faculty and residents on Web sites and through written materials distributed to applicants.

Similarly, if your mission is to train competent community surgeons, focus on breadth and depth of operative experiences, clinical decision-making, and opportunities to develop broad-based skills that will provide graduates with the armamentarium they will need to be successful. Not surprisingly, researchers have demonstrated that residencies conveying a friendly atmosphere, favorable interpersonal skills of enrolled residents, and appealing personali- ties of the faculty were more attractive to candidates.² Laskin & Lesney found that the greatest predictors of resident satisfaction included being satisfied with match outcomes and accuracy of the program in representing itself during the interview process.³ Know that applicants who have been well mentored will be evaluating programs with an eye to: academics (conferences, what graduates are doing, board pass rate and success in acquiring fellowships, research, evaluation); leadership (accreditation status, stability and vision of department leaders); work environment (level specific training, case-load and operative experience, call/vacation schedule, clinic, and nursing); and lifestyle issues (happiness quotient of residents and their spouses/significant others, camaraderie among residents, affordability of housing, parking, and safety). Be prepared to address these areas.

The goal of any interview day is to ensure that applicants leave with a clear picture of the program and no unanswered questions, thus feeling that it was an experience that was well worth their time and effort. However, the role of faculty and residents in the interviewing process requires balance. Noting that increasing pressures on available faculty resources have made recruitment more difficult, Milne and Bellini explored whether residents could partially replace faculty in the interview process. They examined what role the interview itself plays in the overall assessment of an applicant, compared faculty and resident rating of the interview, and asked whether partially substituting residents for faculty had an impact on the match outcome. Two different interview models were developed—a resident-faculty and a resident-faculty-faculty model. The formal interview was found to be highly and equally correlated with the final score for both faculty and residents, and there was no significant difference in the percentage of the applicants that matched from each interview model. They concluded that using residents as interviewers is a viable alternative—residents are knowledgeable about the program and institution, are likely to more easily relate to applicants, are easy to recruit, and are proud to participate.⁴

Predicting which candidates will be successful in your program is certainly more problematic. In 1999, Wagoner and Suriano conducted a comprehensive study of program directors across specialties concerning criteria used in the selection process. They found that program directors in orthopaedic surgery, general surgery, OB/Gyn, and ophthalmology valued these criteria most highly in applicants: grades in required clerkships; numbers of honors grades; class rank; USMLE Step 1 scores; and membership in Alpha Omega Alpha. When looking at involvement in extracurricular activities, all program directors ranked "leadership roles" as highly important. The personal characteristics sought in applicants were (in descending order of importance): compatibility with program; commitment to hard work; fund of knowledge; empathy and compassion; and communication skills.⁵ While researchers have attempted to determine which academic qualifications and personal qualities are predictive of success in residency, results have been mixed.

In a study correlating selection criteria with subsequent performance as residents in orthopaedic surgery, Dirschl et al retrospectively looked at five categories:

• Cognitive (measures of academic achievement to include clinical grades, manuscripts published)
• Potential predictors of motor skill (number of activities that involved gross motors skills, ie: athletics, carpentry, or auto repair)
• Number of activities that entailed fine motor skills (ie: playing a musical instrument, painting, woodworking)
• Leadership (volunteer activities and leadership activities)
• Subjective measures (letters of recommendation)

All indicators were significant in predicting the overall performance of residents. Academic performance in clinical clerkships in medical school was most predictive of subsequent overall performance; election to AOA was moderately predictive; and participation in many activities involving fine motor skills was slightly predictive of psychomotor performance.⁶

Bernstein et al, reviewed orthopaedic resident-selection criteria in depth. They paid particular attention to the importance of the affective domain stating "it is the most common factor leading to discipline or dismissal of a resident and is the most important indicator of his or her ability to function professionally." Clearly the most reasonable approach to evalua-ting a candidate's affective domain begins with personal interaction and direct observation. Unfortunately, the best current method for such prolonged evaluation is through the student performing a clerkship at the Program Director's institution—which is not always feasible or in the best interest of the students' educations.⁷ It has been suggested that an alternative to the "audition elective" is through the evaluation of a candidate's moral reasoning.

While new to general surgery, the role of moral reasoning in the interviewing process and its correlation with performance has been studied in depth by such authors as Kohlberg, Rest, Sheehan, Self, and Baldwin. What is moral reasoning and how is it measured? In short, the method consists of subjects taking a Defining Issues Test which is a paper-and-pencil, objective test that can be group-administered and computer-scored, and takes 20 to 30 minutes to complete.⁸ The test asks subjects to resolve a series of moral dilemmas. Each dilemma is followed by a systematic set of questions that enable the subject to reveal the logic/structure of his/her moral reasoning. Thus, what is being tested is the applicant's capacity for moral reasoning about conflicts in values, and not the applicant's particular set of moral beliefs or values.

Numerous studies in a vast array of medical disciplines have demonstrated that there is a significant relationship between moral reasoning skills and clinical performance and that this provides a better determination of future performance than any of the academic measures currently utilized. Self, et al have suggested that programs currently attempt to assess applicants' moral reasoning in a haphazard manner—typically through personal interview questions.⁸ The questions are designed to get at applicants' values without explicitly acknowledging this purpose. Applicants frequently recognize the underlying purpose of the question and try to "game" the answers. As a result, there is little data to suggest that the questions improve the selection process at all. Furthermore, such questions are generally aimed at capturing an applicant's personal moral values, not their moral reasoning ability. In 1980, Sheehan and colleagues reported that clinical performance among pediatric residents was highly correlated with their levels of moral reasoning as measured by the Defining Issues Test. Residents with the highest levels of moral reasoning rarely were rated by their supervisors to be in the least clinically competent group. Residents with lower moral reasoning scores were seldom, if ever, rated as being superior clinical performers.9 Subsequent studies in family practice, orthopaedic surgery, medical students, dental students, and nursing students have confirmed this relationship.

Recruiting and retaining the best and brightest to the field of surgery is the lifeblood of our surgery residency programs. In an era of exploding technology and limited resources, it is incumbent upon residency programs to adapt and adopt—that is, to adapt tried and true methods of recruitment to those that best utilize the time and energy of faculty and residents, and to adopt new methods of assessing those personal qualities that are predictive of success.

References

1. Surgical Educators' Handbook, Association for Surgical Education, p. 64-73, 2002.
2. Marciani RD, Smith TA, Heaton LJ: Applicants' opinions about the selection process for oral and maxillofacial surgery programs. J Oral Maxillofac Surg, 61: 608-614, 2003.
3. Laskin DM, Lesney RJ, Best AM: The residents' viewpoint of the matching process, factors including their program selection, and satisfaction with the results. J Oral Maxillofac Surg, 61:228-33, 2003.
4. Milne CK, Bellini LM, Ravenell KL, et al: Residents as members of intern selection committees: can they partially replace faculty? Teaching and Learning in Medicine, 15: 242-246, 2003.
5. Wagoner NE, Suriano JR: Program directors' responses to a survey on variables used to select residents in a time of change. Acad Med, 74: 51-58, 1999.
6. Dirshl DR, Dahners LE, Adams GL, et al: Correlating selection criteria with subsequent performance as residents. Clin Orthop, 399: 265-71, 2002.
7. Bernstein AD, Jazrawi LM, et al: Orthopaedic Resident-Selection Criteria. The Journal of Bone and Joint Surgery, 84: 2090-2096, 2002.
8. Self DJ, Baldwin DC: Should moral reasoning serve as a criterion for student and resident selection? Clin Orthop, 1: 115-123, 2000.
9. Sheehan TJ, Husted S, Candee D, et al: Moral judgment as a predictor of clinical performance. Eval Health Professions, 3: 393-404, 1980.