SEARCH
Division of Education
for CONTENT

 

EMAIL   PRINT PRINT

RAP Home

Promoting Excellence in Surgical Care Through Education, Training, and Validation

E-Learning Resource Center

Division of Education > RAP > Previous Articles

Standards-Based Portfolio Serves as a Flexible Tool to Guide Surgical Residents Toward Required Standards of Training Programs

Ruth H Nawotniak, MS, James M Hassett, MD and Merril T Dayton, MD
Department of Surgery, State University of New York at Buffalo

Background

A variety of forces are having a significant impact on surgical training programs that require such programs to be re-engineered. Decreases in clinical contact hours—coupled with increases in documentation and requirements for demonstration of outcomes—pressure program directors to improve efficiency and efficacy of the educational effort. These forces result in multiple challenges to:

  1. create strategies that manage resident time
  2. meet all goals and objectives required of the program
  3. satisfy the requirements for qualification for board certification
  4. document experience
  5. guarantee career development.

This paper describes the development and utilization of a standards-based portfolio to help meet these challenges.

Introduction

In the article :An Evaluation of a Standards-Based Portfolio," Gallagher states: "A portfolio is generally conceptualized as a collection of the student's best work or a collection of work that evidences incremental development." He defines a standards-based portfolio as "a series of student work that seeks to address pre-determined learning outcomes."1 We are using the term standards-based portfolio to mean a specific manner of documentation that supports the work effort of the surgeon-in-training and guides the resident towards the standards that are required for completion of the training program.

Although relatively new in the United States, the standards-based portfolio is a vetted mechanism in the United Kingdom; it is used by the Royal College of Physicians and teaching institutions and training programs in both nursing and physician education in the UK.2,3,4,5,6 The Accreditation Committee for Graduate Medical Education (ACGME) has validated the use of a portfolio system as shown by its presence on their Web site in their "Toolbox of Assessment Methods for the Outcome Project."7

We have taken the concept of a standards-based portfolio system and extended it to be both a career development resource and a tool for managing ongoing procedural training. This initiative grew from a previously developed resident credentialing program. In its present form, it is a compilation of a resident's work efforts and tasks, which are associated with the required components of the resident's educational curriculum and clinical experience. Completion of all tasks results in successful completion of the program.

In the development of a standards-based portfolio, there are four issues:

  1. identification of the standard and the criteria being assessed
  2. the method of assessment or evaluation
  3. thresholds for successful completion of the task
  4. documentation of the effort and outcome.

Each standard includes components from:

  1. the resident's career goals
  2. performance criteria set by the American Board of Surgery (ABS)
  3. core competencies described by the ACGME
  4. procedural experiences in the training program.

We based our standards for completion of the training program by incorporating the following criteria:

The challenge of the standards-based portfolio is to efficiently manage the documentation process. To meet this challenge, we use database computer software programs to track procedural data and educational assignments, as well as commercially available six-sectioned file folders to house each resident's documentation.

Organization

The standards-based portfolio is divided into these sections:

  1. General Information
  2. Credentialing Documentation
  3. Department Credentials
  4. Examination Scores
  5. Professional Growth and Development
  6. Evaluations.

General Information

This section includes documentation of the resident's educational history that will support a request for privileges at health care facilities or for a future fellowship. This documentation includes: the medical school diploma, dean's letter, and transcript; documentation regarding a previous residency for residents who have transferred into the program; and a copy of the certificate of completion of the training program. A notarized copy of the resident's certificate of completion is also included as some health care facilities are now asking for this in their application. Also included is a copy of the resident's application for the American Board of Surgery examination.

Credentialing Documentation

This section contains the work effort of the resident towards meeting the requirements set by the ABS for acceptance into the Board examination and for health care institution privileging requests. A database computer program is used to track the procedural activity of the resident. A Defined Category Report is used to show acquisition of the minimum index case requirements. This is augmented by a copy of the final complete case list, in both paper and disk formats using Excel, and a copy of the Resident Record, a numerical summary report of case acquisition that is (1) required by the ABS with the application for the Qualifying Exam and (2) instrumental in the graduated resident's request for hospital privileges. The case list on disk allows the graduate to support his/her request for privileges in specific surgical procedures for which a hospital may require documentation. To meet the university's requirement to track the ability of the resident to perform bedside procedures independently, we utilize an online database program created by the university. The report generated is kept in the portfolio as documentation. The resident's rotation schedule, which is required by the ABS application process, is kept here as well. The final pieces of documentation kept in this section are evidence of certification in courses such as Basic Life Support, Advanced Cardiac Life Support, and Advanced Trauma Life Support and successful completion of federal and state mandated courses (eg, HIPAA Awareness, Infection Control).

Department Credentials

This section contains documentation that addresses criteria set by the ACGME and the department including:

certificates of mastery in areas of general surgery training as the resident proceeds throughout the training program, (eg, conscious sedation, basic surgical care, resuscitation, critical care, transplantation, general surgery, vascular surgery, pediatric surgery, and thoracic surgery, and

certificates of course completion within the departmental educational curriculum. The PGY1 and 2 residents concentrate on the basic sciences. In this course, specific elements of medical knowledge must be mastered before a certificate can be awarded. Palliative care and geriatric surgery programs have been implemented and a patient safety course is being designed that will be part of the PGY1 educational curriculum. At the PGY3 level, there is a problem-based learning project that is designed to improve the resident's knowledge of and experience with process improvement and quality assurance. At the PGY4 level, basic science is revisited. This time, the PGY4 resident is teaching the course, thereby developing his/her teaching and communication skills. At the PGY5 level, there are courses on risk management, coding, cultural awareness, reporting of sentinel events, and expository writing for medical documentation. The curriculum also includes courses that span the five years of training in verbal/non-verbal communication, which utilize standardized patients as one facet. The resident chooses the best of his/her work effort in those areas to submit for completion of the course. This is attached to the department credential.

Examinations

This section contains test results. These include the United States Medical Licensing Examination (USMLE) scores as submitted with the application and the yearly ABSITE scores. These reports aid in documenting compliance to the standards set by the department and are required for possible future fellowships.

Professional Growth and Development

All resident presentations given at grand rounds, hospital conferences, regional and/or national conferences, as well as poster presentations and publications, are included in this section. As the department continues to develop the management tools to track resident growth and development within the competencies, this section will contain the individual resident's work effort towards his/her successful completion.

Evaluations

We use several types of assessment and evaluation tools including self-assessment, 360, faculty evaluations, and program director evaluations. The faculty evaluates the residents after each rotation. The program director meets with each resident at least twice a year and reviews these evaluations. Prior to these meetings, the resident completes a self-evaluation, which is reviewed at the meeting. This self-evaluation, as well as a summary of the rotation evaluations and the program director's evaluation, is kept here. Also included here is the final exit interview evaluation at the end of training. This type of tracking allows the resident to monitor and review not only the faculty's perception of his/her progress, but also the resident's self-perception as he/she continues to develop. The senior residents will also have a composite evaluation from their peer residents that addresses their development in leadership and teaching skills.

Each of the six competencies of the ACGME's Outcomes Project, as well as the development of the whole physician throughout the five-year program, is extensively documented in the standards-based portfolio. In this utilization, the standards-based portfolio allows documentation of growth and maturation. Multiple competencies are covered in any task or element of the curriculum. Still, there are specific parts of the curriculum that lend themselves to particular competencies.

Growth and certification in medical knowledge is shown through the case logs, bedside credentialing program, successful completion of the basic science curriculum, in-service examinations, and presentations. Progress in applying medical knowledge to patient care is shown in the quality assurance presentations and faculty evaluations. Development of good communication skills, both verbal and written, is evidenced through the communication curriculum that runs through all five years; the third-year expository writing exercises on adverse outcome reporting and chart notation writing; and resident teaching, presentations, and publications. Professionalism is documented through the evaluations of the resident's progression through the communication curriculum and in the education in the processes of risk management and quality assurance. Practice-based learning and systems-based practice are tracked through extensive presentations at hospital-based quality assurance, "Mortality and Morbidity" or outcome conferences, and through the resident's performance in the palliative care and surgical geriatrics courses. They are also found in the multiple assignments at seven different clinical venues.

The portfolio is given to the resident upon successful completion of the training program. It then becomes a resource for career development and job placement as it contains all the basic information needed for the graduated surgeon-in-training to support his/her request for privileges at hospitals or health care entities.

The success of the portfolio as a career development resource has gone from concept to reality. We have begun receiving feedback from our graduated surgeons-in-training as to the effectiveness of the portfolio in helping them to manage their hospital staff applications. Having the documentation required by the health care institutions at their fingertips not only expedites the application process but allows the new surgeon to become a contributing member of a medical staff in a more timely manner.

We use the standards-based portfolio as a dynamic document. As we re-engineer the educational curriculum, the standards-based portfolio will continue to develop. It is a flexible tool for managing the documentation needed to show the growth and development of a surgeon-in-training towards becoming a complete physician and in meeting the standards set by the department, university, ACGME, and ABS for competence.

Summary

A standards-based portfolio brings value to the educational environment and allows documentation of both effort and mastery of the various clinical tasks. Defining the standards is a controlling activity that is valuable when outcomes are measured. It is a flexible tool that can respond to changes in the clinical environment. It can be constructed to meet the standards and needs of any training program. It provides excellent documentation of the core competencies. The standards-based portfolio provides value to the resident, who knows the specific goals and objectives of each task and assignment. It allows the resident to chart progress and can be used to support requests for independent clinical privileges or applications for fellowship and clinical appointments.

References

  1. Gallegher P. An evaluation of a standards based portfolio. Nurse Education Today. July 2001.
  2. Mathers NJ. Challis MC. Howe AC. Portfolios in continuing medical education—effective and efficient? Comment in: Med Educ.:33(7):521-30. July 1999.
  3. Gallagher P. An evaluation of a standards based portfolio. Nurse Education Today 21(3):197-200. Apr 2001.
  4. Fung MF. Wlaker M. Fung KF. An internet-based learning portfolio in resident education: The KOALA multicentre programme. Medical Education 34(6):474-9. June 2000.
  5. Reece SM. Pearce CW. Melillo KD. The faculty portfolio: documenting the scholarship of teaching. Journal of Professional Nursing 17(4):180-6. Jul-Aug 2001.
  6. Brooks BA. Madda M. How to organize a professional portfolio for staff and career development. Journal for Nurses in Staff Development 15(1):5-10, Jan-Feb 1999.
  7. Toolbox of Assessment Methods. <www.acgme.org> Accessed 10/6/2003. Chicago, Illinois 2001.
  8. Program Requirements in Surgery. <www.acgme.org> Accessed 10/6/2003. Chicago, Illinois 2003.
  9. Common Program Requirements. <www.acgme.org> Accessed 10/6/2003. Chicago, Illinois 2003.

 

Revised January 18, 2005