American College Of Surgeons - Inspiring Quality: Highest Standards, Better Outcomes

Surgeons help create a more predictive scorecard

OCTOBER 23, 2017
Clinical Congress Daily Highlights, Monday Second Edition

Surgeon scorecards are designed to improve care, enhance transparency, and empower patients to make more informed decisions. According to James Mossner, University of Michigan Medical School, Ann Arbor, MI, these efforts have yet to do this accurately and effectively.

On Monday afternoon, Mr. Mossner presented the Michigan Surgeon Scorecard, which uses multiple measures to generate a composite score that may provide a more reliable measure of surgeon performance than existing scorecards.

Perhaps the most well-known surgeon scorecard currently used is one developed by ProPublica. But while this scorecard represents an important step in advancing objective measures to evaluate surgeons, it also has several methodological weaknesses. It uses Medicare billing data, fails to control for clinical perioperative patient level comorbidities, and has never been tested for its ability to predict future performance.

Many current scorecards are flawed because they heavily weigh mortality, a relatively rare event in surgery. While important, this metric can render scorecards less sensitive, and it places surgeons who operate on highly morbid patients at a considerable disadvantage.

The Michigan Surgeon Scorecard comprises a composite measure of surgeon performance on colectomies across five domains: survival, morbidity avoidance, anastomotic leak avoidance, surgical site infection (SSI) bundle compliance, and utilization. These domains were selected based on input from surgeons across the state of Michigan.

The reproducibility and predictability of the Michigan Surgeon Scorecard was studied using clinical registry data on 3,387 colon operations from the Michigan Surgical Quality Collaborative. These data revealed that the Michigan Surgeon Scorecard was able to predict future performance with 89 percent accuracy, compared to the 32 percent accuracy achieved by the ProPublica Surgeon Scorecard. As Mr. Mossner explained, “If you were to take a one-star surgeon at one time point, and without any intervention, reevaluate them later, they would still probably be a one-star surgeon 89 percent of the time using our scorecard.”

Mr. Mossner and his team continue to collect data and refine the scorecard. Adding procedure-specific metrics, for example, may produce a more comprehensive score.

These findings highlight the potential to provide domain-specific feedback to surgeons and help improve performance. The presentation concluded by identifying possible uses of this scorecard, such as pairing lower-performing surgeons with higher-performing ones to help improve performance.

Additional Information:
The Scientific Forum, Health Services Research I, was held October 23 at the 2017 Clinical Congress of the American College of Surgeons in San Diego, CA. Program, webcast and audio information is available online at FACS.org/clincon2017.

Return to Index