July 21, 2020
The Surgical Metrics Project Research Advisory Board Members
- Peter Angelos, MD, PhD, FACS
- Megan Applewhite, MD, FACS
- Jo Buyske, MD, FACS
- E. Patchen Dellinger, MD, FACS
- David Hoyt, MD, FACS
- Carlos Pellegrini, MD, FACS, FRCSI(Hon), FRCS(Hon), FRCSEd(Hon)
- Steven Stain, MD, FACS
- Patricia Turner, MD, MBA, FACS
Committee on Surgical Skills Training for Practicing Surgeons
- Barbara Bass, MD, FACS, FRCS(Hon), FRCSI(Hon), FCOSECSA(Hon)
- Ajit Sachdeva, MD, FACS, FRCSC, FSACME
Clinical Congress 2019 provided the opportunity for surgeons to participate in The Surgical Metrics Project. A total of 255 attendees visited The Surgical Metrics Project booth to explore the use of wearable technologies as a means of measuring surgical decision making and surgical technique. Spearheading the effort is Carla M. Pugh, MD, PhD, FACS, coauthor of this article and professor of surgery, and her research team at the Technology Enabled Clinical Improvement (T.E.C.I.) Center, Stanford University, CA. The initial goal of The Surgical Metrics Project is to enable information exchange and facilitate data sharing. It is imperative that surgeons lead the discussion about how their data should be managed and applied.
The Surgical Metrics Project exhibit contained 10 surgical simulation stations where participants were equipped with wearable technologies and asked to “run the bowel” and perform an open repair of any small bowel enterotomies identified in a segment of porcine intestines.
Upon entering The Surgical Metrics Project exhibit, participants completed a one-page demographic survey. A summary of the survey results are shown in Figure 1.
After providing their demographic information, participants were fitted with a wearable electroencephalography (EEG) sensor to record and measure brain activity (see Figures 2a–2c). The sensors were attached to their foreheads, at which point participants were led to the cognitive testing area to undergo a baseline assessment of their cognitive skills.
Figure 1. Survey respondent demographics (n = 255)
Participants were then outfitted with magnetic motion-tracking sensors on their hands and an audio recorder on their lab coat. Once participants were fully instrumented with the wearable technologies (see Figure 2d), they stepped into one of the 10 simulated surgical environments equipped with two video cameras, a stopwatch, one surgical assistant, and all of the surgical tools necessary to operate on their “patient.”
Figure 2. The Surgical Metrics Project in action
The research team used a template and surgical tools to pre-injure the porcine tissue in the lab such that each surgeon was presented with the same case. Surgeons had varying completion times and approaches to the procedure. The team was able to capture all of these preferences and individual decisions using magnetic motion-tracking sensors, audio recorders, video cameras, and EEG sensors.
Upon completion of the simulated surgical procedure, researchers documented the surgeon’s procedure time and performed a leak test to determine the quality of the repair (see Figure 2e).
The standardized operative surgical task and wearable technologies allowed the team to collect baseline data to better understand how surgeons’ decision making, technical skill, and communication strategies contribute to procedural outcomes.
Additional Information
Additional information and videos about The Surgical Metrics Project are available at the following links:
The Clinical Congress News, the daily paper published during the conference, queried participants for their initial reactions to The Surgical Metrics Project. “It’s really important to get baseline data in terms of where surgeons in practice are—in both their cognitive and technical abilities,” said Sabha Ganai, MD, PhD, FACS, Springfield, IL.
Arthur Berg, DO, Hackensack, NJ, said, “This is an interesting way to see and quantify variations of surgeon techniques in terms of different movements and different decisions. It’s really great that they’re actually using objective data—EEG monitoring—to see the different variations in technique, and it will be cool to see the differences between beginners and more highly trained, experienced surgeons.”
“This can help us to understand what the differences are between novices and experienced surgeons. Once we understand the differences, we’ll be able to get novice surgeons looking more like experienced surgeons in a shorter period of time. The value is getting lots of people to do it, so having it here at Clinical Congress is a great chance to get real data that will yield real results,” said David M. Notrica, MD, FACS, FAAP, Phoenix, AZ.
In addition to the anecdotes shared with the Clinical Congress News, several participants engaged in The Surgical Metrics Project conversation on Twitter. A tweet by Andrew S. Wright, MD, FACS, Seattle, WA, described his experience with The Surgical Metrics Project and encouraged others to participate (see photo).
Paving the way for a new frontier of data-driven metrics to advance quality, The Surgical Metrics Project aims to include all members of the surgical community in the conversation regarding data use and data sharing. As part of The Surgical Metrics Project data collection, Dr. Pugh and her research team had the opportunity to gauge participants’ attitudes regarding data and assessment. For the responses, see Figure 3.
Figure 3. Attitudes regarding data and assessment (n = 255)
Ongoing data analysis is now taking place and with guidance from The Surgical Metrics Project Advisory Board (see sidebar), Dr. Pugh and her research team will determine how to distribute feedback to participants. The overall goal is to develop an anonymous database of video, motion, and audio data to facilitate surgical planning, training, and review.
“The Surgical Metrics Project is the foundation of assessing surgical technique,” according to American College of Surgeons Executive Director David B. Hoyt, MD, FACS. “Once validated it can define how we might measure skills acquisition, obtain mastery, and even demonstrate deterioration of skills over time.”
The College is proud to announce the continuation of its strategic partnership with the T.E.C.I. Center and plans to continue offering The Surgical Metrics Project at Clinical Congress.
The T.E.C.I. Center is a research group directed by Dr. Pugh that aims to transform human health and welfare through advances in data science and personalized, data-driven performance metrics for health care providers. More information about the T.E.C.I. Center is available on the Stanford Medicine website.