SEARCH
Division of Education
for CONTENT

 

EMAIL   PRINT PRINT

RAP

Promoting Excellence in Surgical Care Through Education, Training, and Validation

E-Learning Resource Center

Division of Education > RAP > Previous Articles

Visual perception in surgery: How can its challenges be addressed to improve training and performance?

Stan Hamstra, PhD
Associate Professor, Department of Medical Education
University of Michigan, Ann Arbor, MI

Of the five senses, sight and touch are unquestionably the most important for the performance of surgery. No doubt, this is the reason that we find so many papers written on visual-spatial abilities, psychomotor skills, and more recently, the impact of video game experience on surgical performance. Some of these papers are helpful, some provocative and some appear to be simply baffling and esoteric to the practicing surgeon or surgical educator. The aim of this short review is to provide an overview of current research and thinking on visual perception and its implications for surgical performance and training. (I leave the complex topics of touch, haptics, and proprioception to others.)

1) The surgical environment poses many perceptual challenges to the surgeon

Whether you are operating in an open or laparoscopic environment, there are many perceptual constraints that affect performance and learning. In the open surgical environment, the surgeon generally can simply expand the incision or introduce extra light sources or retractors to optimize the view. However, there are certain surgical approaches or conditions that are, by their very nature, perceptually constrained. For example, deep brain access in neurosurgery necessarily constrains the view angle and hence reduces the likelihood of achieving true stereoscopic depth perception. Similarly, visual identification of integrated tumor boundaries in white matter is often difficult, if not impossible. In parathyroid surgery, the visual field is often occluded with blood and anatomical membranes that make it difficult to isolate target tissue boundaries.

In laparoscopic or minimally invasive approaches, visual perceptual constraints are much more profound. The surgeon is constrained by the information the camera provides and his or her underlying knowledge of the surgical anatomy. The camera is not only con-strained by reduced degrees of freedom of movement, but also by limitations in illumination, depth cues, and geometry of approach. For example, a co-axial light source severely restricts the ability to benefit from the depth cue of shading or direction of illumination. Fortunately, the field of vision science has developed a taxonomy for parsing these problems into a manageable framework.

Vision scientists generally agree that the process of visual perception can be broken down into a few major dimensions: luminance, color, motion, depth, and texture (which can be considered a derivative of luminance contrast). And in fact, there are separate neural mechanisms in the human brain that mediate these different dimensions of performance and act more or less independently. That is, if you lose the motion system, you can still process color and depth, and so on. A key concept here is that useful perceptual information is provided by contrast in any of the five domains. For example, luminance contrast can be thought of as the degree of sharpness between black letters and a white page. Reduce the contrast and the perceptual information is reduced. Similarly, motion contrast can be thought of as the relative motion between two objects in the visual field. Consider a camouflaged animal in the wild; reducing the relative motion between the animal and its background reduces the motion contrast and, hence, the amount of perceptual information. (Consider that the term "breaking camouflage" refers specifically to the introduction of relative motion in the visual field.)

By using this framework, we can begin to isolate perceptual constraints in surgery to manageable constructs, which can then be studied in isolation. For example, if the parathyroid surgeon is constrained by a lack of color contrast or texture contrast in his or her surgical view, it may be possible to imagine a method for enhancing contrast in those domains. In this case, depth and motion may be somewhat irrelevant to the task at hand.

Alternatively, depth perception may be the most important domain in laparoscopic surgery and it may be most effective to focus on methods for manipulating variables that could enhance depth contrast, such as motion parallax, shading, occlusion, and stereopsis. (A full list of depth contrast variables that have been identified by vision scientists would also include texture gradient, light source assumptions, relative size of familiar objects, and linear perspective. Thus, these variables form a list of potential candidates for studying how to improve depth perception in laparoscopic surgery.)

Finally, there are other major influences on surgical performance that have been identified as belonging to the class of visual perception but are probably miscategorized. These include simple familiarity with instrumentation and the more complex topic of visuo-proprioceptive interaction (a term that invokes visualization or mental imagery for planning motor actions).

2) How these perceptual challenges can be addressed

So how can surgeons respond to these perceptual challenges? One option is to change the equipment, the other option is to change the person using the equipment, either by selection or training.

 

Changes in instrumentation to compensate for perceptual limitations

There have been many attempts to improve surgical methods by developing innovative changes in instrumentation. There is of course the famous but failed attempt to incorporate stereoscopic vision into the standard laparoscopic environment.1 This approach failed because of limitations in geometry (the distal end of the laparoscopic instrument did not readily sup- port two cameras at a far enough distance from each other) and usability (ie, the surgeon had to wear special goggles and remain fixed in a very narrow range of positions along the line of sight). (It should be noted that stereopsis has been successfully implemented in robotic- assisted surgery because of the controlled environment.) Nevertheless, attempts to improve instrumentation continue. For example, an interesting method to combat the lack of depth perception in minimal-access views is to employ what is essentially an articulating endoscope;2 this can be used to gain more information about the depth contrast of objects in the visual field by allowing for non-axial views. It remains to be seen whether these improvements take hold in the surgical community. Certainly, it seems success will depend on a combined emphasis on usability and an understanding of the principles of visual perception.

Innate differences among individuals in their ability to process the perceptual constraints

There is a considerable amount of debate and dialogue regarding whether or not surgeons are born with innate perceptual abilities which make them more likely to succeed, or whether these perceptual abilities can be learned.

In a number of studies, subjects who appeared to show innate differences in visual-spatial ability performed equally well on surgical tasks following a brief intervention. Also, expert craniofacial surgeons who perform spatially complex surgical tasks on a regular basis were found to have visual-spatial test scores around the norm for the general population. Taken together, these findings suggest that surgical performance is related less to innate perceptual abilities than repeated practice or training during residency and fellowship. It seems that intimate knowledge of and experience with surgical tasks, when combined with competent intraoperative judgment, may overshadow any advantage afforded by superior perceptual ability.

Thus, there is very little evidence to suggest that we should select trainees on the basis of perceptual abilities, whether innate or gained from experience (ie, as in video game usage; see next section).

Perceptual training in surgery

Certainly we know that surgical abilities are learned. Otherwise, we would not have surgical training programs. But should we incorporate perceptual learning into the surgical curriculum? Can perceptual ability be learned?

This question has received considerable attention recently with the publication of a series of papers on whether practice and experience with video games helps one become proficient in laparoscopic surgery. While the intuitive appeal of this question is intriguing, it appears that video game experience is more directly related to developing a familiarity with “screen-mediated task execution” than any particular aspect of visual perception per se. 3,4,5 The main issue seems to be that extensive prior video game experience creates familiarity with an interface that is similar to the interface used in laparoscopic surgery. A more fruitful approach is suggested by the previously discussed concern regarding the specificity of visual perceptual processing into various domains. For example, if we examine in detail what exactly is being practiced in video games, we find that very little of it is directly targeted at the specific visual skills required for performing laparoscopic surgery. Clearly, video games require skill in detecting motion and color contrast. But very few games involve the detailed depth contrast tasks that are required in laparoscopic surgery, and none, to my knowledge, require consistent focused skill development in manipulating objects in specific depth relations. In fact, the bulk of skills required in video game playing have more to do with psychomotor performance and instrument interface skills.

A careful analysis of laparoscopic skills suggests a greater emphasis on depth contrast, specifically the ability to extract three-dimensional (3D) object relations from two dimensions (2D). In fact, this ability has been repeatedly highlighted over the years by those studying laparoscopic skills acquisition. To study this specific ability directly, Sidhu et al. 6 developed a unique method for gauging the perceived depth relations of objects presented in a 2D display. The results of this study provide some support for the claim that visual perceptual skills necessary for understanding 3D structure can be learned or improved with practice. Although this study focused entirely on perceptual abilities and not procedural skills, the results have direct implications for technical skill training. Starting from a theory-guided framework offered by vision research and physics, the authors developed a unique assessment tool to determine the extent of 3D structure that subjects perceived in 2D displays of clinical images. In this case, the images were CT angiograms of abdominal aortic aneurysms (AAA) that were rendered in 2D. Typically, endovascular access and manipulation is guided by these images, with the implication that the vascular surgeon or radiologist infers 3D structure from the 2D information presented. The assessment tool produced a "depth map," representing each subject’s 3D interpretation of the 2D angiogram. Depth maps were found to be significantly different between experts and novices. In addition, following a brief intervention focused on the 3D structure of angiogram representations, novice depth maps were found to be similar to those of the expert group. Thus, it appears that perception of overall surface contour of 3D structures from 2D images can be affected by experience and training. The authors were careful to note that this improvement in perceptual skill did not make the novices expert surgeons; the intervention simply improved one aspect of performance that is necessary to perform adequately. By applying methods of vision science to an important problem in surgery, this research helps us understand the nature of visual perceptual processes involved in the execution of an increasingly common clinical task. This research uses applied tools in vision science to understand the perceptual constraints involved in related areas, such as minimally invasive surgery.

From these results, we can conclude that 3D from 2D can be learned, but it is probably very specific to the context. In other words, if vascular surgery residents learn the 3D anatomy of vascularization, this ability will help them interpret future exposure to 2D representations involving that anatomy. Similarly, it would seem that a detailed understanding of the anatomy involved in laparoscopic surgery might assist the trainee in adequately perceiving the proper depth relations of tissue presented in the typical 2D laparoscopic view. However, there is no evidence either way that such understanding will help the trainee in performing accurately from a 2D view; getting that evidence requires additional research involving psychomotor performance in addition to addressing the perceptual question raised by Sidhu et al. Similarly, it is unclear whether perceptual training in the specific task of making 3D judgments based on 2D presentations would help. Based on other research in specific perceptual learning, it would seem the benefits would be relatively small, very specific, and involve a great deal of practice.7,8 This result follows naturally from the fact that the neural mechanisms involved in these skills are at a "lower level" in the neural pathway that mediates visual perception, and thus, less susceptible to the effects of plasticity or learning. (It is widely held that "higher-level" neural processes, such as those involved in cognition, are much more susceptible to systematic change such as that which occurs in the process of learning. Another way to think of this issue is that the neurons in your retina are not likely to change their processing characteristics, whereas neurons that mediate cognition are changing all the time.)

 

3) Conclusions

It seems that perceptual abilities can be learned to some extent, but whether or not these abilities are great enough to affect performance on real-world tasks remains an open question. Certainly, it seems helpful to do a thorough task analysis of the target surgical performance before isolating a particular skill to practice. In a recent review, Ericsson and colleagues 9 concluded that there is no evidence that innate abilities determine expertise; rather it seems that expertise in any discipline is largely determined by extensive amounts of deliberate practice. It is likely that perceptual abilities are subject to the same forces, with only a minor amount of surgical performance being attributable to perceptual abilities, whether innate or learned.

Most of the conclusions offered in this article are fairly tenuous. That’s because perceptual ability is a difficult area in which to conduct quality research. One way to improve the potential of this research area is to systematically approach the problem from both sides in more detail. That is, what exactly is it about visual perception that we are interested in? Depth perception? Texture contrast? Spatial relations? Or is our interest in something that isn’t even related to visual perception, such as selective attention, or psychomotor skill? In the same vein, what is it about surgical ability that we are exactly interested in? Technical ability? Flow of the operation? Error avoidance? Time? Many of the simulator-based metrics use time and error count in their primary measure of performance. But time alone is well known to be flawed as a measure of performance (you can be fast and careless, or slow and careful), and errors on a simulator-based system do not necessarily correlate with important errors in the live clinical setting with real patients. At this point, it is helpful to use previously validated tools to measure surgical performance, such as Reznick’s OSATS or Fried’s MISTELS for the FLS course. These instruments take into account the majority of important variation in surgical performance and can be adapted easily to suit special situations. Once you have a reliable and specific measure of surgical performance, only then can you begin to research these questions adequately. But the other side of the equation is also critical to determine exactly the specific domain of visual perception that relates to the surgical performance you have measured. At that point, you can investigate whether visual perception is important in predicting surgical performance. So far, this work is only just beginning. Stay tuned.

4) References

  1. Mueller MD, Camartin C, Dreher E, Hänggi W. Three-dimensional laparoscopy. Gadget or progress? A randomized trial on the efficacy of three-dimensional laparoscopy. Surg Endosc. 1999 May;13(5):469-72.
  2. Breedveld P, Hirose S. Design of steerable endoscopes to improve the visual perception of depth during laparoscopic surgery. Mechanical Des, 126(1); 2004.
  3. Rosser JC Jr, Lynch PJ, Cuddihy L, Gentile DA, Klonsky J, Merrell R. The impact of video games on training surgeons in the 21st century. Arch Surg. 2007 Feb;142(2):181-6; discussion 186.
  4. Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J. Impact of hand dominance, gender, and experience with computer games on performance in virtual reality laparoscopy. Surg Endosc. 2003 Jul;17(7):1082-5.
  5. Harper JD, Kaiser S, Ebrahimi K, Lamberton GR, Hadley HR, Ruckle HC, Baldwin DD. Prior video game exposure does not enhance robotic surgical performance. J Endourol. 2007 Oct;21(10):1207-10.
  6. Sidhu RS, Tompa D, Jang RW, Grober ED, Johnston KW, Reznick RK, Hamstra SJ: Interpretation of three-dimensional structure from two-dimensional endovascular images: implications for educators in vascular surgery. Vasc Surg 39(6):1305–1311, 2004.
  7. Fahle M. Specificity of learning curvature, orientation, and vernier discriminations. Vision Res. 1997 Jul;37(14):1885-95.
  8. Yotsumoto Y, Watanabe T, Sasaki Y. Different dynamics of performance and brain activation in the time course of perceptual learning. Neuron. 2008 Mar 27;57(6):827-33.
  9. Ericsson KA, Charness N, Feltovich PJ, Hoffman RR (Eds.) Cambridge Handbook on Expertise and Expert Performance. New York, NY: Cambridge University Press, 2006.

 

Revised December 12, 2008