Three new VIU lab papers on visual search with 3D image stacks prevalent in radiology

Former VIU Postdoc Miguel Lago and current VIU Graduate Student Devi Klein have had their recent work on visual search in 3D image volumes–an increasingly common practice for radiologists working with cutting-edge medical imaging technology–published in three new papers. Together, the work provides the field with critical new knowledge about errors with 3D image stacks, computational models to evaluate 3D search, and possible technological solutions to mitigate these errors.

The first of two first-author papers from Miguel Lago, "Under-exploration of Three-Dimensional Images Leads to Search Errors for Small Salient Targets", is published in Current Biology and details how one common diagnostic error for radiologists working with 3D medical image volumes–the missing of small, yet salient, targets that are easily found when searching 2D images despite the greater information content in 3D volumes–can be explained by a tendency of humans to over-estimate their search coverage (i.e., amount of exploration) while actually under-sampling the 3D volumes (via eye movements; Fig. 1). More information about this novel and important work–which counts VIU members Craig Abbey, Aditya Jonnalagadda, and Miguel Eckstein as coauthors–as well as interviews with the authors can be found in this write-up by The Current.

Dr. Lago's second paper, "Foveated Model Observers for Visual Search in 3D Medical Images", is published in IEEE Transactions in Medical Imaging and shows how the most popular computational models (i.e., model observers) used to evaluated task-based image quality fail to predict 3D search because they do not incorporate the varying resolution/processing across the human field of view (i.e., foveated vision). A newly proposed foveated model observer that incorporates spatially-varying resolution/processing, the concept of eye movements, and the ability to scroll across slices can predict search performance in 3D images and might provide a viable tool to evaluate the image quality of medical imaging modalities that result in 3D image stacks.

Devi Klein's work, "The perceptual influence of 2D synthesized images on 3D search", was presented at the 2021 Society of Photo-optical Instrumentation Engineers (SPIE) Medical Imaging Digital Forum and explores how the use of supplementary 2D synthesized images can help observers mitigate the missing of small targets in 3D search and reduce search times. The work uses eye tracking to show that these synthesized images reduce small-target misses by efficiently guiding observers' eye movements to suspicious locations in the 3D volume (Fig.2). These small signals would otherwise be missed due to the detrimental effects of peripheral vision on signal detectability–especially of small signals–and the typical under-exploration characterized by Dr. Lago's work above. More discussion of this exciting expansion of the 3D image search work can be found at the end of the same article in The Current linked to above discussing Dr. Lago's work.