Wisdom of the Crowds & Group Decisions
Groups of insects, fish, birds, mammals, and primates have been shown to aggregate their individual judgments into group decisions for various tasks. Although some groups seem to have leaders who make decisions alone on behalf of their groups, it is difficult for individuals to outperform aggregations of the entire group’s individual judgments, a phenomenon known as the wisdom of the crowds. Perhaps that is why humans often make important decisions as a group, even if the only expedient but effective group decision mechanism is to use the simple majority voting rule.
Most studies on perceptual collective decision-making have used artificial tasks, where perceptual decisions are limited only by noise that is internal to each observer’s brain (i.e., no external noise). For these contrived scenarios, the maximum wisdom of crowd benefits are specified by the idealized signal detection theory model that treats observers’ internal judgments as normally distributed and statistically independent.
However, such idealized environments are uncommon in (critical) real-world perceptual tasks for which harnessing the wisdom of the crowds is of potential high interest. Perceptual decisions for real world images (i.e., radiological medical images) are often limited by properties inherent to the images. The target of interest might be at a vantage point that makes it difficult to notice, other objects can occlude the target, and noise in the imaging process can reduce the target’s detectability. All observers viewing the same images share the same external sources of variability, which lead to correlations in their judgments and reduce collective integration benefits. Aside from reduced detectability of targets caused by external noise, real world perceptual tasks often include spatial uncertainty, requiring observers to scrutinize large visual areas for potential targets that might not be very visible in the visual periphery. Spatial uncertainty is the case for many life-critical tasks, such as doctors searching for lesions in medical images or intelligence analysts searching for particular objects in satellite and aerial images.
How the search component of perceptual tasks affects the collective integration benefits is, however, not well understood. Recently, we have developed the first theoretical frameworks within signal detection theory that predict the wisdom of crowd benefits for search and other important visual tasks. These models are helping us derive simple algorithms that can provide substantial performance improvements for humans on many important real-world tasks, including life-critical ones such as finding tumors in medical images.
