The human visual system routinely handles vast amounts of information at about \(10^{8}\)  to \(10^{9}\)  bits per second [1]}, [2]}, [3]}, [4]}. An essential mechanism that allows the human visual system to process this amount of information in real time is its ability to selectively focus attention on salient parts of a scene.
Which parts of a scene and what individual patterns particularly attract the viewer's eyes (salient areas) have been the subject of psychological research for decades, and designing computational models for predicting salient areas is a longstanding problem in computer vision. In recent years, we have observed a surge in the development of data-driven models of saliency based on deep neural networks. Such deep models have demonstrated significant performance improvements in comparison to classical models, which are based on hand-crafted features or psychological assumptions, outperforming them on most benchmarks.
However, while there still remains a relatively large gap between deep models and the human visual system (see Table REF ), the performance of deep models appears to have reached a ceiling. This raises the question of what is learned by deep models that drives their superior performance over classical models, and what are the remaining and missing ingredients to attain human-like performance.
Internal representations of deep object recognition models have been visualized and analyzed extensively in recent years. Such efforts, however, are missing for saliency models and it is unclear how saliency models do what they do.
<TABLE>