Hard Attention.
A hard attention model prioritizes task-relevant regions to extract meaningful features from an input. Early attempts to model attention employed image saliency as a priority map. High priority regions were selected using methods such as winner-take-all [1]}, [2]}, [3]}, searching by throwing out all features but the one with minimal activity [4]}, and dynamic routing of information [5]}.
Few used graphical models to model visual attention. [6]} used augmented hidden Markov models to model attention policy. [7]} used a Restricted Boltzmann Machine (RBM) with third-order connections between attention location, glimpse, and the representation of a scene. Motivated by this, [8]} proposed an autoregressive model to compute exact gradients, unlike in an RBM. [9]} used an RBM as a generative model and searched for informative locations using the Hamiltonian Monte Carlo algorithm.
Many used reinforcement learning to train attention models. [10]} used Q-learning with the reward that measures the objectness of the attended region. [11]} estimated rewards using particle filters and employed a policy based on the Gaussian Process and the upper confidence bound. [12]} modeled attention as a partially observable Markov decision process and used a policy gradient algorithm for learning. Later, [13]} extended this approach to multiple objects.
