The emergence of new imaging sensors has opened new frontiers and also brought new challenges for face image analysis systems [1]}. Lenslet light field cameras, hereafter referred only as Light Field (LF) cameras, have recently come into prominence as they are able to simultaneously capture the intensity of light rays coming from multiple directions in space [2]}, [3]}. An LF camera “sees” the visual scene from multiple angles or perspectives, allowing to render a set of 2D Sub-Aperture (SA) images, each corresponding to a 2D image for a different viewpoint of the observed scene. The set of rendered 2D SA images forms a multi-view SA array which offers intra-view/spatial (within each view) and inter-view/angular (across views) correlations that can be effectively exploited for various visual analysis tasks, notably facial image analysis [4]}. In this context, the main challenge of LF-based face and emotion recognition is how to exploit both the intra-view and inter-view relationships to improve the performance of the recognition solutions. This is the recognition paradigm considered in this paper. The LF-based face and emotion recognition can also be formulated as an image set based recognition problem [5]}, where matching is performed between image sets instead of single images.
