
CapsField Solution: A new deep learning solution, called CapsField, is proposed for both face and expression recognition in the wild. CapsField combines convolutional and capsule networks [1]} for exploiting not only spatial information but also the angular information available in LF images. A few solutions based on capsule networks have recently been proposed in the context of facial image analysis, including age and gender [2]}, and expression [3]} recognition, showing that capsule networks are well suited to deal with face pose variations. CapsField goes one step further, for the first time exploiting dynamic routing between capsules for learning the angular variations across the different viewpoints captured in LF images. Additionally, CapsField assigns higher weights to the more relevant features while ignoring the spurious dimensions, to increase its robustness against face appearance and environmental changes.

In the wild face data collection: The captured dataset, named Light Field Faces in the Wild (LFFW), addresses a major weakness in the face and expression recognition research domains, which is the lack of publicly available LF facial images captured in the wild. To the best of the authors’ knowledge, no similar dataset is available, even for private usage. The proposed LFFW dataset consists of LF images captured under several unconstrained acquisition variations, in both indoor and outdoor environments, at different locations, and from different distances. During the acquisition, there were no pre-defined protocol or any stimulus used to induce the desired expression. Additionally, a subset of the LFFW dataset includes facial images with different expressions, which has also been annotated to be used in the context of LF-based facial expression recognition.

Constrained face data collection: This paper also makes available a second LF dataset, called Light Field Face Constrained (LFFC), acquired in constrained conditions, with the same subjects present in LFFW.
The complementary LFFC dataset was acquired between 1 day and 3 years prior to the LFFW images, allowing to study the constrained versus in the wild cross-dataset generalization ability of face recognition solutions. The two complementary face datasets (LFFW and LFFC) offer a unique set of facial data, facilitating a wide range of research possibilities related to face analysis. Both datasets, along with the annotation metadata, will be publicly available to the research community.

Benchmarking: Finally, this paper provides an extensive performance assessment of the proposed face and expression recognition solutions regarding the most competitive state-of-the-art solutions. As new datasets along with novel evaluation protocols are proposed in this paper, the authors had to re-implemented all the benchmarking solutions. The obtained results show that the proposed CapsField solution outperforms the state-of-the-art solutions by a significant margin for both face and expression recognition tasks, for the new face datasets.

