Another main area of research is the creation of datasets, which are not only important for training, but also essential for performance comparison.
Many papers and several surveys [1]}, [2]}, [3]}, [4]}, [5]}, [6]} show the performance of different deep neural networks in challenging environments that achieve impressive results on the LFW [7]} and IJB-C [8]} datasets.
Unfortunately, these datasets include only mixtures of different aspects of face recognition, but it is impossible to evaluate which of these aspects can be considered to be solved and which aspects require more research focus.
Only in rare cases, aspects such as ethnicity [1]} or aging [3]} are evaluated separately.
Also, even though the title of the latest review [6]} suggests to review open-source face recognition frameworks, the authors only investigate different parts of the face recognition pipeline (different datasets, face and facial landmark detection methods, deep learning models and evaluation techniques), but they do not test combinations of different parts since no framework for face recognition is available and maintained – except for the Bob framework [12]}, [13]}, [14]} utilized and advertised in this work.
It is important to note that those surveys only duplicate the results promised by the reviewed papers without having a chance to reproduce these numbers or to change evaluation criteria.
