A 4D Light Field Dataset and CNN Architectures for Material Recognition
We introduce a new light field dataset of materials, and take advantage of the recent success of deep learning to perform material recognition on the 4D light field. Our dataset contains 12 material categories, each with 100 images taken with a Lytro Illum, from which we extract about 30,000 patches in total. To the best of our knowledge, this is the first mid size dataset for light field images. Our main goal is to investigate whether the additional information in a light field (such as multiple sub aperture views and view dependent reflectance effects) can aid material recognition. Since recognition networks have not been trained on 4D images before, we propose and compare several novel CNN architectures to train on light field images. In our experiments, the best performing CNN architecture achieves a 7 boost compared with 2D image classification (70\\rightarrow 77\ These results constitute important baselines that can spur further research in the use of CNNs for light field applications. Upon publication, our dataset also enables other novel applications of light fields, including object detection, image segmentation and view interpolation.
ECCV