Automatic 3D Car Model Alignment for Mixed Image Based Rendering
Image Based Rendering (IBR) allows good quality free viewpoint navigation in urban scenes, but suffers from artifacts on poorly reconstructed objects, e.g. reflective surfaces such as cars. To alleviate this problem, we propose a method that automatically identifies stock 3D models, aligns them in the 3D scene and performs morphing to better capture image contours. We do this by first adapting learning based methods to detect and identify an object class/pose in images. We then propose a method which exploits all available information, namely partial and inaccurate 3D reconstruction, multi view calibration, image contours and the 3D model to achieve accurate object alignment suitable for subsequent morphing. These steps provide models which are well aligned in 3D and to contours in all the images of the multi view dataset, allowing us to use the resulting model in our mixed IBR algorithm. Our results show significant improvement in image quality for free viewpoint IBR, especially when moving far from the captured viewpoints.
2016 Fourth International Conference on 3D Vision (3DV)