@inproceedings{
cai2024phocolens,
title={PhoCoLens: Photorealistic and Consistent Reconstruction in Lensless Imaging},
author={Xin Cai and Zhiyuan You and Hailong Zhang and Jinwei Gu and Wentao Liu and Tianfan Xue},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=XUAcPEaeBU}
}
Lensless cameras offer significant advantages in size, weight, and cost compared to traditional lens-based systems. Without a focusing lens, lensless cameras rely on computational algorithms to recover the scenes from multiplexed measurements. However, current algorithms struggle with inaccurate forward imaging models and insufficient priors to reconstruct high-quality images. To overcome these limitations, we introduce a novel two-stage approach for consistent and photorealistic lensless image reconstruction. The first stage of our approach ensures data consistency by focusing on accurately reconstructing the low-frequency content with a spatially varying deconvolution method that adjusts to changes in the Point Spread Function (PSF) across the camera's field of view. The second stage enhances photorealism by incorporating a generative prior from pre-trained diffusion models. By conditioning on the low-frequency content retrieved in the first stage, the diffusion model effectively reconstructs the high-frequency details that are typically lost in the lensless imaging process, while also maintaining image fidelity. Our method achieves a superior balance between data fidelity and visual quality compared to existing methods, as demonstrated with two popular lensless systems, PhlatCam and DiffuserCam.
PhoCoLens: Photorealistic and Consistent Reconstruction in Lensless Imaging
Xin Cai Zhiyuan You Hailong Zhang Jinwei Gu Wentao Liu Tianfan Xue