Multiscale Voxel Based Decoding For Enhanced Natural Image Reconstruction From Brain Activity
Reconstructing perceived images from human brain activity monitored by
functional magnetic resonance imaging (fMRI) is hard, especially for natural
images. Existing methods often result in blurry and unintelligible
reconstructions with low fidelity. In this study, we present a novel approach
for enhanced image reconstruction, in which existing methods for object
decoding and image reconstruction are merged together. This is achieved by
conditioning the reconstructed image to its decoded image category using a
class-conditional generative adversarial network and neural style transfer. The
results indicate that our approach improves the semantic similarity of the
reconstructed images and can be used as a general framework for enhanced image
reconstruction.