@inproceedings{
le2024monkeysee,
title={MonkeySee: Space-time-resolved reconstructions of natural images from macaque multi-unit activity},
author={Lynn Le and Paolo Papale and K. Seeliger and Antonio Lozano and Thirza Dado and Feng Wang and Pieter R. Roelfsema and Marcel van Gerven and Ya{\u{g}}mur G{\"u}{\c{c}}l{\"u}t{\"u}rk and Umut G{\"u}{\c{c}}l{\"u}},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=OWwdlxwnFN}
}
In this paper, we reconstruct naturalistic images directly from macaque brain signals using a convolutional neural network (CNN) based decoder. We investigate the ability of this CNN-based decoding technique to differentiate among neuronal populations from areas V1, V4, and IT, revealing distinct readout characteristics for each. This research marks a progression from low-level to high-level brain signals, thereby enriching the existing framework for utilizing CNN-based decoders to decode brain activity. Our results demonstrate high-precision reconstructions of naturalistic images, highlighting the efficiency of CNN-based decoders in advancing our knowledge of how the brain's representations translate into pixels. Additionally, we present a novel space-time-resolved decoding technique, demonstrating how temporal resolution in decoding can advance our understanding of neural representations. Moreover, we introduce a learned receptive field layer that sheds light on the CNN-based model's data processing during training, enhancing understanding of its structure and interpretive capacity.
MonkeySee: Space-time-resolved reconstructions of natural images from macaque multi-unit activity
Lynn Le Paolo Papale K. Seeliger Antonio Lozano Thirza Dado Feng Wang Pieter R. Roelfsema Marcel van Gerven Yağmur Güçlütürk Umut Güçlü