task_categories:image-to-image task_categories:object-detection size_categories:100K<n<1M license:other biology neuroscience fmri region:us
---
license: other
dataset_info:
  features:
  - name: subject_id
    dtype: int64
  - name: trial_id
    dtype: int64
  - name: session_id
    dtype: int64
  - name: nsd_id
    dtype: int64
  - name: image
    dtype: image
  - name: activity
    dtype: image
  - name: subject
    dtype: string
  - name: flagged
    dtype: bool
  - name: BOLD5000
    dtype: bool
  - name: shared1000
    dtype: bool
  - name: coco_split
    dtype: string
  - name: coco_id
    dtype: int64
  - name: objects
    struct:
    - name: area
      sequence: int64
    - name: bbox
      sequence:
        sequence: float64
    - name: category
      sequence: string
    - name: iscrowd
      sequence: int64
    - name: segmentation
      list:
      - name: counts
        dtype: string
      - name: poly
        sequence:
          sequence: float64
      - name: size
        sequence: int64
    - name: supercategory
      sequence: string
    - name: target
      sequence: int64
  - name: captions
    sequence: string
  - name: repetitions
    struct:
    - name: subject1_rep0
      dtype: int64
    - name: subject1_rep1
      dtype: int64
    - name: subject1_rep2
      dtype: int64
    - name: subject2_rep0
      dtype: int64
    - name: subject2_rep1
      dtype: int64
    - name: subject2_rep2
      dtype: int64
    - name: subject3_rep0
      dtype: int64
    - name: subject3_rep1
      dtype: int64
    - name: subject3_rep2
      dtype: int64
    - name: subject4_rep0
      dtype: int64
    - name: subject4_rep1
      dtype: int64
    - name: subject4_rep2
      dtype: int64
    - name: subject5_rep0
      dtype: int64
    - name: subject5_rep1
      dtype: int64
    - name: subject5_rep2
      dtype: int64
    - name: subject6_rep0
      dtype: int64
    - name: subject6_rep1
      dtype: int64
    - name: subject6_rep2
      dtype: int64
    - name: subject7_rep0
      dtype: int64
    - name: subject7_rep1
      dtype: int64
    - name: subject7_rep2
      dtype: int64
    - name: subject8_rep0
      dtype: int64
    - name: subject8_rep1
      dtype: int64
    - name: subject8_rep2
      dtype: int64
  splits:
  - name: train
    num_bytes: 26695182666.0
    num_examples: 195000
  - name: test
    num_bytes: 2461280671.0
    num_examples: 18000
  download_size: 22565691383
  dataset_size: 29156463337.0
task_categories:
- image-to-image
- object-detection
tags:
- biology
- neuroscience
- fmri
size_categories:
- 100K<n<1M
---

# NSD-Flat
[[`GitHub`]](https://github.com/clane9/NSD-Flat) [[🤗 `Hugging Face Hub`]](https://huggingface.co/datasets/clane9/NSD-Flat)

A Hugging Face dataset of pre-processed brain activity flat maps from the [Natural Scenes Dataset](https://naturalscenesdataset.org/), constrained to a visual cortex region of interest and rendered as PNG images.

## Load the dataset

Load the dataset from [Hugging Face Hub](https://huggingface.co/datasets/clane9/NSD-Flat)

```python
from datasets import load_dataset

dataset = load_dataset("clane9/NSD-Flat", split="train")
```

## Building the dataset

### 1. Download source data

Run [`download_data.sh`](download_data.sh) to download the required source data:

- NSD stimuli images and presentation info
- COCO annotations
- NSD beta activity maps in fsaverge surface space

```bash
bash download_data.sh
```

### 2. Convert the COCO annotations

Run  [`convert_nsd_annotations.py`](convert_nsd_annotations.py) to crop and reorganize the COCO annotations for NSD.

```bash
python convert_nsd_annotations.py
```

### 3. Generate the dataset

Run [`generate_dataset.py`](generate_dataset.py) to generate the huggingface dataset in Arrow format.

```bash
python generate_dataset.py --img_size 256 --workers 8
```

## Citation

If you find this dataset useful, please consider citing:

```
@article{allen2022massive,
  title     = {A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence},
  author    = {Allen, Emily J and St-Yves, Ghislain and Wu, Yihan and Breedlove, Jesse L and Prince, Jacob S and Dowdle, Logan T and Nau, Matthias and Caron, Brad and Pestilli, Franco and Charest, Ian and others},
  journal   = {Nature neuroscience},
  volume    = {25},
  number    = {1},
  pages     = {116--126},
  year      = {2022},
  publisher = {Nature Publishing Group US New York}
}
```

```
@misc{lane2023nsdflat,
  author       = {Connor Lane},
  title        = {NSD-Flat: Pre-processed brain activity flat maps from the Natural Scenes Dataset},
  howpublished = {\url{https://huggingface.co/datasets/clane9/NSD-Flat}},
  year         = {2023},
}
```

## License

Usage of this dataset constitutes agreement to the [NSD Terms and Conditions](https://cvnlab.slite.page/p/IB6BSeW_7o/Terms-and-Conditions).