@inproceedings{
wang2024eegpt,
title={{EEGPT}: Pretrained Transformer for Universal and Reliable Representation of {EEG} Signals},
author={Guangyu Wang and Wenchao Liu and Yuhong He and Cong Xu and Lin Ma and Haifeng Li},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=lvS2b8CjG5}
}
Electroencephalography (EEG) is crucial for recording brain activity, 
  with applications in medicine, neuroscience, and brain-computer interfaces (BCI). 
  However, challenges such as low signal-to-noise ratio (SNR), high inter-subject variability, and channel mismatch complicate the extraction of robust, 
  universal EEG representations. 
  We propose EEGPT, a novel 10-million-parameter pretrained transformer model designed for universal EEG feature extraction. 
  In EEGPT, a mask-based dual self-supervised learning method for efficient feature extraction is designed. 
  Compared to other mask-based self-supervised learning methods, 
  EEGPT introduces spatio-temporal representation alignment. 
  This involves constructing a self-supervised task based on 
  EEG representations that possess high SNR and rich semantic information, 
  rather than on raw signals. 
  Consequently, this approach mitigates the issue of poor feature quality typically 
  extracted from low SNR signals.
  Additionally, EEGPT's hierarchical structure processes spatial and temporal information separately, 
  reducing computational complexity while increasing flexibility and adaptability for BCI applications. 
  By training on a large mixed multi-task EEG dataset, we fully exploit EEGPT's capabilities.
  The experiment validates the efficacy and scalability of EEGPT, 
  achieving state-of-the-art performance on a range of downstream tasks with linear-probing.
  Our research advances EEG representation learning, offering innovative solutions for bio-signal processing and AI applications.
  The code for this paper is available at: https://github.com/BINE022/EEGPT
EEGPT: Pretrained Transformer for Universal and Reliable Representation of EEG Signals
Guangyu Wang Wenchao Liu Yuhong He Cong Xu Lin Ma Haifeng Li