TractoFormer: A Novel Fiber-level Whole Brain Tractography Analysis Framework Using Spectral Embedding and Vision Transformers
Diffusion MRI tractography is an advanced imaging technique for quantitative
mapping of the brain's structural connectivity. Whole brain tractography (WBT)
data contains over hundreds of thousands of individual fiber streamlines
(estimated brain connections), and this data is usually parcellated to create
compact representations for data analysis applications such as disease
classification. In this paper, we propose a novel parcellation-free WBT
analysis framework, TractoFormer, that leverages tractography information at
the level of individual fiber streamlines and provides a natural mechanism for
interpretation of results using the attention mechanism of transformers.
TractoFormer includes two main contributions. First, we propose a novel and
simple 2D image representation of WBT, TractoEmbedding, to encode 3D fiber
spatial relationships and any feature of interest that can be computed from
individual fibers (such as FA or MD). Second, we design a network based on
vision transformers (ViTs) that includes: 1) data augmentation to overcome
model overfitting on small datasets, 2) identification of discriminative fibers
for interpretation of results, and 3) ensemble learning to leverage fiber
information from different brain regions. In a synthetic data experiment,
TractoFormer successfully identifies discriminative fibers with simulated group
differences. In a disease classification experiment comparing several methods,
TractoFormer achieves the highest accuracy in classifying schizophrenia vs
control. Discriminative fibers are identified in left hemispheric frontal and
parietal superficial white matter regions, which have previously been shown to
be affected in schizophrenia patients.