Human brain function as measured by functional magnetic resonance imaging
 (fMRI), exhibits a rich diversity. In response, understanding the individual variability
 of brain function and its association with behavior has become one of the
 major concerns in modern cognitive neuroscience. Our work is motivated by the
 view that generative models provide a useful tool for understanding this variability.
 To this end, this manuscript presents two novel generative models trained
 on real neuroimaging data which synthesize task-dependent functional brain images.
 Brain images are high dimensional tensors which exhibit structured spatial
 correlations. Thus, both models are 3D conditional Generative Adversarial networks
 (GANs) which apply Convolutional Neural Networks (CNNs) to learn an
 abstraction of brain image representations. Our results show that the generated
 brain images are diverse, yet task dependent. In addition to qualitative evaluation,
 we utilize the generated synthetic brain volumes as additional training data to improve
 downstream fMRI classifiers (also known as decoding, or brain reading).
 Our approach achieves significant improvements for a variety of datasets, classifi-
 cation tasks and evaluation scores. Our classification results provide a quantitative
 evaluation of the quality of the generated images, and also serve as an additional
 contribution of this manuscript. Functional Magnetic Resonance Imaging (fMRI) is a common tool used by cognitive neuroscientists to investigate the properties of brain function in response to stimuli. Classic analysis approaches BID19 focused on analyzing group-averaged brain function images. However, it was discovered that brain activation patterns vary significantly between individuals. Thus, modern analysis now prioritizes understanding the inter-subject variability of brain function (Dubois & Adolphs, 2016; BID6 . Our work is motivated by the view that generative models provide a useful tool for understanding this variability -as they enable the synthesis of a variety of plausible brain images representing different hypothesized individuals, and high-quality generative models can be analyzed to posit potential mechanisms that explain this variability BID10 . The results presented in this paper provide -to our knowledge for the first time, positive results suggesting that it is indeed possible to generate high quality diverse and task dependent brain images.While we can qualitatively evaluate generative brain images, quantitative evaluation allows us to objectively compare between various results. To this end, we utilize the generated synthetic brain volumes as additional training data to improve downstream fMRI classifiers. The use of classifiers to predict behavior associated with brain images is also known as decoding or brain reading BID21 BID16 ). Classifiers such as support vector machines and deep networks have been applied for decoding brain images. For example, Cox & Savoy (2003) attempted to classify which of 10 categories of object a subject was looking at (including similar categories, such as horses and cows) based on limited number of brain images. Besides visual tasks, BID14 distinguished active regions of brains when subjects listened to linguistic words, where the stimuli included five items from each of 12 semantic categories (animals, body parts etc.).Beyond providing a model for individual variability, high quality brain image synthesis addresses pressing data issues in cognitive neuroscience. Progress in the computational neurosciences is stifled by the difficulty of obtaining brain data either because of a limited culture of data sharing, or due to medical privacy regulations BID18 . For the computational neuroscientist, generated images deliver unlimited quantities of high quality brain imaging data that can be used to develop state of the art tools before application to real subjects and/or patients BID21 . This approach of using modern generative models to synthesize data, which in turn accelerates scientific study, has already proven useful in many scientific fields such as particle physics and astronomy BID1 . Our work represent a first application for this approach to neuroscience.One of the promising generative models are Generative Adversarial Networks (GANs) BID7 , capturing complex distributions using a non-cooperative two-player game formulation: a generator produces synthetic data by transforming samples drawn from a simple distribution; a discriminator focuses on distinguishing synthetic and real data. Despite (or possibly due to) the compelling formulation, GAN training is known to be unstable. To address this difficulty various variants have been proposed. Wasserstein GANs (WGANs) formulate the objective using the Wasserstein distance rather than the classical Jenson-Shannon divergence. Improved training of WGANs BID9 applies an additional gradient penalty, which avoids the critical weight clipping in WGANs which might lead to pathological behavior. Dualing GANs restrict the discriminator and formulate the dual objective (Li et al., 2017) . Beyond improving the stability, conditional GANs (Mirza & Osindero, 2014) make it possible to control the data generation process by conditioning the model on additional information. Auxiliary Classifier GANs (AC-GANs) BID15 ) unify a GAN and classifier to a single architecture, employing labels to generate ImageNet samples. 3D GANs BID23 reconstruct 3D objects and BID4 propose to use improved WGAN to enhance the stability of 3D GANs.We make the following contributions in this paper:1. We develop Improved Conditional Wasserstein GANs (ICW-GAN) and Auxiliary Classifier and Discriminator GANs (ACD-GAN), two types of 3D conditional GANs to synthesize fMRI brain data, both of which we find to be stable to train. 2. We assess the qualitative quality and diversity of generated brain volumes. Our results suggest that the proposed models are able to generate high quality task-dependent and diverse 3D brain images. 3. We evaluate our models on three datasets using a series of image classification tasks with support vector machines and deep network classifiers at two levels of brain image resolution. Results show that augmenting training data using synthetic data generated by our models can greatly improve test classification accuracy of brain volumes. Generative models provide a useful tool for understanding the individual variability of brain images. The results of this manuscript show -to our knowledge for the first time, that 3-D conditional GANs, in particular our proposed ICW-GAN and ACD-GAN, can generate high quality diverse and task dependent brain images. We hope our results inspire additional research on generative models for brain imaging data. Beyond qualitative evaluation, we evaluate quantitative performance by using the generated images as additional training data in a predictive model -mixing synthetic and real data to train classifiers. The results show that our synthetic data augmentation can significantly improve classification accuracy -a result which may be of independent interest. Future work will focus on additional qualitative evaluation of the generated images by neuroscience experts and exploration of various applications. We also plan to more throughly investigate the trained models to further explore what it may contribute to the science of individual variability in neuroimaging. Finally, we plan to expand our models to combine data across multiple studies -each of which use different labels, by exploring techniques for merging labels based on the underlying cognitive processes BID17 .6 SUPPLEMENTARY MATERIAL
Two novel GANs are constructed to generate high-quality 3D fMRI brain images and synthetic brain images greatly help to improve downstream classification tasks.
{"src": "scitldr-AIC_train", "paper_id": "BJaU__eCZ"}