Neuroscience theory posits that the brain's visual system coarsely identifies broad object categories via neural activation patterns, with similar objects producing similar neural responses. Artificial neural networks also have internal activation behavior in response to stimuli. We hypothesize that networks exhibiting brain-like activation behavior will demonstrate brain-like characteristics, e.g., stronger generalization capabilities. In this paper we introduce a human-model similarity (HMS) metric, which quantifies the similarity of human fMRI and network activation behavior. To calculate HMS, representational dissimilarity matrices (RDMs) are created as abstractions of activation behavior, measured by the correlations of activations to stimulus pairs. HMS is then the correlation between the fMRI RDM and the neural network RDM across all stimulus pairs. We test the metric on unsupervised predictive coding networks, which specifically model visual perception, and assess the metric for statistical significance over a large range of hyperparameters. Our experiments show that networks with increased human-model similarity are correlated with better performance on two computer vision tasks: next frame prediction and object matching accuracy. Further, HMS identifies networks with high performance on both tasks. An unexpected secondary finding is that the metric can be employed during training as an early-stopping mechanism.
With respect to the intersection between neuroscience and machine learning, the neuroimaging technique of fMRI has been used as ground-truth for designing features @cite_0 , interpreting neural network features @cite_35 @cite_48 , and studying network performance @cite_44 . Fong al @cite_9 recently found that raw fMRI data could be used to weight support vector machines to improve performance, indicating that coarse-level brain data can potentially help machine learning networks generalize. The success of that study, alongside the public release of human fMRI data in RDM form by Nili al @cite_41 further motivated us to use fMRI data as ground-truth in our network evaluation. The specific contributions fMRI data can make in expanding our understanding of neural networks are still to be explored, but to our knowledge this is the first instance of fMRI data being deployed for neural network model search, where the task is to screen different hyperparameter and architecture configurations for models that perform well on a given task. There is significant recent interest in optimization methods, search strategies, and infrastructure for neural network model search @cite_30 @cite_24 @cite_28 @cite_12 @cite_39 . In this context, our work represents a new capability for such searches.
@cite_30 @cite_35 @cite_41 @cite_48 @cite_9 @cite_28 @cite_39 @cite_0 @cite_44 @cite_24 @cite_12
2526782364 2112532472 2621356193 61705880
Recently, neuron activations extracted from a pre-trained convolutional neural network (CNN) show promising performance in various visual tasks. However, due to the domain and task bias, using the features generated from the model pre-trained for image classification as image representations for instance retrieval is problematic. In this paper, we propose quartet-net learning to improve the discriminative power of CNN features for instance retrieval. The general idea is to map the features into a space where the image similarity can be better evaluated. Our network differs from the traditional Siamese-net in two ways. First, we adopt a double-margin contrastive loss with a dynamic margin tuning strategy to train the network which leads to more robust performance. Second, we introduce in the mimic learning regularization to improve the generalization ability of the network by preventing it from overfitting to the training data. Catering for the network learning, we collect a large-scale dataset, namely GeoPair, which consists of 68k matching image pairs and 63k non-matching pairs. Experiments on several standard instance retrieval datasets demonstrate the effectiveness of our method. Over the past decade, functional Magnetic Resonance Imaging (fMRI) has emerged as a powerful new instrument to collect vast quantities of data about activity in the human brain. A typical fMRI experiment can produce a three-dimensional image related to the human subject's brain activity every half second, at a spatial resolution of a few millimeters. As in other modern empirical sciences, this new instrumentation has led to a flood of new data, and a corresponding need for new data analysis methods. We describe recent research applying machine learning methods to the problem of classifying the cognitive state of a human subject based on fRMI data observed over a single time interval. In particular, we present case studies in which we have successfully trained classifiers to distinguish cognitive states such as (1) whether the human subject is looking at a picture or a sentence, (2) whether the subject is reading an ambiguous or non-ambiguous sentence, and (3) whether the word the subject is viewing is a word describing food, people, buildings, etc. This learning problem provides an interesting case study of classifier learning from extremely high dimensional (105 features), extremely sparse (tens of training examples), noisy data. This paper summarizes the results obtained in these three case studies, as well as lessons learned about how to successfully apply machine learning methods to train classifiers in such settings. With the advance in neuraimaging technology, such as functional Magnetic Resonance Imaging (fMRI), more brain image data become available, and mining the brain images emerges as a hot area in data mining communities. In contrast to ordinary images extensively studied in computer vision communities, brain images can be parcellated into different Regions of Interest (ROIs) due to the underlying functional and structural organization of human brain. Distinct diseases are highly relevant to a specific set of ROIs. In this paper, in order to leverage regional patterns to predict diseases and identify relevant ROIs, we present a novel Ensemble Approach on Regionalized Neural Networks (EARNN). EARNN is of a two-stage learning architecture. Its first stage consists of training a set of Regionalized Neural Network models, each of which learns from a brain region by using a proposed stochastic pooling operation. Its second stage adopts an effective EM algorithm to ensemble a mixture of Regionalized Neural Networks and iteratively estimate the mixing coefficients. We test our model on two real fMRI datasets. Experiments demonstrate that by effectively exploiting regional patterns and structural information, EARNN achieves the superior performance in comparison to previous approaches. In addition, according to our domain knowledge, EARNN successfully identifies relevant ROIs of diseases. Research in recent years has provided some evidence of temporal non-stationarity of functional connectivity in resting state fMRI. In this paper, we present a novel methodology that can decode connectivity dynamics into a temporal sequence of hidden network "states" for each subject, using a Hidden Markov Modeling (HMM) framework. Each state is characterized by a unique covariance matrix or whole-brain network. Our model generates these covariance matrices from a common but unknown set of sparse basis networks, which capture the range of functional activity co-variations of regions of interest (ROIs). Distinct hidden states arise due to a variation in the strengths of these basis networks. Thus, our generative model combines a HMM framework with sparse basis learning of positive definite matrices. Results on simulated fMRI data show that our method can effectively recover underlying basis networks as well as hidden states. We apply this method on a normative dataset of resting state fMRI scans. Results indicate that the functional activity of a subject at any point during the scan is composed of combinations of overlapping task-positive negative pairs of networks as revealed by our basis. Distinct hidden temporal states are produced due to a different set of basis networks dominating the covariance pattern in each state.