According to parallel distributed processing (PDP) theory in psychology, neural networks (NN) learn distributed rather than interpretable localist representations. This view has been held so strongly that few researchers have analysed single units to determine if this assumption is correct. However, recent results from psychology, neuroscience and computer science have shown the occasional existence of local codes emerging in artificial and biological neural networks. In this paper, we undertake the first systematic survey of when local codes emerge in a feed-forward neural network, using generated input and output data with known qualities. We find that the number of local codes that emerge from a NN follows a well-defined distribution across the number of hidden layer neurons, with a peak determined by the size of input data, number of examples presented and the sparsity of input data. Using a 1-hot output code drastically decreases the number of local codes on the hidden layer. The number of emergent local codes increases with the percentage of dropout applied to the hidden layer, suggesting that the localist encoding may offer a resilience to noisy networks. This data suggests that localist coding can emerge from feed-forward PDP networks and suggests some of the conditions that may lead to interpretable localist representations in the cortex. The findings highlight how local codes should not be dismissed out of hand.
Localist codes have been found in artificial neural networks, see @cite_1 @cite_16 for full reviews, some examples are @cite_4 @cite_5 ). @cite_8 have shown that PDP models learn localist codes when trained to co-activate multiple items at the same time. Deep networks learn selective codes under some conditions. For example, @cite_14 's found quote mark detectors in RNNs. And there is also some evidence from feed-forward models. For example, @cite_15 have found that probing individual hidden layer neurons (HLNs) with noise and using activation maximisation they can produce a picture of what that neuron will respond most to, and from this they identified HLNs that act as feature detectors, such as those that only responding to creases (in clothing) or eyes or faces and so on.
@cite_14 @cite_4 @cite_8 @cite_1 @cite_5 @cite_15 @cite_16
1951216520 2950789693 2052453836 2580783431 2557495694 2963464195 2141554956
Recurrent Neural Networks (RNNs), and specifically a variant with Long Short-Term Memory (LSTM), are enjoying renewed interest as a result of successful applications in a wide range of machine learning problems that involve sequential data. However, while LSTMs provide exceptional results in practice, the source of their performance and their limitations remain rather poorly understood. Using character-level language models as an interpretable testbed, we aim to bridge this gap by providing an analysis of their representations, predictions and error types. In particular, our experiments reveal the existence of interpretable cells that keep track of long-range dependencies such as line lengths, quotes and brackets. Moreover, our comparative analysis with finite horizon n-gram models traces the source of the LSTM improvements to long-range structural dependencies. Finally, we provide analysis of the remaining errors and suggests areas for further study. We consider the problem of building high- level, class-specific feature detectors from only unlabeled data. For example, is it possible to learn a face detector using only unlabeled images? To answer this, we train a 9-layered locally connected sparse autoencoder with pooling and local contrast normalization on a large dataset of images (the model has 1 bil- lion connections, the dataset has 10 million 200x200 pixel images downloaded from the Internet). We train this network using model parallelism and asynchronous SGD on a clus- ter with 1,000 machines (16,000 cores) for three days. Contrary to what appears to be a widely-held intuition, our experimental re- sults reveal that it is possible to train a face detector without having to label images as containing a face or not. Control experiments show that this feature detector is robust not only to translation but also to scaling and out-of-plane rotation. We also find that the same network is sensitive to other high-level concepts such as cat faces and human bod- ies. Starting with these learned features, we trained our network to obtain 15.8 accu- racy in recognizing 20,000 object categories from ImageNet, a leap of 70 relative im- provement over the previous state-of-the-art. ABSTRACTThere is now a large literature in neuroscience highlighting how some neurons respond highly selectively to high-level information (e.g. cells that respond to specific faces) and a growing literature in psychology and computer science showing that artificial neural networks often learn highly selective representations. Nevertheless, the vast majority of neuroscientists reject “grandmother cell” theories out of hand, and many psychologists reject localist models based on neuroscience. In this review, I detail some of the conceptual confusions regarding grandmother cells that have contributed to this state of affairs, and review the literature of single-unit recording studies in artificial neural networks that may provide insights into why some neurons respond in a highly selective manner. I then briefly review the contributions from leading theorists in psychology and neuroscience. My hope this special issue contributes to a more productive debate on an important issue that has often been character... ABSTRACTSingle-cell recording studies show that some neurons respond to complex visual information (e.g. words, objects, faces) in a highly selective manner, with individual neurons responding to about 0.5 of presented images. Such data have often been taken as inconsistent with “grandmother cell” theories as well as with localist models in psychology. In particular, it is commonly assumed that units in localist models respond to only one input, resulting in greater levels of selectivity than seen in single-cell results. To test this assumption, we recorded unit activity from a localist model of word identification. Our results show that the model can capture the levels of selectivity reported in neuroscience. Accordingly, single-cell data do not rule out localist coding schemes. We propose that the term grandmother cell should be reserved for the hypothesis that the brain implements localist representations: neurons that represent one and only one thing but respond to multiple things. Deep neural networks (DNNs) have demonstrated state-of-the-art results on many pattern recognition tasks, especially vision classification problems. Understanding the inner workings of such computational brains is both fascinating basic science that is interesting in its own right---similar to why we study the human brain---and will enable researchers to further improve DNNs. One path to understanding how a neural network functions internally is to study what each of its neurons has learned to detect. One such method is called activation maximization, which synthesizes an input (e.g. an image) that highly activates a neuron. Here we dramatically improve the qualitative state of the art of activation maximization by harnessing a powerful, learned prior: a deep generator network. The algorithm (1) generates qualitatively state-of-the-art synthetic images that look almost real, (2) reveals the features learned by each neuron in an interpretable way, (3) generalizes well to new datasets and somewhat well to different network architectures without requiring the prior to be relearned, and (4) can be considered as a high-quality generative method (in this case, by generating novel, creative, interesting, recognizable images). A fundamental claim associated with parallel distributed processing (PDP) theories of cognition is that knowledge is coded in a distributed manner in mind and brain. This approach rejects the claim that knowledge is coded in a localist fashion, with words, objects, and simple concepts (e.g. “dog”), that is, coded with their own dedicated representations. One of the putative advantages of this approach is that the theories are biologically plausible. Indeed, advocates of the PDP approach often highlight the close parallels between distributed representations learned in connectionist models and neural coding in brain and often dismiss localist (grandmother cell) theories as biologically implausible. The author reviews a range a data that strongly challenge this claim and shows that localist models provide a better account of single-cell recording studies. The author also contrast local and alternative distributed coding schemes (sparse and coarse coding) and argues that common rejection of grandmother cell theories in neuroscience is due to a misunderstanding about how localist models behave. The author concludes that the localist representations embedded in theories of perception and cognition are consistent with neuroscience; biology only calls into question the distributed representations often learned in PDP models.