One characteristic of human visual perception is the presence of Gestalt phenomena,' that is, that the whole is something other than the sum of its parts. A natural question is whether image-recognition networks show similar effects. Our paper investigates one particular type of Gestalt phenomenon, the law of closure, in the context of a feedforward image classification neural network (NN). This is a robust effect in human perception, but experiments typically rely on measurements (e.g., reaction time) that are not available for artificial neural nets. We describe a protocol for identifying closure effect in NNs, and report on the results of experiments with simple visual stimuli. Our findings suggest that NNs trained with natural images do exhibit closure, in contrast to networks with randomized weights or networks that have been trained on visually random data. Furthermore, the closure effect reflects something beyond good feature extraction; it is correlated with the network's higher layer features and ability to generalize.
Leveraging the experimental framework of classical psychology to study NNs is an important yet under-explored area of research. Recent work in this direction includes investigating shape biases in NNs @cite_28 or measuring abstract reasoning ability using tests designed for humans @cite_48 . Our work aims to continue this effort.
@cite_28 @cite_48
2729557715 2963488527 2803137490 2011348940
Deep neural networks (DNNs) have achieved unprecedented performance on a wide range of complex tasks, rapidly outpacing our understanding of the nature of their solutions. This has caused a recent surge of interest in methods for rendering modern neural systems more interpretable. In this work, we propose to address the interpretability problem in modern DNNs using the rich history of problem descriptions, theories and experimental methods developed by cognitive psychologists to study the human mind. To explore the potential value of these tools, we chose a well-established analysis from developmental psychology that explains how children learn word labels for objects, and applied that analysis to DNNs. Using datasets of stimuli inspired by the original cognitive psychology experiments, we find that state-of-the-art one shot learning models trained on ImageNet exhibit a similar bias to that observed in humans: they prefer to categorize objects according to shape rather than color. The magnitude of this shape bias varies greatly among architecturally identical, but differently seeded models, and even fluctuates within seeds throughout training, despite nearly equivalent classification performance. These results demonstrate the capability of tools from cognitive psychology for exposing hidden computational properties of DNNs, while concurrently providing us with a computational model for human word learning. Deep neural networks (DNNs) have advanced performance on a wide range of complex tasks, rapidly outpacing our understanding of the nature of their solutions. While past work sought to advance our understanding of these models, none has made use of the rich history of problem descriptions, theories, and experimental methods developed by cognitive psychologists to study the human mind. To explore the potential value of these tools, we chose a well-established analysis from developmental psychology that explains how children learn word labels for objects, and applied that analysis to DNNs. Using datasets of stimuli inspired by the original cognitive psychology experiments, we find that state-of-the-art one shot learning models trained on ImageNet exhibit a similar bias to that observed in humans: they prefer to categorize objects according to shape rather than color. The magnitude of this shape bias varies greatly among architecturally identical, but differently seeded models, and even fluctuates within seeds throughout training, despite nearly equivalent classification performance. These results demonstrate the capability of tools from cognitive psychology for exposing hidden computational properties of DNNs, while concurrently providing us with a computational model for human word learning. State-of-the-art deep convolutional networks (DCNs) such as squeeze-and- excitation (SE) residual networks implement a form of attention, also known as contextual guidance, which is derived from global image features. Here, we explore a complementary form of attention, known as visual saliency, which is derived from local image features. We extend the SE module with a novel global-and-local attention (GALA) module which combines both forms of attention -- resulting in state-of-the-art accuracy on ILSVRC. We further describe ClickMe.ai, a large-scale online experiment designed for human participants to identify diagnostic image regions to co-train a GALA network. Adding humans-in-the-loop is shown to significantly improve network accuracy, while also yielding visual features that are more interpretable and more similar to those used by human observers. Abstract In this article we propose an account of the early development of children's knowledge about the mind and report two studies designed to test a part of it. According to this “connections-representations” account, young children begin their discovery of the mental world by learning that they and other people have internal experiences or mental states that connect them cognitively to external objects and events-experiences such as seeing them, hearing them, and wanting them. Later, they realize that the same object can be seriously (other than in pretense) mentally represented in different, seemingly contradictory ways: for example, as A in appearance, B in reality, C according to their perceptual or conceptual perspective, and D according to another person's. The results of both studies confirmed the prediction that 3-year-olds would perform well on appearance-reality and perceptual perspective-taking tasks requiring only an understanding of cognitive connections, but perform poorly on tasks requiring an understanding of seemingly-contrary-to-fact mental representations. To illustrate, children of this age had little difficulty determining that they could hear but could not see a noise-making object located on the experimenter's side of a barrier, and that the experimenter could see it (connections-level tasks). In contrast, they were largely unable to say, for instance, that a toy bear held behind a large elephant mask and emitting a cat sound looked like an elephant, sounded like a cat, and really was a toy bear (representations-level tasks)—even though the experimenter had actually told them previously what it looked like, sounded like, and really was. The article concludes with speculations about the possible origins of connections and representations knowledge and observations about the significance of these acquisitions for the child's development.