This paper presents an information theoretic approach to the concept of intelligence in the computational sense. We introduce a probabilistic framework from which computation alintelligence is shown to be an entropy minimizing process at the local level. Using this new scheme, we develop a simple data driven clustering example and discuss its applications.
Many sources claim to have computational theories of intelligence, but for the most part these theories'' merely act to describe certain aspects of intelligence @cite_6 . For example, Meyer in @cite_12 suggests that performance on multiple tasks is dependent on adaptive executive control, but makes no claim on the emergence of such characteristics. Others discuss how data is aggregated. This type of analysis is especially relevant in computer vision and image recognition @cite_9 .
@cite_9 @cite_12 @cite_6
1974302391 2806422867 1497599070 1748815599
Abstract : Persistent controversies about human multiple task performance suggest that research on it will benefit from increased use of precise computational models. Toward this objective, the present report outlines a comprehensive theoretical framework for understanding and predicting the performance of concurrent perceptual motor and cognitive tasks. The framework involves an Executive Process Interactive Control (EPIC) architecture, which has component modules that process information at perceptual, cognitive, and motor levels. On the basis of EPIC, computational models that use a production system formalism may be constructed to simulate multiple task performance under a variety of conditions. These models account well for reaction time data from representative paradigms such as the psychological refractory period (PRP) procedure. With modest numbers of parameters, good fits between empirical and simulated reaction times support several key conclusions: (1) at a cognitive level, people can apply distinct sets of production rules simultaneously for executing the procedures of multiple tasks; (2) there is no immutable central response selection or decision bottleneck; (3) people's capacity to process information and take action at peripheral perceptual motor levels is limited; (4) to cope with such limits and to satisfy task priorities, flexible scheduling strategies are used; (5) these strategies are mediated by executive cognitive processes that coordinate concurrent tasks adaptively. The initial success of EPIC and models based on it suggest that they may help characterize multiple task performance across many domains, including ones that have substantial practical relevance. During the 60s and 70s, AI researchers explored intuitions about intelligence by writing programs that displayed intelligent behavior. Many good ideas came out from this work but programs written by hand were not robust or general. After the 80s, research increasingly shifted to the development of learners capable of inferring behavior and functions from experience and data, and solvers capable of tackling well-defined but intractable models like SAT, classical planning, Bayesian networks, and POMDPs. The learning approach has achieved considerable success but results in black boxes that do not have the flexibility, transparency, and generality of their model-based counterparts. Model-based approaches, on the other hand, require models and scalable algorithms. Model-free learners and model-based solvers have close parallels with Systems 1 and 2 in current theories of the human mind: the first, a fast, opaque, and inflexible intuitive mind; the second, a slow, transparent, and flexible analytical mind. In this paper, I review developments in AI and draw on these theories to discuss the gap between model-free learners and model-based solvers, a gap that needs to be bridged in order to have intelligent systems that are robust and general. A number of psychophysical studies concerning the detection, localization and recognition of objects in the visual field have suggested a two-stage theory of human visual perception. The first stage is the “preattentive” mode, in which simple features are processed rapidly and in parallel over the entire visual field. In the second, “attentive” mode, a specialized processing focus, usually called the focus of attention, is directed to particular locations in the visual field. The analysis of complex forms and the recognition of objects are associated with this second stage.1 The computational justification for such a hypothesis comes from the realization that while it is possible to imagine specific algorithms performing tasks such as shape analysis and recognition at specific locations, it is difficult to imagine these algorithms operating in parallel over the whole visual scene, since such an approach will quickly lead to a combinatorial explosion in terms of required computational resources.2 This is essentially the major critique of Minsky and Papert to a universal application of perceptrons in visual perception.3 Taken together, these empirical and theoretical studies suggest that beyond a certain preprocessing stage, the analysis of visual information proceeds in a sequence of operations, each one applied to a selected location (or locations). Although much of the hubris and hyperbole surrounding the 1990's Internet has softened to a reasonable level, the inexorable momentum of information growth continues unabated. This wealth of information provides resources for adapting to the problems posed by our increasingly complex world, but the simple availability of more information does not guarantee its successful transformation into valuable knowledge that shapes, guides, and improves our activity. When faced with something like the analysis of sense-making behavior on the web, traditional research models tell us a lot about learning and performance with browser operations, but very little about how people will actively navigate and search through information structures, what information they will choose to consume, and what conceptual models they will induce about the landscape of cyberspace. Thus, it is fortunate that a new field of research, Adaptive Information Interaction (AII), is becoming possible. AII centers on the problems of understanding and improving human-information interaction. It is about how people will best shape themselves to their information environments, and how information environments can best be shaped to people. Its roots lie in human-computer interaction (HCI), information retrieval, and the behavioral and social sciences. This book is about Information Foraging Theory (IFT), a new theory in Adaptive Information Interaction that is one example of a recent flourish of theories in adaptationist psychology that draw upon evolutionary-ecological theory in biology. IFT assumes that people (indeed, all organisms) are ecologically rational, and that human information-seeking mechanisms and strategies adapt the structure of the information environments in which they operate. Its main aim is to create technology that is better shaped to users. Information Foraging Theory will be of interest to student and professional researchers in HCI and cognitive psychology.