We investigate different approaches to integrating object recognition and planning in a tabletop manipulation domain with the set of objects used in the 2012 RoboCup@Work competition. Results of our preliminary experiments show that, with some approaches, close integration of perception and planning improves the quality of plans, as well as the computation times of feasible plans.
Various planning techniques have been used for efficient visual processing management in earlier studies @cite_10 @cite_5 @cite_6 @cite_9 . A survey of such works can be found in the context of the recent introduction of planning for perception in the context of cognitive robotics @cite_8 . The current report distinguishes itself in that it is an empirical investigation of embedding of perceptual processing into a task planning problem, rather than an application of planning to perceptual processing and also aims to investigate how this integration might also improve efficiency of planning. In that sense, a more relevant related work reports a Prolog-based decision making system that utilizes external computations for generating and evaluating perceptual hypotheses, such as the missing objects on a table @cite_1 , though it should be emphasized that the current report is an empirical investigation of different ways of embedding such external computations.
@cite_8 @cite_9 @cite_1 @cite_6 @cite_5 @cite_10
2135624041 1974302391 1497599070 2186752256
Flexible, general-purpose robots need to autonomously tailor their sensing and information processing to the task at hand. We pose this challenge as the task of planning under uncertainty. In our domain, the goal is to plan a sequence of visual operators to apply on regions of interest (ROIs) in images of a scene, so that a human and a robot can jointly manipulate and converse about objects on a tabletop. We pose visual processing management as an instance of probabilistic sequential decision making, and specifically as a Partially Observable Markov Decision Process (POMDP). The POMDP formulation uses models that quantitatively capture the unreliability of the operators and enable a robot to reason precisely about the trade-offs between plan reliability and plan execution time. Since planning in practical-sized POMDPs is intractable, we partially ameliorate this intractability for visual processing by defining a novel hierarchical POMDP based on the cognitive requirements of the corresponding planning task. We compare our hierarchical POMDP planning system (HiPPo) with a non-hierarchical POMDP formulation and the Continual Planning (CP) framework that handles uncertainty in a qualitative manner. We show empirically that HiPPo and CP outperform the naive application of all visual operators on all ROIs. The key result is that the POMDP methods produce more robust plans than CP or the naive visual processing. In summary, visual processing problems represent a challenging domain for planning techniques and our hierarchical POMDP-based approach for visual processing management opens up a promising new line of research. Abstract : Persistent controversies about human multiple task performance suggest that research on it will benefit from increased use of precise computational models. Toward this objective, the present report outlines a comprehensive theoretical framework for understanding and predicting the performance of concurrent perceptual motor and cognitive tasks. The framework involves an Executive Process Interactive Control (EPIC) architecture, which has component modules that process information at perceptual, cognitive, and motor levels. On the basis of EPIC, computational models that use a production system formalism may be constructed to simulate multiple task performance under a variety of conditions. These models account well for reaction time data from representative paradigms such as the psychological refractory period (PRP) procedure. With modest numbers of parameters, good fits between empirical and simulated reaction times support several key conclusions: (1) at a cognitive level, people can apply distinct sets of production rules simultaneously for executing the procedures of multiple tasks; (2) there is no immutable central response selection or decision bottleneck; (3) people's capacity to process information and take action at peripheral perceptual motor levels is limited; (4) to cope with such limits and to satisfy task priorities, flexible scheduling strategies are used; (5) these strategies are mediated by executive cognitive processes that coordinate concurrent tasks adaptively. The initial success of EPIC and models based on it suggest that they may help characterize multiple task performance across many domains, including ones that have substantial practical relevance. A number of psychophysical studies concerning the detection, localization and recognition of objects in the visual field have suggested a two-stage theory of human visual perception. The first stage is the “preattentive” mode, in which simple features are processed rapidly and in parallel over the entire visual field. In the second, “attentive” mode, a specialized processing focus, usually called the focus of attention, is directed to particular locations in the visual field. The analysis of complex forms and the recognition of objects are associated with this second stage.1 The computational justification for such a hypothesis comes from the realization that while it is possible to imagine specific algorithms performing tasks such as shape analysis and recognition at specific locations, it is difficult to imagine these algorithms operating in parallel over the whole visual scene, since such an approach will quickly lead to a combinatorial explosion in terms of required computational resources.2 This is essentially the major critique of Minsky and Papert to a universal application of perceptrons in visual perception.3 Taken together, these empirical and theoretical studies suggest that beyond a certain preprocessing stage, the analysis of visual information proceeds in a sequence of operations, each one applied to a selected location (or locations). A major bottleneck in the realization of autonomous robotic agents performing complex manipulation tasks are the requirements that these tasks impose onto perception mechanisms. There is a strong need to scale robot perception capabilities along two dimensions: First, the variations of appearances and perceptual properties that real-world objects exhibit. Second, the variety of perceptual tasks, like categorizing and localizing, decomposing objects into their functional parts, perceiving the affordances they provide. This paper, addresses this need by organizing perception into a two-stage process. First, a pervasive and 'calm' perceptual component runs continually and interprets the incoming image stream to form a general purpose hybrid (symbolic sub-symbolic) belief state. This is used by the second component, the task-directed perception subsystem, to perform the respective perception tasks in a more informed way. We describe and discuss the first component and explain how it can manage realistic belief states, form a memory of past perceptual experiences, and compute valuable perceptual attributes without delaying plan execution. It does so by exploiting that perception is not a one-shot task but rather a secondary task that is pervasively and calmly performed throughout the lifetime of the robot. We show system operating on a leading-edge manipulation platform.