Mobile robots, performing long-term manipulation activities in human environments, have to perceive a wide variety of objects possessing very different visual characteristics and need to reliably keep track of these throughout the execution of a task. In order to be efficient, robot perception capabilities need to go beyond what is currently perceivable and should be able to answer queries about both current and past scenes. In this paper we investigate a perception system for long-term robot manipulation that keeps track of the changing environment and builds a representation of the perceived world. Specifically we introduce an amortized component that spreads perception tasks throughout the execution cycle. The resulting query driven perception system asynchronously integrates results from logged images into a symbolic and numeric (what we call sub-symbolic) representation that forms the perceptual belief state of the robot.
The most comprehensive work on object belief states was done by Blodow @cite_2 , laying the ground work for what is necessary to create and manage such representations. @cite_11 present a Markov Logic Network based framework dynamically resolving the entities of objects. The idea of entity resolution is further developed by @cite_17 , where a pervasive calm perception component for the framework @cite_0 is built.
@cite_0 @cite_11 @cite_17 @cite_2
1499297823 2133815132 2186752256
We present RoboSherlock, an open source software framework for implementing perception systems for robots performing human-scale everyday manipulation tasks. In RoboSherlock, perception and interpretation of realistic scenes is formulated as an unstructured information management (UIM) problem. The application of the UIM principle supports the implementation of perception systems that can answer task-relevant queries about objects in a scene, boost object recognition performance by combining the strengths of multiple perception algorithms, support knowledge-enabled reasoning about objects and enable automatic and knowledge-driven generation of processing pipelines. We demonstrate the potential of the proposed framework by three feasibility studies of systems for real-world scene perception that have been built on top of RoboSherlock. Knowing precisely where objects are located enables a robot to perform its tasks both more efficiently and more reliably. To acquire the respective knowledge and to effectively use it as a resource, a robot has to go through the world with “open eyes”. Specifically, it has to become environment-aware by keeping track of where objects of interest are located and explicitly represent their geometrical properties. In this paper, we propose to equip robots with a perception system that passively monitors the environment using a 3D data acquisition system, identifying objects that might become the subject of future manipulation tasks. Our system encompasses a 3D semantic mapping and reconstruction pipeline and a storage and data merging unit for perceived information that provides on-demand modeling and comparison capabilities. Based on probabilistic logical models, we address the important perceptual subtask of object identity resolution, i.e. inferring which observations refer to which entities in the real world (perceptual anchoring). Our system can be used as a bootstrapping system for the generation of object-centric knowledge and can, in this way, be used as a mid-level perception system that enables activity recognition, scene recognition and high-level planning. A major bottleneck in the realization of autonomous robotic agents performing complex manipulation tasks are the requirements that these tasks impose onto perception mechanisms. There is a strong need to scale robot perception capabilities along two dimensions: First, the variations of appearances and perceptual properties that real-world objects exhibit. Second, the variety of perceptual tasks, like categorizing and localizing, decomposing objects into their functional parts, perceiving the affordances they provide. This paper, addresses this need by organizing perception into a two-stage process. First, a pervasive and 'calm' perceptual component runs continually and interprets the incoming image stream to form a general purpose hybrid (symbolic sub-symbolic) belief state. This is used by the second component, the task-directed perception subsystem, to perform the respective perception tasks in a more informed way. We describe and discuss the first component and explain how it can manage realistic belief states, form a memory of past perceptual experiences, and compute valuable perceptual attributes without delaying plan execution. It does so by exploiting that perception is not a one-shot task but rather a secondary task that is pervasively and calmly performed throughout the lifetime of the robot. We show system operating on a leading-edge manipulation platform.