Advances in sensor technologies, object detection algorithms, planning frameworks and hardware designs have motivated the deployment of robots in warehouse automation. A variety of such applications, like order fulfillment or packing tasks, require picking objects from unstructured piles and carefully arranging them in bins or containers. Desirable solutions need to be low-cost, easily deployable and controllable, making minimalistic hardware choices desirable. The challenge in designing an effective solution to this problem relates to appropriately integrating multiple components, so as to achieve a robust pipeline that minimizes failure conditions. The current work proposes a complete pipeline for solving such packing tasks, given access only to RGB-D data and a single robot arm with a vacuum-based end-effector, which is also used as a pushing finger. To achieve the desired level of robustness, three key manipulation primitives are identified, which take advantage of the environment and simple operations to successfully pack multiple cubic objects. The overall approach is demonstrated to be robust to execution and perception errors. The impact of each manipulation primitive is evaluated by considering different versions of the proposed pipeline, which incrementally introduce reasoning about object poses and corrective manipulation actions.
The 3D bin packing problem, which is NP-hard @cite_5 @cite_3 , requires objects of different or similar volumes to be packed into a cubical bin. Most strategies search for @math -optimal solutions via greedy algorithms @cite_20 . While bin packing is studied extensively, to the best of the authors' knowledge there are few attempts to deploy bin packing solutions on real robots, where inaccuracies in vision and control are taken into account. Such inaccuracies have been considered in the context of efforts relating to the Amazon Robotics Challenge @cite_13 @cite_12 @cite_21 @cite_11 @cite_0 @cite_6 but most of these systems do not deal with bin packing. Most deployments of automatic packing use mechanical components, such as conveyor trays, that are specifically designed for certain products @cite_10 , rendering them difficult to customize and deploy. Industrial packing systems also assume that the objects are already mechanically sorted before packing. While this work makes the assumption that the objects have the same dimensions, the setup is more challenging as the objects are randomly thrown in the initial bin.
@cite_10 @cite_21 @cite_3 @cite_6 @cite_0 @cite_5 @cite_13 @cite_12 @cite_20 @cite_11
1548119923 2963394817 2101057470 2221752211 2551030640 2529170537 2890982024 2068863978 2766201361
This paper describes a strawberry-harvesting robot, a packing robot, and a movable bench system. The harvesting and packing operations in strawberry production require harder, more time-consuming work compared to other operations such as transplanting and chemical spraying, making automation of these tasks desirable. Since harvesting and packing operation account for half of total working hours, automation of these tasks are strongly desired. First of all, based on the findings of many studies on strawberry-harvesting robots for soil culture and elevated substrate culture, our institute of the Bio-oriented Technology Research Advancement Institution and Shibuya Seiki developed a commercial model of a strawberry-harvesting robot, which is chiefly composed a cylindrical manipulator, machine vision, an end-effector, and traveling platform. The results showed an average 54.9 harvesting success rate, 8.6 s cycle time of picking operation, and 102.5 m h work efficiency in hanging-type growing beds in an experimental greenhouse. Secondly, a prototype automatic packing robot consisting of a supply unit and a packing unit was developed. The supply unit picks up strawberries from a harvesting container, and the packing unit sucks each fruit from calyx side and locates its orientation into a tray. Performance testing showed that automatic packing had a task success rate of 97.3 , with a process time per fruit of 7.3 s. Thirdly, a movable bench system was developed, which makes planting beds rotate in longitudinal and lateral ways. This system brought high density production and labour saving operation at a fixed position, such as crop maintenance and harvesting. By setting up the main body of a strawberry-harvesting robot on working space, unmanned operation technique was developed and tested in an experimental greenhouse. Field experiments of these new automation technologies were conducted and gave a potential of practical use. Ahstract-The Amazon Robotics Challenge enlisted sixteen teams to each design a pick-and-place robot for autonomous warehousing, addressing development in robotic vision and manipulation. This paper presents the design of our custom-built, cost-effective, Cartesian robot system Cartman, which won first place in the competition finals by stowing 14 (out of 16) and picking all 9 items in 27 minutes, scoring a total of 272 points. We highlight our experience-centred design methodology and key aspects of our system that contributed to our competitiveness. We believe these aspects are crucial to building robust and effective robotic systems. The problem addressed in this paper is that of orthogonally packing a given set of rectangular-shaped items into the minimum number of three-dimensional rectangular bins. The problem is strongly NP-hard and extremely difficult to solve in practice. Lower bounds are discussed, and it is proved that the asymptotic worst-case performance ratio of the continuous lower bound is ?. An exact algorithm for filling a single bin is developed, leading to the definition of an exact branch-and-bound algorithm for the three-dimensional bin packing problem, which also incorporates original approximation algorithms. Extensive computational results, involving instances with up to 90 items, are presented: It is shown that many instances can be solved to optimality within a reasonable time limit. An important logistics application of robotics involves manipulators that pick-and-place objects placed in warehouse shelves. A critical aspect of this task corresponds to detecting the pose of a known object in the shelf using visual data. Solving this problem can be assisted by the use of an RGBD sensor, which also provides depth information beyond visual data. Nevertheless, it remains a challenging problem since multiple issues need to be addressed, such as low illumination inside shelves, clutter, texture-less and reflective objects as well as the limitations of depth sensors. This letter provides a new rich dataset for advancing the state-of-the-art in RGBD-based 3D object pose estimation, which is focused on the challenges that arise when solving warehouse pick-and-place tasks. The publicly available dataset includes thousands of images and corresponding ground truth data for the objects used during the first Amazon Picking Challenge at different poses and clutter conditions. Each image is accompanied with ground truth information to assist in the evaluation of algorithms for object detection. To show the utility of the dataset, a recent algorithm for RGBD-based pose estimation is evaluated in this letter. Given the measured performance of the algorithm on the dataset, this letter shows how it is possible to devise modifications and improvements to increase the accuracy of pose estimation algorithms. This process can be easily applied to a variety of different methodologies for object pose detection and improve performance in the domain of warehouse pick-and-place. This paper studies two end-effector modalities for warehouse picking: (i) a recently developed, underactuated three-finger hand and (ii) a custom built, vacuum-based gripper. The two systems differ on how they pick objects. The first tool provides increased flexibility, while the vacuum alternative is simpler and smaller. The aim is to show how the end-effector influences the success rate and speed of robotic picking. For the study, the same planning process is followed for known poses of multiple objects with different geometries and characteristics. The resulting trajectories are executed on a real system showing that, under different conditions, different types of end-effectors can be beneficial. This motivates the development of hybrid solutions. This paper presents an overview of the inaugural Amazon Picking Challenge along with a summary of a survey conducted among the 26 participating teams. The challenge goal was to design an autonomous robot to pick items from a warehouse shelf. This task is currently performed by human workers, and there is hope that robots can someday help increase efficiency and throughput while lowering cost. We report on a 28-question survey posed to the teams to learn about each team’s background, mechanism design, perception apparatus, planning, and control approach. We identify trends in this data, correlate it with each team’s success in the competition, and discuss observations and lessons learned based on survey results and the authors’ personal experiences during the challenge. Note to Practitioners —Perception, motion planning, grasping, and robotic system engineering have reached a level of maturity that makes it possible to explore automating simple warehouse tasks in semistructured environments that involve high-mix, low-volume picking applications. This survey summarizes lessons learned from the first Amazon Picking Challenge, highlighting mechanism design, perception, and motion planning algorithms, as well as software engineering practices that were most successful in solving a simplified order fulfillment task. While the choice of mechanism mostly affects execution speed, the competition demonstrated the systems challenges of robotics and illustrated the importance of combining reactive control with deliberative planning. Robotic picking from cluttered bins is a demanding task, for which Amazon Robotics holds challenges. The 2017 Amazon Robotics Challenge (ARC) required stowing items into a storage system, picking specific items, and packing them into boxes. In this paper, we describe the entry of team NimbRo Picking. Our deep object perception pipeline can be quickly and efficiently adapted to new items using a custom turntable capture system and transfer learning. It produces high-quality item segments, on which grasp poses are found. A planning component coordinates manipulation actions between two robot arms, minimizing execution time. The system has been demonstrated successfully at ARC, where our team reached second places in both the picking task and the final stow-and-pick task. We also evaluate individual components. We prove that the First Fit bin packing algorithm is stable under the input distribution Uk − 2; k for all k ≥ 3, settling an open question from the recent survey by Coffman, Garey, and Johnson [“Approximation algorithms for bin backing: A survey,” Approximation algorithms for NP-hard problems, D. Hochbaum (Editor), PWS, Boston, 1996]. Our proof generalizes the multidimensional Markov chain analysis used by Kenyon, Sinclair, and Rabani to prove that Best Fit is also stable under these distributions [Proc Seventh Annual ACM-SIAM Symposium on Discrete Algorithms, 1995, pp. 351–358]. Our proof is motivated by an analysis of Random Fit, a new simple packing algorithm related to First Fit, that is interesting in its own right. We show that Random Fit is stable under the input distributions Uk− 2; k, as well as present worst case bounds and some results on distributions Uk− 1; k and Uk; k for Random Fit. © 2000 John Wiley & Sons, Inc. Random Struct. Alg., 16, 240–259, 2000 Correspondence to: Michael Mitzenmacher. *Most of this work was done while at the Max-Planch-Institut fur Informatik, Saarbrucken, Germany. † A substantial portion of this research was done while at the Computer Science Department, UC Berkeley and Digital Equipment Corporation Systems Research Center. Contract grant sponsor: National Science Foundation. Contract grant number: CCR-9505448. © 2000 John Wiley & Sons, Inc. This work proposes a process for efficiently searching over combinations of individual object 6D pose hypotheses in cluttered scenes, especially in cases involving occlusions and objects resting on each other. The initial set of candidate object poses is generated from state-of-the-art object detection and global point cloud registration techniques. The best-scored pose per object by using these techniques may not be accurate due to overlaps and occlusions. Nevertheless, experimental indications provided in this work show that object poses with lower ranks may be closer to the real poses than ones with high ranks according to registration techniques. This motivates a global optimization process for improving these poses by taking into account scene-level physical interactions between objects. It also implies that the Cartesian product of candidate poses for interacting objects must be searched so as to identify the best scene-level hypothesis. To perform the search efficiently, the candidate poses for each object are clustered so as to reduce their number but still keep a sufficient diversity. Then, searching over the combinations of candidate object poses is performed through a Monte Carlo Tree Search (MCTS) process that uses the similarity between the observed depth image of the scene and a rendering of the scene given the hypothesized pose as a score that guides the search procedure. MCTS handles in a principled way the tradeoff between fine-tuning the most promising poses and exploring new ones, by using the Upper Confidence Bound (UCB) technique. Experimental results indicate that this process is able to quickly identify in cluttered scenes physically-consistent object poses that are significantly closer to ground truth compared to poses found by point cloud registration methods.