The problem of autonomous transportation in industrial scenarios is receiving a renewed interest due to the way it can revolutionise internal logistics, especially in unstructured environments. This paper presents a novel architecture allowing a robot to detect, localise, and track multiple pallets using machine learning techniques based on an on-board 2D laser rangefinder. The architecture is composed of two main components: the first stage is a pallet detector employing a Faster Region-based Convolutional Neural Network (Faster R-CNN) detector cascaded with a CNN-based classifier; the second stage is a Kalman filter for localising and tracking detected pallets, which we also use to defer commitment to a pallet detected in the first step until sufficient confidence has been acquired via a sequential data acquisition process. For fine-tuning the CNNs, the architecture has been systematically evaluated using a real-world dataset containing 340 labeled 2D scans, which have been made freely available in an online repository. Detection performance has been assessed on the basis of the average accuracy over k-fold cross-validation, and it scored 99.58 in our tests. Concerning pallet localisation and tracking, experiments have been performed in a scenario where the robot is approaching the pallet to fork. Although data have been originally acquired by considering only one pallet, artificial data have been generated as well to mimic the presence of multiple targets in the robot workspace. Our experimental results confirm that the system is capable of identifying, localising and tracking pallets with a high success rate while being robust to false positives.
An approach for the automated pallet detection combining stereo reconstruction and object detection from monocular images has been presented in @cite_35 . Improvements and extensions for a stereo camera system responsible for autonomous load handling were presented, by the same authors, in @cite_42 . However, the use of stereovision and structure-from-motion algorithms can hardly fit with real-time requirements typically needed when autonomous vehicles are present. The work described in @cite_19 introduces a method to identify pallets using color segmentation in real time. However, such a method is prone to the presence of false positives, unless assumptions about pallets colour are posed. A comparison between two common 3D vision technologies, namely the photonic mixer device (PMD) and typical stereo camera systems, was presented in @cite_15 . The authors conclude that the PMD system is characterised by a greater accuracy than a typical stereo camera system. On the basis of such an insight, a solution for pallet loading and de-palletising detection employing a PMD camera has been introduced in @cite_44 . Again, such approach requires the introduction of hardware, at the expense of cost and maintenance.
@cite_35 @cite_42 @cite_44 @cite_19 @cite_15
1964262869 1985023071 1948310751 169439271
In this study, a novel approach for the detection of parcel loading positions on a pallet is presented. This approach was realized as a substantial change in comparison with traditional system design of contour detection in de-palletizing processes. Complex 3D-vision systems, costly laser scanners or throughput decreasing local sensor solutions integrated in grippers are substituted by a low-cost photonic mixing device (PMD) camera. By combining PMD technology and a predetermined model of loading situations, stored during assembling the pallet, this approach can compensate for the drawbacks of each respective system. An essential part of the approach are computer-graphics methods specific to the given problem to both detect the deviation between the nominal and the actual loading position and if necessary an automated correction of the packaging scheme. From an economic point of view, this approach can decrease the costs of mandatory contour checking in automated de-palletizing processes. The paper presents a method for automatically detecting pallets and estimating their position and orientation. For detection we use a sliding window approach with efficient candidate generation, fast integral features and a boosted classifier. Specific information regarding the detection task such as region of interest, pallet dimensions and pallet structure can be used to speed up and validate the detection process. Stereo reconstruction is employed for depth estimation by applying Semi-Global Matching aggregation with Census descriptors. Offline test results show that successful detection is possible under 0.5 seconds. We present newly added modules for our autonomous load handling system. The stereo camera system provides information about the scene in front of the automated forklift. A new alternative module for pallet detection is described. Several processing modules for unloading operations are also presented. Our system is evaluated by means of detection rate and by performing field tests. The tests show that it is capable of providing a sufficiently accurate position of the pallets in order to perform loading and unloading operations in multiple scenarios. State of the art visual SLAM systems have recently been presented which are capable of accurate, large-scale and real-time performance, but most of these require stereo vision. Important application areas in robotics and beyond open up if similar performance can be demonstrated using monocular vision, since a single camera will always be cheaper, more compact and easier to calibrate than a multi-camera rig. With high quality estimation, a single camera moving through a static scene of course effectively provides its own stereo geometry via frames distributed over time. However, a classic issue with monocular visual SLAM is that due to the purely projective nature of a single camera, motion estimates and map structure can only be recovered up to scale. Without the known inter-camera distance of a stereo rig to serve as an anchor, the scale of locally constructed map portions and the corresponding motion estimates is therefore liable to drift over time. In this paper we describe a new near real-time visual SLAM system which adopts the continuous keyframe optimisation approach of the best current stereo systems, but accounts for the additional challenges presented by monocular input. In particular, we present a new pose-graph optimisation technique which allows for the efficient correction of rotation, translation and scale drift at loop closures. Especially, we describe the Lie group of similarity transformations and its relation to the corresponding Lie algebra. We also present in detail the system’s new image processing front-end which is able accurately to track hundreds of features per frame, and a filter-based approach for feature initialisation within keyframe-based SLAM. Our approach is proven via large-scale simulation and real-world experiments where a camera completes large looped trajectories.