This letter presents a three-dimensional (3-D) visual detection method for the challenging task of detecting peduncles of sweet peppers (Capsicum annuum) in the field. Cutting the peduncle cleanly is one of the most difficult stages of the harvesting process, where the peduncle is the part of the crop that attaches it to the main stem of the plant. Accurate peduncle detection in 3-D space is, therefore, a vital step in reliable autonomous harvesting of sweet peppers, as this can lead to precise cutting while avoiding damage to the surrounding plant. This letter makes use of both color and geometry information acquired from an RGB-D sensor and utilizes a supervised-learning approach for the peduncle detection task. The performance of the proposed method is demonstrated and evaluated by using qualitative and quantitative results [the area-under-the-curve (AUC) of the detection precision-recall curve]. We are able to achieve an AUC of 0.71 for peduncle detection on field-grown sweet peppers. We release a set of manually annotated 3-D sweet pepper and peduncle images to assist the research community in performing further research on this topic.
Strawberry peduncle detection was reported by @cite_29 . The region of interest (ROI) was pre-defined using prior knowledge and the boundary point between a fruit and peduncle was detected using colour information. The inclination of the peduncle - the angle between the vertical axis and the boundary point - was computed. It is possible to easily distinguish the boundary point for red strawberry but is a challenge to exploit this approach for green sweet pepper.
@cite_29
2501369945 2087877962 2064245748 2064352738
This paper presents a novel approach to fruit detection using deep convolutional neural networks. The aim is to build an accurate, fast and reliable fruit detection system, which is a vital element of an autonomous agricultural robotic platform; it is a key element for fruit yield estimation and automated harvesting. Recent work in deep neural networks has led to the development of a state-of-the-art object detector termed Faster Region-based CNN (Faster R-CNN). We adapt this model, through transfer learning, for the task of fruit detection using imagery obtained from two modalities: colour (RGB) and Near-Infrared (NIR). Early and late fusion methods are explored for combining the multi-modal (RGB and NIR) information. This leads to a novel multi-modal Faster R-CNN model, which achieves state-of-the-art results compared to prior work with the F1 score, which takes into account both precision and recall performances improving from 0 . 807 to 0 . 838 for the detection of sweet pepper. In addition to improved accuracy, this approach is also much quicker to deploy for new fruits, as it requires bounding box annotation rather than pixel-level annotation (annotating bounding boxes is approximately an order of magnitude quicker to perform). The model is retrained to perform the detection of seven fruits, with the entire process taking four hours to annotate and train the new model per fruit. Yield mapping for tree crops by mechanical harvesting requires automatic detection and counting of fruits in tree canopy. However, partial occlusion, shape irregularity, varying illumination, multiple sizes and similarity with the background make fruit identification a very difficult task to achieve. Therefore, immature green citrus-fruit detection within a green canopy is a challenging task due to all the above-mentioned problems. A novel algorithmic technique was used to detect immature green citrus fruit in tree canopy under natural outdoor conditions. Shape analysis and texture classification were two integral parts of the algorithm. Shape analysis was conducted to detect as many fruits as possible. Texture classification by a support vector machine (SVM), Canny edge detection combined with a graph-based connected component algorithm and Hough line detection, were used to remove false positives. Next, keypoints were detected using a scale invariant feature transform (SIFT) algorithm and to further remove false positives. A majority voting scheme was implemented to make the algorithm more robust. The algorithm was able to accurately detect and count 80.4 of citrus fruit in a validation set of images acquired from a citrus grove under natural outdoor conditions. The algorithm could be further improved to provide growers early yield estimation so that growers can manage grove more efficiently on a site-specific basis to increase yield and profit. We developed a strawberry-harvesting robot, consisting of a cylindrical manipulator, end-effector, machine vision unit, storage unit and travelling unit, for application to an elevated substrate culture. The robot was based on the development concepts of night operation, peduncle handling and task sharing with workers, to overcome the robotic harvesting problems identified by previous studies, such as low work efficiency, low success rate, fruit damage, difficulty of detection in unstable illumination and high cost. In functional tests, the machine vision assessments of fruit maturity agreed with human assessments for the Amaotome and Beni-hoppe cultivars, but the performance for Amaotome was significantly better. Moreover, the machine vision unit correctly detected a peduncle of the target fruit at a rate of 60 . In harvesting tests conducted throughout the harvest season on target fruits with a maturity of 80 or more, the successful harvesting rate of the system was 41.3 when fruits were picked using a suction device before cutting the peduncle, while the rate was 34.9 when fruits were picked without suction. There were no significant differences between the two picking methods in terms of unsuccessful picking rates. The execution time for the successful harvest of a single fruit, including the time taken to transfer the harvested fruit to a tray, was 11.5 s. Fully automated yield estimation of intact fruits prior to harvesting provides various benefits to farmers. Until now, several studies have been conducted to estimate fruit yield using image-processing technologies. However, most of these techniques require thresholds for features such as color, shape and size. In addition, their performance strongly depends on the thresholds used, although optimal thresholds tend to vary with images. Furthermore, most of these techniques have attempted to detect only mature and immature fruits, although the number of young fruits is more important for the prediction of long-term fluctuations in yield. In this study, we aimed to develop a method to accurately detect individual intact tomato fruits including mature, immature and young fruits on a plant using a conventional RGB digital camera in conjunction with machine learning approaches. The developed method did not require an adjustment of threshold values for fruit detection from each image because image segmentation was conducted based on classification models generated in accordance with the color, shape, texture and size of the images. The results of fruit detection in the test images showed that the developed method achieved a recall of 0.80, while the precision was 0.88. The recall values of mature, immature and young fruits were 1.00, 0.80 and 0.78, respectively.