Autonomous harvesting may provide a viable solution to mounting labor pressures in the United States's strawberry industry. However, due to bottlenecks in machine perception and economic viability, a profitable and commercially adopted strawberry harvesting system remains elusive. In this research, we explore the feasibility of using deep reinforcement learning to overcome these bottlenecks and develop a practical algorithm to address the sub-objective of viewpoint optimization, or the development of a control policy to direct a camera to favorable vantage points for autonomous harvesting. We evaluate the algorithm's performance in a custom, open-source simulated environment and observe encouraging results. Our trained agent yields 8.7 times higher returns than random actions and 8.8 percent faster exploration than our best baseline policy, which uses visual servoing. Visual investigation shows the agent is able to fixate on favorable viewpoints, despite having no explicit means to propagate information through time. Overall, we conclude that deep reinforcement learning is a promising area of research to advance the state of the art in autonomous strawberry harvesting.
This paper is the first of a two-part series on the topic of visual servo control using computer vision data in the servo loop to control the motion of a robot. In this paper, we describe the basic techniques that are by now well established in the field. We first give a general overview of the formulation of the visual servo control problem. We then describe the two archetypal visual servo control schemes: image-based and position-based visual servo control. Finally, we discuss performance and stability issues that pertain to these two schemes, motivating the second article in the series, in which we consider advanced techniques The main contribution of this paper is in the development of vision-based estimation and control system for robotic fruit harvesting and rigorous stability analysis to guarantee performance of the closed-loop system. The presented cooperative visual servo controller benefits from the large field-of-view of a fixed camera and the accuracy of a camera-in-hand (CiH). Computationally inexpensive perspective transformation-based range estimation method obtains 3D fruit position using a monocular camera to enable real-time manipulator control. A rotation controller is developed to orient the robot such that the target fruit selected by the fixed camera can be viewed by the CiH attached to the end-effector. Subsequently, the end-effector can be servoed to the target fruit location using the presented pursuit guidance based hybrid translation controller. Lyapunov-based stability analysis guarantees global exponential regulation of the end-effector. Numerical simulations verify the feasibility of the developed controller while the performance is evaluated on a seven degrees-of-freedom kinematically redundant manipulator using an artificial citrus tree. The position of the fruit was randomly selected, and the closed-loop visual servo control experiment was performed 21 times to analyze the repeatability and accuracy of the developed controller. With 95 confidence level the expected position of the robot end-effector is observed to lie within the confidence ellipsoid. The accuracy of the controller was observed to be about 15mm, thus making the system suitable for harvesting medium and large varieties of citrus fruit but may limit operation for small varieties such as page and blood oranges.