We present a coarse-to-fine approach based semi-autonomous teleoperation system using vision guidance. The system is optimized for long range teleoperation tasks under time-delay network conditions and does not require prior knowledge of the remote scene. Our system initializes with a self exploration behavior that senses the remote surroundings through a freely mounted eye-in-hand web cam. The self exploration stage estimates hand-eye calibration and provides a telepresence interface via real-time 3D geometric reconstruction. The human operator is able to specify a visual task through the interface and a coarse-to-fine controller guides the remote robot enabling our system to work in high latency networks. Large motions are guided by coarse 3D estimation, whereas fine motions use image cues (IBVS). Network data transmission cost is minimized by sending only sparse points and a final image to the human side. Experiments from Singapore to Canada on multiple tasks were conducted to show our system's capability to work in long range teleoperation tasks.
Long range teleoperation control has lots of work done using Internet connections @cite_2 , @cite_26 . While it provides a possibility for convenient operation regardless of location, it typically omits a common limitation in practice: network conditions. Low bandwidth and high latency networks are more common, for example, in tasks such as space missions (e.g., 6 to 44 minutes delay in Martian Rover @cite_5 ) where an Internet connection is not available.
@cite_5 @cite_26 @cite_2
2037986392 2626574340 2588104504 2076447811
Abstract Internet telerobotics has emerged in recent decade with direct control and supervisory control as the main teleoperation paradigms. Both paradigms, however, are difficult to use on applications operating in the unknown and dynamic real world, while they do not provide adequate feeling of interaction or a human-friendly control interface to human operator. This paper proposes a novel interactive control (i.e., active supervisory control) paradigm: telecommanding, which is used for Internet-based wheeled robot teleoperation. Telecommanding involves two parts: basic telecommanding using joystick commands, and advanced telecommanding using linguistic commands. Each joystick or linguistic command is designed to perform an independent task and is defined with multiple events (non-time action references), and the corresponding response functions. This event-driven mechanism enables the robot to deliberately respond to expected events while to reactively respond to unexpected events. Assisted by up-to-date media streaming technologies, telecommanding can help a novice operator to easily control an Internet robot navigating in an unknown and dynamic real world. Experiments, including an Internet-based teleoperation test over 1500 km from Beijing to Hong Kong, demonstrate the promising performance. Controlling robots in real-time over a wireless interface present fundamental challenges for forthcoming fifth generation wireless networks. Mission critical real-time applications such as telesurgery over the tactile Internet require a communication link that is both ultra-reliable and low-latency, and that simultaneously serving multiple devices and applications. Wireless performance requirements for these applications surpass the capabilities of current wireless cellular standards. The prevailing ambitions for the fifth generation wireless specifications go beyond higher throughput and embrace the wireless performance demands of mission critical real-time applications in robotics and the Internet of Things. To accommodate these demands, changes have to be made across all layers of the wireless infrastructure. The fifth generation wireless standards are far from finalized but massive Multiple-Input Multiple- Output has surfaced as a strong radio access technology candidate and has great potential to cope with all these stringent requirements. In this paper, we investigate how Ultra-Reliable and Low-Latency Communication with massive MIMO can be achieved for bilateral teleoperation, an integral part of the tactile Internet. We conclude through simulation what the performance bounds are for massive MIMO and thus how to configure such a system for near deterministic latency and what the inherit trade-offs are. Robot teleoperation systems face a common set of challenges including latency, low-dimensional user commands, and asymmetric control inputs. User control with Brain---Computer Interfaces (BCIs) exacerbates these problems through especially noisy and erratic low-dimensional motion commands due to the difficulty in decoding neural activity. We introduce a general framework to address these challenges through a combination of computer vision, user intent inference, and arbitration between the human input and autonomous control schemes. Adjustable levels of assistance allow the system to balance the operators capabilities and their perception of control authority. Additionally, a custom servo controller design allow for safe interactions of the robotic arm with the environment. We present experimental results demonstrating significant performance improvement using our shared-control assistance framework on adapted rehabilitation benchmarks with two subjects at various timepoints relative to their implantation with intracortical BCIs. Our results indicate that shared assistance mitigates perceived user difficulty in using a seven-degree of freedom robotic arm as a prosthetic and enables successful performance on previously infeasible tasks. We showcase the extensibility of our architecture with applications to quality-of-life tasks such as opening a door, pouring liquids from containers, and manipulation with objects previously unknown to the system in densely cluttered environments. Today’s Internet technology provides a convenient way for us to develop an integrated network environment for the diversified applications of different robotic systems. To be successful in real‐world applications, Internet‐based robots require a high degree of autonomy and local intelligence to deal with the restricted bandwidth and arbitrary transmission delay of the Internet. This paper describes the first step toward building such an Internet‐based robotic system for teleoperation in the University of Essex. The system has a standard network protocol and an interactive human‐machine interface. Using a Web browser, a remote operator can control the mobile robot to navigate in our laboratory with visual feedback and a simulated environment map via the Internet. The employment of an intuitive user interface enables Internet users to control the mobile robot and implement useful tasks remotely. Although at its first stage, the developed system has the potential to be extended to many real‐world applications such as tele‐manufacturing, tele‐training and tele‐service.