We consider UAV IoT aerial sensing that delivers multiple VR AR immersive communication sessions to remote users. The UAV swarm is spatially distributed over a wide area of interest, and each UAV captures a viewpoint of the scene below it. The remote users are interested in visual immersive navigation of specific subareas scenes of interest, reconstructed on their respective VR AR devices from the captured data. The reconstruction quality of the immersive scene representations at the users will depend on the sampling sensing rates associated with each UAV. There is a limit on the aggregate amount of data that the UAV swarm can sample and send towards the users, stemming from physical transmission capacity constraints. Similarly, each VR AR application has minimum reconstruction quality requirements for its own session. We propose an optimization framework that makes three contributions in this context. First, we select the optimal sampling rates to be used by each UAV, such that the system and application constraints are not exceed, while the priority weighted reconstruction quality across all VR AR sessions is maximized. Then, we design an optimal scalable source-channel signal representation that instills into the captured data inherent rate adaptivity, unequal error protection, and minimum required redundancy. Finally, the UAV transmission efficiency is enhanced by the use of small-form-factor multi-beam directional antennas and optimal power link scheduling across the scalable signal representation layers. Our experiments demonstrate competitive advantages over conventional methods for visual sensing. This is a first-of-its-kind study of an emerging application of prospectively broad societ al impact.
VR AR immersive communication via UAV IoT sensing is a new topic that has not been studied before. Closely related areas include ground-based multi-camera wireless sensing for multi-view systems @cite_50 , immersive telecollaboration @cite_7 @cite_11 @cite_40 , multi-view video coding communication @cite_29 @cite_42 @cite_44 @cite_19 , and 360 @math video streaming @cite_25 @cite_28 . Another related area is graph-based signal processing for time-varying point clouds used in AR applications @cite_36 @cite_10 .
@cite_7 @cite_28 @cite_36 @cite_29 @cite_42 @cite_44 @cite_19 @cite_40 @cite_50 @cite_10 @cite_25 @cite_11
926607442 2774492100 2952242169 1987648924
Cameras provide a rich source of information while being passive, cheap and lightweight for small Unmanned Aerial Vehicles (UAVs). In this work we present the first implementation of receding horizon control, which is widely used in ground vehicles, with monocular vision as the only sensing mode for autonomous UAV flight in dense clutter. Two key contributions make this possible: novel coupling of perception and control via relevant and diverse, multiple interpretations of the scene around the robot, leveraging recent advances in machine learning to showcase anytime budgeted cost-sensitive feature selection, and fast non-linear regression for monocular depth prediction. We empirically demonstrate the efficacy of our novel pipeline via real world experiments of more than 2 kms through dense trees with an off-the-shelf quadrotor. Moreover our pipeline is designed to combine information from other modalities like stereo and lidar. Mobile wireless sensor networks have been extensively deployed for enhancing environmental monitoring and surveillance. The availability of low-cost mobile robots equipped with a variety of sensors makes them promising in target coverage tasks. They are particularly suitable where quick, inexpensive, or nonlasting visual sensing solutions are required. In this paper, we consider the problem of low complexity target tracking to cover and follow moving targets using flying robots. We tackle this problem by clustering targets while estimating the camera location and orientation for each cluster separately through a cover-set coverage method. We also leverage partial knowledge of target mobility to enhance the efficiency of our proposed algorithms. Three computationally efficient approaches are developed: predictive fuzzy , predictive incremental fuzzy , and local incremental fuzzy . The objective is to find a compromise among coverage efficiency, traveled distance, number of drones required, and complexity. The targets move according to one of the following three possible mobility patterns: random waypoint, Manhattan grid, and reference point group mobility patterns. The feasibility of our algorithms and their performance are also tested on a real-world indoor testbed called drone-be-gone , using Parrot AR.Drone quadcopters. The deployment confirms the results obtained with simulations and highlights the suitability of the proposed solutions for real-time applications. In this paper, the efficient deployment and mobility of multiple unmanned aerial vehicles (UAVs), used as aerial base stations to collect data from ground Internet of Things (IoT) devices, is investigated. In particular, to enable reliable uplink communications for IoT devices with a minimum total transmit power, a novel framework is proposed for jointly optimizing the three-dimensional (3D) placement and mobility of the UAVs, device-UAV association, and uplink power control. First, given the locations of active IoT devices at each time instant, the optimal UAVs' locations and associations are determined. Next, to dynamically serve the IoT devices in a time-varying network, the optimal mobility patterns of the UAVs are analyzed. To this end, based on the activation process of the IoT devices, the time instances at which the UAVs must update their locations are derived. Moreover, the optimal 3D trajectory of each UAV is obtained in a way that the total energy used for the mobility of the UAVs is minimized while serving the IoT devices. Simulation results show that, using the proposed approach, the total transmit power of the IoT devices is reduced by 45 compared to a case in which stationary aerial base stations are deployed. In addition, the proposed approach can yield a maximum of 28 enhanced system reliability compared to the stationary case. The results also reveal an inherent tradeoff between the number of update times, the mobility of the UAVs, and the transmit power of the IoT devices. In essence, a higher number of updates can lead to lower transmit powers for the IoT devices at the cost of an increased mobility for the UAVs. We present a system for accurate real-time mapping of complex and arbitrary indoor scenes in variable lighting conditions, using only a moving low-cost depth camera and commodity graphics hardware. We fuse all of the depth data streamed from a Kinect sensor into a single global implicit surface model of the observed scene in real-time. The current sensor pose is simultaneously obtained by tracking the live depth frame relative to the global model using a coarse-to-fine iterative closest point (ICP) algorithm, which uses all of the observed depth data available. We demonstrate the advantages of tracking against the growing full surface model compared with frame-to-frame tracking, obtaining tracking and mapping results in constant time within room sized scenes with limited drift and high accuracy. We also show both qualitative and quantitative results relating to various aspects of our tracking and mapping system. Modelling of natural scenes, in real-time with only commodity sensor and GPU hardware, promises an exciting step forward in augmented reality (AR), in particular, it allows dense surfaces to be reconstructed in real-time, with a level of detail and robustness beyond any solution yet presented using passive computer vision.