Novel view synthesis aims to synthesize new images from different viewpoints of given images. Most of previous works focus on generating novel views of certain objects with a fixed background. However, for some applications, such as virtual reality or robotic manipulations, large changes in background may occur due to the egomotion of the camera. Generated images of a large-scale environment from novel views may be distorted if the structure of the environment is not considered. In this work, we propose a novel fully convolutional network, that can take advantage of the structural information explicitly by incorporating the inverse depth features. The inverse depth features are obtained from CNNs trained with sparse labeled depth values. This framework can easily fuse multiple images from different viewpoints. To fill the missing textures in the generated image, adversarial loss is applied, which can also improve the overall image quality. Our method is evaluated on the KITTI dataset. The results show that our method can generate novel views of large-scale scene without distortion. The effectiveness of our approach is demonstrated through qualitative and quantitative evaluation.
The ability to predict the future movements of a robots' surroundings is one of the crucial requirements for decision-making and planning in robotics. Most related works anticipate the movement of certain objects in the scene under the assumption that the background is fixed @cite_22 . However, in the real world, robot's view constantly changing as it moves, which introduces additional challenges when solving this problem.
@cite_22
2281874712 2403419699 1988994102 2591342762
For robotic systems to realise lifelong autonomy they must be able to navigate accurately in changing environments. In this thesis we describe, implement and validate a new approach to the problem of long-term navigation. To begin, we present our stereo visual odometry system which provides highly accurate pose estimation. Our approach combines several techniques found in existing implementations and a recently published image descriptor that simplifies the solution architecture. The performance and versatility of our system is demonstrated through testing on multiple datasets. Equipped with our visual odometry system, we describe a new approach to the problem of lifelong navigation. We learn a model whose complexity varies naturally in accordance with the variation of scene appearance. As the robot repeatedly traverses its workspace, it accumulates distinct visual experiences that, in concert, implicitly represent the scene variation - each experience captures a visual mode. When operating in a previously visited area, we continually try to localise in these previous experiences while simultaneously running the visual odometry. Failure to localise in a sufficient number of prior experiences indicates an insufficient model of the workspace and instigates the laying down of the live image sequence as a new distinct experience. In this way, over time we can capture the typical temporally varying appearance of an environment and the number of experiences required tends to a constant. Although we focus on vision as a primary sensor, the ideas we present here are equally applicable to other sensor modalities. We demonstrate our approach working on a road vehicle operating over a three month period at different times of day, in different weather and lighting conditions. To facilitate interaction with people, robots must not only recognize current actions, but also infer a person's intentions and future behavior. Recent advances in depth camera technology have significantly improved human motion tracking. However, the inherent high dimensionality of interacting with the physical world makes efficiently forecasting human intention and future behavior a challenging task. Predictive methods that estimate uncertainty are therefore critical for supporting appropriate robotic responses to the many ambiguities posed within the human-robot interaction setting. We address these two challenges, high dimensionality and uncertainty, by employing predictive inverse optimal control methods to estimate a probabilistic model of human motion trajectories. Our inverse optimal control formulation estimates quadratic cost functions that best rationalize observed trajectories framed as solutions to linear-quadratic regularization problems. The formulation calibrates its uncertainty from observed motion trajectories, and is efficient in high-dimensional state spaces with linear dynamics. We demonstrate its effectiveness on a task of anticipating the future trajectories, target locations and activity intentions of hand motions. This paper presents a strategy for planning robot motions in dynamic, uncertain environments (DUEs). Successful and efficient robot operation in such environments requires reasoning about the future evolution and uncertainties of the states of the moving agents and obstacles. A novel procedure to account for future information gathering (and the quality of that information) in the planning process is presented. To approximately solve the stochastic dynamic programming problem that is associated with DUE planning, we present a partially closed-loop receding horizon control algorithm whose solution integrates prediction, estimation, and planning while also accounting for chance constraints that arise from the uncertain locations of the robot and obstacles. Simulation results in simple static and dynamic scenarios illustrate the benefit of the algorithm over classical approaches. The approach is also applied to more complicated scenarios, including agents with complex, multimodal behaviors, basic robot-agent interaction, and agent information gathering. There is an inherent need for autonomous cars, drones, and other robots to have a notion of how their environment behaves and to anticipate changes in the near future. In this work, we focus on anticipating future appearance given the current frame of a video. Existing work focuses on either predicting the future appearance as the next frame of a video, or predicting future motion as optical flow or motion trajectories starting from a single video frame. This work stretches the ability of CNNs (Convolutional Neural Networks) to predict an anticipation of appearance at an arbitrarily given future time, not necessarily the next video frame. We condition our predicted future appearance on a continuous time variable that allows us to anticipate future frames at a given temporal distance, directly from the input video frame. We show that CNNs can learn an intrinsic representation of typical appearance changes over time and successfully generate realistic predictions at a deliberate time difference in the near future.