BACKGROUND

Increasingly, vehicles, such as cars, airplanes, robots, etc., are being equipped with multiple external cameras to provide to the operator of the vehicle external views of the area surrounding the vehicle. These external views are commonly used to help maneuver the vehicle, such as when backing up or parking a car. Multiple camera views may be stitched together to form an external surround view around the vehicle. However, external views of areas which are not within a field of view of any cameras of such systems may not be available. Additionally, generating these multi-camera views requires multiple cameras, failure of one or more cameras can hinder operations of such systems. Therefore, it is desirable to have an improved technique for sensor fusion based perceptually enhanced surround view.

SUMMARY

This disclosure relates to a method, the method including obtaining a first location of a vehicle, the vehicle having two or more cameras disposed about the vehicle, and wherein each camera is associated with a physical camera pose indicating where each camera is located with respect to the vehicle, capturing, by a first camera, a first image of a first area in a first field of view of the first camera, associating the first image with the first location of the vehicle when the first image was captured, moving the vehicle in a direction so that the first area is in an expected second field of view of a second camera, wherein the second camera is not capturing images, obtaining a second location of the vehicle, determining a temporal camera pose based on a first physical camera pose of the first camera, a second physical camera pose of the second camera, and the second location of the vehicle, and rendering a view of the first area from the expected second field of view of the second camera based on the temporal camera pose and the first image.

Another aspect of the present disclosure relates to a non-transitory program storage device including instructions stored thereon to cause one or more processors to obtain a first location of a vehicle, the vehicle having two or more cameras disposed about the vehicle, and wherein each camera is associated with a physical camera pose indicating where each camera is located with respect to the vehicle, receive, from a first camera, a first image of a first area in a first field of view of the first camera, associate the first image with the first location of the vehicle when the first image was captured, obtain a second location of the vehicle after the vehicle has moved in a direction so that the first area is in an expected second field of view of a second camera, wherein the second camera is not capturing images, determine a temporal camera pose based on a first physical camera pose of the first camera, a second physical camera pose of the second camera, and the second location of the vehicle, and render a view of the first area based on the temporal camera pose and the first image.

Another aspect of the present disclosure relates to a system for rendering a view around a vehicle, the system including two or more cameras disposed about the vehicle, wherein each camera is associated with a physical camera pose indicating where each camera is located with respect to the vehicle, a memory, and one or more processors operatively coupled to the memory, and the one or more cameras, wherein the one or more processors are configured to execute non-transitory instructions causing the one or more processors to obtaining a first location of the vehicle, capture, by a first camera, a first image of a first area in a first field of view of the first camera, associate the first image with the first location of the vehicle when the first image was captured, obtain a second location of the vehicle after the vehicle has moved in a direction so that the first area is in an expected second field of view of a second camera, wherein the second camera is not capturing images, determine a temporal camera pose based on a first physical camera pose of the first camera, a second physical camera pose of the second camera, and the second location of the vehicle, and render a view of the first area based on the temporal camera pose and the first image.

It may be understood that while techniques herein are discussed in the context of visible light cameras and determining poses for physical and virtual cameras using a bowl shape, nothing in this disclosure is meant to limit these techniques to such sensors and techniques for determining poses. Rather, the techniques discussed herein are readily applicable across a broad range of sensor devices, including, non-visible light or electromagnetic sensors, including infrared, near infrared, or cameras capable of capturing images across a wide range of electromagnetic frequencies. The techniques discussed here are also further applicable to other ways of determining poses for physical and virtual cameras.

