1. A method of improving depth sensor data in real-time based on scene sensor data, the method comprising: estimating, with one or more processor circuits, a homography H between a depth image and a scene image of a depth-scene image pair included in a plurality of depth-scene image pairs generated by a depth-scene image sensor pair, the depth-scene image sensor pair including a depth sensor and a scene sensor; aligning, with the one or more processor circuits, depth images of the depth-scene image pairs with corresponding scene images based on the homography H, wherein for each depth-scene image pair, the depth image is warped based on the homography H such that locations of depths in the depth image are aligned with locations of corresponding pixels in the scene image; and correcting, with the one or more processor circuits, at least some of the aligned depth images based on image data from one or more of the scene images.