1. A method for reconstructing a physical space, comprising: obtaining a plurality of images of the physical space using a camera carried by a rig while the rig moves in the physical space through a plurality of poses; determining an orientation and a coordinate system of each pose; partitioning the physical space into a plurality of volumes, each volume of the plurality of volumes comprises vertices; for each volume of the plurality of volumes, obtaining a depth map of the physical space using a depth sensor carried by the rig; providing a pose graph in a first state, the pose graph in the first state comprising a plurality of nodes and links between nodes of the plurality of nodes, each node of the plurality of nodes is associated with: one of the poses, one of the plurality of images associated with the one of the poses and one of the coordinate systems associated with the one of the poses, and the links define correspondences between the plurality of images and transforms between the coordinate systems; for each volume of the plurality of volumes, anchoring the vertices of the volume to at least two different points in the pose graph in the first state, the vertices of the volume comprise shared vertices, the shared vertices are shared with an adjacent volume of the plurality of volumes when the pose graph is in the first state; updating the pose graph to a second state, the updating comprising updating one or more of the nodes and one or more of the links; and for each volume of the plurality of volumes, maintaining the shared vertices when the pose graph is in the second state, wherein the maintaining the shared vertices results in changing a shape of at least one volume of the plurality of volumes.