1. A non-transitory machine-readable storage medium containing program code that when processed by a processor causes a method to be performed, the method comprising: receiving a first image that was captured by a camera at a first time, the first image showing an object in view of the camera at the first time; analyzing the first image to determine, for each feature in a set of features of the object, a first image position for the feature at the first time; receiving a second image that was captured by the camera at a second time that occurs after the first time, the second image showing the object in view of the camera at the second time: analyzing the second image to determine, for each feature in the set of features of the object, a second image position for the feature at the second time; determining, based on the first image positions for at least a first subset of the set of features of the object at the first time and the second image positions for the at least the subset of the set of features of the object at the second time, whether the object moved out of plane relative to the camera from the first time to the second time; in response to determining that the object moved out of plane relative to the camera from the first time to the second time, determining an amount of out-of-plane movement of the object from the first time to the second time based on changes in (i) distances between pairs of features of the object in at least a second subset of the set of features of the object at the first time and (ii) distances between the pairs of features of the object at the second time; and adjusting a position of a lens of the camera based on the amount of out-of-plane movement of the object from the first time to the second time, so as to focus the camera on the object.