1. A system for automatically collecting and labeling data about a physical object, comprising: a mechanical device configured to maneuver the object within a three-dimensional (3D) workspace in a real-world environment; one or more sensors configured to capture and output images of the object when it is within one or more prescribed regions of the workspace; a computing device; and a computer program having program modules executable by the computing device, the computing device being directed by the program modules of the computer program to, calibrate the sensors to the coordinate system of the mechanical device, for each of a plurality of different object poses within said regions of the workspace, use the mechanical device to maneuver the object into the object pose, input an image of the object from each of the sensors while the object is in the object pose, input data specifying the object pose, the data specifying the pose being in the coordinate system of the mechanical device, and label the image of the object input from each of the sensors with the data specifying the object pose and with information identifying the object, and generate a database for the object, said database comprising the labeled image of the object input from each of the sensors for each of said different object poses.