1. A method for optimizing object detection performed by autonomous vehicles, the method comprising: receiving, by one or more computing devices having one or more processors, sensor data captured by a sensor of an autonomous vehicle, wherein the sensor data includes one or more first object labels applied by the autonomous vehicle, each of the one or more first object labels corresponding to an object of the sensor data; receiving, by the one or more computing devices, one or more second object labels for the sensor data; determining, by the one or more computing devices, a difference between a first value indicating a quantity of the one or more first object labels applied by the autonomous vehicle and a second value indicating a quantity of the one or more second object labels; determining, by the one or more computing devices, whether the autonomous vehicle has met a predetermined missed object threshold corresponding to a maximum allowable quantity of object labels not applied by the autonomous vehicle by comparing the difference to the threshold; and when the autonomous vehicle has met the threshold, generating a recommendation for an adjustment of at least one object detection parameter of at least one of a plurality of sensors in order to optimize object detection.