We examined whether failures to generalize are restricted to the smaller and more distant cars by calculating AP@0.5IOU as a function of distance. Specifically, we evaluated the performance of networks trained on different public and synthetic datasets on the synthetic images in the ISETAuto, which include metadata describing the distance to each of the cars. Failures to generalize were present for cars at all distances (Figure 6). This suggests that the limited generalization is not associated with the distance or size of the car in the image.
[t!](topskip=0pt, botskip=0pt, midskip=0pt)[width=3 in]fig6.png
Performance (AP@0.5IOU) for a network model trained on different real and synthetic datasets (legend). The networks are evaluated using the ISETAuto dataset which includes metadata about the distance to each car [1]}. The ordering of the generalization remains consistent for cars at all distances.
