1. A method for classifying objects in an image, comprising the steps of: applying an appearance classifier to the image to determine candidates of the objects and statistics associated with the candidates, wherein the appearance classifier uses a set of windows, and the candidates are in selected windows; and applying a context classifier only to the selected windows of the image to determine an identity, and location of objects in the image, wherein the steps are performed in a processor, wherein the context classifier uses object context features and global context features and wherein the object context features encode relative information between candidates including a size of the candidate divided by a mean of the size of all candidates, a score divided by a mean score, the score divided by a maximal score, location divided by a mean location of all candidates, and average distance from all other candidates divided by an average of candidate pair-wise distances.