1. A vision system for a vehicle, said vision system comprising: a camera disposed at a vehicle equipped with said vision system, said camera comprising an image sensor and a lens, wherein, when said camera is disposed at the equipped vehicle, said lens is exposed to the environment exterior the equipped vehicle; wherein said camera has an exterior field of view exterior the equipped vehicle and is operable to capture image data representative of a scene occurring in the field of view of said camera; an image processor operable to process multiple frames of image data captured by said camera; wherein said image processor is operable to process captured image data to detect a blob in a frame of captured image data; wherein, responsive to processing a first frame of captured image data, and responsive to said image processor determining with a confidence level of at least 80% a first threshold likelihood that a detected blob is indicative of a contaminant, said image processor adjusts processing when processing subsequent frames of captured image data; wherein adjustment of processing when processing the subsequent frames of captured image data comprises focusing in the subsequent frames on image data associated with the blob detected in the first frame; and wherein, responsive to said image processor determining with a confidence level of at least 80% a second threshold likelihood that the detected blob is indicative of a contaminant when processing subsequent frames of captured image data, said image processor determines that the detected blob is representative of a contaminant at said lens of said camera.