1. A driver assistance system for a vehicle, said driver assistance system comprising: an imaging device comprising an array of photosensing pixels, wherein said imaging device is disposed at least partially within an exterior rearview mirror assembly that attaches at a side of a vehicle equipped with said driver assistance system; wherein, with said exterior rearview mirror assembly attached at the side of the equipped vehicle, said imaging device has a field of view forward of said exterior rearview mirror assembly in a direction towards the front of the equipped vehicle and has a field of view rearward of said exterior rearview mirror assembly in a direction towards the rear of the equipped vehicle and has a field of view sideward of said exterior rearview mirror assembly in a direction away from the side of the equipped vehicle at which said exterior rearview mirror assembly is attached; a control for processing image data captured by said imaging device; wherein said control is operable to determine that said imaging device is misaligned when said imaging device is disposed at least partially within the exterior rearview mirror assembly at the side of the equipped vehicle; and wherein said control, responsive to a determination of misalignment of said imaging device, is operable to at least partially compensate for misalignment of said imaging device by at least one of (i) adjusting processing of captured image data and (ii) adjusting said captured image data.