1. A spectral imaging system for imaging a scene or object, the spectral imaging system comprising: a spectral image sensor unit that receives a projected image of the scene or object with a field of view moving such that the image is projected along a path over the spectral image sensor unit resulting from the movement; the spectral image sensor unit comprising at least two sets of light sensor elements in a focal plane, each of the light sensor elements comprising a filter that adapts the spectral response of the light sensor element; wherein, within each set of the at least two sets of light sensor elements, there are light sensor elements with at least two types of different spectral responses; wherein each type of light sensor element is positioned in an elongated section having a width of at least one light sensor element, the spectral responses in the at least two sets of light sensor elements being essentially identical; wherein the elongated sections have an orientation and position on the spectral image sensor unit relative to a direction of movement so that at least part of the scene passes the same spectral responses at least twice; wherein a readout from each sensor line is sampled at a predetermined rate, the predetermined sampling rate being chosen relative to the movement so as to provide at least one sample from each line from each position in the scene in the direction of movement, each position in the scene being sampled in each spectral response at least twice; and wherein a plurality of readings of a position in the scene obtained with light sensor elements having essentially identical spectral responses and different viewing angles are averaged.