1. A 3D image capture device comprising: a light transmitting section with m transmitting areas (where m is an integer that is equal to or greater than two) that have mutually different spectral transmittance characteristics; an image sensor which is arranged to receive light rays that have been transmitted through the m transmitting areas and in which a plurality of unit elements are arranged, each said unit element including n photosensitive cells (where n is an integer that is equal to or greater than m) and n transmitting filters with mutually different spectral transmittance characteristics which are arranged so as to face the n photosensitive cells; an imaging section which produces an image on the imaging area of the image sensor; and an image processing section which modifies an n×m matrix that is defined by the respective spectral transmittance characteristics of the m transmitting areas and the n transmitting filters by changing the angle between at least two out of m column vectors of the n×m matrix and which generates multi-viewpoint images represented by light rays that have been incident on at least two of the m transmitting areas based on the modified n×m matrix and n photoelectrically converted signals supplied from the n photosensitive cells.