1. An imaging apparatus comprising: an image pickup device comprising a plurality of photoelectric conversion elements arrayed in a predetermined first direction and a second direction intersecting with the first direction; a color filter that is provided above a plurality of pixels configured by the plurality of photoelectric conversion elements, the color filter having a repeatedly disposed basic array pattern of array lines that each include in the second direction at least one of each of a first filter corresponding to a first color that contributes most to obtaining a brightness signal or each of a plurality of second filters respectively corresponding to 2 or more second colors other than the first color, and include a plurality of matching array lines that have the same array as each other along the second direction, and at least one array line that has a different array for the first filter and the second filters in the second direction to that of the matching array lines; a line image data generation section that reads pixel signals of the plurality of pixels at a set cycle from the image pickup device, and, from the read pixel signals, generates, from out of the plurality of pixels, line image data of each of the matching array lines in the basic any pattern; a pixel summing section that sums together pixels that are the same color pixel as each other in the generated line image data for the matching array lines; and an image data generation section that generates image data based on the line image data that has been pixel summed.