1. An image processing apparatus for extracting a line segment included in an image comprising: a density gradient direction determining section for determining a density gradient direction, in which density changes, of each processing unit composed of a predetermined number of pixels of an image; and a line segment extracting section, which regards a couple of processing units with an opposite density gradient direction to each other as a processing unit pair, for extracting a processing unit group including a plurality of processing units which are allocated in a row in a direction perpendicular direction to the density gradient directions, as a line segment; and wherein the line segment extracting section regards a plurality of processing unit pairs allocated in a row in a direction perpendicular to the density gradient directions as a cluster of processing unit pairs, and generates the processing unit group by grouping two clusters of processing unit pairs, of which an endpoint of one cluster of processing unit pairs and an endpoint of the other cluster of processing unit pairs are allocated each other within a fourth threshold distance and also a difference in slopes in a direction perpendicular to the density gradient directions between the two clusters of processing unit pairs is within a fifth threshold.