1. An image processing apparatus comprising: a line segment detecting unit configured to detect vertical line segments and horizontal line segments in an image; a facing-lines candidate generating unit configured to generate vertical facing-lines candidates and horizontal facing-lines candidates from the vertical line segments and the horizontal line segments, the vertical facing-lines candidates and horizontal facing-lines candidates being candidates for pairs of facing lines configuring quadrangle areas in the image; a rectangle candidate generating unit configured to generate a plurality of pairs of one of the vertical facing-lines candidates and one of the horizontal facing-lines candidates, and to generate the quadrangle areas as rectangle candidates, the quadrangle areas having intersecting points of each pair of the vertical facing-lines candidates and the horizontal facing-lines candidates as four corner points; and a calculating unit configured to calculate likelihood of each of the rectangle candidates based on a relationship between line segments constituting the vertical facing-lines candidates and the horizontal facing-lines candidates and the rectangle candidates.