1. A character recognition apparatus, which extracts a character image from an original image including the character image and recognizes characters with respect to the character image, comprising: a background discriminating section that discriminates the background area based on the dispersion amount of color component and the frequency distribution with respect to color information in the original image, a non-character line discriminating section that discriminates non-character lines other than the characters and obtains non-character line information with respect to the non-character lines based on, in the original image, the frequency distribution of pixels having color information different from that of pixels constituting the background area and continuous pixels each having the same color characteristics and continuing more than a predetermined number, a first non-character line removed image creating section that creates a first non-character line removed image in which the non-character lines are removed from the original image by, in the original image, replacing the non-character lines, which are discriminated by the non-character line discriminating section, with pixels having the same color component as that of the background, a first character area discriminating section that discriminates, in the first non-character line removed image, which is created by the first non-character line removed image creating section, the area other than the background area including the continuous pixels each having the same color characteristics and continuing more than a predetermined number, an enlarged image creating section that creates an enlarged image based on the original image, a second non-character line removed image creating section that creates a second non-character line removed image in which non-character lines are removed from the enlarged image by, in the enlarged image, which is created by the enlarged image creating section, replacing the non-character lines with a background pixels, which have the same color component as that of the background, based on the non-character line information, which is obtained by the non-character line discriminating section, an interference judgment section that judges, in the original image, whether or not the characters and the non-character line interfere with each other based on the non-character line information, which is obtained by the non-character line discriminating section and the character area, which is discriminated by the first character area discriminating section, a character image restoring section that, when the interference judgment section judges that the character image and the non-character line interfere with each other, restores the character image with respect to the interference point between the character image and the non-character line in the second non-character line removed image based on the difference between the color information with respect to the character image and the color information with respect to the non-character line, a second character area discriminating section that discriminates the area larger than a predetermined size, which is, in the second non-character line removed image, the area other than the background area including the continuous pixels each having the same color characteristics and continuing more than a predetermined number, as the character area, and a character recognizing section that digitizes the character area, where is discriminated by the second character area discriminating section and recognizes the characters.