1. An image processing apparatus comprising: an input unit configured to input a document image obtained by reading a document; a vectorization unit configured to vectorize an outline of a character portion in the document image into an outline vector by using vector data; a storage unit configured to store outline font data; a determination unit configured to determine whether a size of a character represented by the outline vector to be output is not more than a predetermined size; a replacement unit configured to replace the outline vector with a character code indicating the character and size information indicating the size of the character to be output in a case where said determination unit determines that the size of the character to be output is not more than the predetermined size; and a generation unit configured to read out the outline font data from the storage unit based on the character code and generate image data by using the outline font data and the size information in the case where said determination unit determines that the size of the character to be output is not more than the predetermined size, and generate image data by using the outline vector and the size information in a case where said determination unit determines that the size of the character to be output is more than the predetermined size.