1. An image processing apparatus comprising: a decoding unit configured to decode a bit stream to generate an image; and a prediction unit configured to predict the image generated by the decoding unit, by reference to a list of reference images generated based on layer reference image information that controls reference images for each layer, the reference images being referable in a layer direction and referred in predicting the image that is the bit stream subjected to decoding processing.