1. An image decoder for decoding encoded data so as to form a decoded image, comprising: a decoding circuit configured to decode, from the encoded data, (i) an encode parameter including prediction mode information and (ii) prediction residual data; a first selecting circuit configured to select, from a plurality of prediction sets, one prediction set to be used with respect to each target block to be intra-predicted, wherein the prediction set includes, as prediction mode candidates, a plurality of prediction modes including at least one direction prediction mode, wherein each of the plurality of prediction sets includes the same number of the plurality of prediction modes; a second selecting circuit configured to select one prediction mode from the plurality of prediction modes included in the prediction set selected by the first selector; a predicting circuit configured to form, on the basis of the prediction mode thus selected, a predicted image of the each target block to be intra-predicted; and an adding circuit configured to form a decoded image of the each target block to be intra-predicted by adding the predicted image formed by the predicting circuit and the prediction residual data obtained by decoding the encoded data, wherein the predicting circuit configured to refer to a rightmost one of pixels of a decoded block in a case where the decoded block is positioned on the left of the each target block to be intra-predicted.