1. An image decoder for decoding encoded image data, the image decoder comprising a processor and a memory, the processor, working together with the memory, being configured to: decode, from the encoded image data, (i) a parameter including prediction mode information and (ii) prediction residual data for each block of plurality of blocks to be intra-predicted; select, from a plurality of prediction sets, one prediction set to be used with respect to each block to be intra-predicted, wherein each prediction set of the plurality of prediction sets includes, as prediction mode candidates, a plurality of prediction modes including at least one direction prediction mode, wherein each of the plurality of prediction sets includes the same number of the plurality of prediction modes; select, based on the parameter including the prediction mode information, one prediction mode from the plurality of prediction modes included in the selected prediction set; form, on the basis of the selected prediction mode, a predicted image of each block to be intra-predicted; and form a decoded image of each block to be intra-predicted by adding, for each block to be intra-predicted, the predicted image of the block and the prediction residual data for the block wherein the forming of the predicted image includes referring to a rightmost one of pixels of a decoded block in a case where the decoded block is positioned on the left of the block to be intra-predicted.