1. An apparatus of decoding image, the apparatus comprising: an entropy decoder which extracts information that indicates an intra prediction mode applied to a current block to be decoded, from a bitstream; a reference pixel determining unit which determines one of neighboring pixels adjacent to the current block and filtered neighboring pixels filtered from the neighboring pixels as reference pixels, based on at least one of a size of the current block and an intra prediction mode of the current block; an intra prediction performing unit which performs intra prediction on the current block using the extracted information and the determined reference pixels, wherein the image is split into a plurality of maximum coding units, according to information about a maximum size of a coding unit, the maximum coding unit is hierarchically split into one or more coding units of depths according to split information, a coding unit of a current depth is one of rectangular data units split from a coding unit of an upper depth, when the split information indicates a split for the current depth, the coding unit of the current depth is split into coding units of a lower depth, independently from neighboring coding units, and the coding unit of the current depth is split into at least one prediction unit.