1. An image forming apparatus, comprising: an image reading unit that reads an image of a document as image data of a plurality of color components used in a predetermined color system; a recording unit that forms an image on a recording medium based on image data obtained by a reading operation of the image reading unit; a detection unit that detects a predetermined operation restricting dot pattern formed on the document based on the image data obtained by the reading operation of the image reading unit, and detects a color of the operation restricting dot pattern when the operation restricting dot pattern is detected, the detection unit includes: a plurality of dot pattern detection units that are allocated respectively to image data of each of the plurality of color components, and, based on image data of the respective allocated color components, detect the operation restricting dot pattern of the allocated color component; and a dot pattern color determination unit that determines that the operation restricting dot pattern is formed on the document when any one of the plurality of dot pattern detection units detects the operation restricting dot pattern of the respective allocated color component, and determines that the color of the operation restricting dot pattern is formed on the document from a combination of presence or absence of detecting the color component of the operation restricting dot pattern by the plurality of dot pattern detection units; a storage unit that stores, beforehand, a correspondence relationship between the color of the operation restricting dot pattern and a restriction detail for an operation of the image forming apparatus, the correspondence relationship defines a relationship that allocates different restriction details to the operation restricting dot pattern according to the colors of the operation restricting dot pattern; and a control unit that reads, from the storage unit, the restriction detail corresponding to the color of the operation restricting dot pattern when the operation restricting dot pattern is formed and the color of the restricting dot pattern is determined by the detection unit, and then restricts the operation of the image forming apparatus in accordance with the restriction detail.