1. A speech recognizer control system comprising: a speech input element to which a speech utterance is input from a user; a speech recognition element which recognizes the content of the speech utterance input to the speech input element; a device controlling element which identifies a device to be controlled among a plurality of devices on the basis of at least the content of the speech utterance recognized by the speech recognition element and which controls an operation of the identified device; and a state change storing element which stores, as first auxiliary information for identifying a device to be controlled, a state change other than at least a state change caused by a speech utterance from the user among the state changes of operations in the individual devices of the plurality of devices, wherein the device controlling element executes first device identification processing for identifying the device to be controlled on the basis of the content of the speech utterance recognized by the speech recognition element, and if the device cannot be identified by the first device identification processing, then the device controlling element uses at least the first auxiliary information stored in the state change storing element to identify the device.