1. A speech recognizer control system comprising: a speech input element to which a speech utterance is inputted from a first user; a speech recognition element which recognizes a content of the speech utterance input to the speech input element; a device controlling element which identifies a device to be controlled among a plurality of devices on the basis of at least the content of the speech utterance recognized by the speech recognition element, wherein the content of the speech utterance controls an operation of the identified device; and a state change storing element storing a first auxiliary information for identifying a device to be controlled, wherein the first auxiliary information comprises an identification of an active state corresponding to at least a state change not caused by a speech utterance from the first user, wherein the state change is stored as an active state until a predetermined amount of time elapses from a time at which the state change occurred, wherein the first auxiliary information further comprises an identification of at least one of a plurality of predetermined types of human senses associated with the content of the speech utterance; wherein the device controlling element executes first device identification processing for identifying the device to be controlled on the basis of the content of the speech utterance recognized by the speech recognition element; and if the device cannot be identified by the first device identification processing, then the device controlling element uses at least the first auxiliary information stored in the state change storing element to identify the device to be controlled.