1. A method of authenticating a telephone caller, the method comprising: receiving, by a processor of an authentication server, audio data including speech of the telephone caller; analyzing, by the processor, the audio data to identify at least one characteristic of the speech of the telephone caller; comparing, by the processor, the at least one characteristic to a plurality of word clusters, each word cluster comprising at least one associated characteristic, and each word cluster being associated with one of a plurality of demographics; determining, by the processor, a most similar word cluster of the plurality of word clusters to the audio data based on a similarity of the at least one characteristic of the speech of the telephone caller and the at least one associated characteristic of the most similar cluster; receiving, by the processor, a purported identity of the telephone caller, the purported identity including caller demographic data; comparing, by the processor, the caller demographic data to the demographic associated with the most similar word cluster; and identifying, by the processor, the telephone caller as at least one of: likely having the purported identity in response to determining the caller demographic data matches the demographic associated with the most similar word cluster, and unlikely to have the purported identity in response to determining the caller demographic data matches a demographic associated with a word cluster different from the most similar word cluster.