1. An acoustic model learning device comprising: a first variation model learning unit that estimates a parameter defining a first variation model indicating a variation in a speech for each type of a first environment factor by using a plurality of sample speech data acquired for each combination of one of a plurality of types of the first environment factor and one of plurality of types of a second environment factor, the first environment factor being one of a plurality of environment factors that change and thereby cause a variation in a speech, and the second environment factor being another of the plurality of environment factors; a second variation model learning unit that, using the plurality of sample speech data, with respect to each type of the second environment factor, estimates a parameter defining a second variation model indicating a variation in a speech; and an environment-independent acoustic model learning unit that, using the plurality of sample speech data, estimates a parameter defining an environment-independent acoustic model not specified as any type of the first environment factor and the second environment factor, wherein each of the learning units estimates each parameter in such a way that an integrated degree of fitness obtained by integrating a degree of fitness of the first variation model to the sample speech data, a degree of fitness of the second variation model to the sample speech data, and a degree of fitness of the environment-independent acoustic model to the sample speech data becomes the maximum, wherein the first variation model and the second variation model are each defined by a two-stage affine transformation.