1. A method for adapting acoustic models in a speech recognition system, the method comprising: estimating noise in a portion of a speech signal; determining a first estimated variance scaling vector using an estimated 2-order polynomial and the noise estimation, wherein the estimated 2-order polynomial represents a priori knowledge of a dependency of a variance scaling vector on noise; determining a second estimated variance scaling vector using statistics from prior portions of the speech signal; determining a variance scaling factor using the first estimated variance scaling vector and the second estimated variance scaling vector; and using the variance scaling factor to adapt an acoustic model.