Deviance (statistics)

In statistics, deviance is a goodness-of-fit statistic for a statistical model; it is often used for statistical hypothesis testing. It is a generalization of the idea of using the sum of squares of residuals (RSS) in ordinary least squares to cases where model-fitting is achieved by maximum likelihood. It plays an important role in exponential dispersion models and generalized linear models.
Definition.
The unit deviance formula_1 is a bivariate function that satisfies the following conditions:
The total deviance formula_4 of a model with predictions formula_5 of the observation formula_6 is the sum of its unit deviances: formula_7.
The (total) deviance for a model "M"0 with estimates formula_8, based on a dataset "y", may be constructed by its likelihood as:
Here formula_10 denotes the fitted values of the parameters in the model "M"0, while formula_11 denotes the fitted parameters for the "saturated model": both sets of fitted values are implicitly functions of the observations "y". Here, the saturated model is a model with a parameter for every observation so that the data are fitted exactly. This expression is simply 2 times the log-likelihood ratio of the full model compared to the reduced model. The deviance is used to compare two models – in particular in the case of generalized linear models (GLM) where it has a similar role to residual variance from ANOVA in linear models (RSS).
Suppose in the framework of the GLM, we have two nested models, "M1" and "M2". In particular, suppose that "M1" contains the parameters in "M2", and "k" additional parameters. Then, under the null hypothesis that "M2" is the true model, the difference between the deviances for the two models follows, based on Wilks' theorem, an approximate chi-squared distribution with "k"-degrees of freedom. This can be used for hypothesis testing on the deviance.
Some usage of the term "deviance" can be confusing. According to Collett:
Examples.
The unit deviance for the Poisson distribution is formula_14, the unit deviance for the Normal distribution is given by formula_15.