Residual sum of squares

In statistics, the residual sum of squares (RSS), also known as the sum of squared residuals (SSR) or the sum of squared estimate of errors (SSE), is the sum of the squares of residuals (deviations predicted from actual empirical values of data). It is a measure of the discrepancy between the data and an estimation model, such as a linear regression. A small RSS indicates a tight fit of the model to the data. It is used as an optimality criterion in parameter selection and model selection.
In general, total sum of squares = explained sum of squares + residual sum of squares. For a proof of this in the multivariate ordinary least squares (OLS) case, see partitioning in the general OLS model.
One explanatory variable.
In a model with a single explanatory variable, RSS is given by:
where "y""i" is the "i"th value of the variable to be predicted, "x""i" is the "i"th value of the explanatory variable, and formula_2 is the predicted value of "y""i" (also termed formula_3).
In a standard linear simple regression model, formula_4, where formula_5 and formula_6 are coefficients, "y" and "x" are the regressand and the regressor, respectively, and ε is the error term. The sum of squares of residuals is the sum of squares of formula_7; that is
where formula_9 is the estimated value of the constant term formula_5 and formula_11 is the estimated value of the slope coefficient formula_6.
Matrix expression for the OLS residual sum of squares.
The general regression model with observations and explanators, the first of which is a constant unit vector whose coefficient is the regression intercept, is
where is an "n" × 1 vector of dependent variable observations, each column of the "n" × "k" matrix is a vector of observations on one of the "k" explanators, formula_14 is a "k" × 1 vector of true coefficients, and is an "n"× 1 vector of the true underlying errors. The ordinary least squares estimator for formula_6 is
The residual vector formula_19 = formula_20; so the residual sum of squares is:
(equivalent to the square of the norm of residuals). In full:
where is the hat matrix, or the projection matrix in linear regression.
Relation with Pearson's product-moment correlation.
The least-squares regression line is given by
where formula_24 and formula_25, where formula_26 and formula_27
Therefore,
where formula_29
The Pearson product-moment correlation is given by formula_30 therefore, formula_31