Least-squares function approximation

In mathematics, least squares function approximation applies the principle of least squares to function approximation, by means of a weighted sum of other functions. The best approximation can be defined as that which minimizes the difference between the original function and the approximation; for a least-squares approach the quality of the approximation is measured in terms of the squared differences between the two.
Functional analysis.
A generalization to approximation of a data set is the approximation of a function by a sum of other functions, usually an orthogonal set: 
with the set of functions {formula_2} an orthonormal set over the interval of interest, : see also Fejér's theorem. The coefficients {formula_3} are selected to make the magnitude of the difference ||||2 as small as possible. For example, the magnitude, or norm, of a function over the can be defined by:
where the ‘*’ denotes complex conjugate in the case of complex functions. The extension of Pythagoras' theorem in this manner leads to function spaces and the notion of Lebesgue measure, an idea of “space” more general than the original basis of Euclidean geometry. The satisfy orthonormality relations:
where "δ""ij" is the Kronecker delta. Substituting function into these equations then leads to
the "n"-dimensional Pythagorean theorem:
The coefficients {"a""j"} making ||"f" − "f""n"||2 as small as possible are found to be:
The generalization of the "n"-dimensional Pythagorean theorem to "infinite-dimensional " real inner product spaces is known as Parseval's identity or Parseval's equation. Particular examples of such a representation of a function are the Fourier series and the generalized Fourier series.
Further discussion.
Using linear algebra.
It follows that one can find a "best" approximation of another function by minimizing the area between two functions, a continuous function formula_8 on formula_9 and a function formula_10 where formula_11 is a subspace of formula_12:
all within the subspace formula_11. Due to the frequent difficulty of evaluating integrands involving absolute value, one can instead define
as an adequate criterion for obtaining the least squares approximation, function formula_16, of formula_8 with respect to the inner product space formula_11.
As such, formula_19 or, equivalently, formula_20, can thus be written in vector form:
In other words, the least squares approximation of formula_8 is the function formula_23 closest to formula_8 in terms of the inner product formula_25. Furthermore, this can be applied with a theorem: