Constrained least squares

In constrained least squares one solves a linear least squares problem with an additional constraint on the solution. 
I.e., the unconstrained equation formula_1 must be fit as closely as possible (in the least squares sense) while ensuring that some other property of formula_2 is maintained.
There are often special-purpose algorithms for solving such problems efficiently. Some examples of constraints are given below:
When the constraint only applies to some of the variables, the mixed problem may be solved using separable least squares by letting formula_14 and formula_15 represent the unconstrained (1) and constrained (2) components. Then substituting the least-squares solution for formula_16, i.e.
(where + indicates the Moore–Penrose pseudoinverse) back into the original expression gives (following some rearrangement) an equation that can be solved as a purely constrained problem in formula_18.
where formula_20 is a projection matrix. Following the constrained estimation of formula_21 the vector formula_22 is obtained from the expression above.