Linear Least Squares with Non Negativity Constraint I am interested in the linear least squares problem:
$$\min_x \|Ax-b\|^2$$
Without constraint, the problem can be directly solved. With an additional linear equality constraint, the problem can be directly solved too, thanks to a Lagrange multiplier.
However, I could not find a direct way to solve this problem with a linear inequality constraint. The problem belongs to quadratic programming, and the methods mentioned on Wikipedia involve an iterative procedure. I also know about Karush–Kuhn–Tucker conditions, but I cannot deal with them in this particular problem since the primal and dual feasability conditions, and the complementary slackness conditions, are numerous and not helpful in an abstract setting.

Let us assume the linear inequality constraint is indeed a simple enforcement of non-negativity:
$$x\geq0$$
Is there a direct method which could be directly applied to this simpler case? The only hope I could find so far lies in the method explained in a technical report by Gene H. Golub and Michael A. Saunders, released in May 1969, called Linear Least Squares and Quadratic Programming, and which was linked in another question.

Solving regular Least Squares problem using Gradient Descent is easy.
How will it be different with the constraint applied?
Well, you will have to project the solution of each iteration onto the set of the constraints.
Here is the code:
%% Solution by Projected Gradient Descent

vX = zeros([numCols, 1]);

for ii = 1:numIterations
    vX = vX - ((stepSize / sqrt(ii)) * mA.' * (mA * vX - vB));
    vX = max(vX, 0);
end

The full code and validation against CVX is in my Mathematics Q561696 GitHub Repository.
Fast Algorithm for the Solution of Large Scale Non Negativity Constrained Least Squares Problems
There is a paper called Fast Algorithm for the Solution of Large Scale Non Negativity Constrained Least Squares Problems. The ideas there might be used here. The MATLAB Code is given as well - FCNNLS - Fast Combinatorial Non Negative Least Squares.
