Q:

Using least squares with an incomplete quadratic basis

I am currently using least squares (LS) to compute gradients/derivatives. I am using LS with an incomplete quadratic basis:
$$\phi = \begin{bmatrix}
1 & x & y & z & 0.5x^2 & 0.5y^2 & 0.5z^2
\end{bmatrix}
$$
rather than the full basis:
$$\phi = \begin{bmatrix}
1 & x & y & z & xy & xz & yz & 0.5x^2 & 0.5y^2 & 0.5z^2
\end{bmatrix}
$$
What are the main disadvantages in using an incomplete quadratic basis over the full quadratic basis? 

A:

The isosurfaces associated to the incomplete basis are spheres, while the complete basis yields quadrics.
Hence the former are more appropriate for isotropic phenomena, with no coupling between the variables.
Notice that the complete basis has more degrees of freedom and will always achieve smaller residual error, possibly giving you a false feeling of a better fit (which can be overfitting).

