Determine whether origin is a local extrema of a given function Consider the function $f(x,y)=x^6-2x^2y-x^4y+2y^2$ defined from $\mathbb{R}^2$ to $\mathbb{R}$.
Then $(0,0)$ is a critical point.
Also $f_{xx}$ at origin is zero, $f_{yy}$ at origin is $4$ and $f_{xy}$ at origin is $0.$ The value of the determinant is zero. So it cannot be concluded whether the origin is local minimum or maximum or saddle point.
How can we determine for this particular problem whether the origin is local maximum or local minimum or a saddle point?
Thanks in advance.

$f(x,\lambda x^2)=x^4(2\lambda(\lambda-1) + (1- \lambda)x^2)$ for all $x \in \mathbb R$.
Therefore $$f(x, \frac{1}{2}x) = x^4(-\frac{1}{2} + \epsilon(x)x^2)$$ is negative in a neighborhood of $(0,0)$. While
$$f(x,2x) = x^4 (4 - \delta(x)x^2)$$ is positive in a neighborhood of $(0,0)$.
You have a saddle point.
