Nature of critical points of $f(x,y,z)$ How can I study the nature of the critical points of $f(x,y,z)=3xy^2+6y^2+x^2z+3z^2$?
Since
$$
\begin{cases}
f'_x=3y^2+2xz\\
f'_y=6xy+12y\\
f'_z=x^2+6z
\end{cases}
$$
the only critical point is $(0,0,0)$.
The Hessian matrix is
$$
H_f(0,0,0)=\left(
\begin{matrix}
0 & 0 & 0\\
0 & 12 & 0\\
0 & 0 & 6
\end{matrix}\right),
$$
whose determinant is zero. Now I'm stuck, because I've tried to study $f$ along many restriction, and every time I realized that $O$ is a minimum, but I am not able to prove (or disprove) that $O$ is indeed a minimum.
What can I do now?

Let $f(x,y,z)=3xy^2+6y^2+x^2z+3z^2$ then $f\in C^{2}({\bf R})$. For $(x,y,z)\in {\bf R}^{3}$ then $\nabla f(x,y,z)=(0,0,0)\iff \begin{cases}2xz+3y^{2}=0,\\6xy+12y=0,\\x^{2}+6z=0 \end{cases}$. Thus ${\bf 0}:=(0,0,0)$ is indeed the only critical point. The Hessian matrix $H_f$ at ${\bf 0}$ is given by $H_f({\bf 0})=\left.\begin{pmatrix} 2z&6y&2x\\6y&6x+12&0\\2x&0&6\end{pmatrix}\right|_{{\bf 0}}=\begin{pmatrix}0&0&0\\0&12&0\\0&0&6\end{pmatrix}$, then $\det(H_f({\bf 0}))=0$ and the Hessian-test is inconclusive.
Thus, we need to review in the definition directely, considering that $f({\bf 0})=0$.

*

*If $x=y=0$, then $3z^{2}=f(x,y,z)>f({\bf 0})=0$ if $z\not=0$, so $f({\bf 0})$ is not local maximum.

*If $z=-x^{3}$ and $y=0$, then $x^{5}(3x-1)=f(x,y,z)<f({\bf 0})=0$ if $0<x<\frac{1}{3}$, so $f({\bf 0})$ is not a local mimimum.

Recall $f$ has a local minimum at $p$ iff there exists $B_\delta(a)\cap {\rm dom}(f)$ such that $f(x)\geqslant f(p)$ over that ball. Therefore, $f({\bf 0})=0$ can not be a extreme value of $f$, then it is a saddle point.
