Q:

Show that given function is identically zero

Let $D \subset \mathbb{R}^{2}$ be and open and bounded set and $u \in C^{2}(D)\cup C^{0}(\overline{D})$ be a solution of $$ -\bigtriangleup u + u^{3} + uu_{x}^{3} + u_{y}^{2} = 0 $$ in D and $$u \equiv 0$$ in $\partial D$
Prove that u is identically 0 in $\overline{D}$.
As a first attempt I used the divergence theorem: $$\int_{D}\bigtriangleup u =\int_{\partial D}<\bigtriangledown u, n>$$ where n is the normal vector. From this we have that $$ \int_{D} u^{3} + uu_{x}^{3} + u_{y}^{2} = 0 $$ because if $u(x) = 0,  \forall x \in \partial D$ then $\bigtriangledown u$ must be $0$. 
I have absolutely no clue how to proceed from here with this question. Does anyone have any clues?

A:

To solve this explore the consequences of having a critical point inside the domain. The following observations are all you need:

If $u\not\equiv 0$ then there has to be a critical point inside the domain where $\nabla u = 0$. At this point the PDE reads $\Delta u = u^3$.
At a local maximum (minimum) we have $\Delta u \leq 0$ ($\Delta u \geq 0$).

This kind of argument is similar to what is often used to prove maximum principles for certain PDEs.

Another method that often works with PDEs involving the Laplacian is to multiply by $u$ and integrate over $D$ using integration by parts (instead of just integrating the PDE directly as you tried). This does not work on this PDE, however if the PDE was slightly different it would. For example if $-\Delta u + u^3 + uu_x^2 + u u_y^2 = 0$ then this would lead to
$$\int_D (\nabla u)^2 + u^4 + (uu_x)^2 + (uu_y)^2{\rm d}x = 0$$
And since all the terms in the integrand is positive they each have to be identical to zero which gives you $u\equiv 0$. As I said this does not work here, but it's a very useful method to know about.

