Give a formula for the volume of the solid under a surface $z=xy$ and a triangle? Give a formula for the volume of the solid with unit density lying under the surface $z = xy$ and above the triangle in the $xy$-plane with vertices $(0, 1, 0)$, $(1, 1, 0)$ and $(0, 2, 0)$.


What is the correct way to do this?

Attempt:

The volume $V$ lying beneath the surface $z = f(x,y)$ and above the region

$a\leq x\leq b,g_{1}(x)\leq y\leq g_{2}(x)$

$V=\iint_{D}f(x,y)dA=\int_{a}^{b}\int_{g_1(x)}^{g_2(x)}f(x,y)dydx$

In the triangle with vertices $(0, 1, 0)$, $(1, 1, 0)$ and $(0, 2, 0)$, $x$ lies between

$0$ and $1$

The $y$ component could lie between $y = 1$ and the line joining the points $(1, 1)$ and $(0,2)$. And it could have the equation $y = -x + 2$.

$D= \left \{ (x,y):0\leq x\leq 1,1\leq y\leq -x+2 \right \}$

$V=\iint_{D} (xy)dA=\int_{0}^{1}\int_{1}^{ -x+2}xydydx$
Here's another way to get answer to this question:


Rename the vertices: $A_1 = (0,1,0), A_2 = (1,1,0), A_3 = (0,2,0)$.
Find area $$A_x = [\min{(A_{1x},A_{2x},A_{3x})} .. \max{(A_{1x},A_{2x},A_{3x})}] = [\min(0,1,0) .. \max(0,1,0)] = [0..1]$$
$$A_y = [\min{(A_{1y},A_{2y},A_{3y})} .. \max{(A_{1y},A_{2y},A_{3y})}] = [\min(1,1,2) .. \max(1,1,2)] = [1..2]$$

$A=[0..1]\times[1..2]$. (bounding rectangle)
Find subset of A which is inside the triangle, $K \subset A$: 
$$ K = \{ (x,y) | isInsideTriangle(x,y) \wedge (x,y) \in A \}$$, where isInsideTriangle can be found from url https://stackoverflow.com/questions/2049582/how-to-determine-if-a-point-is-in-a-2d-triangle
Note that $z=xy$ can be written in form $z-xy=0$ which is of form $f(x,y,z)=0$.
Use implicit function and heightmap substitution $z=f(x,y)$: $$f(x,y,z) = f(x,y,f(x,y)) = 0$$
$$ f(x,y)-xy = 0$$, $$ f(x,y) = xy $$
Calculate area of the triangle $$ A_{tri} = \frac{1}{2} A_x \cdot A_y = \frac{1}{2} 1 \cdot 1 = \frac{1}{2} $$
Calculate average of f(x,y) by random sampling, utilizing the area $K$ and a sum $av_f(K) = \frac{\sum_{i=1}^{k} f(x_i,y_i)}{k}$, where $f(x_i,y_i)$ is random sample taken from $(x_i,y_i) \in K, i \in [1..k], k&gt;10000$.
Note that $\int_K{f(x,y)dK} = A_{tri} \times av_f(K) $ where $av_f(K)$ is the average of $f(x,y)$ in area $K$.
What you did looks good providing some light modifications:


The volume is a 3d integral. So I would start saying that your aim is to compute the volume V of the domain $\mathcal{V}=\{(x,y,z) \in \mathbb{R}^3 \ | 0\leq x\leq 1,1\leq y\leq -x+2, 0 \le z \le xy \}$.
Then $V= \int \int \int_\mathcal{V} dx dy dz=\int \int_Dxy dx dy$.
Then you have a small issue of using $x$ as both a bound of integration and a variable in the function to be integrated which is confusing.