Surface integral first kind I have to calculate surface integral:$$\iint_D zdD$$ where $S$ is part of cone $z=\sqrt{x^2+y^2}, 1<z<2$ cut out by $(x^2+y^2)^2=2(x^2-y^2)$. I am only not sure how to use part that $1<z<2$. I thought to plug this in the formula and get $\iint_S \sqrt{x^2+y^2}\sqrt{1+(\frac{x}{x^2+y^2})^2+(\frac{y}{x^2+y^2})^2} dS$. And then using given cylinder, and converting to polar coordinates I get $-\frac{\pi}{4}\le \phi\le \frac{\pi}{4}, 0\le r\le 2\sqrt{\cos(2\phi)}$ it shouldn't be hard to calculate this integral. Is this correct or there is something I'm missing?

While most of what you have mentioned is correct, you have not considered that $z \geq 1$. The limits of $\theta$ will also be different. $-\frac{\pi}{4} \leq \theta \leq \frac{\pi}{4}$ is valid only when $z$ starts from $0$. Also just to notice, there is no surface area of the cone cut out by the cylinder above $z = \sqrt2$.
Cone surface is given by $z = \rho$; Cylinder is given by $\rho = \sqrt{2\cos2\theta}$
Maximum radius of the cylinder's cross section is $\sqrt2 \ $ at $\theta = 0$ so for $z \gt \sqrt2$, the cylinder is completely inside the cone.
For a cone with $z=r, \ |r'_{\rho} \times r'_{\theta}| = \rho\sqrt2$.
At $z = 1, \rho = 1 = \sqrt{2 \cos2\theta} \implies -\frac{\pi}{6} \leq \theta \leq \frac{\pi}{6}$. As $z$ increases, $\rho$ increases and range of $\theta$ that cuts out the surface area of the cone reduces. So we cannot have $\theta \gt \frac{\pi}{6}$.
Also note that there is one more petal between $\frac{3\pi}{4} \leq \theta \leq \frac{5\pi}{4}$. So we need to multiply integral by $2$.
Substituting $z = \rho$ and multiplying by factor $\rho \sqrt2$, the integral becomes,
$S = 2 \sqrt2 \displaystyle \int_{-\pi/6}^{\pi/6} \int_{1}^{\sqrt{2\cos (2\theta)}} \rho^2 \  d\rho \  d\theta$
Alternatively,
$S = 2\sqrt2 \displaystyle \int_1^\sqrt2 \int_{-\frac{1}{2} \arccos\big(\frac{\rho^2}{2}\big)}^{\frac{1}{2} \arccos\big(\frac{\rho^2}{2}\big)} \rho^2 \ d\theta \  d\rho$
