Flux through cone and hyperboloid I want to find the flux of the vector field
$$F(x,y,z)=(x+z,2x+2y,3y+3z)$$
through the body defined by

$$x^2/16+y^2/4-z^2/3\le 1 \quad\mbox{with} \quad 0\le z \lt 3$$
$$x^2/16+y^2/4-(z-7)^2/4\le 0 \quad \mbox{with}\quad 3\le z \lt 7.$$

I tried to apply the divergence theorem to calculate the flux throw the cone but the integral becomes too difficult because of the limits.
As you already found for this vector field, $ \ \nabla  \cdot  \mathbf{F} \ = \ 6 \ $ , and it is entirely understandable that one would wish to find the flux through this compound surface by applying the Divergence Theorem.



The graph above (exposing the interior of the enclosed volume) presents the situation:  a hyperboloid of one sheet, with its "throat" aligned on the $ z-$ axis and having an elliptical cross-section, meets the lower nappe of a cone of elliptical cross-section, also with its symmetry axis on the $ \ z-$ axis, on the plane $ \ z = 3 \ $ ; there, the two surfaces meet in the same ellipse.

All of the "horizontal" cross-sections of these surfaces are similar ellipses, as we will demonstrate shortly.  Because of the axial alignment of the surfaces, we can reduce the volume calculation to integration in the single variable $ \ z \ $ .  It will be helpful to write the equations of the surfaces as

$$ \frac{x^2}{16} \ + \ \frac{y^2}{4} \ = \ 1 \ + \frac{z^2}{3} \ = \ \sigma_h^2(z) \ \ \ \text{and} $$

$$ \frac{x^2}{16} \ + \ \frac{y^2}{4} \ = \ \frac{(z-7)^2}{4} \ = \ \sigma_c^2(z)\ \ , $$

where we will call the expressions on the right-hand sides of the equations "scaling factors" for the hyperboloid and the cone.  If we then write the surface equations as

$$ \frac{x^2}{16\sigma_h^2} \ + \ \frac{y^2}{4\sigma_h^2} \ = \ 1  \ \ \text{and} \ \  \frac{x^2}{16\sigma_c^2} \ + \ \frac{y^2}{4\sigma_c^2} \ = \ 1 , $$

we see that the cross-sections are ellipses centered on the $ \ z-$ axis, with "major axes" parallel to the $ \ x-$ axis of lengths $ \ 4 \ \sigma_h \ $ and $ \ 4 \ \sigma_c \ $ , and "minor axes" parallel to the $ \ y-$ axis of lengths $ \ 2 \ \sigma_h \ $ and $ \ 2 \ \sigma_c \ $ , respectively.  The infinitesimal volume "slices" along the $ \ z-$ axis are thus $ \ 8 \ \pi \ \sigma_h^2 \ $ and 
$ \ 8 \ \pi \ \sigma_c^2 \ $ . [We can easily check that the ellipses "match up" at $ \ z = 3 \ $ , where $ \ \sigma_h^2 \ = \ \sigma_c^2 \ = \ 4 \ $ . ]

We can now write the integral for the volume enclosed by this compound surface as

$$ 8  \pi \ \ \left[ \ \int_0^3  \sigma_h^2(z) \ \ dz \ \ + \ \ \int_3^7  \sigma_c^2(z) \ \ dz \ \right] $$

$$ = \ \ 8  \pi \ \ \left[ \ \int_0^3  \left( 1 \ + \ \frac{z^2}{3}  \right) \ \ dz \ \ + \ \ \int_3^7  \frac{(z-7)^2}{4} \ \ dz \ \right] $$

$$ = \ \ 8  \pi \ \ \left[ \   \left( z \ +  \frac{z^3}{9}  \right) \ \vert_0^3  \ \ + \ \  \left( \frac{(z-7)^3}{12} \right)  \ \vert_3^7  \ \right] $$

$$ = \ \ 8  \pi \ \ \left[ \   \left( 3 \ +  \frac{3^3}{9}  \right)   \ \ - \ \  \left( \frac{(3-7)^3}{12} \right)  \ \right] \ = \ 8  \pi \  ( \ 3 \ + \ 3      \ + \  \frac{64}{12} \ ) \ = \ \frac{272 \pi}{3} \ \ . $$

Hence, by the Divergence Theorem, 

$$ \iint_S \ \mathbf{F} \cdot {n} \ \ dS \ \ = \ \ 6 \ \iiint_V \ \nabla \cdot \mathbf{F} \ \ dV \  =  \ 6 \ \cdot \ \frac{272 \pi}{3} \ = \ 544 \pi \ \ . $$
Hint: After converting it into a triple integral via divergence theorem, apply a "polar" change of coordinates of the form $x = 4r \cos{\theta}, y = 2 r \sin{\theta}$. You want to use this change of coordinates because the cross-sections of the given surface are ellipses. Do not forget that you'll also have to calculate the Jacobian for this change of coordinates.