Optimization—Finding the Area of the Largest Isoceles Triangle 
I managed to solve $(a)$. Since the area of a triangle is determined by $\frac{1}{2}$ base $\times$ height, and we already know the height, we just have to solve for the base. Using Pythagorean theorem, we can deduce base as $\sqrt{36-h^2}$. Therefore, the area is $(6+h)(\sqrt{36-h^2})$. To find $h$, we will take the derivative of the area equation and set that equal to zero. It turns out $h = 3$. Plugging $h$ into the area equation gives us an area of $27\sqrt{3}$.
Can you guys help me do $(b)$? I believe for $(c)$, the largest type of triangle of maximum area is an equilateral.
Thanks.

Part b looks to be the tough part and c would likely quickly follow from it.  It is possible, however, to verify your guess from what you've already done.  One of the legs of that lower right triangle is $h$ which is equal to $3$.  The hypoteneuse has length $6$.  Therefore, the angle in between is $60$ degrees.  The supplementary angle is then $120$ and the other $2$ angles of the isosceles triangle on the right are $30$ each.  That makes the angles of the large triangle $60$ degrees each.
The only thing I can think of for part b is to cut the larger triangle into several smaller ones. If you draw an altitude from the center of the circle to the base of the isosceles triangle right of center, you can calculate the lengths of the base and height through trigonometric formulae given $\alpha$ and the length of the radii. So the isosceles portion at the right has area of
$$\frac12bh=\frac12(6\sin\alpha)(12\cos\alpha)=36\sin\alpha\cos\alpha$$
That just leaves the area of the lower right triangle.  Once that's calculated, double everything to account for the congruent left half of the big triangle.  For this, either use the inscribed angle theorem (had to look up the name) or just note that the isosceles triangle on the right has angles of $\alpha,\alpha$ and $180-2\alpha$, making the supplementary angle $2\alpha$.
