Green's Theorem in polar coordinates - what am I doing wrong? Let $$\vec F = M\hat i + N\hat j = (3x^2y^2)\hat i + (2x^2 + 2x^3y)\hat j$$
be a vector field, and C be the counterclockwise circle centered at $(a,0)$, with a radius of $a$. Find $\oint_C M\,dx + N\,dy$.
Applying Green's Theorem: 
$$\oint_C M\,dx + N\,dy = \iint_R N_x-M_y\,dA = 4\iint_R x\,dx$$
Using center of mass with uniform density in place of direct computation:
$$4\iint_R x\,dx=4\frac{\iint_R x\,dx}{\iint_R\,dx}\iint_R\,dx = 4 \bar x *Area(R) = 4\pi a^3$$
The problem I'm having is when I try to do the double integral by direct computation in polar coordinates. I've checked the intermediate steps with an integral calculator to make sure there wasn't a simple calculation error, which there wasn't, so I'm guessing something is wrong with the setup.
The first thing I did was get an expression for $x$ and $y$ in terms of $\theta$. I did this by noting that, for any ray coming from the origin, it will intersect the circle and form an isosceles triangle, where the first leg of length $a$ is coincidental with the $x$ axis, and the second leg, call it $l$, is the line segment of length $a$ going through the circle's origin and the point of intersection.
From this, the angle between the two legs should be $\pi-2\theta$, since there are two equivalent $\theta$s, and the sum of the angles in a triangle is $\pi$ radians.
The angle $\alpha$ between $l$ and the $x$ axis is $\pi - (\pi - 2\theta)$, since the angle between the two legs of the isosceles triangle plus the angle between $l$ and the $x$ axis should be $\pi$. This makes it easy to get an expression for $x$ and $y$ using vectors. If the circle were at the origin, then we would have $\vec r = (a\,cos(\alpha),a\,sin(\alpha)=(a\,cos(2\theta), a\,sin(2\theta)$, to which we add the offset vector $(a,0)$ to obtain $\vec r = (a(1+cos(2\theta), a\,sin(2\theta)$.
I did this another way to confirm, by using the law of cosines, and it seems correct from that standpoint as well. If the angle between the two sides of the isosceles triangle is $\pi - 2\theta$, then $r$ should be $\sqrt{2a^2(1-cos(\pi - 2\theta)}$, or equivalently, $\sqrt{2a^2(1+cos(2\theta))}$, since $cos(\pi-x)=-cos(x)$. Then, the unit vector extending in the direction of the ray from the origin is $\hat u = (cos(\theta), sin(\theta))$, and the ray should be $r \hat u$.
I graphed both of these representations for the $x$ coordinate, namely, $x=a(1+cos(2\theta))$ and $x=cos(\theta) \sqrt{2a^2(1+cos(2\theta))}$, and they are equivalent on $-\frac{\pi}{2} \le x \le \frac{\pi}{2}$.
Finally, I set up the integral:
$$\int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} \int_0^{\sqrt{2a^2(1+cos(2\theta))}}r(4a(1+cos(2\theta)))\,dr\,d\theta$$
Evaluating the inner integral leaves:
$$\int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} 4a^3(1+cos(2\theta))^2\,d\theta = 6 \pi a^3$$
Interestingly, if I evaluate this integral without the square, i.e.,
$$\int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} 4a^3(1+cos(2\theta))\,d\theta$$
then I get the correct answer, although I have no clue at this point how I would get such an expression.
Any insight would be much appreciated.

Here is a method that produces a value of $4\pi a^3$. 
The set of points in $R$ is given by 
$$ \{ \theta \in [-\frac{\pi}{2}, \frac{\pi}{2}] , r \in [0,2a\cos(\theta)] \} .$$
In polar coordinates, $x = r\cos(\theta)$. So, the integral becomes 
$$ \int_{-\pi/2}^{\pi/2} \int_{0}^{2a\cos(\theta)} 4(r\cos(\theta))r\ dr\ d\theta .$$
Evaluating the inner integral, this is equal to
$$ \frac{32a^3}{3} \int_{-\pi/2}^{\pi/2} \cos^4(\theta)\ d\theta $$
Since $\int_{-\pi/2}^{\pi/2} \cos^4(\theta)\ d\theta$, the result is $4\pi a^3$ as expected. 

Unfortunately, I can't figure out why your method doesn't work. Your integral
$$ \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} \int_0^{\sqrt{2a^2(1+cos(2\theta))}}r(4a(1+cos(2\theta)))\,dr\,d\theta $$
should be equivalent to my integral because:


*

*$\sqrt{2a^2(1+cos(2\theta))} = \sqrt{4a^2 \cos(\theta)^2} = 2a\cos(\theta)$ on the interval $\theta \in [-\pi/2,\pi/2]$. 

*$a (1+\cos(2\theta)) = 2a \cos(\theta)^2 = r\cos(\theta)$. 


The problem seems to be the manipulation in (2). Running (your integral)
Integrate[4 r (a (1 + Cos[2 t])), {t, -Pi/2, Pi/2}, {r, 0, 2 a Cos[t]}]

in Mathematica returns $6a^3 \pi$. Running (my integral)
Integrate[4 r^2 Cos[t], {t, -Pi/2, Pi/2}, {r, 0, 2 a Cos[t]}]

in Mathematica returns $4a^3 \pi$. The limits of integration are the same, but they produce two different values.
Maybe somebody else can figure out why they produce two different results.

Update: The issue is that we cannot assume $r = 2a \cos(\theta)$. This is because $2a \cos(\theta)$ is the maximum value $r$ obtains, but $r$ varies over the region $R$. Specifically, $r \in [0, 2a\cos(\theta)$. 
It's like trying to compute the area of a circle of radius $a$, but instead of writing
$$ \int_0^{2\pi} \int_0^a r\ dr\ d\theta = \pi a^2$$
you write 
$$ \int_0^{2\pi} \int_0^a a\ dr\ d\theta = 2\pi a^2$$
and get an answer that is too big. 
