Find all orthogonal $3\times 3$ matrices of the form... Find all orthogonal $3\times 3$ matrices of the form
\begin{bmatrix}a&b&0\\c&d&1\\e&f&0\end{bmatrix}

Using the fact that $A^TA$ = $I_n$, I set that all up and ended up with the following system of equations:
$$\left\{\begin{array}{l}a^2 + e^2 = 1\\
ab + ef = 0\\
b^2 + f^2 = 1\end{array}\right.$$
I know I can let things equal the sine and cosine of theta, but I'm not exactly sure how to write this answer down on paper. There has to be tons of possibilities, right? How many exactly?

You have 
\begin{align}
a^2+e^2=b^2+f^2&=1\\
c=d=ab+ef&=0.
\end{align}
The first equation represents the lengths of the (unit) vectors $\pmatrix{a\\e}$ and $\pmatrix{b\\f}$ and the second equation represents the scalar product of these vectors, showing that they are perpendicular. 
Wihtout loss of generality, let $\pmatrix{a\\e}=\pmatrix{\cos\theta\\\sin\theta}$ for some $\theta\in\mathbb{R}.$ Then $\pmatrix{b\\f}$ can be either $\pmatrix{-\sin\theta\\\cos\theta}$ or $\pmatrix{\sin\theta\\-\cos\theta}$ (draw it if you do not see why this is). 
This means that the matrix you seek is either of the two matrices
$$\pmatrix{\cos\theta & \mp\sin\theta & 0 \\ 0 & 0 & 1 \\ \sin\theta  &\pm\cos\theta & 0}.$$
