How many ways can the vertices of an equilateral triangle be colored using three different colors? Its not $3^3$ because some of the colorings are equivalent. How would I apply Burnside's theorem to this?

In this case it suffices to select the number of vertices of each color (since all vertices are connected amongst themselves). Use stars and bars
Look at the three vertices as stars and we need to bars to separate the stars, thus the answer is $\binom{5}{2}=10$

Edit: I assume two triangles are the same if you can take the two triangles and move them in $3d$ space so that they look the same. In other words this is when the group acting on the triangle is $D_3$. The answer when the triangle can only be moved in $2d$ space is different, since one triangle becomes two different triangles (namely $RGB$ splits into $RGB$ and $RBG$. So in this case the answer is $11$. This corresponds to when the group that is acting is $Z_3$.

Also notice problems of this type are usually solved via Burnside’s lemma, you can look up the same thing for the faces of cubes and there the lemma actually is useful. In this case however it is not useful because it is very simple.
