Combination of $k$ vertices from the $n$ vertices of $n$-sided regular polygon Select $k$ vertices from the $n$ vertices of an $n$-sided regular polygon. For two of this kind of $k$-combination $A$ and $B$, they are treated as an identical one if $A$ can transform to $B$ by rotating around the center of the polygon. What is the expression of total number of this $k$-combination for arbitrary $k$ and $n$?
Alternatively, two $k$-combination $A$ and $B$ are treated as an identical one if $A$ can transform to $B$ by joint operation of rotating around the center of the polygon and reflecting about an axis passing through the center. What is the total number of $k$-combination in this case?

This is an application of Burnside Lemma, you take $X:=\mathcal{P}_k(V_0,...,V_{n-1})$ ($V_0,...,V_{n-1}$ are the vertices of your polygon). Then $\mathbb{D}_n$ acts on the vertices of you regular polygon and hence on the set $X$. Suppose you take a subgroup $G$ of $\mathbb{D}_n$ then you want to know the cardinal of $X/G$ (the elements of $X$ modulo identification with respect to the action of $G$). Burnside's lemma will then be :
$$|X/G|=\frac{1}{|G|}\sum_{g\in G}|Fix(g)| $$
Take $G=\mathbb{Z}_n$ the subgroup of rotations then it is generated by $R$ rotation of angle $\frac{2\pi}{n}$. If $A\in X$ is fixed by $R^p$ for all $V_i\in R^p$ :
$$R^pV_i=V_{p+i\text{ mod } n} $$
Then one sees that if $l:=\frac{n}{gcd(p,n)}$ is the order of $p$ mod $n$ then $<R^p>$-action on $A$ will split $A$ in $\frac{k}{l}$ orbits of size $l$. 
If $n/gcd(p,n)$ does not divide $k$ then $Fix(R^p)=0$.
If $n/gcd(p,n)$ divides $k$ then any fixed $k$-combination is uniquely determined by a choice of a $k\times gcd(p,n)/n$-combination of pairwise non-$R^p$-conjugate vertices. Setting :
$$V=\{V_0,...,V_{n-1}\}$$
You are looking for $k\times gcd(p,n)/n$ combination of $V/G$ which is of cardinal $gcd(p,n)$, hence, in that case you have :
$$|Fix(R^p)|=\begin{pmatrix}gcd(p,n)\\ k\times gcd(p,n)/n \end{pmatrix} $$
Finally :
$$|X/\mathbb{Z}_n|=\frac{1}{n}\sum_{p=0}^{n-1}|Fix(R^p)| $$
I don't see how to compute it explicitely but for $k=4$ and $n=6$ I get $3$ classes which seems to do the job...
For the case where $G=\mathbb{D}_n$ it suffices to see on which condition a symmmetry stabilizes an element take $s$ be such a symmetry, up to a global rotation, you can assume that the symmetry is given by the $x$-axis, well now to compute $|Fix(s)|$ you need to split into different cases depending on the parity of both $k$ and $n$, I will let you finish this... 
