Star polygon
In geometry, a star polygon is a type of non-convex polygon. Only the regular star polygons have been studied in any depth; star polygons in general appear not to have been formally defined, however certain notable ones can arise through truncation operations on regular simple and star polygons.

Branko Grünbaum identified two primary definitions used by Kepler, one being the regular star polygons with intersecting edges that don't generate new vertices, and the second being simple isotoxal concave polygons.

The first usage is included in polygrams which includes polygons like the pentagram but also compound figures like the hexagram.

Etymology
Star polygon names combine a numeral prefix, such as penta-, with the Greek suffix -gram (in this case generating the word pentagram). The prefix is normally a Greek cardinal, but synonyms using other prefixes exist. For example, a nine-pointed polygon or enneagram is also known as a nonagram, using the ordinal nona from Latin. The -gram suffix derives from γραμμή (grammḗ) meaning a line.

Regular star polygon

A "regular star polygon" is a self-intersecting, equilateral equiangular polygon.

A regular star polygon is denoted by its Schläfli symbol {p/q}, where p (the number of vertices) and q (the density) are relatively prime (they share no factors) and q ≥ 2.

The symmetry group of {n/k} is dihedral group Dn of order 2n, independent of k.

Regular star polygons were first studied systematically by Thomas Bradwardine, and later Kepler.

Construction via vertex connection 

Regular star polygons can be created by connecting one vertex of a simple, regular, p-sided polygon to another, non-adjacent vertex and continuing the process until the original vertex is reached again. Alternatively for integers p and q, it can be considered as being constructed by connecting every qth point out of p points regularly spaced in a circular placement. For instance, in a regular pentagon, a five-pointed star can be obtained by drawing a line from the first to the third vertex, from the third vertex to the fifth vertex, from the fifth vertex to the second vertex, from the second vertex to the fourth vertex, and from the fourth vertex to the first vertex. 

If q is greater than half of p, then the construction will result in the same polygon as p-q; connecting every third vertex of the pentagon will yield an identical result to that of connecting every second vertex. However, the vertices will be reached in the opposite direction, which makes a difference when retrograde polygons are incorporated in higher-dimensional polytopes. For example, an antiprism formed from a prograde pentagram {5/2} results in a pentagrammic antiprism; the analogous construction from a retrograde "crossed pentagram" {5/3} results in a pentagrammic crossed-antiprism. Another example is the tetrahemihexahedron, which can be seen as a "crossed triangle" {3/2} cuploid.

Degenerate regular star polygons
If p and q are not coprime, a degenerate polygon will result with coinciding vertices and edges. For example {6/2} will appear as a triangle, but can be labeled with two sets of vertices 1-6. This should be seen not as two overlapping triangles, but a double-winding of a single unicursal hexagon.

Construction via stellation 

Alternatively, a regular star polygon can also be obtained as a sequence of stellations of a convex regular core polygon. Constructions based on stellation also allow for regular polygonal compounds to be obtained in cases where the density and amount of vertices are not coprime. When constructing star polygons from stellation, however, if q is greater than p/2, the lines will instead diverge infinitely, and if q is equal to p/2, the lines will be parallel, with both resulting in no further intersection in Euclidean space. However, it may be possible to construct some such polygons in spherical space, similarly to the monogon and digon; such polygons do not yet appear to have been studied in detail.

Simple isotoxal star polygons 
When the intersecting lines are removed, the star polygons are no longer regular, but can be seen as simple concave isotoxal 2n-gons, alternating vertices at two different radii, which do not necessarily have to match the regular star polygon angles. Branko Grünbaum in Tilings and Patterns represents these stars as |n/d| that match the geometry of polygram {n/d} with a notation {nα} more generally, representing an n-sided star with each internal angle α<180°(1-2/n) degrees. For |n/d|, the inner vertices have an exterior angle, β, as 360°(d-1)/n.

Examples in tilings
These polygons are often seen in tiling patterns. The parametric angle α (degrees or radians) can be chosen to match internal angles of neighboring polygons in a tessellation pattern. Johannes Kepler in his 1619 work Harmonices Mundi, including among other period tilings, nonperiodic tilings like that three regular pentagons, and a regular star pentagon (5.5.5.5/2) can fit around a vertex, and related to modern penrose tilings.

Interiors of star polygons

The interior of a star polygon may be treated in different ways. Three such treatments are illustrated for a pentagram. Branko Grunbaum and Geoffrey Shephard consider two of them, as regular star polygons and concave isogonal 2n-gons.

These include:
 Where a side occurs, one side is treated as outside and the other as inside. This is shown in the left hand illustration and commonly occurs in computer vector graphics rendering.
The number of times that the polygonal curve winds around a given region determines its density. The exterior is given a density of 0, and any region of density > 0 is treated as internal. This is shown in the central illustration and commonly occurs in the mathematical treatment of polyhedra. (However, for non-orientable polyhedra density can only be considered modulo 2 and hence the first treatment is sometimes used instead in those cases for consistency.)
 Where a line may be drawn between two sides, the region in which the line lies is treated as inside the figure. This is shown in the right hand illustration and commonly occurs when making a physical model.

When the area of the polygon is calculated, each of these approaches yields a different answer.

Star polygons in art and culture

Star polygons feature prominently in art and culture. Such polygons may or may not be regular but they are always highly symmetrical. Examples include:
The {5/2} star pentagon (pentagram) is also known as a pentalpha or pentangle, and historically has been considered by many magical and religious cults to have occult significance.
The {7/2} and {7/3} star polygons (heptagrams) also have occult significance, particularly in the Kabbalah and in Wicca.
The {8/3} star polygon (octagram), is frequent geometrical motifs in Mughal Islamic art and architecture; the first is on the emblem of Azerbaijan.
An eleven pointed star called the hendecagram was used on the tomb of Shah Nemat Ollah Vali.