Abecedarius

An abecedarius (also abecedary and abecedarian) is a special type of acrostic in which the first letter of every word, strophe or verse follows the order of the letters in the alphabet.
Etymology.
"Abecedarius" (or "abecedarium") is a Medieval Latin word meaning "ABC primer", derived by adding the suffix "-arius" (-a, -um) to the names of the first four letters of the alphabet (a+b+c+d). According to the OED, the earliest use of its English cognate, "abecedary", dates back at least to the mid-15 century, preceding the first usage of "abecedarian" which, as an adjective meaning "arranged in alphabetical order", can be first attested in 1665. The related adjective "alphabetic" (from Ancient Greek) has been used interchangeably with "abecedarian" since at least the 17th century.
Origins.
The abecedarius is most probably the oldest type of acrostic. Its origins have been linked to either the sacred nature of letters and the mystical significance of these types of arrangements or its didactic use as a mnemonic and instructive device for children. Indeed, this second type of abecedarii, mostly in the form of rhythmic arrangements or songs, is still popular and widely used tool to teach children the alphabet or other concepts.
Use in literature.
Ancient literature.
The oldest abecedarii found are of Semitic origin. In fact, all of the confirmed acrostics in the Hebrew Bible are alphabetic. These occur in four of the five chapters that make up the "Book of Lamentations", in the praise of the good wife in Proverbs 31:10-31, and in Psalms 25, 34, 37, 111, 112, 119, and 145.
Notable among the acrostic psalms is the long Psalm 119, which typically is printed in subsections named after the 22 letters of the Hebrew alphabet, each section consisting of 8 verses, each of which begins with the same letter of the alphabet and the entire psalm consisting of 22 x 8 = 176 verses; and Psalm 145, which is recited three times a day in the Jewish services.
Some acrostic psalms are technically imperfect. E.g., Psalm 9 and Psalm 10 appear to constitute a single acrostic psalm together, but the length assigned to each letter is unequal and five of the twenty-two letters of the Hebrew alphabet are not represented and the sequence of two letters is reversed. In Psalm 25, one Hebrew letter is not represented; the following letter (resh) repeated. In Psalm 34, the current final verse, 23, does fit verse 22 in content, but makes the line too long. In Psalms 37 and 111, the numbering of verses and the division into lines are interfering with each other; as a result, in Psalm 37, for the letters dalet and kaph, there is only one verse, and the letter ayin is not represented. Psalm 111 and 112 have 22 lines, but 10 verses. Psalm 145 does not represent the letter nun, having 21 verses, but one Qumran manuscript of this psalm does have that missing line, which agrees with the Septuagint.
Medieval literature.
Written around 393, St. Augustine's well-known abecedarian psalm against the Donatists is the earliest known example of medieval rhythmical verse. 
Another example is Old Polish poem "Skarga umierającego" ("A Dying Man Complaint"). Such poems are important historical sources for development of orthography. They can be proofs that some letters were used long ago and later were totally forgotten. Bishop Constantine of Preslav's abecedarius from 9th century documents the early Slavic alphabet. Nowadays abecedariuses are written chiefly for children.
The most famous example of abecedarius in English literature is Geoffrey Chaucer's ABC.
Modern literature.
One of the most famous and complex modern examples of alliterative verse in the English language is Alaric Watts's abecedarius "The Siege of Belgrade" which loosely chronicles the historical event in 29 lines, each of the first 26 not only beginning with the consecutive letters of the alphabet, but also composed only of words beginning with the respective letter:
<poem style="margin-left: 5em;">An Austrian army, awfully arrayed,
Boldly by battery besieged Belgrade.
Cossack commanders cannonading come,
Dealing destruction's devastating doom. 
Every endeavor engineers essay,
For fame, for fortune fighting - furious fray!</poem>
Even though rarely used, some authors have preferred to use the term "abecedarius" for poems which follow Watts' arrangement, considering the "alphabet-in-acrostic" form just a loose application, as can be witnessed in these self-referential lines:
<poem style="margin-left: 5em;">An abecedarius always alliterates
Blindly blunders, but blooms:
Comes crawling craftily, cantering crazily,
Daring, doubtless, dark dooms.</poem>
Contemporary literature.
In the words of the American poet and critic Edward Hirsch, "[t]he abecedarian has been revived in contemporary poetry with experimental force", because, "[p]aradoxically, the arbitrary structure triggers verbal extravagances". Hirsch names Harryette Mullen's "Sleeping with the Dictionary" (2002), Carolyn Forché's poem "On Earth" (2003), Barbara Hamby's "The Alphabet of Desire" (2006) and Karl Elder's "Gilgamesh at the Bellagio" (2007) as few modern examples structured in accordance with different variations of the basic abecedarian sequence, where the adherence to the form produces unusual and interesting aesthetic results.
In the case of Forché's "forty-seven page poem", for example, the rigorous alphabetical order "guides not only the stanzas, but also the words themselves":
<poem style="margin-left: 5em;">languid at the edge of the sea
lays itself open to immensity
leaf-cutter ants bearing yellow trumpet flowers
 along the road
left everything left all usual worlds behind
library, lilac, linens, litany.</poem>
Mary Jo Bang's verse collection "The Bride of E" uses the abecedarian as an organizing principle, as do Jessica Greenbaum's “A Poem for S.”, Tom Disch’s “Abecedary”, and Matthea Harvey’s sequence “The Future of Terror/The Terror of Future”.
Children's literature.
Some of the best-known and loved abecedarians have been written for children, such as Dr. Seuss's "ABC" or the roughly half-dozen alphabet books of Edward Gorey, the most notorious among them "The Gashlycrumb Tinies". However, even the most experimental authors of the twentieth century have authored children's or quasi-children abecedarians. Written in an attempt to compose "a birthday book [she] would have liked as a child", "", Gertrude Stein's intended follow-up to her first children's book, "The World Is Round", has been described as "a romp through the alphabet" and an "unusual alphabet book". Also, Djuna Barnes' last book, "Creatures in an Alphabet" is a collection of rhyming quatrains about different animals, ordered, albeit loosely, in an alphabet sequence.
Related concepts.
Iroha mojigusari.
Iroha mojigusari is a Japanese poetic form, a "specialized version" of the abecedarius, in which the first line begins with the first and ends with the second character of the alphabet, the second one begins with the third and ends with the fourth character of the alphabet, and so on, "until all the letters of the alphabet have been used in order".
Alphabet poem.
Invented by Paul West, a British-born American novelist, poet, and essayist, the alphabet poem is a stricter modern variation of the abecedarius. It consists of 13 lines, each consisting of two words, each word starting with a letter which follows the initial letter of the preceding word. West introduced the alphabet poem in his book "Alphabet Poetry", a cycle of 26 poems, the first of which starts with AB and ends with YZ, the second one starting with BC and ending with ZA, and the last one starting with ZA and ending in XY. Due to the constraints, West allows himself few liberties here and there, as can be witnessed in this example:
<poem style="margin-left: 5em;">Artichokes, Bubbly,
Caviar, Dishes
Epicures Favor,
Gourmets Hail;
Ices, Juicy
Kickshaws, Luxurious
Mousses, Nibblesome
Octopus, Pheasant,
Quiches, Sweets,
Treats Utterly
Vanquish Weightwatchers:
Xenodochy's 
Yum-yum!</poem>