Recent work has proposed a variety of different prompting techniques: chain-of-thought prompting [1]}, interactive question answering [2]}, aggregation-based schemes like self-consistency [3]} and ThinkSum [4]}. We consider all these works as instances of LMP (also discussed under the term of prompt programming [5]}, [6]}), where the goal is to leverage the reasoning abilities of a pre-trained model to achieve a specific task. A few select works have identified this trend, and propose novel LM-focused programming systems: PromptChainer [7]}, OpenPrompt [8]}, and PromptSource [9]} provide integrated development environments for LM interaction. The latter two even support a simple templating language akin to LMQL  top-level string semantics. However, neither of the projects implements constraints or control flow like LMQL  does.
Finally, [10]} discuss the idea of language model cascades, relating LM querying to probabilistic programming, which opens up interesting avenues for future work, also in the more general context of language model programming and LMQL.
