Large Language Models (LLMs)  went from non-existent to ubiquitous in the machine learning discourse within a few years. Due to the fast pace of the field, it is difficult to identify the remaining challenges and already fruitful application areas. In this paper, we aim to establish a systematic set of open problems and application successes so that ML researchers can comprehend the field’s current state more quickly and become productive.11footnotetext:Equal contribution.22footnotetext:{jean.kaddour,robert.mchardy}.20@ucl.ac.uk, joshua.harris@ukhsa.gov.uk