LMs for OpenAI. Overall, OpenAI has explored two major approaches to further improving the GPT-3 model, _i.e._, training on code data and alignment with human preference, which are detailed as follows.