The NLU models include a classic CRF model, triangular CRF extension (Tri-CRF) [1]} and Bi-RNN with attention (Att-RNN) [2]}. Our recent findings on medication information extraction from EHRs in English has shown that transformer-based language models can be extremely competitive [3]}. Even though, there are no available language models trained specifically on biomedical domain, we have decided to include Flaubert, a general-purpose pre-trained transformer language model for French [4]}.
