For our `text' features (Table REF ), we use the pretrained contextual sentence embeddings of BERT [1]}. Our embedding dimension is 768 for each speaker's response. When predicting ACQI and IQ we concatenate the embeddings for both consumer and dialog system for this turn and the previous. This results in a \(4\times 768 = 3072\)  dimensional vector. When the previous utterance is unavailable, we use the zero vector
of the appropriate dimension. This feature vector was used to represent a system where the model uses only surface textual features for the previous two turns per user.
