A few recent works, on the other hand, use GNNs to compute the knowledge graph representation [24, 20, 27] . These high-level knowledge graph representations are particularly important for question answering task [22, 10, <cite>17]</cite> . We use pre-trained representations to initialize the model and then update them using a relational and bi-directional GNN model.