We are currently collecting the Semantic Textual Similarity (STS) database in spoken format and the next step will be to investigate whether the model presented here also learns to capture sentence level semantic information and understand language in a deeper sense than recognising word presence. The work presented in <cite>[15]</cite> has made the first efforts in this regard and we aim to extend this to a larger database with sentences from multiple domains.