In order to better understand the components that lead to effective representations, we propose a lightweight version of InferSent<cite> (Conneau et al., 2017)</cite> , called InferLite, that does not use any recurrent layers and operates on a collection of pre-trained word embeddings.