Leveraging supervision given by some general task aided in obtaining state-of-the-art sentence representations <cite>(Conneau et al., 2017)</cite> . However, every supervised learning tasks is prone to overfit.