There is clear utility in stacking additional layers; however, even with 4 stacked layers the RNNs still underperform the CNN-based embeddings of<cite> [14]</cite> until we begin adding fully connected layers.