Figure 8: Input/output cosine similarity score of each token per layer for text ”Write a quiz about bits”. The cosine similarity scores of the 16 newly added layers(layers 28-44), as depicted in the lower figure, are observed to be nearly 1.