The above-described strategy has an overall selection ratio of \(R=R_{dedup}*R_{proto}\) and intends to diversify the distribution of our data locally and globally. For brevity we refer to this method as **D4**, a shorthand for _Document De-Duplication and Diversification_. Throughout this work, we choose \(R_{dedup}=0.75\) and vary \(R_{proto}\) (we discuss this choice in Section A.1). In Section 4, we compare the performance of D4 to baseline training and other methods, and in Section 4.4 we analyze D4 and show that reclustering after semantic de-duplication indeed reduces the impact of duplicate-driven clusters (see Figure 7).