The impact of EOS on class decision boundaries is graphically illustrated in Figure  REF  for cross-entropy on the CIFAR-10 dataset with the automobile (depicted in blue) and trucks (orange) classes. Compared to the extreme majority class, autos and trucks are imbalanced at 1.7:1 and 100:1 ratios, respectively. When compared to each other, the imbalance level is 60:1, with trucks as the minority class. These two classes were selected due to their perceived similarity and potential for overlap. Figure  REF  uses the t-distributed Stochastic Neighbor Embedding (t-SNE) method [1]} to visualize the class decision boundary. The t-SNE method is effective at preserving local distances in high-dimensional data, such as neural network embeddings, when those embeddings are converted to a two-dimensional visualization [2]}, [3]}. In the t-SNE plots, the local structure of the minority class examples (trucks) in the baseline, SMOTE, Borderline SMOTE and Balanced SVM cases are uneven, with varying levels of density, such that there are intra-class gaps in data points. In the baseline and Borderline SMOTE cases, there are gaps in the auto class, which is also over-sampled. Only in the EOS visualization is the local structure of the classes represented with uniform, dense points. Additionally, in the EOS t-SNE illustration, there is a wider local boundary between the nearest enemy points when compared to the other methods. We hypothesize that this improved representation is due to EOS expanding the range of feature embeddings and closing the generalization gap, which results in a denser, more uniform class manifold in embedding space (RQ3 answered).
