We mixed and matched datasets and models to better understand the effect of topology under a variety of scenarios. In our experiments, we used a 2-layer feed-forward model, a 2-layer CNN model and Resnet18 [1]}. We also utilized MNIST [2]}, FEMNIST [3]}, CIFAR-10 and CIFAR-100 [4]} datasets for our training data. We used 1000 uniformly sampled images from FMNIST [5]} as global data when conducting MNIST/FEMNIST experiments. Similarly, we used 1000 samples of Tinyimagenet [6]} as global data when working with CIFAR-10/CIFAR-100. In the case of quantity-based label skew, each node is randomly assigned 2 classes while samples are equally distributed across the nodes. This is done to simulate a severe and strict quantity skew. In the case of distribution-based label skew, each party is allocated a proportion of the samples of each label according to Dirichlet distribution. I.e. Each node \(v\)  is allotted \(p_{k,v} \sim Dir_N (\beta = 0.5, 0.1)\)  proportion of the instances of class \(k\) . Here, the concentration parameter \(\beta \)  determines the extent to which the labels are skewed (i.e. smaller \(\beta \)  produces a higher skew). For feature skew and mixed non-IID settings, we set the feature noise level \(\eta =0.2, 0.3\)  respectively. The noise levels specify the different levels of Gaussian noise added to the peer local datasets. Specifically, given noise level \(\eta \) , we apply noise \(\widetilde{x_i} \sim Gau(\eta \; . \; N_i/N)\) , to each peer \(N_i\)  , where \(Gau(\eta \; . \; N_i/N)\)  is a Gaussian distribution with mean 0 and variance \(\eta \; . \; N_i/N\) .
