It is noted that one of the key factors behind designing an ensemble is the diversity among the ensemble members. The preliminary results seen in this work suggests that diversity in model architectures results in a better performing ensemble as compared to diversity through randomized initialization. Further analysis may be required, for example analyzing the final states in the parameter space attained by each ensemble [1]}, to investigate this hypothesis.
