To evaluate PRESTO, we use a benchmark collection of 10 datasets from the UCI Machine Learning repository. Overall, we rely heavily on the results presented in [1]} and follow their methodology closely. As such, we follow closely the protocol they used, which originated in  [2]} and was used subsequently in [3]}, [4]}, and [5]}. For each dataset, we set aside 10% as a test set, and split the remainder 80-20 for training / validation. The training/validation split is used for hyper-parameter optimization (including the number of boosting stages for each base classifier). Then the model is retrained on the combined training/validation data for the best choice of parameters and evaluated on the test set. As in  [1]}, we ran 20 trials for each dataset, except for Protein (5) and YearMSD (1). The latter was a particularly large dataset with a single train/test split, and therefore required special handling.
