Despite their promise, ML-based solutions face a significant practical challenge: obtaining representative and labeled training data is infeasible for many organizations. On the one hand, commercial datasets with these properties exist, but are unattainable for many due to financial constraints [1]}, [2]}, with the licensing needed can cost $400k/year.
On the other hand, home-grown datasets face scaling and labeling challenges, e.g., benign samples are often closed-source or copyright-protected, and labeling involves error-prone manual analysis to reverse engineer each binary. Consequently, many security practitioners and researchers only have access to small datasets that lack the heterogeneity seen in the wild [3]}. For example, we find that recent ML detectors achieve accuracies of only 60-71% when trained on small public datasets [4]}, [5]} versus their large, commercial counterparts; such degradations are unacceptable given that single-digit accuracy improvements (and any new detected malware) are celebrated by malware analysts [6]}. We note that these small datasets are also realistic in representing the challenge applying ML to targeted or otherwise unique malware families of interst (e.g., targeted banking malware) rather than broad and indiscriminate malware.
