Optimization of Decision Tree Evaluation Using SIMD Instructions
Decision forest (decision tree ensemble) is one of the most popular machine
learning algorithms. To use large models on big data, like document scoring
with learning-to-rank models, we need to evaluate these models efficiently. In
this paper, we explore MatrixNet, the ancestor of the popular CatBoost library.
Both libraries use the SSE instruction set for scoring on CPU. This paper
investigates the opportunities given by the AVX instruction set to evaluate
models more efficiently. We achieved 35% speedup on the binarization stage
(nodes conditions comparison), and 20% speedup on the trees apply stage on the
ranking model.