Truly Unordered Probabilistic Rule Sets for Multi-class Classification
Rule set learning has long been studied and has recently been frequently
revisited due to the need for interpretable models. Still, existing methods
have several shortcomings: 1) most recent methods require a binary feature
matrix as input, learning rules directly from numeric variables is
understudied; 2) existing methods impose orders among rules, either explicitly
or implicitly, which harms interpretability; and 3) currently no method exists
for learning probabilistic rule sets for multi-class target variables (there is
only a method for probabilistic rule lists).
  We propose TURS, for Truly Unordered Rule Sets, which addresses these
shortcomings. We first formalise the problem of learning truly unordered rule
sets. To resolve conflicts caused by overlapping rules, i.e., instances covered
by multiple rules, we propose a novel approach that exploits the probabilistic
properties of our rule sets. We next develop a two-phase heuristic algorithm
that learns rule sets by carefully growing rules. An important innovation is
that we use a surrogate score to take the global potential of the rule set into
account when learning a local rule.
  Finally, we empirically demonstrate that, compared to non-probabilistic and
(explicitly or implicitly) ordered state-of-the-art methods, our method learns
rule sets that not only have better interpretability (i.e., they are smaller
and truly unordered), but also better predictive performance.