
Batch size: the sample size of mini-batch.

LR: learning rate.

\(\gamma \) : the hyperparameter used to update the sparsity in each leaf (Eq. REF ).

Steps: the total number of training steps. We find 2000 works well across our datasets.

LR warmup steps: we do the learning rate warmup [1]} that linearly increases the learning rate from 0 to 1e-3 in the first 1000 steps.

Smoothing \(\delta \) : the smoothing count for our volume and data ratio estimation.

Per tree dropout: the dropout noise we use for the update of each tree.

Arch: we adopt the GAMAtt architecture form the NodeGAM [2]}.

No. layer: the number of layers of trees.

No. trees: the number of trees per layer.

Addi tree dim: the dimension of the tree's output. If more than 0, it appends an additional dimension in the output of each tree.

Tree depth: the depth of tree.

Dim Attention: since we use the GAMAtt architecture, this determines the size of the attention embedding. We find tuning more than 32 will lead to insufficient memory in our GPU, and in general 8-16 works well.

Column Subsample (\(\rho \) ): this controls how many proportion of features a tree can operate on.

Temp annealing steps (K), Min Temp: these control how fast the temperature linearly decays from 1 to the mininum temperature (0.1) in K steps. After K training steps, the entmax or entmoid become max or step functions in the model.

<TABLE>