The form of the activation function in (c) includes both ReLU activation (\(t^{(l)}=0 \mathbf {1}_{n_l}\) ) and identity map (\(t^{(l)}=-\infty \mathbf {1}_{n_l}\) , so that \(\sigma ^{(l)}(x)=x\) ) as special cases, where \(\mathbf {1}_{n_l}\)  is vector \((1,\ldots ,1)^\top \)  having \(n_l\)  entries.
We only consider the ReLU activation function in this section as we are generalizing the model in [1]} where the tropical algebra makes the most sense for ReLU activations. In addition, much of the theory literature on neural networks has focused on ReLU networks [2]}, [3]}, [1]}.
