An analytical definition of computation unit utilization \(\eta ^{state}\)  is directly derived as a ratio of a number of unique tensor-states and a computation unit capacity. One could aim at optimizing a network architecture search (NAS) [1]} such that every computation unit reaches maximum utilization \(\eta ^{state}=1.0\)  over all training data points and all predicted labels.
A statistical definition of computation unit utilization is derived from a tensor-state distribution shape and aims at maximum normalized entropy of the state distribution for each predicted label (entropy-based utilization) \(\eta ^{entropy}\)  or at minimum Kullback–Leibler (KL) divergence of the state distribution from a uniform distribution allocated per predicted label \(\eta ^{KLDiv}\) . The use of Entropy and KL divergence [2]} are borrowed from the source coding theorem [3]}.
