A CapsNet is typically composed of an encoder and a decoder part, where the latter is a reconstruction network with 3 fully connected layers [1]}. In the previously-discussed experiments, we have removed the decoder. The main limitation of our work arise when computing the entropy of CapsNets trained with the decoder. We observed that the entropy of a CapsNets+REM is almost the same as that of a CapsNet+Q shown in Tables REF  and REF .
Indeed, when the decoder is used, the activity vector of an output capsule encodes richer representations of the input. Sabour et al. [1]} introduced the decoder to boost the routing performance on MNIST by enforcing the pose encoding a capsule. They also show that, when a perturbed activity vector is feeded to the decoder, such perturbation affects the reconstruction. So capsules representations are equivariant, meaning that transformations applied to the input are described by continuous changes in the output vector. In order to verify if output capsules of a trained CapsNet+REM without the decoder (so with low entropy) are still equivariant, we stacked on top of it the reconstruction network, without training the encoder. We can see in Fig. REF  that the output capsules are still equivariant to many transformations so we successfully obtained a CapsNet+REM with low entropy but with equivariant representations encoded into capsules.
<FIGURE>