Figure REF  shows the average recognition rates for the first three experiments. Initially, different HPNN architectures were tested to reach a generalization performance equivalent to the standard CNN reported by Khorrami et al. [1]} (CNN in Figure REF ). The architecture has three pyramidal layers with 4, 8 and 8 sub-layers, followed by two dense layers with 40 and 8 neurons. The HPNN has achieved the same recognition rate on CK+ as the CNN, i.e. 81.8%, but contains far less trainable parameters: 113,520 compared to 1,335,956 of the CNN. The PyraNet-3 with three pyramidal layers, but with only one sub-layer, has achieved the lowest recognition rate of 72.90%. If we remove the last two pyramidal layers of the PyraNet-3, the recognition rate increases about 4% as shown in Figure REF  by the PyraNet-1. This result shows the problem of going deeper with pyramidal architectures using the original approach by Phung et al. [2]}.
