The reader may have noticed that in this paper we only work on the manifolds for the last three fully connected layers though AlexNet has convolutional layers as well. One way is to repeat the above work for convolutional layers by taking every output of them as a long vector as in [1]}. However to our opinion, unlike the fully connected layers which is straightforward to be treated as vectors in high dimension, convolutional layer is more of matrix type, where the vector-form feature descriptor may break the geometric structure of the pixel matrix space. Redefine manifold for convolutional layers is also part of our future work.
