The resemblance between the methods used in studying quantum-many body physics and in machine learning has drawn considerable attention. In particular, tensor networks (TNs) and deep learning architectures bear striking similarities to the extent that TNs can be used for machine learning. Previous results used one-dimensional TNs in image recognition, showing limited scalability and a request of high bond dimension. In this work, we train two-dimensional hierarchical TNs to solve image recognition problems, using a training algorithm derived from the multipartite entanglement renormalization ansatz (MERA). This approach overcomes scalability issues and implies novel mathematical connections among quantum many-body physics, quantum information theory, and machine learning. While keeping the TN unitary in the training phase, TN states can be defined, which optimally encodes each class of the images into a quantum many-body state. We study the quantum features of the TN states, including quantum entanglement and fidelity. We suggest these quantities could be novel properties that characterize the image classes, as well as the machine learning tasks. Our work could be further applied to identifying possible quantum properties of certain artificial intelligence methods. Over the past years, we have witnessed a booming progress in applying quantum theories and technologies to realistic problems. Paradigmatic examples include quantum simulators BID31 and quantum computers (Steane, 1998; BID16 BID2 aimed at tackling challenging problems that are beyond the capability of classical digital computations. The power of these methods stems from the properties quantum many-body systems.Tensor networks (TNs) belong to the most powerful numerical tools for studying quantum manybody systems BID22 BID13 BID26 . The main challenge lies in the exponential growth of the Hilbert space with the system size, making exact descriptions of such quantum states impossible even for systems as small as O(10 2 ) electrons. To break the "exponential wall", TNs were suggested as an efficient ansatz that lowers the computational cost to a polynomial dependence on the system size. Astonishing achievements have been made in studying, e.g. spins, bosons, fermions, anyons, gauge fields, and so on Cirac & Verstraete, 2009; BID23 BID26 BID26 . TNs are also exploited to predict interactions that are used to design quantum simulators BID25 .As TNs allowed the numerical treatment of difficult physical systems by providing layers of abstraction, deep learning achieved similar striking advances in automated feature extraction and pattern recognition BID19 . The resemblance between the two approaches is beyond superficial. At theoretical level, there is a mapping between deep learning and the renormalization group BID1 , which in turn connects holography and deep learning BID37 BID10 , and also allows studying network design from the perspective of quantum entanglement BID20 . In turn, neural networks can represent quantum states BID3 BID4 BID15 BID11 .Most recently, TNs have been applied to solve machine learning problems such as dimensionality reduction BID5 , handwriting recognition BID30 BID12 . Through a feature mapping, an image described as classical information is transferred into a product state defined in a Hilbert space. Then these states are acted onto a TN, giving an output vector that determines the classification of the images into a predefined number of classes. Going further with this clue, it can be seen that when using a vector space for solving image recognition problems, one faces a similar "exponential wall" as in quantum many-body systems. For recognizing an object in the real world, there exist infinite possibilities since the shapes and colors change, in principle, continuously. An image or a gray-scale photo provides an approximation, where the total number of possibilities is lowered to 256 N per channel, with N describing the number of pixels, and it is assumed to be fixed for simplicity. Similar to the applications in quantum physics, TNs show a promising way to lower such an exponentially large space to a polynomial one.This work contributes in two aspects. Firstly, we derive an efficient quantum-inspired learning algorithm based on a hierarchical representation that is known as tree TN (TTN) (see, e.g., BID21 ). Compared with Refs. BID30 BID12 where a onedimensional (1D) TN (called matrix product state (MPS) (Östlund & Rommer, 1995) ) is used, TTN suits more the two-dimensional (2D) nature of images. The algorithm is inspired by the multipartite entanglement renormalization ansatz (MERA) approach BID35 BID36 BID7 BID9 , where the tensors in the TN are kept to be unitary during the training. We test the algorithm on both the MNIST (handwriting recognition with binary images) and CIFAR (recognition of color images) databases and obtain accuracies comparable to the performance of convolutional neural networks. More importantly, the TN states can then be defined that optimally encodes each class of images as a quantum many-body state, which is akin to the study of a duality between probabilistic graphical models and TNs BID27 . We contrast the bond dimension and model complexity , with results indicating that a growing bond dimension overfits the data. we study the representation in the different layers in the hierarchical TN with t-SNE ( BID32 , and find that the level of abstraction changes the same way as in a deep convolutional neural network BID18 or a deep belief network BID14 , and the highest level of the hierarchy allows for a clear separation of the classes. Finally, we show that the fidelities between each two TN states from the two different image classes are low, and we calculate the entanglement entropy of each TN state, which gives an indication of the difficulty of each class. We continued the forays into using tensor networks for machine learning, focusing on hierarchical, two-dimensional tree tensor networks that we found a natural fit for image recognition problems. This proved a scalable approach that had a high precision, and we can conclude the following observations:• The limitation of representation power (learnability) of the TTNs model strongly depends on the input bond (physical indexes). And, the virtual bond (geometrical indexes) determine how well the TTNs approximate this limitation.• A hierarchical tensor network exhibits the same increase level of abstraction as a deep convolutional neural network or a deep belief network.• Fidelity can give us an insight how difficult it is to tell two classes apart.• Entanglement entropy has potential to characterize the difficulty of representing a class of problems.In future work, we plan to use fidelity-based training in an unsupervised setting and applying the trained TTN to recover damaged or compressed images and using entanglement entropy to characterize the accuracy.
This approach overcomes scalability issues and implies novel mathematical connections among quantum many-body physics, quantum information theory, and machine learning.
{"src": "scitldr-AIC_train", "paper_id": "ryF-cQ6T-"}