Deep Convolution Neural Networks (CNNs), rooted by the pioneer work of  \cite{Hinton1986,LeCun1985,Alex2012}, and summarized in \cite{LeCunBengioHinton2015},   have been shown to be very useful in a variety of fields.   The  state-of-the art CNN machines such as image rest net \cite{He_2016_CVPR} are described by real value inputs and kernel convolutions followed by the local and  non-linear rectified linear outputs.   Understanding the role of these layers, the accuracy and limitations of them,  as well as making them more efficient (fewer parameters)  are all ongoing research questions. 
 
  Inspired in quantum theory, we propose the use of complex value kernel functions, followed by the local non-linear  absolute (modulus) operator square. We argue that an advantage of quantum inspired complex kernels is robustness to realistic unpredictable scenarios (such as clutter noise,  data deformations). We study a concrete problem of shape detection and show that when multiple overlapping shapes are deformed and/or clutter noise is added, a convolution layer with quantum inspired complex kernels outperforms the statistical/classical kernel counterpart and a "Bayesian shape estimator" . The superior performance is due to the quantum phenomena of interference, not present in classical CNNs.   The convolution process in machine learning maybe summarized as follows. Given an input f L−1 (x) ≥ 0 to a convolution layer L, it produces an output DISPLAYFORM0 From g L (y) a local and non-linear function is applied, f L (y) = f (g L (y)), e.g., f = ReLu (rectified linear units) or f = |.| , the magnitude operator. This output is then the input to the next convolution layer (L+1) or simply the output of the whole process. We can also write a discrete form of these convolutions, as it is implemented in computers. We write g DISPLAYFORM1 , where the continuous variables y, x becomes the integers i, j respectively, the kernel function K(y − x) → w ij becomes the weights of the CNN and the integral over dx becomes the sum over j.These kernels are learned from data so that an error (or optimization criteria) is minimized. The kernels used today a real value functions. We show how our understanding of the optimization criteria "dictate" the construction of the quantum inspired complex value kernel. In order to concentrate and study our proposal of quantum inspired kernels, we simplify the problem as much as possible hoping to identify the crux of the limitation of current use of real value kernels.We place known shapes in an image, at any location, and in the presence of deformation and clutter noise. These shapes may have been learned by a CNN. Our main focus is on the feedforward performance, when new inputs are presented. Due to this focus, we are able to construct a Bayesian a posteriori probability model to the problem, which is based on real value prior and likelihood models, and compare it to the quantum inspired kernel method.The main advantage of the quantum inspired method over existing methods is its high resistance to deviations from the model, such as data deformation, multiple objects (shapes) overlapping, clutter noise. The main new factor is the quantum interference phenomenon BID1 BID0 , and we argue it is a desired phenomena for building convolution networks. It can be carried out by developing complex value kernels driven by classic data driven optimization criteria. Here we demonstrate its strength on a shape detection problem where we can compare it to state of the art classical convolution techniques. We also can compare to the MAP estimator of the Bayesian model for the shape detection problem.To be clear, we do not provide (yet) a recipe on how to build kernels for the full CNN framework for machine learning, and so the title of this paper reflects that. Here, we plant a seed on the topic of building complex value kernels inspired in quantum theory, by demonstrating that for a given one layer problem of shape detection (where the classic data optimization criteria is well defined), we can build such complex value kernel and demonstrate the relevance of the interference phenomena.To our knowledge such a demonstration is a new contribution to the field. We also speculate on how this process can be generalized. Deep Convolution Neural Networks (CNNs), rooted on the pioneer work of BID8 ; BID4 ; BID3 , and summarized in BID5 , have been shown to be very useful in a variety of fields.Inspired in quantum theory, we investigated the use of complex value kernel functions, followed by the local non-linear absolute (modulus) operator square. We studied a concrete problem of . For each of the figures 5a, 5b,5c we vary we vary b = 1 2 a, a, 2a (or center displacements δµ = 0.25, 0.5, 1), respectively. These figures depict ratios Q(a, b, ) × (blue) for ∈ (0.047, 0.2802) and H(a, b, α) × ← − α (red) for ← − α ∈ (22.727, 2.769) (The reverse arrow implies the x-axis start at the maximum value and decreases thereafter). All plots have 200 points, with uniform steps in their respective range. Note that our proposed parameter value is = 0.1401, the solution to equation FORMULA42 , and indeed gives a high ratio. Also, α = 2.769 is the smallest value to yield all Hough votes in the center. Clearly the quantum ratio outperforms the best classical Hough method, which does not vary much across α values. As the center displacement increases, the quantum method probability, for = 0.1401, decreases much faster than the Hough method probability. Final figure 5d display values of |ψ| 2 (µ * ) × (at the true center) in blue, for ∈ (0.047, 0.2802), with 200 uniform steps. In red, V (µ * ) × ← − α for ← − α ∈ (22.727, 2.769), with 200 uniform steps. DISPLAYFORM0 shape detection and showed that when multiple overlapping shapes are deformed and/or clutter noise is added, a convolution layer with quantum inspired complex kernels outperforms the statistical/classical kernel counterpart and a "Bayesian shape estimator". It is worth to mention that the Bayesian shape estimator is the best method as long as the data satisfy the model assumptions. Once we add multiple shapes, or add clutter noise (not uniform noise), the Bayesian method breaks down rather easily, but not the quantum method nor the statistical version of it (the Hough method being an approximation to it). An analysis comparing the Quantum method to the Hough method was carried out to demonstrate the superior accuracy performance of the quantum method, due to the quantum phenomena of interference, not present in the classical CNN.We have not focused on the problem of learning the shapes here. Given the proposed quantum kernel method, the standard techniques of gradient descent method should also work to learn the kernels, since complex value kernels are also continuous and differentiable. Each layer of the networks carries twice as many parameters, since complex numbers are a compact notation for two numbers, but the trust of the work is to suggest that they may perform better and reduce the size of the entire network. These are just speculations and more investigation of the details that entice such a construction are needed. Note that many articles in the past have mentioned "quantum" and "neural networks" together. Several of them use Schrödinger equation, a quantum physics modeling of the world. Here in no point we visited a concept in physics (forces, energies), as Schrödinger equation would imply, the only model is the one of shapes (computer vision model). Quantum theory is here used as an alternative statistical method, a purely mathematical construction that can be applied to different models and fields, as long as it brings benefits. Also, in our search, we did not find an article that explores the phenomena of interference and demonstrate its advantage in neural networks. The task of brining quantum ideas to this field must require demonstrations of its utility, and we think we did that here.
A quantum inspired kernel for convolution network, exhibiting interference phenomena,  can be very useful (and compared it with real value  counterpart).
{"src": "scitldr-AIC_train", "paper_id": "HyyHX4gZM"}