One of the main problems encountered so far with recurrent neural networks is that they struggle to retain long-time information dependencies in their recurrent connections. Neural Turing Machines (NTMs) attempt to mitigate this issue by providing the neural network with an external portion of memory, in which information can be stored and manipulated later on. The whole mechanism is differentiable end-to-end, allowing the network to learn how to utilise this long-term memory via SGD. This allows NTMs to infer simple algorithms directly from data sequences. Nonetheless, the model can be hard to train due to a large number of parameters and interacting components and little related work is present. In this work we use a NTM to learn and generalise two arithmetical tasks: binary addition and multiplication. These tasks are two fundamental algorithmic examples in computer science, and are a lot more challenging than the previously explored ones, with which we aim to shed some light on the capabilities on this neural model.
Previous works investigated the task of learning binary arithmetic with neural networks. About the binary addition task, @cite_21 showed three depth-optimal feedforward neural networks able to perform @math -bits sums. More recently, @cite_11 developed a new model to perform addition in a more parallel way. Similarly, the multiplication task was studied in @cite_5 , where a model optimal in depth was developed. Also @cite_17 studied this task, developing a model that is more efficient and fast than the previous ones. @cite_7 studied both the problems using feedforward neural networks, founding solutions that are optimal in the depth of the network and bounds polinomially the number of neurons and synapses.
@cite_11 @cite_7 @cite_21 @cite_5 @cite_17
190377865 2032690046 2141469882 2579422619
Addition is the most commonly used arithmetic operation and is the speed-limiting element in the core of arithmetic logic unit (ALU) in a microprocessor. Perceptron of feedforward neural networks, inspired by the threshold logic unit neuron model of McCulloch and Pitts, is one of the most important aspects of artificial neural networks (ANN). This paper proposes a design of neural network parallel adder (NNPA) under the framework of multi-layer perceptron (MLP) of binary feedforward neural networks (BFNN). The DNA-like learning algorithm proposed by the present authors is successfully used for training the weight-threshold values of NNPA. Moreover, the efficiency of NNPA is compared with that of the conventional adder such as carry-ripple adder and carry-look-ahead adder. It is shown that some advantages of ANN such as synchronous, parallel and fast speed in information processing are sufficiently taken by the current NNPA. Abstract We design new feed-forward multi-layered neural networks which perform different elementary arithmetic operations, such as bit shifting, addition of N p -bit numbers, and multiplication of two n -bit numbers. All the structures are optimal in depth and are polynomially bounded in the number of neurons and in the number of synapses. The whole set of synaptic couplings and thresholds are obtained exactly. An artificial neural network (ANN) is commonly modeled by a threshold circuit, a network of interconnected processing units called linear threshold gates. The depth of a network represents the number of unit delays or the time for parallel computation. The Size of a circuit is the number of gates and measures the amount of hardware. It was known that traditional logic circuits consisting of only unbounded fan-in AND, OR, NOT gates would require at least Ω(log n log log n) depth to compute common arithmetic functions such as the product or the quotient of two n-bit numbers, unless we allow the size (and fan-in) to increase exponentially (in n). We show in this paper that ANNs can be much more powerful than traditional logic circuits. In particular, we prove that that iterated addition can be computed by depth-2 ANN, and multiplication and division can be computed by depth-3 ANNs with polynomial size and polynomially bounded integer weights, respectively. Moreover, it follows from known lower bound results that these ANNs are optimal in depth. We also indicate that these techniques can be applied to construct polynomial-size depth-3 ANN for powering, and depth-4 ANN for multiple product. Almost all signal processing applications demand a considerable number of multiplications. As addition, multiplication is also the speed-limiting element in the core of arithmetic logic unit (ALU) in a microprocessor. In this paper, the perceptron of artificial neural network (ANN) is used to build a binary multiplier, named neural network binary multiplier(NNBM). It is shown that many advantages of ANN such as synchronous, parallel and fast speed on processing information are taken by the multiplier.