TECHNICAL FIELD

Embodiments described herein generally relate to improvements in information-processing performance for machine-learning systems having numerous practical applications, such as image processing systems, complex data centers, self-driving vehicles, security systems, medical treatment systems, transaction systems, and the like. Certain embodiments relate particularly to artificial neural networks (ANNs).

BACKGROUND

Machine learning, deep learning in particular, is receiving more attention by researchers and system developers due its successful application to automated perception, such as machine vision, speech recognition, motion understanding, and automated control (e.g., autonomous motor vehicles, drones, and robots). Modern multi-layered neural networks have become the framework of choice for deep learning. Conventional neural networks are mostly based on the computational operations of real-number calculus.

Quaternion algebras, based on a multi-dimensional complex number representation, has drawn attention across digital signal processing applications (motion-tracking, image processing, and control) due to the significant reduction in parameters and in operations and more accurate physics representation (singularity-free rotations) compared to one-dimensional real or two-dimensional complex algebras. Because quaternion (QT) operations necessitate reconciliation across geometry, calculus, interpolation, and algebra, to date, quaternions have not been well adapted to deep multi-layered neural networks. In particular, training neural networks requires the optimization of a cost (or loss) function via backpropagation—a process that computes the gradient (partial derivative) of the error with respect to parameters of a layer, and propagates the composite gradient to previous layers to signal a modification to the input parameters.

There have been attempts to incorporate quaternions in machine-learning applications to make use of their desirable properties. However, those approaches either perform coordinate-wise real-valued gradient-based learning, or entirely forgo training of hidden layers. Conventional coordinate-wise real-number calculus applied to quaternions fails to satisfy standard product or chain rules of calculus for quaternions, and tends to dissociate the relation between the non-scalar components of the quaternion. Consequently, pseudo-gradients are generated in place of quaternion differentials for encoding of error in the backpropagation algorithm.

