Choice of Activation and Optimisation Functions. 
The ReLU activation function was chosen over other activation functions, as the
reduced likelihood of vanishing gradient allows for a constant gradient
resulting in faster learning. Further, the sparsity of representations are shown
to be more beneficial than dense representations, as seen in other activations
such as sigmoids [1]}, [2]}, [3]}.
The softmax activation function, combined with categorical
cross-entropy [4]} for the loss function, was chosen due
to the simple fact that this is a multi-class classification problem. Simply, a
sample can belong to one of the 7 classes, with each class corresponding to one
of the robot movements. As well as this, the Adam optimiser was an ideal
candidate. It is an extension to the Stochastic Gradient Descent (SGD) method,
based on adaptive estimation of first- and second-order
moments [5]}. Specifically, it allows for the updating of
network weights iteratively based on the training data, and fits best with the
weighted sample sets in opposition to other tried methods such as standard SGD,
RMSProp and SGD + Nesterov Momentum.
