Semmelhack et al. (2014) have achieved high classification accuracy in distinguishing swim bouts of zebrafish using a Support Vector Machine (SVM). Convolutional Neural Networks (CNNs) have reached superior performance in various image recognition tasks over SVMs, but these powerful networks remain a black box. Reaching better transparency helps to build trust in their classifications and makes learned features interpretable to experts. Using a recently developed technique called Deep Taylor Decomposition, we generated heatmaps to highlight input regions of high relevance for predictions. We find that our CNN makes predictions by analyzing the steadiness of the tail's trunk, which markedly differs from the manually extracted features used by Semmelhack et al. (2014). We further uncovered that the network paid attention to experimental artifacts. Removing these artifacts ensured the validity of predictions. After correction, our best CNN beats the SVM by 6.12%, achieving a classification accuracy of 96.32%. Our work thus demonstrates the utility of AI explainability for CNNs. In the study by Semmelhack et al. (2014) , a well-performing classifier allowed to correlate neural interventions with behavioral changes. Support Vector Machines (SVMs) were commonly applied to such classification tasks, relying on feature engineering by domain experts. In recent years, Convolutional Neural Networks (CNNs) have proven to reach high accuracies in classification tasks on images and videos reducing the need for manual feature engineering. After Lecun & Bengio (1995) introduced them in the 90s, CNNs had their break-through in the competition ILSVRC2012 with the architecture of . Since then, more and more sophisticated architectures have been designed enabling them to identify increasingly abstract features. This development has become possible due to the availability of larger training sets, computing resources, GPU training implementations, and better regularization techniques, such as Dropout ; Zeiler & Fergus (2014) ). While these more complex deep neural network architectures achieved better results, they also kept their learnt features hidden if not further analyzed. This caused CNNs to come with significant drawbacks: a lack of trust in their classifications, missing interpretability of learned features in the application domain, and the absence of hints as to what data could enhance performance (Molnar (2019) ). Explaining the decisions made by CNNs might even become a legal requirement in certain applications (Alber et al. (2018) ). In order to overcome these drawbacks, subsequent research has developed approaches to shed light on the inner workings of CNNs. These approaches have been successfully used for uncovering how CNNs might learn unintended spurious correlations, termed "Clever Hans" predictions (Lapuschkin et al. (2019) ). Such predictions could even become harmful if the predictions entailed decisions with severe consequences (Leslie (2019) ). Also, since deep neural networks have become a popular machine learning technique in applied domains, spurious correlations would undermine scientific discoveries. This paper focuses on zebrafish research as an applied domain of AI explainability, considering that the research community around this organism has grown immensely. The zebrafish is an excellent model organism for vertebrates, including humans, due to the following four reasons: The genetic codes of humans and zebrafish are about 70% orthologue (Howe et al. (2013) ). The fish are translucent which allows non-invasive observation of changes in the organism (Bianco et al. (2011) ). Furthermore, zebrafish are relatively cheap to maintain, produce plenty of offspring, and develop rapidly. Finally, they are capable of recovering their brain structures within days after brain injury (Kishimoto et al. (2011) ; Kizil et al. (2012) ). In this paper, we adapt CNNs to work on highly controlled zebrafish video recordings and show the utility of a recently developed AI explainability technique on this task. We train the network on optical flow for binary classifying swim bouts and achieve superior performance when compared to the current state-of-the-art in bout classification (Semmelhack et al. (2014) ). We then create heatmaps over the videos with the "iNNvestigate" toolbox (Alber et al. (2018) ) which highlight the areas that our CNN pays attention to when making a prediction. The resulting heatmaps show that our CNN learns reasonable features which are very different from those manually composed by Semmelhack et al. (2014) . We trained a two-stream Convolutional Neural Network (CNN) on recordings of larval zebrafish to classify prey and spontaneous swim bouts. We then visualized the learned weights by generating relevance heatmaps showing which regions of the input the network focuses on while performing its classifications. We find that our CNN is capable of learning highly discriminating tail features. These features seem to be quite different from the ones used in the SVM classification by Semmelhack et al. (2014) -the previous state-of-the-art in bout classification. The heatmaps further uncovered a "Clever Hans" type of correlation. After removing this spurious correlation and retraining the network, the network reached a test accuracy of 96.32%, which is 6.12% points better than the accuracy achieved by Semmelhack et al. (2014) . Judging from the test accuracy, our CNN has learned better discriminating features than those used for the SVM by Semmelhack et al. (2014), and has thus beaten manual feature engineering in this application domain. Steadiness of the fish's trunk as differentiating feature. The relevance heatmaps and high accuracy show that the network achieves correct classifications by looking for salient features in the trunk of the tail while largely disregarding the tip. A sharp and clear relevance profile confined to the edges of the trunk gives a clear sign of a prey bout. The opposite speaks for a spontaneous bout. Here, attention spreads out to capture the strong vertical oscillation of the trunk. For this reason we conclude that the CNN makes its predictions based on the steadiness of the trunk. We believe our interpretation of learned features to be in line with existing research on the kinematics of prey bouts. As shown by Borla et al. (2002) and McElligott & O'Malley (2005) , prey bouts require fine control of the tail's axial kinematics to perform precise swim movements. Zebrafish noticeably reduce their yaw rotation and stabilize the positioning of their head to make a targeted move at their prey. Such precise movements are not required in spontaneous swim bouts. The heatmaps indicate that the network has found clear evidence for these kinds of motion in the trunk of the tail. Furthermore, we argue that the CNN has learned features which are very different from the ones identified by Semmelhack et al. (2014) . All of their features -as outlined in Section 2 -, except the second one, rely on information from the tip of the tail and a complete sequence of frames. However, many optical flow frames do not depict the tip of the tail because of its small size and high speed. This might have happened due to suboptimal parameter settings which could not handle the sometimes long distances which the tip traveled between frames. Also, subsamples include only 85 of the original 150 frames for each video. Due to its higher performance, we conclude not only that the CNN has learned a different set of features, but also that these features must bear higher discriminative power. Origin of the "Clever Hans" correlation. The telltale motion in the top left corner stems from a substance called agarose, which the fish's head was embedded in to keep it steady. It is quite curious that, while not visible to human eyes, the agarose seems to be moving each time the fish performed a spontaneous swim bout, but not so for a prey bout. We speculate that this correlation was unintentionally introduced by the experimenters who might have tapped the petri dish to induce the fish to perform a spontaneous swim bout. Future work. Calculating and storing optical flow is expensive. If we attained similar performance on original frames, training would be considerably cheaper. While we can confirm the findings by that the spatial stream by itself reaches a fairly competitive accuracy, it provides only very minor improvement to the overall network. Yet, this stream is probably looking for very similar features as the temporal stream, because it focuses largely on the upper half of the tail, just like the temporal stream. If that is the case, we should see improved performance when giving the spatial stream a sequence of frames. It should be interesting to probe whether the spatial stream could then match or even surpass the performance of the temporal stream. Furthermore, CNNs such as the one used in this paper could be used to investigate brain recovery in larval zebrafish. It has been shown on a cellular level that zebrafish can heal their brain within days after a lesion. However, this needs to be proven on a behavioral level (Krakauer et al. (2017) ). Future work could perform a lesion study on the optic tectum in zebrafish (McDowell et al. (2004) ; Roeser & Baier (2003) ), a brain region responsible for translating visual input into motor output. CNNs could then assess swim bouts of recovered fish and give a measure for potential behavioral changes. Insights from relevance heatmaps would be required if the CNN were not able not distinguish recovered fish from healthy ones.
We demonstrate the utility of a recent AI explainability technique by visualizing the learned features of a CNN trained on binary classification of zebrafish movements.
{"src": "scitldr-AIC_train", "paper_id": "rJgQkT4twH"}