State of the art sound event classification relies in neural networks to learn the associations between class labels and audio recordings within a dataset. These datasets typically define an ontology to create a structure that relates these sound classes with more abstract super classes. Hence, the ontology serves as a source of domain knowledge representation of sounds. However, the ontology information is rarely considered, and specially under explored to model neural network architectures.
 We propose two ontology-based neural network architectures for sound event classification. We defined a framework to design simple network architectures that preserve an ontological structure. The networks are trained and evaluated using two of the most common sound event classification datasets. Results show an improvement in classification performance demonstrating the benefits of including the ontological information. Humans can identify a large number of sounds in their environments e.g., a baby crying, a wailing ambulance siren, microwave bell. These sounds can be related to more abstract categories that aid interpretation e.g., humans, emergency vehicles, home. These relations and structures can be represented by ontologies BID0 , which are defined for most of the available datasets for sound event classification (SEC). However, sound event classification rarely exploits this additional available information. Moreover, although neural networks are the state of the art for SEC BID1 BID2 BID3 , they are rarely designed considering such ontologies.An ontology is a formal representation of domain knowledge through categories and relationships that can provide structure to the training data and the neural network architecture. The most common type of ontologies are based on abstraction hierarchies defined by linguistics, where a super category represents its subcategories. Generally, the taxonomies are defined by either nouns or verbs e.g., animal contains dog and cat, dog contains dog barking and dog howling. Examples of datasets are ESC-50 BID4 , UrbanSounds BID5 , DCASE BID6 , AudioSet BID7 . Another taxonomy can be defined by interactions between objects and materials, actions and descriptors e.g., contains Scraping, which contains Scraping Rapidly and Scraping a Board BID8 BID9 BID10 . Another example of this type is given by physical properties, such as frequency and time patterns BID11 BID12 BID13 . There are multiple benefits of considering hierarchical relations in sound event classifiers. They can allow the classifier to back-off to more general categories when encountering ambiguity among subcategories. They can disambiguate classes that are acoustically similar, but not semantically. They can be used to penalize classification differently, where miss classifying sounds from different super classes is worse than within the same super class. Lastly, they can be used as domain knowledge to model neural networks. In fact, ontological information has been evaluated in computer vision BID14 and music BID15 , but has rarely been used for sound event classification.Ontology-based network architectures have showed improvement in performance along with other benefits. Authors in BID16 proposed an ontology-based deep restricted Boltzmann machine for textual topic classification. The architecture replicates the tree-like structure adding intermediate layers to model the transformation from a super class to its sub classes. Authors showed improved performance and reduced overfitting in training data. Another example used a perceptron for each node of the hierarchy, which classified whether an image corresponded to such class or not BID17 . Authors showed an improvement in performance due to the ability of class disambiguation by comparing predictions of classes and sub classes. Motivated by these approaches and by the flexibility to adapt structures in a deep learning model we propose our ontology-based networks detailed in the following section. In this paper we proposed a framework to design neural networks for sound event classification using hierarchical ontologies. We have shown two methods to add such structure into deep learning models in a simple manner without adding more learnable parameters. We used a Feed-forward Network with an ontological layer to relate predictions of different levels in the hierarchy. Additionally, we proposed a Siamese neural Network to compute ontology-based embeddings to preserve the ontology in an embedding space. The embeddings plots showed clusters of super classes containing different sub classes. Our results in the datasets and MSoS challenge improved over the baselines. We expect that our results pave the path to further explore ontologies and other relations, which is fundamental for sound event classification due to wide acoustic diversity and limited lexicalized terms to describe sounds.
We present ontology-based neural network architectures for sound event classification.
{"src": "scitldr-AIC_train", "paper_id": "HkGv2NMTjQ"}