<commit_before>---
layout:			basic
title:			"Swara"
subtitle:       "Musical machine learning"
category:		"Projects"
date:			2016-09-23 04:20:00
author:			"<NAME>"
header-img:		"img/swara-bg.jpg"
---

# Swara
[Swara](https://en.wikipedia.org/wiki/Swara) is my attempt at bringing musical machine learning to the masses. My ultimate objective is to build exciting musical applications such as algorithmic composition, fingerprinting, and musical translation. This is a current area of active research conducted by the likes of [Google Magenta](https://magenta.tensorflow.org/welcome-to-magenta) and [Jukedeck](https://www.jukedeck.com/). In this article, I'll describe my approach to the development of these musical applications and the various musical and machine learning models that enable them. The code is publicly available on [Github](https://github.com/ashwin153/swara).

# Musical Modeling
The first goal of Swara was to develop ```swara-music```, a library for programmatically representing music. This library acts as a liason between musicians and mathematicians; it provides mathematicians access to the various musical elements of a song without obfuscating their underlying musical meaning. In other words, the library provides a convenient representation for songs that both a mathematician and a musician can understand and appreciate. For example, a waltz tempo can be constructed using the library in the following manner.

```java
// Waltz Tempo.
final Tempo waltz = new Tempo.Builder()
    .withSignature(3, 4)
    .withBpm(88)
    .build();
```

### Terminology
The ```swara-music``` library currently supports the following musical elements: ```Song```, ```Fragment```, ```Key```, ```Tempo```, ```Phrase```, ```Voice```, ```Chord```, and ```Note```. These simple musical primitives can be combined to form highly complex musical arrangements. For example, this [code](https://gist.github.com/ashwin153/d86292dbfc346b48d7e8f9e79db463fd) produces the following fragment of sheet music (rendered using [MuseScore 2](https://musescore.org/en/2.0)).

<img align="center" src="/img/sample-song.png"/>

This simple song fragment highlights some of the cool features of the ```swara-music``` library including: key and tempo changes, polyphony, multiple instruments, dynamics, and accidentals. The real beauty is in the code snippet that generated the song fragment - it reads just like music! 

### I/O
In order to facilitate interoperability, the ```swara-music``` library exposes the ```MusicReader``` and ```MusicWriter``` interfaces for writing and reading musical elements to and from various file types. The library currently supports MIDI and JSON (for python/js interoperability).

### Future Work
Under current development is support for the extraction of various musical features. Musical features are derived attributes of music such as genre and melody that are superfluous to the construction of the music itself. For example, a song is _composed_ of notes and the notes _form_ a chord progression. In this example, the chord progression is a derived property of the notes themselves and is a musical feature. Current musical features under development include ```Genre```, ```Dynamics```, ```Harmony```, ```Melody```, and ```Rhythm```.

# Machine Learning
The second goal of Swara was to write generalized machine learning models built on [Apache Commons Math](http://commons.apache.org/proper/commons-math/). The ```swara-learn``` library provides a set of machine learning models built in Java from first principles. **Disclaimer:** these implementations make no claim to be high-performance. In fact, I'm certain there are far faster and far more memory efficient versions of the models I created. This library is simply the result of geniue curiosity into the inner-workings of artificial intelligence. I may have written all the code, but this is by no means a solo effort. Special thanks to [Stanford CS-229](http://cs229.stanford.edu) for its informative and rigorous explanations. Under current development are the following machine learning models.

### Markov Models
- ```DiscreteMarkovChain```
- ```HiddenMarkovModel```

### Neural Networks
- ```ForwardLayer```
- ```LstmLayer```
- ```ConvolutionalLayer```
- ```PoolingLayer```

# Applications
The third goal of Swara was to use the ```swara-music``` and ```swara-learn``` libraries to build exciting musical technologies. Under current development are the following applications in the ```swara-core``` library:

### Algorithmic Composition
Algorithmic composition is the application of machine learning techniques to generating original scores of music. Under current development are: a ```MarkovComposer``` built on Discrete Markov Chains, a ```NeuralComposer``` built on LSTM Neural Networks, and a ```RandomComposer``` built for benchmarking the other methods.

### Fingerprinting
Fingerprinting is the processes of generating a musical fingerprint; or a set of descriptors that uniquely define a piece of music. This musical fingerprint can then be used to perform musical identification and search. Under current development is the ```ShazamFingerprinter``` built off the [Shazam](http://www.ee.columbia.edu/~dpwe/papers/Wang03-shazam.pdf) paper.

### Musical Translation
Musical translation is the application of machine learning techniques to extracting musical information from audio input sources. It is the musical analogue of speech-to-text. Under current development is the ```MarkovTranslator``` built on Hidden Markov Models.

Cover photograph thanks to the lord himself [Kanye West](http://wallpapercave.com/kanye-west-graduation-wallpaper). <commit_msg>Fixed language<commit_after>---
layout:layoutbasic
title:layout"Swara"
subtitle:       "Musical machine learning"
category:learning"Projects"
date:learning2016-09-23 04:20:00
author:20"<NAME>"
header-img:header"img/swara-bg.jpg"
---

# Swara
[Swara](https://en.wikipedia.org/wiki/Swara) is my attempt at bringing musical machine learning to the masses. My ultimate objective is to build exciting musical applications such as algorithmic composition, fingerprinting, and musical translation. This is an area of active research (see, for example, [Google Magenta](https://magenta.tensorflow.org/welcome-to-magenta) and [Jukedeck](https://www.jukedeck.com/)). In this article, I'll describe my approach to the development of these musical applications and the various musical and machine learning models that enable them. The code is publicly available on [Github](https://github.com/ashwin153/swara).

# Musical Modeling
The first goal of Swara was to develop ```swara-music```, a library for programmatically representing music. The library provides a convenient representation for songs that both a mathematician and a musician can understand and appreciate. It acts as a liason between mathematicians and musicians by providing the former with access to the various musical elements of a song without obfuscating their underlying musical meaning. For example, a waltz tempo can be constructed using the library in the following manner.

```java
// Waltz Tempo.
final Tempo waltz = new Tempo.Builder()
    .withSignature(3, 4)
	    .withBpm(88)
		    .build();
			```

### Terminology
The ```swara-music``` library currently supports the following musical elements: ```Song```, ```Fragment```, ```Key```, ```Tempo```, ```Phrase```, ```Voice```, ```Chord```, and ```Note```. These simple musical primitives can be combined to form highly complex musical arrangements. For example, this [code](https://gist.github.com/ashwin153/d86292dbfc346b48d7e8f9e79db463fd) produces the following fragment of sheet music (rendered using [MuseScore 2](https://musescore.org/en/2.0)).

<img align="center" src="/img/sample-song.png"/>

This simple song fragment highlights some of the cool features of the ```swara-music``` library including: key and tempo changes, polyphony, multiple instruments, dynamics, and accidentals. The real beauty is in the code snippet that generated the song fragment - it reads just like music! 

### I/O
In order to facilitate interoperability, the ```swara-music``` library exposes the ```MusicReader``` and ```MusicWriter``` interfaces for writing and reading musical elements to and from various file types. The library currently supports MIDI and JSON (for python/js interoperability).

### Future Work
Under current development is support for the extraction of various musical features. Musical features are derived attributes of music such as genre and melody that are superfluous to the construction of the music itself. For example, a song is _composed_ of notes and the notes _form_ a chord progression. In this example, the chord progression is a derived property of the notes themselves and is a musical feature. Current musical features under development include ```Genre```, ```Dynamics```, ```Harmony```, ```Melody```, and ```Rhythm```.

# Machine Learning
The second goal of Swara was to write generalized machine learning models built on [Apache Commons Math](http://commons.apache.org/proper/commons-math/). The ```swara-learn``` library provides a set of machine learning models built in Java from first principles. **Disclaimer:** these are  not high-performance implementations; in fact, I'm more than certain that faster and more memory-efficient versions of these models exist. This library is simply the result of my genuine curiosity into the inner workings of artificial intelligence. Although I have written all the code, this is by no means a solo effort. Special thanks are due to [Stanford CS-229](http://cs229.stanford.edu) for its informative and rigorous explanations. Under current development are the following machine learning models:

### Markov Models
- ```DiscreteMarkovChain```
- ```HiddenMarkovModel```

### Neural Networks
- ```ForwardLayer```
- ```LstmLayer```
- ```ConvolutionalLayer```
- ```PoolingLayer```

# Applications
The third goal of Swara was to use the ```swara-music``` and ```swara-learn``` libraries to build exciting musical technologies. Under current development are the following applications in the ```swara-core``` library:

### Algorithmic Composition
Algorithmic composition is the application of machine learning techniques to generating original scores of music. Under current development are: a ```MarkovComposer``` built on Discrete Markov Chains, a ```NeuralComposer``` built on LSTM Neural Networks, and a ```RandomComposer``` built for benchmarking the other methods.

### Fingerprinting
Fingerprinting is the processes of generating a musical fingerprint; or a set of descriptors that uniquely define a piece of music. This musical fingerprint can then be used to perform musical identification and search. Under current development is the ```ShazamFingerprinter``` built off the [Shazam](http://www.ee.columbia.edu/~dpwe/papers/Wang03-shazam.pdf) paper.

### Musical Translation
Musical translation is the application of machine learning techniques to extracting musical information from audio input sources. It is the musical analogue of speech-to-text. Under current development is the ```MarkovTranslator``` built on Hidden Markov Models.

Cover photograph courtesy of the Lord himself [Kanye West](http://wallpapercave.com/kanye-west-graduation-wallpaper). 
{'fraction_non_alphanumeric': 0.09335219236209336, 'fraction_numerical': 0.009547383309759547, 'mean_word_length': 5.281510272071071, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 12, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '327235', 'n_tokens_mistral': 2973, 'n_tokens_neox': 2802, 'n_words': 1373}