<commit_before># Open Transcoding Platform

### made by <NAME> for Emerson College (and the world)

The Open Transcoding Platform is a basic farm of nodes that can transcode video to different formats. People can upload video files, which are then transcoded into smaller web-friendly versions for HTML5/Flash Media Server playback.

There is one "master" and at least one "farmer". The master is the web frontend and controller. The farmers simply ask for files to transcode.

This is based entirely on Median's original transcode farm, which I wrote, except this lets you download the end results. This platform is more of an entry point with the difficult video transcoding part done for you, it's up to you to fit this into an existing system which would require it. It is important to note that there is no login system included with this, but there are hooks for it.

Median actually uses this software to transcode different versions of uploaded videos, utilizing 12+ farmers, which range from dedicated IBM Blades to vSphere VMs. I've included a "tier" system in the Farmer schema to differentiate fast encoders and slow encoders if you want to.

Also, just to give credit where it's due, the web interfaces uses the [Foundation framework](http://foundation.zurb.com/) to make it very browser-friendly.

## Technology Required

### For master

- Debian-based distro (Ubuntu 12.04 Server used in production)
- lighttpd 1.4.28+ (1.4.31 used in production)
- PHP 5.3+ (5.4.6 used in production)
- MongoDB 2.0+ (2.2.0 replica set used in production)
- node.js 0.8+ (0.8.8 used in production)
- npm 1.1+ (1.1.59 used in production)
- node module formidable 1.0+ (1.0.11 used in production)
- enough disk space to temporarily store files (250GB+ recommended)

### For farmer node

- Debian-based distro (Ubuntu 11.10 and 12.04 used in production)
- node.js 0.8+ (0.8.8 used in production)
- HandBrakeCLI 0.9+ (0.9.8 used in production)
- enough disk space to temporarily store files (10GB+ recommended)

## Installing

Detailed installation instructions can be found in the /docs/ folder.

## Learning More

Read all the information in the /docs/ folder.

## Features yet to be done

* end-user API (not the Farmer API)
* Help files for the web interface (are they needed?)<commit_msg>- being ridiculous with the readme file<commit_after># Open Transcoding Platform

### made by <NAME> for Emerson College (and the world)

The Open Transcoding Platform is a basic farm of nodes that can transcode video to different formats. People can upload video files, which are then transcoded into smaller web-friendly versions for HTML5/Flash Media Server playback. I built this because I couldn't find anything to do this well enough for my needs.

There is one "master" and at least one "farmer". The master is the web frontend and controller. The farmers simply ask for files to transcode.

This is based entirely on [Median](http://median.emerson.edu/)'s original transcode farm, which I wrote, except this lets you download the end results. This platform is more of an entry point with the difficult video transcoding part done for you, it's up to you to fit this into an existing system which would require it. It is important to note that there is no login system included with this, but there are hooks for it.

Median actually uses this software to transcode different versions of uploaded videos, utilizing 12+ farmers, which range from dedicated IBM Blades to vSphere VMs. I've included a "tier" system in the Farmer schema to differentiate fast encoders and slow encoders if you want to.

## Technology Required

### For master

- Debian-based distro (Ubuntu 12.04 Server used in production)
- lighttpd 1.4.28+ (1.4.31 used in production)
- PHP 5.3+ (5.4.6 used in production)
- MongoDB 2.0+ (2.2.0 replica set used in production)
- node.js 0.8+ (0.8.8 used in production)
- npm 1.1+ (1.1.59 used in production)
- node module formidable 1.0+ (1.0.11 used in production)
- enough disk space to temporarily store files (250GB+ recommended)

### For farmer node

- Debian-based distro (Ubuntu 11.10 and 12.04 used in production)
- node.js 0.8+ (0.8.8 used in production)
- HandBrakeCLI 0.9+ (0.9.8 used in production)
- enough disk space to temporarily store files (10GB+ recommended)

## Installing

Detailed installation instructions can be found in the /docs/ folder.

## Learning More

Read all the information in the /docs/ folder.

## Technology used

- the web interface uses the [Foundation framework](http://foundation.zurb.com/) and jQuery to make it very browser-friendly
- the upload script uses [Formidable](https://github.com/felixge/node-formidable) for huge file uploads
- the master server uses lighttpd, nodejs, mongodb, haproxy, ffmpeg, all of which are awesome projects

## Features yet to be done

* end-user API (not the Farmer API)
* Help files for the web interface (are they needed?)
{'fraction_non_alphanumeric': 0.0717050315746588, 'fraction_numerical': 0.026481971888368303, 'mean_word_length': 4.0880829015544045, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1806681', 'n_tokens_mistral': 1434, 'n_tokens_neox': 1318, 'n_words': 729}