<commit_before><!--
    This Source Code Form is subject to the terms of the Mozilla Public
    License, v. 2.0. If a copy of the MPL was not distributed with this
    file, You can obtain one at http://mozilla.org/MPL/2.0/.
-->

<!--
    Copyright (c) 2014, Joyent, Inc.
-->

# Marlin: Manta Compute Engine

Repository: <<EMAIL>:marlin.git>
Browsing: <https://mo.joyent.com/marlin>
Who: <NAME>
Docs: <https://mo.joyent.com/docs/marlin>
Tickets/bugs: <https://devhub.joyent.com/jira/browse/MANTA>


# Overview

For overview on the purpose, design, and usage of Marlin, see
docs/index.restdown (or https://mo.joyent.com/docs/marlin).

This document describes the development workflow.


# Quick start for using the tools

Since it's useful to run the Marlin client tools (mrjob, mrjobreport, mrerrors,
etc.) on an OS X laptop, the build system includes hacks to allow just those
bits to build on OS X.  To use these tools:

    $ git clone <EMAIL>:marlin.git
    ...<commit_msg>TOOLS-625 update README<commit_after><!--
    This Source Code Form is subject to the terms of the Mozilla Public
    License, v. 2.0. If a copy of the MPL was not distributed with this
    file, You can obtain one at http://mozilla.org/MPL/2.0/.
-->

<!--
    Copyright (c) 2014, Joyent, Inc.
-->

# manta-marlin

This repository is part of the Joyent Manta project.  For contribution
guidelines, issues, and general documentation, visit the main
[Manta](http://github.com/joyent/manta) project page.

Marlin is the Manta compute engine.  It comprises a service, an agent, and a set
of client tools for executing arbitrary user jobs on objects stored in Manta.


# Overview

For overview on the purpose, design, and usage of Marlin, see
docs/index.restdown (or https://mo.joyent.com/docs/marlin).

This document describes the development workflow.


# Quick start for using the tools

Since it's useful to run the Marlin client tools (mrjob, mrjobreport, mrerrors,
etc.) on an OS X laptop, the build system includes hacks to allow just those
bits to build on OS X.  To use these tools:

    $ git clone <EMAIL>:marlin.git
    ...
{'fraction_non_alphanumeric': 0.09919316563834836, 'fraction_numerical': 0.009017560512577124, 'mean_word_length': 3.4472573839662446, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4751401', 'n_tokens_mistral': 689, 'n_tokens_neox': 646, 'n_words': 281}