<commit_before># Machi

  [Travis-CI](http://travis-ci.org/basho/machi) :: ![Travis-CI](https://secure.travis-ci.org/basho/machi.png)

Our goal is a robust & reliable, distributed, highly available(*),
large file store based upon write-once registers, append-only files,
Chain Replication, and client-server style architecture.  All members
of the cluster store all of the files.  Distributed load
balancing/sharding of files is __outside__ of the scope of this
system.  However, it is a high priority that this system be able to
integrate easily into systems that do provide distributed load
balancing, e.g., Riak Core.  Although strong consistency is a major
feature of Chain Replication, first use cases will focus mainly on
eventual consistency features --- strong consistency design will be
discussed in a separate design document (read more below).

The ability for Machi to maintain strong consistency will make it
attractive as a toolkit for building things like CORFU and Tango as
well as better-known open source software such as Kafka's file
replication.  (See the bibliography of the [Machi high level design
doc](./doc/high-level-machi.pdf) for further references.)

(*) Capable of operating in "AP mode" or "CP mode" relative to the
  CAP Theorem.

## Status: mid-June 2015: work is underway

The two major design documents for Machi are now ready or nearly ready
for internal Basho and external party review.  Please see the
[doc](./doc) directory's [README](./doc) for details

* Machi high level design
* Machi chain self-management design

The work of implementing first draft of Machi is now underway.  The
code from the [prototype/demo-day-hack](prototype/demo-day-hack/) directory is
being used as the initial scaffolding.

* The chain manager is ready for "AP mode" use in eventual
  consistency use cases.

* All Machi client/server protocols are based on
  [Protocol Buffers](https://developers.google.com/protocol-buffers/docs/overview).
    * The current specification for Machi's protocols can be found at
      [https://github.com/basho/machi/blob/master/src/machi.proto](https://github.com/basho/machi/blob/master/src/machi.proto).
    * The Machi PB protocol is not yet stable.  Expect change!
    * The Erlang language client implementation of the high-level
      protocol flavor is very brittle (e.g., very little error
      handling yet).
    * The Erlang language client implementation of the low-level
      protocol flavor are still a work-in-progress ... but they are
      more robust than the high-level library's implementation.

If you'd like to work on a protocol such as Thrift, UBF,
msgpack over UDP, or some other protocol, let us know by
[opening an issue](./issues/new) to discuss it.

## Contributing to Machi: source code, documentation, etc.

Basho Technologies, Inc. as committed to licensing all work for Machi
under the
[Apache Public License version 2](./LICENSE).  All authors of source code
and documentation who agree with these licensing terms are welcome to
contribute their ideas in any form: suggested design or features,
documentation, and source code.

Machi is still a very young project within Basho, with a small team of
developers; please bear with us as we grow out of "toddler" stage into
a more mature open source software project.
We invite all contributors to review the
[CONTRIBUTING.md](./CONTRIBUTING.md) document for guidelines for
working with the Basho development team.

## A brief survey of this directories in this repository

* A list of Frequently Asked Questions, a.k.a.<commit_msg>Update README.me for mid-Oct 2015<commit_after># Machi

  [Travis-CI](http://travis-ci.org/basho/machi) :: ![Travis-CI](https://secure.travis-ci.org/basho/machi.png)

Our goal is a robust & reliable, distributed, highly available(*),
large file store based upon write-once registers, append-only files,
Chain Replication, and client-server style architecture.  All members
of the cluster store all of the files.  Distributed load
balancing/sharding of files is __outside__ of the scope of this
system.  However, it is a high priority that this system be able to
integrate easily into systems that do provide distributed load
balancing, e.g., Riak Core.  Although strong consistency is a major
feature of Chain Replication, first use cases will focus mainly on
eventual consistency features --- strong consistency design will be
discussed in a separate design document (read more below).

The ability for Machi to maintain strong consistency will make it
attractive as a toolkit for building things like CORFU and Tango as
well as better-known open source software such as Kafka's file
replication.  (See the bibliography of the [Machi high level design
doc](./doc/high-level-machi.pdf) for further references.)

    (*) When operating in strong consistency mode (supporting
    sequential or linearizable semantics), the availability of the
    system is restricted to quorum majority availability.  When in
    eventual consistency mode, service can be provided by any
    available server.

## Status: mid-October 2015: work is underway

* The chain manager is ready for both eventual consistency use ("AP
  mode") and strong constency use ("CP mode").  Both modes use a new
  consensus technique, Humming Consensus.
    * Scott will be
      [http://ricon.io/agenda/#managing-chain-replication-metadata-with-humming-consensus](speaking about Humming Consensus)
      at the [http://ricon.io](Ricon 2015 conference) in San Francisco,
      CA, USA on Thursday, November 5th, 2015.
    * Implementation of the file repair process for strong consistency
      is still in progress.

* All Machi client/server protocols are based on
  [Protocol Buffers](https://developers.google.com/protocol-buffers/docs/overview).
    * The current specification for Machi's protocols can be found at
      [https://github.com/basho/machi/blob/master/src/machi.proto](https://github.com/basho/machi/blob/master/src/machi.proto).
    * The Machi PB protocol is not yet stable.  Expect change!
    * The Erlang language client implementation of the high-level
      protocol flavor is brittle (e.g., little error handling yet).

If you'd like to work on a protocol such as Thrift, UBF,
msgpack over UDP, or some other protocol, let us know by
[opening an issue to discuss it](./issues/new).

The two major design documents for Machi are now mostly stable.
Please see the [doc](./doc) directory's [README](./doc) for details.

## Contributing to Machi: source code, documentation, etc.

Basho Technologies, Inc. as committed to licensing all work for Machi
under the
[Apache Public License version 2](./LICENSE).  All authors of source code
and documentation who agree with these licensing terms are welcome to
contribute their ideas in any form: suggested design or features,
documentation, and source code.

Machi is still a very young project within Basho, with a small team of
developers; please bear with us as we grow out of "toddler" stage into
a more mature open source software project.
We invite all contributors to review the
[CONTRIBUTING.md](./CONTRIBUTING.md) document for guidelines for
working with the Basho development team.

## A brief survey of this directories in this repository

* A list of Frequently Asked Questions, a.k.a.
{'fraction_non_alphanumeric': 0.06974513565360373, 'fraction_numerical': 0.0031515483694162785, 'mean_word_length': 4.1473906911142455, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '273557', 'n_tokens_mistral': 2001, 'n_tokens_neox': 1907, 'n_words': 957}