<commit_before># Graphalytics MapReduce V2 platform extension


## Getting started

Please refer to the documentation of the Graphalytics core (`graphalytics` repository) for an introduction to using Graphalytics.


## MapReduce V2-specific benchmark configuration

The `mapreducev2` benchmark runs on Hadoop version 2.4.1 or later (may work for earlier versions, this has not been verified). Before launching the benchmark, configure your Hadoop cluster to operate in pseudo-distributed or distributed mode. You must also increase the maximum number of job counters by setting the following property in `$HADOOP_CONF_DIR/mapred-site.xml`:

 - `mapreduce.job.counters.limit`: Set this to a large value, e.g. `100000`.

Next, edit the `mapreducev2`-specific configuration file for Graphalytics, `config/mapreducev2.properties`, and change the following settings:

 - `mapreducev2.reducer-count`: Set to an appropriate number of reducers for your Hadoop deployment (note: variable number of reducers per graph/algorithm is not yet supported).
 - `hadoop.home`: Set to the root of your Hadoop installation (`$HADOOP_HOME`).

Ensure that Hadoop is running before starting the benchmark.
<commit_msg>[meta] Add Jenkins build status to README<commit_after># Graphalytics MapReduce V2 platform extension

[![Build Status](http://jenkins.tribler.org/buildStatus/icon?job=Graphalytics_MapReduceV2_master_tester)](http://jenkins.tribler.org/job/Graphalytics_MapReduceV2_master_tester/)


## Getting started

Please refer to the documentation of the Graphalytics core (`graphalytics` repository) for an introduction to using Graphalytics.


## MapReduce V2-specific benchmark configuration

The `mapreducev2` benchmark runs on Hadoop version 2.4.1 or later (may work for earlier versions, this has not been verified). Before launching the benchmark, configure your Hadoop cluster to operate in pseudo-distributed or distributed mode. You must also increase the maximum number of job counters by setting the following property in `$HADOOP_CONF_DIR/mapred-site.xml`:

 - `mapreduce.job.counters.limit`: Set this to a large value, e.g. `100000`.

Next, edit the `mapreducev2`-specific configuration file for Graphalytics, `config/mapreducev2.properties`, and change the following settings:

 - `mapreducev2.reducer-count`: Set to an appropriate number of reducers for your Hadoop deployment (note: variable number of reducers per graph/algorithm is not yet supported).
 - `hadoop.home`: Set to the root of your Hadoop installation (`$HADOOP_HOME`).

Ensure that Hadoop is running before starting the benchmark.

{'fraction_non_alphanumeric': 0.07162110127069696, 'fraction_numerical': 0.012321909896033885, 'mean_word_length': 5.352078239608802, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4566652', 'n_tokens_mistral': 740, 'n_tokens_neox': 682, 'n_words': 300}