<commit_before>Hadoop-BAM: a library for manipulation of BAM (Binary Alignment/Map) and
BGZF-compressed files using the Hadoop MapReduce framework, and command line
tools in the vein of SAMtools.

------------
Dependencies
------------

Hadoop 0.20.2 or later in the 0.20 or 1.0 series. The unstable 0.21, 0.22, and
0.23 releases will not work with this version of Hadoop-BAM.

Picard SAM-JDK. Version 1.56 is provided in the form of sam-1.56.jar and
picard-1.56.jar. Later versions may also work but have not been tested.

Availability:
	Hadoop - http://hadoop.apache.org/
	Picard - http://picard.sourceforge.net/

In order to use all the functionality of Hadoop-BAM, you need to have Picard's
"sam-1.56.jar" and "picard-1.56.jar" (assuming version 1.56) and Hadoop's
"hadoop-0.20.2-core.jar" (assuming version 0.20.2) in the CLASSPATH environment
variable. For running under Hadoop, the Picard .jars should also be in the
HADOOP_CLASSPATH setting in hadoop-env.sh.

------------
Installation
------------

A precompiled "hadoop-bam.jar" is provided. You may also build it yourself
using the commands below.

The easiest way to compile Hadoop-BAM is to use Apache Ant (version 1.7.1 or
greater) with the following command and all relevant .jar files in CLASSPATH:

	ant jar

Note that this building step depends on an additional .jar file, provided by
Hadoop as "lib/commons-cli-1.2.jar" (assuming Hadoop 0.20.2). Ordinarily, when
using Hadoop-BAM under Hadoop, this need not be provided in CLASSPATH or
HADOOP_CLASSPATH, since Hadoop loads it by itself; but compilation does not
happen under Hadoop, so here it is necessary.

The previous command will create the 'hadoop-bam.jar' file. For Javadoc
documentation, run:

	ant javadoc

Documentation can then be found in the "doc" subdirectory.

-------------
Library usage
-------------

Hadoop-BAM provides the standard set of Hadoop file format classes for BAM
files: a FileInputFormat and a RecordReader, and a FileOutputFormat and a
RecordWriter. These are BAMInputFormat, BAMRecordReader, BAMOutputFormat, and
BAMRecordWriter, respectively.

Note that Hadoop-BAM is based around the newer Hadoop API introduced in the
0.20 Hadoop releases instead of the older, deprecated API.

See the Javadoc as well as the command line plugins' source code (in
src/fi/tkk/ics/hadoop/bam/cli/plugins/*.java) for more information. In
particular, for MapReduce usage examples, see for example
src/fi/tkk/ics/hadoop/bam/cli/plugins/Sort.java and
src/fi/tkk/ics/hadoop/bam/cli/plugins/chipster/Summarize.java.

------------------
Command-line usage
------------------

Hadoop-BAM can be used as a command-line tool, with functionality in the form
of plugins that provide commands to which hadoop-bam.jar is a frontend.
Hadoop-BAM provides some commands of its own, but any others found in the Java
class path will be used as well.

Running under Hadoop
....................<commit_msg>README: redo installation section, move .jar info<commit_after>Hadoop-BAM: a library for manipulation of BAM (Binary Alignment/Map) and
BGZF-compressed files using the Hadoop MapReduce framework, and command line
tools in the vein of SAMtools.

------------
Dependencies
------------

Hadoop 0.20.2 or later in the 0.20 or 1.0 series. The unstable 0.21, 0.22, and
0.23 releases will not work with this version of Hadoop-BAM.

Picard SAM-JDK. Version 1.56 is provided in the form of sam-1.56.jar and
picard-1.56.jar. Later versions may also work but have not been tested.

Availability:
   Hadoop - http://hadoop.apache.org/
   Picard - http://picard.sourceforge.net/

------------
Installation
------------

A precompiled "hadoop-bam.jar" is provided. You may also build it yourself
using the commands below.

The easiest way to compile Hadoop-BAM is to use Apache Ant (version 1.7.1 or
greater) with the following simple command:

   ant

Note that for this to work, either the HADOOP_HOME environment variable should
be set to point to the main Hadoop directory, or the relevant .jar files should
be in the CLASSPATH environment variable. If one wants to use CLASSPATH
directly, it should contain the following .jars:

   - hadoop-0.20.2-core.jar   For Hadoop 0.20.2; the appropriate core .jar file
                              for other releases.

   - sam-1.56.jar             Part of Picard, provided in this Hadoop-BAM
                              distribution; later versions may or may not work.

   - picard-1.56.jar          Ditto.

   - commons-cli-1.2.jar      Apache Commons CLI; provided by Hadoop in its
                              lib/ subdirectory. Hadoop loads it when used so
                              this should never be necessary when using
                              Hadoop-BAM, but it is required in this build
                              step.

The previous command will create the 'hadoop-bam.jar' file. For Javadoc
documentation, run:

   ant javadoc

Documentation can then be found in the "doc" subdirectory.

Finally, to run tests, which require version 4 of the JUnit test framework
(available at http://junit.sourceforge.net/), run:

   ant run-tests

-------------
Library usage
-------------

Hadoop-BAM provides the standard set of Hadoop file format classes for BAM
files: a FileInputFormat and a RecordReader, and a FileOutputFormat and a
RecordWriter. These are BAMInputFormat, BAMRecordReader, BAMOutputFormat, and
BAMRecordWriter, respectively.

Note that Hadoop-BAM is based around the newer Hadoop API introduced in the
0.20 Hadoop releases instead of the older, deprecated API.

See the Javadoc as well as the command line plugins' source code (in
src/fi/tkk/ics/hadoop/bam/cli/plugins/*.java) for more information. In
particular, for MapReduce usage examples, see for example
src/fi/tkk/ics/hadoop/bam/cli/plugins/Sort.java and
src/fi/tkk/ics/hadoop/bam/cli/plugins/chipster/Summarize.java.

------------------
Command-line usage
------------------

Hadoop-BAM can be used as a command-line tool, with functionality in the form
of plugins that provide commands to which hadoop-bam.jar is a frontend.
Hadoop-BAM provides some commands of its own, but any others found in the Java
class path will be used as well.

Running under Hadoop
....................
{'fraction_non_alphanumeric': 0.10173299101412067, 'fraction_numerical': 0.01700898587933248, 'mean_word_length': 2.859442724458204, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6829423', 'n_tokens_mistral': 1918, 'n_tokens_neox': 1779, 'n_words': 790}