<commit_before># What is XStream?

[![Gitter](https://badges.gitter.im/Join Chat.svg)](https://gitter.im/benas/xstream?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[![MIT license](http://img.shields.io/badge/license-MIT-brightgreen.svg?style=flat)](http://opensource.org/licenses/MIT)
[![Maven Central](https://maven-badges.herokuapp.com/maven-central/io.github.benas/xstream/badge.svg?style=flat)](http://search.maven.org/#artifactdetails|io.github.benas|xstream|0.1|)
[![Build Status](https://travis-ci.org/benas/xstream.svg?branch=master)](https://travis-ci.org/benas/xstream)

XStream (uniX Stream) is an extension of the Java 8 Stream API to process data pipelines the Unix way.
It provides a set of components that mimic Unix processing commands.

# Features

* 100% compatible with Java 8 Streams
* Intuitive, flexible and extensible API
* A toolbox of reusable components
* No dependencies
* Free and open source

# How to use it?

You can use XStream in two ways:

#### 1. Either unixifiy your stream and process it the unix way:

```java
Stream<String> stream = Stream.of("foo", "bar", "bar", "baz");

XStream.unixify(stream)
        .grep("a")
        .sort()
        .uniq()
        .nl()
        .to(stdOut());
        
// prints:
// 1 bar
// 2 baz
```

#### 2. Or write your pipelines as you read them

```java
// cat input.txt | grep a | sort | uniq | nl > output.txt

XStream.cat("input.txt")
        .pipe(grep("a"))
        .pipe(sort())
        .pipe(uniq())
        .pipe(nl())
        .to(file("output.txt"));
```

# Where to find it?

Add the following maven dependency to your project:

 ```xml
<dependency>
    <groupId>io.github.benas</groupId>
    <artifactId>xstream</artifactId>
    <version>0.1</version>
</dependency>
 ```

Or [download the jar file](https://oss.sonatype.org/content/groups/public/io/github/benas/xstream/0.1/xstream.jar) and add it to your application's classpath.

# Components library

XStream provides a toolbox of reusable components that mimic Unix commands (and more).
Components are inspired by the [Unix philosophy](https://en.wikipedia.org/wiki/Unix_philosophy#Mike_Gancarz:_The_UNIX_Philosophy) and are intended to be:

* Small
* Portable
* Do one thing and do it well
* Side-effect free

Here are some of the built-in components:

![](https://github.com/benas/xstream/raw/master/xstream.jpeg)

You can find a complete reference of components in the [wiki page](https://github.com/benas/xstream/wiki).

# How to extend it ?

The `Stage` interface represents a stage of the pipeline:

```java<commit_msg>fix download link<commit_after># What is XStream?

[![Gitter](https://badges.gitter.im/Join Chat.svg)](https://gitter.im/benas/xstream?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[![MIT license](http://img.shields.io/badge/license-MIT-brightgreen.svg?style=flat)](http://opensource.org/licenses/MIT)
[![Maven Central](https://maven-badges.herokuapp.com/maven-central/io.github.benas/xstream/badge.svg?style=flat)](http://search.maven.org/#artifactdetails|io.github.benas|xstream|0.1|)
[![Build Status](https://travis-ci.org/benas/xstream.svg?branch=master)](https://travis-ci.org/benas/xstream)

XStream (uniX Stream) is an extension of the Java 8 Stream API to process data pipelines the Unix way.
It provides a set of components that mimic Unix processing commands.

# Features

* 100% compatible with Java 8 Streams
* Intuitive, flexible and extensible API
* A toolbox of reusable components
* No dependencies
* Free and open source

# How to use it?

You can use XStream in two ways:

#### 1. Either unixifiy your stream and process it the unix way:

```java
Stream<String> stream = Stream.of("foo", "bar", "bar", "baz");

XStream.unixify(stream)
        .grep("a")
        .sort()
        .uniq()
        .nl()
        .to(stdOut());
        
// prints:
// 1 bar
// 2 baz
```

#### 2. Or write your pipelines as you read them

```java
// cat input.txt | grep a | sort | uniq | nl > output.txt

XStream.cat("input.txt")
        .pipe(grep("a"))
        .pipe(sort())
        .pipe(uniq())
        .pipe(nl())
        .to(file("output.txt"));
```

# Where to find it?

Add the following maven dependency to your project:

 ```xml
<dependency>
    <groupId>io.github.benas</groupId>
    <artifactId>xstream</artifactId>
    <version>0.1</version>
</dependency>
 ```

Or [download the jar file](https://oss.sonatype.org/content/groups/public/io/github/benas/xstream/0.1/xstream-0.1.jar) and add it to your application's classpath.

# Components library

XStream provides a toolbox of reusable components that mimic Unix commands (and more).
Components are inspired by the [Unix philosophy](https://en.wikipedia.org/wiki/Unix_philosophy#Mike_Gancarz:_The_UNIX_Philosophy) and are intended to be:

* Small
* Portable
* Do one thing and do it well
* Side-effect free

Here are some of the built-in components:

![](https://github.com/benas/xstream/raw/master/xstream.jpeg)

You can find a complete reference of components in the [wiki page](https://github.com/benas/xstream/wiki).

# How to extend it ?

The `Stage` interface represents a stage of the pipeline:

```java
{'fraction_non_alphanumeric': 0.14450867052023122, 'fraction_numerical': 0.006165703275529865, 'mean_word_length': 3.9203791469194313, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 23, 'https://': 18, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '842412', 'n_tokens_mistral': 1809, 'n_tokens_neox': 1714, 'n_words': 463}