<commit_before># NtParser

A simple and performant [N-Triples][N-TRIPLES] parser for the JVM.


## Installing

NtParser is published to [Maven Central][mvn]:


```xml
<dependency>
    <groupId>de.knutwalker</groupId>
    <artifactId>ntparser</artifactId>
    <version>0.1</version>
</dependency>
```


```scala
libraryDependencies += "de.knutwalker" % "ntparser" % "0.1"
```


```groovy
compile 'de.knutwalker:ntparser:0.1'
```


## Using


```scala
import de.knutwalker.ntparser.StrictNtParser

StrictNtParser("my-graph.nt") foreach { stmt ⇒
  println(s"Statement: $stmt")
}
```


```java
import de.knutwalker.ntparser.Statement;
import de.knutwalker.ntparser.StrictNtParser;
import java.util.Iterator;

Iterator<Statement> statementIterator = StrictNtParser.parse("my-graph.nt");
while (statementIterator.hasNext()) {
  Statement stmt = statementIterator.next();
  System.out.println("Statement: " + stmt);
}
```


### in-depth usage

There are two static parsers, `de.knutwalker.ntparser.StrictNtParser` and `de.knutwalker.ntparser.NonStrictNtParser`.

Both take either a file name, an `InputStream`, an `Iterable` of String,
or an `Iterator` of String and always return an `Iterator` of `Statement`.
The `apply` methods are Scala API, the `parse` methods are Java API
(and are thus consuming and producing Java `Iterator`s).
The file name may be a path to any file or the file name of a resource file.
The parser can read plain nt files, as well as gzipped and bzipped files.
The `Iterable` and `Iterator` should represent single lines, that is, at-most one N-Triple per element.

The `StrictNtParser` will halt with an Exception at the first parse error.
The `NonStrictNtParser` will just log exceptions and continue parsing.

As an alternative, you can instantiate a `new de.knutwalker.ntparser.NtParser`.
The `NtParser` has several `parse*` methods, that each take a String and assume,
that this is _one_ line of some bigger document and that is correctly delimited.
One invocation of any `parse*` method will parse at-most _one_ statement.


## Specification conformity

NtParser passes [all tests provided by the W3C][rdf-test-cases].


## Roadmap

- parse into different data types, e.g. Jena Model


[N-TRIPLES]: http://www.w3.org/TR/2014/REC-n-triples-20140225/ "RDF 1.1 N-Triples"
[mvn]: http://search.maven.org/#search|ga|1|g%3A%22de.knutwalker%22%20AND%20%28a%3A%22ntparser_2.11%22%20OR%20a%3A%22ntparser_2.10%22%29
[rdf-test-cases]: http://www.w3.org/TR/2014/NOTE-rdf11-testcases-20140225/
<commit_msg>Correct dependency information<commit_after># NtParser

A simple and performant [N-Triples][N-TRIPLES] parser for the JVM.


## Installing

NtParser is published to [Maven Central][mvn]:


```xml
<!-- for Scala 2.10 -->
<dependency>
    <groupId>de.knutwalker</groupId>
    <artifactId>ntparser_2.10</artifactId>
    <version>0.1</version>
</dependency>
<!-- for Scala 2.11 -->
<dependency>
    <groupId>de.knutwalker</groupId>
    <artifactId>ntparser_2.11</artifactId>
    <version>0.1</version>
</dependency>
```


```scala
libraryDependencies += "de.knutwalker" %% "ntparser" % "0.1"
```


## Using


```scala
import de.knutwalker.ntparser.StrictNtParser

StrictNtParser("my-graph.nt") foreach { stmt ⇒
  println(s"Statement: $stmt")
}
```


```java
import de.knutwalker.ntparser.Statement;
import de.knutwalker.ntparser.StrictNtParser;
import java.util.Iterator;

Iterator<Statement> statementIterator = StrictNtParser.parse("my-graph.nt");
while (statementIterator.hasNext()) {
  Statement stmt = statementIterator.next();
  System.out.println("Statement: " + stmt);
}
```


### in-depth usage

There are two static parsers, `de.knutwalker.ntparser.StrictNtParser` and `de.knutwalker.ntparser.NonStrictNtParser`.

Both take either a file name, an `InputStream`, an `Iterable` of String,
or an `Iterator` of String and always return an `Iterator` of `Statement`.
The `apply` methods are Scala API, the `parse` methods are Java API
(and are thus consuming and producing Java `Iterator`s).
The file name may be a path to any file or the file name of a resource file.
The parser can read plain nt files, as well as gzipped and bzipped files.
The `Iterable` and `Iterator` should represent single lines, that is, at-most one N-Triple per element.

The `StrictNtParser` will halt with an Exception at the first parse error.
The `NonStrictNtParser` will just log exceptions and continue parsing.

As an alternative, you can instantiate a `new de.knutwalker.ntparser.NtParser`.
The `NtParser` has several `parse*` methods, that each take a String and assume,
that this is _one_ line of some bigger document and that is correctly delimited.
One invocation of any `parse*` method will parse at-most _one_ statement.


## Specification conformity

NtParser passes [all tests provided by the W3C][rdf-test-cases].


## Roadmap

- parse into different data types, e.g. Jena Model


[N-TRIPLES]: http://www.w3.org/TR/2014/REC-n-triples-20140225/ "RDF 1.1 N-Triples"
[mvn]: http://search.maven.org/#search|ga|1|g%3A%22de.knutwalker%22%20AND%20%28a%3A%22ntparser_2.11%22%20OR%20a%3A%22ntparser_2.10%22%29
[rdf-test-cases]: http://www.w3.org/TR/2014/NOTE-rdf11-testcases-20140225/

{'fraction_non_alphanumeric': 0.1258673862760216, 'fraction_numerical': 0.029683885890516577, 'mean_word_length': 4.817264573991031, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 0, 'lorem ipsum': 0, 'www.': 4, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7089504', 'n_tokens_mistral': 1924, 'n_tokens_neox': 1739, 'n_words': 522}