<commit_before>galimatias
==========

[![Build Status](https://travis-ci.org/smola/galimatias.png?branch=master)](https://travis-ci.org/smola/galimatias)
[![Coverage Status](https://coveralls.io/repos/smola/galimatias/badge.png?branch=master)](https://coveralls.io/r/smola/galimatias?branch=master)

galimatias is a URL parsing and normalization library written in Java.

Design goals:

- Parse URLs as browsers do, with optional normalization to RFC 3986 or RFC 2396.
- Stay as close as possible to WHATWG's [URL Standard](http://url.spec.whatwg.org/). In fact, the parser is largely based on it.
- Convenient fluent API with immutable URL objects.
- Interoperable with java.net.URL and java.net.URI.
- Zero dependencies.
- **I have not decided on the scope of galimatias with respect to non-HTTP(S) URIs. [I would like to read about your use cases here](https://github.com/smola/galimatias/issues/8).**

But, why?
---------

galimatias started out of frustration with java.net.URL and java.net.URI. Both of them are good for basic use cases, but severely broken for others:

- **[java.net.URL.equals() is broken.](http://stackoverflow.com/a/3771123/205607)**

- **java.net.URI can pase only RFC 2396 URI syntax.** `java.net.URI` will only parse a URI if it's strictly compliant with RFC 2396. Most URLs found in the wild do not comply with any syntax standard, and RFC 2396 is outdated anyway.

- **java.net.URI is not protocol-aware.** `http://example.com`, `http://example.com/` and `http://example.com:80` are different entities.

- **Manipulation is a pain.** I haven't seen any URL manipulation code using `java.net.URL` or `java.net.URI` that is simple and concise.

- **Not IDN ready.** Java has IDN support with `java.net.IDN`, but this does not apply to `java.net.URL` or `java.net.URI`.

Setup
-----

There are **no stable releases** of galimatias yet. But you can get the latest development version. If you do, please, let me know about your experience or any problem you find.

### Using Maven

Add to your pom.xml `<dependencies>` section:

```xml
<dependency>
  <groupId>io.mola</groupId>
  <artifactId>galimatias</artifactId>
  <version>0.0.1-SNAPSHOT</version>
</dependency>
```

You will also need to add the Sonatype OSS Snapshots repository to your `<repositories>` section:


```xml
<repository>
	<id>sonatype-oss-snapshots</id>
	<url>https://oss.sonatype.org/content/repositories/snapshots</url>
	<snapshots>
		<enabled>true</enabled>
	</snapshots>		
</repository>
```

### Standalone

Download the latest jar [from Sonatype OSS](https://oss.sonatype.org/content/groups/public/io/mola/galimatias/0.0.1-SNAPSHOT/) and drop it into your classpath.

Getting started
---------------

### Parse a URL

```java
// Parse
String urlString = //...
URL url;
try {
  url = URL.parse(urlString);
} catch (GalimatiasParseException ex) {
  // Do something with non-recoverable parsing error
}
```<commit_msg>Update README for 0.0.1 release.<commit_after>galimatias
==========

[![Build Status](https://travis-ci.org/smola/galimatias.png?branch=master)](https://travis-ci.org/smola/galimatias)
[![Coverage Status](https://coveralls.io/repos/smola/galimatias/badge.png?branch=master)](https://coveralls.io/r/smola/galimatias?branch=master)

galimatias is a URL parsing and normalization library written in Java.

Design goals:

- Parse URLs as browsers do, with optional normalization to RFC 3986 or RFC 2396.
- Stay as close as possible to WHATWG's [URL Standard](http://url.spec.whatwg.org/). In fact, the parser is largely based on it.
- Convenient fluent API with immutable URL objects.
- Interoperable with java.net.URL and java.net.URI.
- Zero dependencies.
- **I have not decided on the scope of galimatias with respect to non-HTTP(S) URIs. [I would like to read about your use cases here](https://github.com/smola/galimatias/issues/8).**

But, why?
---------

galimatias started out of frustration with java.net.URL and java.net.URI. Both of them are good for basic use cases, but severely broken for others:

- **[java.net.URL.equals() is broken.](http://stackoverflow.com/a/3771123/205607)**

- **java.net.URI can pase only RFC 2396 URI syntax.** `java.net.URI` will only parse a URI if it's strictly compliant with RFC 2396. Most URLs found in the wild do not comply with any syntax standard, and RFC 2396 is outdated anyway.

- **java.net.URI is not protocol-aware.** `http://example.com`, `http://example.com/` and `http://example.com:80` are different entities.

- **Manipulation is a pain.** I haven't seen any URL manipulation code using `java.net.URL` or `java.net.URI` that is simple and concise.

- **Not IDN ready.** Java has IDN support with `java.net.IDN`, but this does not apply to `java.net.URL` or `java.net.URI`.

Setup with Maven
----------------

galimatias is available at Maven Central. Just add to your pom.xml `<dependencies>` section:

```xml
<dependency>
  <groupId>io.mola</groupId>
  <artifactId>galimatias</artifactId>
  <version>0.0.1</version>
</dependency>
```

Development snapshots are also available at Sonatype OSS Snapshots repository.

Getting started
---------------

### Parse a URL

```java
// Parse
String urlString = //...
URL url;
try {
  url = URL.parse(urlString);
} catch (GalimatiasParseException ex) {
  // Do something with non-recoverable parsing error
}
```
{'fraction_non_alphanumeric': 0.14291079812206572, 'fraction_numerical': 0.015774647887323943, 'mean_word_length': 3.9360518999073215, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 12, 'lorem ipsum': 0, 'www.': 0, 'xml': 5}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1708038', 'n_tokens_mistral': 1797, 'n_tokens_neox': 1675, 'n_words': 575}