<commit_before>sleekhtml
=========

Cleans &amp; grooms HTML document from unnecessary white-space, HTML tags, comments &amp; other elements.

[![GoDoc](http://godoc.org/github.com/ernestas-poskus/sleekhtml?status.svg)](http://godoc.org/github.com/ernestas-poskus/sleekhtml)
[![MITLicense](http://img.shields.io/badge/license-MIT-blue.svg)](http://opensource.org/licenses/MIT)

## Benchmark (default options)

Website | Raw wget (bytes) | Sanitized (bytes)
------- | ---------------- | ----------------
example.com | 12876 | 531
google.com | 116772 | 2781
economist.com | 194588 | 62278

## Tags
Use tags to get desired output

Defaults:
```go
IgnoredHTMLTags: []string{
	"script", "style", "iframe", "input",
	"form", "svg", "select", "label",
	"fieldset", "frame", "frameset", "noframes",
	"noembed", "embed", "applet", "object",
	"base",<commit_msg>Add Build status to README<commit_after>sleekhtml
=========

Cleans &amp; grooms HTML document from unnecessary white-space, HTML tags, comments &amp; other elements.

[![Build Status](https://travis-ci.org/ernestas-poskus/sleekhtml.svg)](https://travis-ci.org/ernestas-poskus/sleekhtml)
[![GoDoc](http://godoc.org/github.com/ernestas-poskus/sleekhtml?status.svg)](http://godoc.org/github.com/ernestas-poskus/sleekhtml)
[![MITLicense](http://img.shields.io/badge/license-MIT-blue.svg)](http://opensource.org/licenses/MIT)

## Benchmark (default options)

Website | Raw wget (bytes) | Sanitized (bytes)
------- | ---------------- | ----------------
example.com | 12876 | 531
google.com | 116772 | 2781
economist.com | 194588 | 62278

## Tags
Use tags to get desired output

Defaults:
```go
IgnoredHTMLTags: []string{
	"script", "style", "iframe", "input",
	"form", "svg", "select", "label",
	"fieldset", "frame", "frameset", "noframes",
	"noembed", "embed", "applet", "object",
	"base",
{'fraction_non_alphanumeric': 0.22645617855198694, 'fraction_numerical': 0.031573217201959713, 'mean_word_length': 4.251428571428572, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '273660', 'n_tokens_mistral': 711, 'n_tokens_neox': 647, 'n_words': 131}