<commit_before># node-vcdiff

[![Build Status](https://travis-ci.org/baranov1ch/node-vcdiff.svg?branch=master)](https://travis-ci.org/baranov1ch/node-vcdiff)

node.js wrapper for [open-vcdiff](http://code.google.com/p/open-vcdiff)

## Notes

If you want to create SDCH-aware server, better look at [node-sdch](http://github.com/baranov1ch/node-sdch) and [connect-sdch](http://github.com/baranov1ch/connect-sdch)
which use this library internally and serve SDCH-encoded content which could
be comprehended by Chromium-based browsers.

You can use [femtozip](http://github.com/gtoubassi/femtozip) to create dictionaries.

## Quick Examples

```javascript
var vcd = require('vcdiff');

var dictionary = new Buffer(
  'Yo dawg I heard you like common substrings in your documents so we put ' +
  'them in your vcdiff dictionary so you can compress while you compress');

var hashed = new vcd.HashedDictionary(dictionary);

var testData =
  'Yo dawg I heard you like common substrings somewhere else so we put ' +
  'them in your vcdiff dictionary so you can decompress while you decompress'
```

Sync API

```javascript<commit_msg>Update README to mention SInGe dict generator<commit_after># node-vcdiff

[![Build Status](https://travis-ci.org/baranov1ch/node-vcdiff.svg?branch=master)](https://travis-ci.org/baranov1ch/node-vcdiff)

node.js wrapper for [open-vcdiff](http://code.google.com/p/open-vcdiff)

## Notes

If you want to create SDCH-aware server, better look at [node-sdch](http://github.com/baranov1ch/node-sdch) and [connect-sdch](http://github.com/baranov1ch/connect-sdch)
which use this library internally and serve SDCH-encoded content which could
be comprehended by Chromium-based browsers.

You can use [femtozip](http://github.com/gtoubassi/femtozip) or [experimental incremental generator](https://github.com/cscenter/SInGe/tree/master/src/dict_builder) to create dictionaries.
Femtozip took quite a long time to generate (works for minutes and hours) dictionary. SInGe generator is much faster (seconds) and the same volume, since it never allocates during generation and uses linear algorithm based on suffix automaton. Quality is similar, since algorithm is essentially the same.

## Quick Examples

```javascript
var vcd = require('vcdiff');

var dictionary = new Buffer(
  'Yo dawg I heard you like common substrings in your documents so we put ' +
  'them in your vcdiff dictionary so you can compress while you compress');

var hashed = new vcd.HashedDictionary(dictionary);

var testData =
  'Yo dawg I heard you like common substrings somewhere else so we put ' +
  'them in your vcdiff dictionary so you can decompress while you decompress'
```

Sync API

```javascript
{'fraction_non_alphanumeric': 0.10204081632653061, 'fraction_numerical': 0.002968460111317254, 'mean_word_length': 4.860869565217391, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2154630', 'n_tokens_mistral': 862, 'n_tokens_neox': 831, 'n_words': 286}