<commit_before># cbor

Encode and parse data in the Concise Binary Object Representation (CBOR) data format ([RFC7049](http://tools.ietf.org/html/rfc7049)).

## Supported Node.js versions

This project now only supports versions of Node that the Node team is
[currently supporting](https://github.com/nodejs/Release#release-schedule).
Ava's [support
statement](https://github.com/avajs/ava/blob/master/docs/support-statement.md)
is what we will be using as well.  Currently, that means Node `10`+ is
required.  If you need to support an older version of Node (back to version
6), use cbor version 5.2.x, which will get nothing but security updates from
here on out.

## Installation:

```bash
$ npm install --save cbor
# optional, if you need big floats or big decimals:
$ npm install --save bignumber.js
```

**NOTE**
If you are going to use this on the web, use [cbor-web](../cbor-web) instead.

## Documentation:

See the full API [documentation](http://hildjj.github.io/node-cbor/).

For a command-line interface, see [cbor-cli](../cbor-cli).

Example:
```javascript
var cbor = require('cbor');
var assert = require('assert');

var encoded = cbor.encode(true); // returns <Buffer f5>
cbor.decodeFirst(encoded, function(error, obj) {
  // error != null if there was an error
  // obj is the unpacked object
  assert.ok(obj === true);
});

// Use integers as keys?
var m = new Map();
m.set(1, 2);
encoded = cbor.encode(m); // <Buffer a1 01 02>
```

Allows streaming as well:

```javascript
var cbor = require('cbor');
var fs = require('fs');

var d = new cbor.Decoder();
d.on('data', function(obj){
  console.log(obj);
});

var s = fs.createReadStream('foo');
s.pipe(d);

var d2 = new cbor.Decoder({input: '00', encoding: 'hex'});
d.on('data', function(obj){
  console.log(obj);
});
```

There is also support for synchronous decodes:

```javascript
try {
  console.log(cbor.decodeFirstSync('02')); // 2
  console.log(cbor.decodeAllSync('0202')); // [2, 2]
} catch (e) {
  // throws on invalid input
}
```

The sync encoding and decoding are exported as a
[leveldb encoding](https://github.com/Level/levelup#custom_encodings), as
`cbor.leveldb`.

## highWaterMark

The synchronous routines for encoding and decoding will have problems with
objects that are larger than 16kB, which the default buffer size for Node
streams.  There are a few ways to fix this:

1) pass in a `highWaterMark` option with the value of the largest buffer size you think you will need:

```javascript
cbor.encodeOne(Buffer.alloc(40000), {highWaterMark: 65535})
```

2) use stream mode.  Catch the `data`, `finish`, and `error` events.  Make sure to call `end()` when you're done.

```javascript
const enc = new cbor.Encoder()
enc.on('data', buf => /* send the data somewhere */)
enc.on('error', console.error)
enc.on('finish', () => /* tell the consumer we are finished */)

enc.end(['foo', 1, false])
```

3) use `encodeAsync()`, which uses the approach from approach 2 to return a memory-inefficient promise for a Buffer.

## Supported types

The following types are supported for encoding:

* boolean
* number (including -0, NaN, and ±Infinity)
* string
* Array, Set (encoded as Array)
* Object (including null), Map
* undefined
* Buffer
* Date,
* RegExp
* URL
* TypedArrays, ArrayBuffer, DataView
* Map, Set
* BigInt
* [bignumber](https://github.com/MikeMcl/bignumber.js) (If you install it.  It is needed for big float and big decimal types)

Decoding supports the above types, including the following CBOR tag numbers:

| Tag | Generated Type      |
|-----|---------------------|
| 0   | Date                |
| 1   | Date                |
| 2   | BigInt              |
| 3   | BigInt              |
| 4   | bignumber           |
| 5   | bignumber           |
| 21  | Tagged, with toJSON |
| 22  | Tagged, with toJSON |
| 23  | Tagged, with toJSON |
| 32  | URL                 |
| 33  | Tagged              |
| 34  | Tagged              |
| 35  | RegExp              |

## Adding new Encoders

There are several ways to add a new encoder:

### `encodeCBOR` method

This is the easiest approach, if you can modify the class being encoded.  Add an
`encodeCBOR` method to your class, which takes a single parameter of the encoder
currently being used.  Your method should return `true` on success, else `false`.
Your method may call `encoder.push(buffer)` or `encoder.pushAny(any)` as needed.

For example:<commit_msg>Update docs with more tags<commit_after># cbor

Encode and parse data in the Concise Binary Object Representation (CBOR) data format ([RFC7049](http://tools.ietf.org/html/rfc7049)).

## Supported Node.js versions

This project now only supports versions of Node that the Node team is
[currently supporting](https://github.com/nodejs/Release#release-schedule).
Ava's [support
statement](https://github.com/avajs/ava/blob/master/docs/support-statement.md)
is what we will be using as well.  Currently, that means Node `10`+ is
required.  If you need to support an older version of Node (back to version
6), use cbor version 5.2.x, which will get nothing but security updates from
here on out.

## Installation:

```bash
$ npm install --save cbor
# optional, if you need big floats or big decimals:
$ npm install --save bignumber.js
```

**NOTE**
If you are going to use this on the web, use [cbor-web](../cbor-web) instead.

## Documentation:

See the full API [documentation](http://hildjj.github.io/node-cbor/).

For a command-line interface, see [cbor-cli](../cbor-cli).

Example:
```javascript
var cbor = require('cbor');
var assert = require('assert');

var encoded = cbor.encode(true); // returns <Buffer f5>
cbor.decodeFirst(encoded, function(error, obj) {
  // error != null if there was an error
  // obj is the unpacked object
  assert.ok(obj === true);
});

// Use integers as keys?
var m = new Map();
m.set(1, 2);
encoded = cbor.encode(m); // <Buffer a1 01 02>
```

Allows streaming as well:

```javascript
var cbor = require('cbor');
var fs = require('fs');

var d = new cbor.Decoder();
d.on('data', function(obj){
  console.log(obj);
});

var s = fs.createReadStream('foo');
s.pipe(d);

var d2 = new cbor.Decoder({input: '00', encoding: 'hex'});
d.on('data', function(obj){
  console.log(obj);
});
```

There is also support for synchronous decodes:

```javascript
try {
  console.log(cbor.decodeFirstSync('02')); // 2
  console.log(cbor.decodeAllSync('0202')); // [2, 2]
} catch (e) {
  // throws on invalid input
}
```

The sync encoding and decoding are exported as a
[leveldb encoding](https://github.com/Level/levelup#custom_encodings), as
`cbor.leveldb`.

## highWaterMark

The synchronous routines for encoding and decoding will have problems with
objects that are larger than 16kB, which the default buffer size for Node
streams.  There are a few ways to fix this:

1) pass in a `highWaterMark` option with the value of the largest buffer size you think you will need:

```javascript
cbor.encodeOne(Buffer.alloc(40000), {highWaterMark: 65535})
```

2) use stream mode.  Catch the `data`, `finish`, and `error` events.  Make sure to call `end()` when you're done.

```javascript
const enc = new cbor.Encoder()
enc.on('data', buf => /* send the data somewhere */)
enc.on('error', console.error)
enc.on('finish', () => /* tell the consumer we are finished */)

enc.end(['foo', 1, false])
```

3) use `encodeAsync()`, which uses the approach from approach 2 to return a memory-inefficient promise for a Buffer.

## Supported types

The following types are supported for encoding:

* boolean
* number (including -0, NaN, and ±Infinity)
* string
* Array, Set (encoded as Array)
* Object (including null), Map
* undefined
* Buffer
* Date,
* RegExp
* URL
* TypedArrays, ArrayBuffer, DataView
* Map, Set
* BigInt
* [bignumber](https://github.com/MikeMcl/bignumber.js) (If you install it.  It is needed for big float and big decimal types)

Decoding supports the above types, including the following CBOR tag numbers:

| Tag | Generated Type      |
|-----|---------------------|
| 0   | Date                |
| 1   | Date                |
| 2   | BigInt              |
| 3   | BigInt              |
| 4   | bignumber           |
| 5   | bignumber           |
| 21  | Tagged, with toJSON |
| 22  | Tagged, with toJSON |
| 23  | Tagged, with toJSON |
| 32  | URL                 |
| 33  | Tagged              |
| 34  | Tagged              |
| 35  | RegExp              |
| 64  | Uint8Array          |
| 65  | Uint16Array         |
| 66  | Uint32Array         |
| 67  | BigUint64Array      |
| 68  | Uint8ClampedArray   |
| 69  | Uint16Array         |
| 70  | Uint32Array         |
| 71  | BigUint64Array      |
| 72  | Int8Array           |
| 73  | Int16Array          |
| 74  | Int32Array          |
| 75  | BigInt64Array       |
| 77  | Int16Array          |
| 78  | Int32Array          |
| 79  | BigInt64Array       |
| 81  | Float32Array        |
| 82  | Float64Array        |
| 85  | Float32Array        |
| 86  | Float64Array        |
| 258 | Set                 |

## Adding new Encoders

There are several ways to add a new encoder:

### `encodeCBOR` method

This is the easiest approach, if you can modify the class being encoded.  Add an
`encodeCBOR` method to your class, which takes a single parameter of the encoder
currently being used.  Your method should return `true` on success, else `false`.
Your method may call `encoder.push(buffer)` or `encoder.pushAny(any)` as needed.

For example:
{'fraction_non_alphanumeric': 0.1321576543078721, 'fraction_numerical': 0.02315946032083289, 'mean_word_length': 3.1380219780219782, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 11, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7425512', 'n_tokens_mistral': 3176, 'n_tokens_neox': 2935, 'n_words': 1079}