<commit_before>---
title: CSV Parser
layout: page
tags: ['intro','page']
pageOrder: 1
---

[![Build Status](https://secure.travis-ci.org/wdavidw/node-csv-parse.png)][travis-csv-parse]

This package is a parser converting CSV text input into arrays or objects. It
implements the Node.js [`stream.Transform`API][stream_transform]. It also
provides a simple callback-based API for convenience. It is both extremely easy
to use and powerful. It was first released in 2010 and is used against big data
sets by a large community.

## Features

*   Follow the Node.js streaming API
*   Simplicity with the optional callback API
*   Support delimiters, quotes, escape characters and comments
*   Line breaks discovery
*   Support big datasets
*   Complete test coverage and samples for inspiration
*   no external dependencies
*   to be used conjointly with `csv-generate`, `stream-transform` and `csv-stringify`

## Usage

Run `npm install csv` to install the full CSV package or run
`npm install csv-parse` if you are only interested by the CSV parser.

Use the callback style API for simplicity or the stream based API for
scalability. You may also mix the two styles. For example, the
[fs_read.js example][fs_read] pipe a file stream reader and get the results
inside a callback.

For additionnal usage and example, you may refer to
[example page](/parse/examples/),
[the "samples" folder][parse-samples] and [the "test" folder][parse-test].

## Parser options

*   `delimiter` (char)   
    Set the field delimiter. One character only, defaults to comma.   
*   `rowDelimiter` (chars|constant)   
    String used to delimit record rows or a special value; special constants are
    'auto', 'unix', 'mac', 'windows', 'unicode'; defaults to 'auto' (discovered
    in source or 'unix' if no source is specified).   
*   `quote` (char)   
    Optionnal character surrounding a field, one character only, defaults to
    double quotes.   
*   `escape` (char)   
    Set the escape character, one character only, defaults to double quotes.   
*   `columns` (array|boolean|function)
    List of fields as an array, a user defined callback accepting the first line
    and returning the column names or true if autodiscovered in the first CSV
    line, default to null, affect the result data set in the sense that records
    will be objects instead of arrays.   
*   `comment` (char)   
    Treat all the characteres after this one as a comment, default to '#'.   
*   `objname` (string)   
    Name of header-record title to name objects by.   
*   `relax` (boolean)   
    Preserve quotes inside unquoted field.   
*   `skip_empty_lines` (boolean)   
    Dont generate empty values for empty lines.   
*   `trim` (boolean)   
    If true, ignore whitespace immediately around the delimiter, defaults to
    false.   
*   `ltrim` (boolean)   
    If true, ignore whitespace immediately following the delimiter (i.e.
    left-trim all fields), defaults to false.   
*   `rtrim` (boolean)   
    If true, ignore whitespace immediately preceding the delimiter (i.e.
    right-trim all fields), defaults to false.   
*   `auto_parse` (boolean)   
    If true, the parser will attempt to convert read data types to native types.   

All options are optional.

## Migration

Most of the generator is imported from its parent project [CSV][csv] in an
effort to split it between the generator, the parser, the transformer and the
stringifier.

The "record" has disappeared, you are encouraged to use the "readable" event
conjointly with the "read" function as documented above and in the
[Stream API][stream_transform].

[csv]: https://github.com/wdavidw/node-csv
[travis-csv-parse]: http://travis-ci.org/wdavidw/node-csv-parse
[stream_transform]: http://nodejs.org/api/stream.html#stream_class_stream_transform
[fs_read]: https://github.com/wdavidw/node-csv-parse/tree/master/samples/fs_read.js
[parse-samples]: https://github.com/wdavidw/node-csv-parse/tree/master/samples
[parse-test]: https://github.com/wdavidw/node-csv-parse/tree/master/test
[stream_transform]: http://nodejs.org/api/stream.html#stream_class_stream_transform
<commit_msg>Update docs for trim, ltrim and rtrim to clarify that they don't apply to quoted values.<commit_after>---
title: CSV Parser
layout: page
tags: ['intro','page']
pageOrder: 1
---

[![Build Status](https://secure.travis-ci.org/wdavidw/node-csv-parse.png)][travis-csv-parse]

This package is a parser converting CSV text input into arrays or objects. It
implements the Node.js [`stream.Transform`API][stream_transform]. It also
provides a simple callback-based API for convenience. It is both extremely easy
to use and powerful. It was first released in 2010 and is used against big data
sets by a large community.

## Features

*   Follow the Node.js streaming API
*   Simplicity with the optional callback API
*   Support delimiters, quotes, escape characters and comments
*   Line breaks discovery
*   Support big datasets
*   Complete test coverage and samples for inspiration
*   no external dependencies
*   to be used conjointly with `csv-generate`, `stream-transform` and `csv-stringify`

## Usage

Run `npm install csv` to install the full CSV package or run
`npm install csv-parse` if you are only interested by the CSV parser.

Use the callback style API for simplicity or the stream based API for
scalability. You may also mix the two styles. For example, the
[fs_read.js example][fs_read] pipe a file stream reader and get the results
inside a callback.

For additionnal usage and example, you may refer to
[example page](/parse/examples/),
[the "samples" folder][parse-samples] and [the "test" folder][parse-test].

## Parser options

*   `delimiter` (char)   
    Set the field delimiter. One character only, defaults to comma.   
*   `rowDelimiter` (chars|constant)   
    String used to delimit record rows or a special value; special constants are
    'auto', 'unix', 'mac', 'windows', 'unicode'; defaults to 'auto' (discovered
    in source or 'unix' if no source is specified).   
*   `quote` (char)   
    Optionnal character surrounding a field, one character only, defaults to
    double quotes.   
*   `escape` (char)   
    Set the escape character, one character only, defaults to double quotes.   
*   `columns` (array|boolean|function)
    List of fields as an array, a user defined callback accepting the first line
    and returning the column names or true if autodiscovered in the first CSV
    line, default to null, affect the result data set in the sense that records
    will be objects instead of arrays.   
*   `comment` (char)   
    Treat all the characteres after this one as a comment, default to '#'.   
*   `objname` (string)   
    Name of header-record title to name objects by.   
*   `relax` (boolean)   
    Preserve quotes inside unquoted field.   
*   `skip_empty_lines` (boolean)   
    Dont generate empty values for empty lines.   
*   `trim` (boolean)   
    If true, ignore whitespace immediately around the delimiter, defaults to
    false. Does not remove whitespace in a quoted field.   
*   `ltrim` (boolean)   
    If true, ignore whitespace immediately following the delimiter (i.e.
    left-trim all fields), defaults to false. Does not remove whitespace in a quoted field.
*   `rtrim` (boolean)   
    If true, ignore whitespace immediately preceding the delimiter (i.e.
    right-trim all fields), defaults to false.  Does not remove whitespace in a quoted field.
*   `auto_parse` (boolean)   
    If true, the parser will attempt to convert read data types to native types.   

All options are optional.

## Migration

Most of the generator is imported from its parent project [CSV][csv] in an
effort to split it between the generator, the parser, the transformer and the
stringifier.

The "record" has disappeared, you are encouraged to use the "readable" event
conjointly with the "read" function as documented above and in the
[Stream API][stream_transform].

[csv]: https://github.com/wdavidw/node-csv
[travis-csv-parse]: http://travis-ci.org/wdavidw/node-csv-parse
[stream_transform]: http://nodejs.org/api/stream.html#stream_class_stream_transform
[fs_read]: https://github.com/wdavidw/node-csv-parse/tree/master/samples/fs_read.js
[parse-samples]: https://github.com/wdavidw/node-csv-parse/tree/master/samples
[parse-test]: https://github.com/wdavidw/node-csv-parse/tree/master/test
[stream_transform]: http://nodejs.org/api/stream.html#stream_class_stream_transform

{'fraction_non_alphanumeric': 0.08813799621928166, 'fraction_numerical': 0.0011814744801512287, 'mean_word_length': 3.8014747589336357, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1059780', 'n_tokens_mistral': 2422, 'n_tokens_neox': 2392, 'n_words': 1013}