<commit_before># Scuff

[![NPM version](http://img.shields.io/npm/v/scuff.svg)](https://www.npmjs.org/package/scuff)

One stop shop for web scraping with node. A lightweight wrapper for phantom, request, and cheerio.

## Functions
### static(url, callback, options)

Uses request to get an HTML body of static webpage. Returns a function, `$`, that has jQuery like functionality in node.

__Arguments__

* `url` - URL of the page you'd like to scrape.
* `callback` - Function with arguments `error` and a cheerio function (jQuery implementation in node) to easily navigate through the HTML or XML structure.
* `options` - `options.retries` allows you to set the number of attempts to get the requested data's HTML content. Utilizes exponential backoff to reduce and eliminate the number of network errors. Options.retries accepts a `number` between 1 and 5. `options.xml` accepts a truthy or falsey value indicating an XML feed.

### dynamic(url, callback, options)

Uses phantom to get HTML body of a dynamic site (includes dynamic content rendered by JavaScript). Returns a function, `$`, that has jQuery like functionality in node.

__Arguments__

* `url` - URL of the page you'd like to scrape.
* `callback` - Function with arguments `error` and a cheerio function (jQuery implementation in node) to easily navigate through the HTML or XML structure. Includes dynamic content rendered by JavaScript.
* `options` - `options.retries` allows you to set the number of attempts to get the requested data's HTML content. Utilizes exponential backoff to reduce and eliminate the number of network errors. Options.retries accepts a `number` between 1 and 5.<commit_msg>updates readme to link to cheerio documentation<commit_after># Scuff

[![NPM version](http://img.shields.io/npm/v/scuff.svg)](https://www.npmjs.org/package/scuff)

One stop shop for web scraping with node. A lightweight wrapper for phantom, request, and cheerio.

## Functions
### static(url, callback, options)

Uses request to get an HTML body of static webpage. Returns a cheerio function, `$`, that has jQuery like functionality in node ([see cheerio's documentation](https://www.npmjs.com/package/cheerio "cheerio documentation")).

__Arguments__

* `url` - URL of the page you'd like to scrape.
* `callback` - Function with arguments `error` and a cheerio function ([jQuery implementation in node](https://www.npmjs.com/package/cheerio)) to easily navigate through the HTML or XML structure.
* `options` - `options.retries` allows you to set the number of attempts to get the requested data's HTML content. Utilizes exponential backoff to reduce and eliminate the number of network errors. Options.retries accepts a `number` between 1 and 5. `options.xml` accepts a truthy or falsey value indicating an XML feed.

### dynamic(url, callback, options)

Uses phantom to get HTML body of a dynamic site (includes dynamic content rendered by JavaScript). Returns a cheerio function, `$`, that has jQuery like functionality in node ([see cheerio's documentation](https://www.npmjs.com/package/cheerio "cheerio documentation")).

__Arguments__

* `url` - URL of the page you'd like to scrape.
* `callback` - Function with arguments `error` and a cheerio function ([jQuery implementation in node](https://www.npmjs.com/package/cheerio)) to easily navigate through the HTML or XML structure. Includes dynamic content rendered by JavaScript.
* `options` - `options.retries` allows you to set the number of attempts to get the requested data's HTML content. Utilizes exponential backoff to reduce and eliminate the number of network errors. Options.retries accepts a `number` between 1 and 5.
{'fraction_non_alphanumeric': 0.08488499452354874, 'fraction_numerical': 0.002190580503833516, 'mean_word_length': 4.73469387755102, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 6, 'lorem ipsum': 0, 'www.': 6, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3558342', 'n_tokens_mistral': 987, 'n_tokens_neox': 924, 'n_words': 473}