<commit_before># `scrape-metadata`

> HTML metadata scraper

# install

```shell
npm install --save scrape-metadata
```

# usage

```js
const scrape = require('scrape-metadata')
scrape('https://ponyfoo.com', (err, meta) => {
  console.log(meta)
})
```

# example response

Here's what an example response might look like in a website heavily decorated with metadata.

```json
{
  "title": "Pony Foo — JavaScript consulting, modularity, front-end architecture, performance, and more. Authored by <NAME>",
  "charset": "utf-8",
  "theme-color": "#e92c6c",
  "twitterWidgetsLink-color": "#e92c6c",
  "author": "<NAME> <<EMAIL>>",
  "description": "Pony Foo is a technical blog maintained by <NAME>, where he shares his thoughts on JavaScript and the web. Nico likes writing, public speaking, and open-source.",
  "ogSiteName": "Pony Foo",
  "ogUrl": "https://ponyfoo.com/",
  "twitterTitle": "Pony Foo — JavaScript consulting, modularity, front-end architecture, performance, and more. Authored by <NAME>",
  "twitterDescription": "Pony Foo is a technical blog maintained by <NAME>, where he shares his thoughts on JavaScript and the web. Nico likes writing, public speaking, and open-source.",
  "twitterCard": "summary_large_image",
  "twitterSite": "@ponyfoo",
  "twitterCreator": "@nzgb",
  "twitterWidgetsCsp": "on",
  "twitterWidgetsDnt": "on",
  "ogImage": "https://ponyfoo.com/img/banners/branded.9d42b711.png",
  "twitterImageSrc": "https://ponyfoo.com/img/banners/branded.9d42b711.png",
  "images": [
    "https://i.imgur.com/GKoh78o.jpg",
    "https://i.imgur.com/TjJ7TAS.jpg",
    "https://twemoji.maxcdn.com/2/72x72/1f510.png",
    "https://i.imgur.com/wWiTEO4.jpg",
    "https://lh5.googleusercontent.com/FekVGkoPt2gq8VzN5XLarr93psyFuI-_O0bUwJWZVLh9gzQiOJaXavUPuroyMjoPFRfmveefETIbMADUX3R_5Ij7_HIH7xxXj70L6M0CTbVGEjbVSkbYX7RwabOuESpPEFyJY-Br",
    "https://i.imgur.com/J2MHHUM.jpg",
    "https://i.imgur.com/URl6zyr.jpg",
    "https://i.imgur.com/BFQAdJy.jpg",
    "https://i.imgur.com/QnzjRhT.png",
    "https://i.imgur.com/ZpyYMV1.png",
    "https://i.imgur.com/fFN8WA5.jpg",<commit_msg>updated response example<commit_after># `scrape-metadata`

> HTML metadata scraper

# install

```shell
npm install --save scrape-metadata
```

# usage

```js
const scrape = require('scrape-metadata')
scrape('https://ponyfoo.com', (err, meta) => {
  console.log(meta)
})
```

# example response

Here's what an example response might look like in a website heavily decorated with metadata.

```json
{
  "title": "Pony Foo — JavaScript consulting, modularity, front-end architecture, performance, and more. Authored by <NAME>",
  "charset": "utf-8",
  "themeColor": "#e92c6c",
  "twitterWidgetsLinkColor": "#e92c6c",
  "author": "<NAME> <<EMAIL>>",
  "description": "Pony Foo is a technical blog maintained by <NAME>, where he shares his thoughts on JavaScript and the web. Nico likes writing, public speaking, and open-source.",
  "ogSiteName": "Pony Foo",
  "ogUrl": "https://ponyfoo.com/",
  "twitterTitle": "P<NAME> — JavaScript consulting, modularity, front-end architecture, performance, and more. Authored by <NAME>",
  "twitterDescription": "P<NAME> is a technical blog maintained by <NAME>, where he shares his thoughts on JavaScript and the web. Nico likes writing, public speaking, and open-source.",
  "twitterCard": "summary_large_image",
  "twitterSite": "@ponyfoo",
  "twitterCreator": "@nzgb",
  "twitterWidgetsCsp": "on",
  "twitterWidgetsDnt": "on",
  "ogImage": "https://ponyfoo.com/img/banners/branded.9d42b711.png",
  "twitterImageSrc": "https://ponyfoo.com/img/banners/branded.9d42b711.png",
  "images": [
    "https://i.imgur.com/GKoh78o.jpg",
    "https://i.imgur.com/TjJ7TAS.jpg",
    "https://twemoji.maxcdn.com/2/72x72/1f510.png",
    "https://i.imgur.com/wWiTEO4.jpg",
    "https://lh5.googleusercontent.com/FekVGkoPt2gq8VzN5XLarr93psyFuI-_O0bUwJWZVLh9gzQiOJaXavUPuroyMjoPFRfmveefETIbMADUX3R_5Ij7_HIH7xxXj70L6M0CTbVGEjbVSkbYX7RwabOuESpPEFyJY-Br",
    "https://i.imgur.com/J2MHHUM.jpg",
    "https://i.imgur.com/URl6zyr.jpg",
    "https://i.imgur.com/BFQAdJy.jpg",
    "https://i.imgur.com/QnzjRhT.png",
    "https://i.imgur.com/ZpyYMV1.png",
    "https://i.imgur.com/fFN8WA5.jpg",
{'fraction_non_alphanumeric': 0.1649631566436891, 'fraction_numerical': 0.025671499881150462, 'mean_word_length': 4.618157543391188, 'pattern_counts': {'":': 36, '<': 19, '<?xml version=': 0, '>': 23, 'https://': 30, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '552851', 'n_tokens_mistral': 1679, 'n_tokens_neox': 1588, 'n_words': 323}