<commit_before># twitter-text-js

A JavaScript utility that provides text processing routines for Tweets.  This library conforms to a common test suite shared by many other implementations, particularly twitter-text.gem (Ruby).  The library provides autolinking and extraction for URLs, usernames, lists, and hashtags.

## NPM Users

Install it with: `npm install twitter-text`

The `twttr.txt` namespace is exported, making it available as such:

``` js
var twitter = require('twitter-text')
twitter.autoLink(twitter.htmlEscape('#hello < @world >'))
```

## Extraction Examples

    // basic extraction
    var usernames = twttr.txt.extract_mentioned_screen_names("Mentioning @twitter and @jack")
    // usernames == ["twitter", "jack"]

## Auto-linking Examples

    twttr.txt.autoLink("link @user, please #request");
    twttr.txt.autoLink("link @user, and expand url... http://t.co/0JG5Mcq", { 
        urlEntities: [
            {
              "url": "http://t.co/0JG5Mcq",
              "display_url": "blog.twitter.com/2011/05/twitte…",
              "expanded_url": "http://blog.twitter.com/2011/05/twitter-for-mac-update.html",
              "indices": [
                30,
                48
              ]
            }
        ]});

See https://dev.twitter.com/docs/tweet-entities for more info on tweet entities.<commit_msg>Cleaned up autolink example for urlEntities.<commit_after># twitter-text-js

A JavaScript utility that provides text processing routines for Tweets.  This library conforms to a common test suite shared by many other implementations, particularly twitter-text.gem (Ruby).  The library provides autolinking and extraction for URLs, usernames, lists, and hashtags.

## NPM Users

Install it with: `npm install twitter-text`

The `twttr.txt` namespace is exported, making it available as such:

``` js
var twitter = require('twitter-text')
twitter.autoLink(twitter.htmlEscape('#hello < @world >'))
```

## Extraction Examples

    // basic extraction
    var usernames = twttr.txt.extract_mentioned_screen_names("Mentioning @twitter and @jack")
    // usernames == ["twitter", "jack"]

## Auto-linking Examples

    twttr.txt.autoLink("link @user, please #request");
    
    twttr.txt.autoLink("link @user, and expand url... http://t.co/0JG5Mcq", { 
        urlEntities: [
            {
              "url": "http://t.co/0JG5Mcq",
              "display_url": "blog.twitter.com/2011/05/twitte…",
              "expanded_url": "http://blog.twitter.com/2011/05/twitter-for-mac-update.html",
              "indices": [
                30,
                48
              ]
            }
        ]});

See [Tweet Entities](https://dev.twitter.com/docs/tweet-entities) for more info getting url entities from Twitter's API.
{'fraction_non_alphanumeric': 0.125544267053701, 'fraction_numerical': 0.01451378809869376, 'mean_word_length': 2.7663934426229506, 'pattern_counts': {'":': 8, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '692419', 'n_tokens_mistral': 862, 'n_tokens_neox': 806, 'n_words': 222}