<commit_before># websearch.js
A simple interface to websearch. It uses a deprecated but still functioning Google API. T
he reason, why we use this deprecated API is that there is no proper alternative for this API (for complete websearch). 
The Custom Search API only offers a defined subset of the web to search, not the complete Google index.

# Usage

    websearch.search('mother', function (error, result) {
        console.log('search term', result.term);
        console.log('urls', result.urls);
    });

# TODO

- bulk searcher, that step by step goes through a list of terms and returns the results in one callback, should support web and book search<commit_msg>added note about concurrent usage<commit_after># websearch.js

A simple interface to websearch. It uses a deprecated but still functioning Google API. 
The reason, why we use this deprecated API is that there is no proper alternative for this API (for complete websearch). 
The Custom Search API only offers a defined subset of the web to search, not the complete Google index.

The web-searcher can only run one search simultaneously. This is by design! If you run a number of multiple concurrent 
queries you may trigger Google's abuse detection. If you are using their search, you should make sure you are not
putting unnecessary stress on their servers. 

# Usage

    websearch.search('mother', function (error, result) {
        console.log('search term', result.term);
        console.log('urls', result.urls);
    });

# TODO

- bulk searcher, that step by step goes through a list of terms and returns the results in one callback, should support web and book search
{'fraction_non_alphanumeric': 0.06014580801944107, 'fraction_numerical': 0.0, 'mean_word_length': 3.945945945945946, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3414385', 'n_tokens_mistral': 408, 'n_tokens_neox': 402, 'n_words': 234}