<commit_before># Needl

**Take back your privacy. Lose yourself in the haystack.**

Your ISP is most likely tracking your browsing habits and selling them to marketing agencies (albeit anonymised). Or worse, making your browsing history available to law enforcement at the hint of a Subpoena. **Needl** will generate legitimate random Internet traffic in order to conceal your "real" traffic, essentially making your data the **Needl**e in the haystack. The goal is to make it harder for your ISP, government, etc to track your browsing habits.

On 16th November 2016 the UK's IP Bill passed parliament and is set to become law. ISPs will be required to keep a 12-month web history and bulk collection of your data.

Implemented modules:

- **Google**: generates a random search string, searches Google and clicks on a random result.
- **Alexa**: visits a website from the Alexa Top 1 Million list.
- **Twitter**: generates a popular English name and visits their profile; performs random keyword searches
- **DNS**: produces random DNS queries.

Modules in the roadmap (_feel free to get stuck in_):

- **WhatsApp**
- **Spotify**
- **Facebook Messenger**

**Warning**: the Alexa list contains a lot of porn websites. If this is of concern then don't use Needl until adult content filtering is added.

## Installation

Needl will work on pretty much any Linux system with Python 3.0+. A simple `sudo apt-get install python3 python3-pip` will take all of your troubles away.

You can then install Needl in 4 simple steps:

1. `cd /opt`
2. `git clone https://github.com/eth0izzle/needl.git`
3. `sudo make install` or `sudo pip3 install -r requirements.txt`
4. `python3 needl.py --daemon` _(todo: [write service scripts](https://github.com/eth0izzle/needl/issues/1))_

## Usage

Needl runs as a daemon and will happily sit in the background chomping away 24/7, 365. Each module (task) has scheduled actions, for example random DNS queries will happen every 60 seconds. You can configure the intervals within `./data/settings.yaml`.<commit_msg>Added screenshot to README<commit_after># Needl

**Take back your privacy. Lose yourself in the haystack.**

Your ISP is most likely tracking your browsing habits and selling them to marketing agencies (albeit anonymised). Or worse, making your browsing history available to law enforcement at the hint of a Subpoena. **Needl** will generate legitimate random Internet traffic in order to conceal your "real" traffic, essentially making your data the **Needl**e in the haystack. The goal is to make it harder for your ISP, government, etc to track your browsing habits.

![Demo](https://pbs.twimg.com/media/CxeQsH5XAAAp_vN.jpg:large)

**On 16th November 2016 the UK's IP Bill passed parliament and is set to become law. ISPs will be required to keep a 12-month web history and bulk collection of your data.**

Implemented modules:

- **Google**: generates a random search string, searches Google and clicks on a random result.
- **Alexa**: visits a website from the Alexa Top 1 Million list.
- **Twitter**: generates a popular English name and visits their profile; performs random keyword searches
- **DNS**: produces random DNS queries.

Modules in the roadmap (_feel free to get stuck in_):

- **WhatsApp**
- **Spotify**
- **Facebook Messenger**

**Warning**: the Alexa list contains a lot of porn websites. If this is of concern then don't use Needl until adult content filtering is added.

## Installation

Needl will work on pretty much any Linux system with Python 3.0+. A simple `sudo apt-get install python3 python3-pip` will take all of your troubles away.

You can then install Needl in 4 simple steps:

1. `cd /opt`
2. `git clone https://github.com/eth0izzle/needl.git`
3. `sudo make install` or `sudo pip3 install -r requirements.txt`
4. `python3 needl.py --daemon` _(todo: [write service scripts](https://github.com/eth0izzle/needl/issues/1))_

## Usage

Needl runs as a daemon and will happily sit in the background chomping away 24/7, 365. Each module (task) has scheduled actions, for example random DNS queries will happen every 60 seconds. You can configure the intervals within `./data/settings.yaml`.
{'fraction_non_alphanumeric': 0.08074981975486661, 'fraction_numerical': 0.01514059120403749, 'mean_word_length': 4.370322580645161, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1103961', 'n_tokens_mistral': 1186, 'n_tokens_neox': 1091, 'n_words': 607}