<commit_before># Motivation

Playing with extracting knowledge from slack metadata. For example, try to find something out about channels
or people by looking at how they are related on slack.

# Prerequisites

* python3, virtualenv, pip
* a slack api token (API_TOKEN below)
    * see https://api.slack.com/bot-users, specifically https://my.slack.com/services/new/bot

# Install

    virtualenv venv
    source ./venv/bin/activate

    pip install -r requirements.txt

# Usage

## Setup defaults

    export API_TOKEN="your-api-token"

## Get base information

Only include those channels written to in last 3 months, and with a Jaccard distance of 0.7 or less

    TS=$(date -v-3m +%s) python3 crawl.py --token $API_TOKEN --distance 0.7 --recency $TS --out crawl.json

## Visualise channel similarity

    python3 visualise.py --in crawl.json --out vis.json
    python3 -m http.server 8000
    open http://localhost:8000
    
## Recommend channels

    Assuming some user-name:

    python3 recommend.py --token $API_TOKEN --in crawl.json --user user-name<commit_msg>some small fixes to README<commit_after># Motivation

Playing with extracting knowledge from slack metadata. For example, try to find something out about channels
or people by looking at how they are related on slack.

# Prerequisites

* python3, virtualenv, pip
* a slack api token (API_TOKEN below)
    * see https://api.slack.com/bot-users, specifically https://my.slack.com/services/new/bot

# Install

    virtualenv venv
    source ./venv/bin/activate

    pip3 install -r requirements.txt

# Usage

## Setup defaults

    export API_TOKEN="your-api-token"

## Get base information

Only include those channels written to in last 3 months, and with a Jaccard distance of 0.7 or less

    python3 crawl.py --token $API_TOKEN --distance 0.7 --recency $(date -v-3m +%s) --out crawl.json

## Visualise channel similarity

    python3 visualise.py --in crawl.json --out vis.json
    python3 -m http.server 8000 &
    open http://localhost:8000
    
## Recommend channels

    Assuming some user-name:

    python3 recommend.py --token $API_TOKEN --in crawl.json --user user-name
{'fraction_non_alphanumeric': 0.09260120986505352, 'fraction_numerical': 0.01814797580269893, 'mean_word_length': 3.223968565815324, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1931863', 'n_tokens_mistral': 687, 'n_tokens_neox': 626, 'n_words': 242}