<commit_before># thedigitalbrothers redirect

A small app used to redirect thedigitalbrothers.ch to hofratsuess.ch

## Preparing the urls

```bash
# Getting all the links that exist between the 2 websites
wget -r --spider --domains=hofratsuess.ch,thedigitalbrothers.ch http://hofratsuess.ch/ -nv -o links.txt -H

# Isolating thedigitalbrothers urls
cat links.txt | grep "thedigitalbrothers" > links_thedigitalbrothers.txt

# Extract the urls from the log file
cat links_thedigitalbrothers.txt | awk '{gsub("URL\:",""); print}' | awk '{print $3}' | grep -v '^$' | grep 'http://' > urls_thedigitalbrothers.txt
```<commit_msg>adding a description for what this command line calls are being used<commit_after># thedigitalbrothers redirect

A small app used to redirect thedigitalbrothers.ch to hofratsuess.ch

## Preparing the urls

Below is a recipe to extract all urls with the site to be redirected. It's being used after the redirection is in place to check that all links point to their respective new targets.

```bash
# Getting all the links that exist between the 2 websites
wget -r --spider --domains=hofratsuess.ch,thedigitalbrothers.ch http://hofratsuess.ch/ -nv -o links.txt -H

# Isolating thedigitalbrothers urls
cat links.txt | grep "thedigitalbrothers" > links_thedigitalbrothers.txt

# Extract the urls from the log file
cat links_thedigitalbrothers.txt | awk '{gsub("URL\:",""); print}' | awk '{print $3}' | grep -v '^$' | grep 'http://' > urls_thedigitalbrothers.txt
```
{'fraction_non_alphanumeric': 0.10572390572390572, 'fraction_numerical': 0.0026936026936026937, 'mean_word_length': 4.737451737451737, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4381386', 'n_tokens_mistral': 512, 'n_tokens_neox': 477, 'n_words': 171}