<commit_before># `nsndswap`

A Python program that parses [this website](http://xzazupsilon.webs.com/nsnd.html) and [this one](https://wheals.github.io/canwc/nsnd.html), and outputs [gexf files](https://gephi.org/gexf/format/).

# How to use

Run `run.sh`. For the most part, the copious logs can be ignored. This creates four files in the working directory: `homestuck.gexf`, `canwc.gexf`, `almost_everything.gexf`, and `everything.gexf`, containing different subsets of the data used. All node positions are randomized so that a positioning algorithm in Gephi has better luck.

# Output files

The following files are output to the `output/` directory:

- `homestuck` contains details of all songs on the first page (Homestuck soundtrack, unofficialmspafans, Homestuck Gaiden, and miscellaneous other things).
- `canwc` contains details of all songs on the second page (the Cool and New Web Comic soundtrack).
- `almost_everything` contains both of the previous files' data.
- `everything` contains the previous file's data and a few additions of my own (which you can find in `viko_nsnd.py`, if you're interested in their raw form).

These files are dumped in four formats:

- `.gexf` - directed graphs of references
- `.txt` - simple _ad hoc_ plain-text format
- `.titles.txt` - the titles, one per line
- `.reverse.txt` - the format in `.txt`, but showing incoming references rather than outgoing<commit_msg>Remove outdated info about output files from README.md<commit_after># `nsndswap`

A Python program that parses [this website](http://xzazupsilon.webs.com/nsnd.html) and [this one](https://wheals.github.io/canwc/nsnd.html), and outputs [gexf files](https://gephi.org/gexf/format/).

# How to use

Run `run.sh`. For the most part, the copious logs can be ignored.

# Output files

The following files are output to the `output/` directory:

- `homestuck` contains details of all songs on the first page (Homestuck soundtrack, unofficialmspafans, Homestuck Gaiden, and miscellaneous other things).
- `canwc` contains details of all songs on the second page (the Cool and New Web Comic soundtrack).
- `almost_everything` contains both of the previous files' data.
- `everything` contains the previous file's data and a few additions of my own (which you can find in `viko_nsnd.py`, if you're interested in their raw form).

These files are dumped in four formats:

- `.gexf` - directed graphs of references
- `.txt` - simple _ad hoc_ plain-text format
- `.titles.txt` - the titles, one per line
- `.reverse.txt` - the format in `.txt`, but showing incoming references rather than outgoing
{'fraction_non_alphanumeric': 0.09626492106276473, 'fraction_numerical': 0.0, 'mean_word_length': 4.345679012345679, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '548297', 'n_tokens_mistral': 783, 'n_tokens_neox': 740, 'n_words': 337}