<commit_before># Lists Juggle Browser

An application that takes the wealth of data from Lists Juggler, and tries to
distill it down into a number of reports that answer questions the community might
have.

Here's the source of the Lists Juggler frontend and data:
https://github.com/lhayhurst/xwlists

Huge props to the guys who built that!

I'm also using images from the X-Wing Fandom Wikia, linking to the source from
wherever I use an image.

## Application Stack

The application is written in Ruby on Rails and uses PostgreSQL as its database.

Preferably, use something like RVM to handle your Rubies and Gemsets. Then checkout
the repository, make sure you're using Ruby 2.4, and...

```bash
bundle
rake db:create db:migrate
```

...and done. It might not work as easily on Windows, because Windows.

Importing all the data (takes a while):

```bash
rake sync:lists_juggler
```

## Current Status

Currently, the application can import data from Lists Juggler through their HTML lists
and CSV exports, and wrangle it into new tables to hold everything. There's also some
pre-computation of values for statistics.

Next up will be a little UI for the whole thing, and then lots of heavy lifting with
the data.<commit_msg>Readme changed<commit_after># Lists Juggle Browser

An application that takes the wealth of data from Lists Juggler, and tries to
distill it down into a number of reports that answer questions the community might
have.

Here's the source of the Lists Juggler frontend and data:
https://github.com/lhayhurst/xwlists

Huge props to the guys who built that!

I'm also using images from the X-Wing Fandom Wikia, linking to the source from
wherever I use an image.

And finally, thanks to everybody who helps with the X-Wing Miniatures Font:
https://github.com/geordanr/xwing-miniatures-font

## Application Stack

The application is written in Ruby on Rails and uses PostgreSQL as its database.

Preferably, use something like RVM to handle your Rubies and Gemsets. Then checkout
the repository, make sure you're using Ruby 2.4, and...

```bash
bundle
rake db:create db:migrate
```

...and done. It might not work as easily on Windows, because Windows.

Importing all the data (takes a while):

```bash
rake sync:all
rake sync:rebuild_wikia_images
```

You'll need to fix some Wikia links afterwards (from detail pages), but
most should be correct.
{'fraction_non_alphanumeric': 0.05773282764433207, 'fraction_numerical': 0.001685630004214075, 'mean_word_length': 4.3589164785553045, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3969371', 'n_tokens_mistral': 694, 'n_tokens_neox': 657, 'n_words': 358}