<commit_before>anime-scraper
=============

.. image:: banner.png

This is a simple collection of web scrapers that extract the MP4 download URLs from anime stream/download websites.
I made it because of my love for anime and put it on GitHub to share with my IRL friends who are crazy about anime like me.

This scraper simply fetches the download URLs for the episodes. It can also add the download URLs directly to IDM (using the idman
command line utility) with the correct episode names.

**Note**: You must add the directory where IDM is installed (C:\Program Files (x86)\Internet Download Manager by default) to your system
environment variable PATH before trying to use this script to add downloads to IDM.

Usage
-----

::

    usage: pdl.py [-h] [--missing] [--auto] [--start START] [--end END] url

    positional arguments:
      url                   URL to the page of the list of episodes of the anime

    optional arguments:
      -h, --help            show this help message and exit
      --missing, -m         Fetch downloads URLs only for episodes not present in
                            this directory
      --auto, -a            Automatically add the downloads to IDM using the
                            current directory as the download location
      --start START, -s START
                            The episode number to start fetching from
      --end END, -e END     The episode number to stop fetching at

Example:

To fetch episodes 10-20 of the anime Clannad:<commit_msg>Updated support for watchanime.me<commit_after>anime-scraper
=============

.. image:: banner.png

This is a simple collection of web scrapers that extract the MP4 download URLs from anime stream/download websites.
I made it because of my love for anime and put it on GitHub to share with my IRL friends who are crazy about anime like me.

This scraper simply fetches the download URLs for the episodes. It can also add the download URLs directly to IDM (using the idman command line utility) with the correct episode numbers.

**Note**: You must add the directory where IDM is installed (C:/Program Files (x86)/Internet Download Manager by default) to your system
environment variable PATH before trying to use this script to add downloads to IDM.

Usage
-----

::

    usage: pdl.py [-h] [--missing] [--auto] [--start START] [--end END] url

    positional arguments:
      url                   URL to the page of the list of episodes of the anime

    optional arguments:
      -h, --help            show this help message and exit
      --missing, -m         Fetch downloads URLs only for episodes not present in
                            this directory
      --auto, -a            Automatically add the downloads to IDM using the
                            current directory as the download location
      --start START, -s START
                            The episode number to start fetching from
      --end END, -e END     The episode number to stop fetching at

Example:

To fetch episodes 10-20 of the anime Clannad:
{'fraction_non_alphanumeric': 0.06278763971071663, 'fraction_numerical': 0.0046022353714661405, 'mean_word_length': 2.3624309392265195, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 18, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1613050', 'n_tokens_mistral': 768, 'n_tokens_neox': 718, 'n_words': 420}