<commit_before># NewsCroller

This application is a lightweight news crawler bot that does not use a database.
It will use Twitter as a notification destination.
It has confirmed the operation in Raspbian operating at Raspberry pi.
![add separate image](./view.png "add separate image.")

# Usage
It will set up a CSV format file that describes the NewsCroller.sh and crawled away.
It is registered in the crontab.
Described example , please refer to the target_list_hatebu.txt.
~~~
0 * * * * /home/hoge/NewsCroller.sh /home/hoge/target_list_hatebu.txt > /dev/null 2>&
~~~

# Config
###slado_replace.sh
You can change the header of the message.
~~~
MSG_BOT='[自動ツイート]'
~~~
<commit_msg>added a description about rss file.<commit_after># NewsCroller

This application is a lightweight news crawler bot that does not use a database.
It will use Twitter as a notification destination.
It has confirmed the operation in Raspbian operating at Raspberry pi.
![add separate image](./view.png "add separate image.")

# Usage
It will set up a CSV format file that describes the NewsCroller.sh and crawled away.
It is registered in the crontab.
Described example , please refer to the target_list_hatebu.txt.
It supports the RSS as the analysis data.
~~~
0 * * * * /home/hoge/NewsCroller.sh /home/hoge/target_list_hatebu.txt > /dev/null 2>&
~~~

# Config
###slado_replace.sh
You can change the header of the message.
~~~
MSG_BOT='[自動ツイート]'
~~~

{'fraction_non_alphanumeric': 0.09217877094972067, 'fraction_numerical': 0.002793296089385475, 'mean_word_length': 4.387218045112782, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6139812', 'n_tokens_mistral': 458, 'n_tokens_neox': 434, 'n_words': 185}