<commit_before># Delbot™

It understands your voice commands, searches news and knowledge sources, and summarizes and reads out content to you.

Check out the demo [video](https://youtu.be/iVmj1gHOF0w) or read the article on [LinkedIn](https://www.linkedin.com/pulse/delbot-building-bot-from-scratch-python-shail-deliwala).

# How to Run
1. Install the required [packages](https://github.com/shaildeliwala/delbot#libraries).
2. Open a command prompt and navigate to root folder of project.
3. Enter `python app.py` in command prompt to launch web service.
4. Go to http://localhost:5000 (or whichever IP and port you specified).

# Index
1. [Introduction](https://github.com/shaildeliwala/delbot#introduction)
2. [Overview](https://github.com/shaildeliwala/delbot#overview)
    1. [News](https://github.com/shaildeliwala/delbot#news)
    2. [Knowledge](https://github.com/shaildeliwala/delbot#knowledge)
3. [How It Works](https://github.com/shaildeliwala/delbot#how-it-works)
    1. [News Queries](https://github.com/shaildeliwala/delbot#news-queries)
        1. [Parts of speech and tags](https://github.com/shaildeliwala/delbot#parts-of-speech-and-tags)
        2. [Noun chunks](https://github.com/shaildeliwala/delbot#noun-chunks)
        3. [Adpositions? Did you mean prepositions?](https://github.com/shaildeliwala/delbot#adpositions-did-you-mean-prepositions)
        4. [Implementation](https://github.com/shaildeliwala/delbot#implementation)
    2. [Knowledge Queries](https://github.com/shaildeliwala/delbot#knowledge-queries)
        1. [Parts of speech and tags](https://github.com/shaildeliwala/delbot#parts-of-speech-and-tags-1)
        2. [Noun chunks](https://github.com/shaildeliwala/delbot#noun-chunks-1)
        3. [Auxiliary verbs (or their absence)](https://github.com/shaildeliwala/delbot#auxiliary-verbs-or-their-absence)
        4. [Implementation](https://github.com/shaildeliwala/delbot#implementation-1)
4. [Summarization](https://github.com/shaildeliwala/delbot#summarization)
5. [Libraries](https://github.com/shaildeliwala/delbot#libraries)
6. [Web App](https://github.com/shaildeliwala/delbot#web-app-optional)
7. [Limitations](https://github.com/shaildeliwala/delbot#limitations)
8. [Conclusion and Future Work](https://github.com/shaildeliwala/delbot#conclusion-and-future-work)
9. [Demo](https://github.com/shaildeliwala/delbot#demo)
10. [References and Links](https://github.com/shaildeliwala/delbot#references-and-links)

# Introduction
Bots remain a hot topic. Everyone is talking about them.

How about building one from scratch? The simple one we will build today will understand and answer questions like:
- What is the latest news on Star Wars in the New York Times?<commit_msg>Added roadmap<commit_after># Delbot™

It understands your voice commands, searches news and knowledge sources, and summarizes and reads out content to you.

Check out the demo [video](https://youtu.be/iVmj1gHOF0w) or read the article on [LinkedIn](https://www.linkedin.com/pulse/delbot-building-bot-from-scratch-python-shail-deliwala).

# How to Run
1. Install the required [packages](https://github.com/shaildeliwala/delbot#libraries).
2. Open a command prompt and navigate to root folder of project.
3. Enter `python app.py` in command prompt to launch web service.
4. Go to http://localhost:5000 (or whichever IP and port you specified).

# Roadmap
1. Statistical model to determine category such as _who_, _why_, _what_, and _when_ of knowledge questions.
2. Headlines-only news request.
3. Better UI!

# Index
1. [Introduction](https://github.com/shaildeliwala/delbot#introduction)
2. [Overview](https://github.com/shaildeliwala/delbot#overview)
    1. [News](https://github.com/shaildeliwala/delbot#news)
    2. [Knowledge](https://github.com/shaildeliwala/delbot#knowledge)
3. [How It Works](https://github.com/shaildeliwala/delbot#how-it-works)
    1. [News Queries](https://github.com/shaildeliwala/delbot#news-queries)
        1. [Parts of speech and tags](https://github.com/shaildeliwala/delbot#parts-of-speech-and-tags)
        2. [Noun chunks](https://github.com/shaildeliwala/delbot#noun-chunks)
        3. [Adpositions? Did you mean prepositions?](https://github.com/shaildeliwala/delbot#adpositions-did-you-mean-prepositions)
        4. [Implementation](https://github.com/shaildeliwala/delbot#implementation)
    2. [Knowledge Queries](https://github.com/shaildeliwala/delbot#knowledge-queries)
        1. [Parts of speech and tags](https://github.com/shaildeliwala/delbot#parts-of-speech-and-tags-1)
        2. [Noun chunks](https://github.com/shaildeliwala/delbot#noun-chunks-1)
        3. [Auxiliary verbs (or their absence)](https://github.com/shaildeliwala/delbot#auxiliary-verbs-or-their-absence)
        4. [Implementation](https://github.com/shaildeliwala/delbot#implementation-1)
4. [Summarization](https://github.com/shaildeliwala/delbot#summarization)
5. [Libraries](https://github.com/shaildeliwala/delbot#libraries)
6. [Web App](https://github.com/shaildeliwala/delbot#web-app-optional)
7. [Limitations](https://github.com/shaildeliwala/delbot#limitations)
8. [Conclusion and Future Work](https://github.com/shaildeliwala/delbot#conclusion-and-future-work)
9. [Demo](https://github.com/shaildeliwala/delbot#demo)
10. [References and Links](https://github.com/shaildeliwala/delbot#references-and-links)

# Introduction
Bots remain a hot topic. Everyone is talking about them.

How about building one from scratch? The simple one we will build today will understand and answer questions like:
- What is the latest news on Star Wars in the New York Times?
{'fraction_non_alphanumeric': 0.13515443670773075, 'fraction_numerical': 0.01339046598821639, 'mean_word_length': 4.635814889336016, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 50, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1576645', 'n_tokens_mistral': 1905, 'n_tokens_neox': 1840, 'n_words': 402}