<jupyter_start><jupyter_text>Building a Simple Bot with the Rasa Stack![alt text](https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTaX3LNhGcAe1HnPZSuWS0oH6af0LJHXcH7If1sQgLCFAT1chNGFg) Please make a copy of this notebook first!If you're interested in building chatbots or voice apps with the [Rasa Stack](https://https://rasa.com/products/rasa-stack/), you've come to the right place! In this notebook you'll build a complete chatbot with Rasa Stack, without having to install anything on your computer. Using Colaboratory, everything will get installed in a cloud machine for you, so you can focus on creating an interesting bot. We'll build a relatively simple bot, that just asks you about your mood and tries to cheer you up if you're feeling a bit down. The tutorial consists of three parts:*   Part 1: We'll start with a basic bot that can handle multi-turn dialogues but not understand natural language like "I'm doing well"*   Part 2: We'll add the abilitiy to understand natural language so that the bot can generalise to similar inputs that it hasn't seen before*   Part 3: We'll give you further resources so you can build your own.  Part 1: Build a basic bot  InstallationFirst we'll have to install Rasa Core on the server running this notebook. In this first step, your bot won't understand natural language yet. This happens in Step 2. **Run the cell below, this will take a couple of minutes to complete so this is a good time to scroll through the rest of the notebook.**<jupyter_code>!pip install rasa_core;
!echo "Europe/Berlin" > /etc/timezone

import logging, io, json, warnings
logging.basicConfig(level="INFO")
warnings.filterwarnings('ignore')<jupyter_output>Collecting rasa_core
[?25l  Downloading https://files.pythonhosted.org/packages/2f/29/bdef80a262a00b550b313b835848b61b08f5bca232f17076d450b823af60/rasa_core-0.13.4-py3-none-any.whl (204kB)
[K    100% |████████████████████████████████| 215kB 28.6MB/s 
[?25hCollecting ruamel.yaml~=0.15.0 (from rasa_core)
[?25l  Downloading https://files.pythonhosted.org/packages/36/e1/cc2fa400fa5ffde3efa834ceb15c464075586de05ca3c553753dcd6f1d3b/ruamel.yaml-0.15.89-cp36-cp36m-manylinux1_x86_64.whl (651kB)
[K    100% |████████████████████████████████| 655kB 20.2MB/s 
[?25hCollecting python-dateutil~=2.7 (from rasa_core)
[?25l  Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB)
[K    100% |████████████████████████████████| 235kB 34.7MB/s 
[?25hCollecting twilio~=6.0 (from rasa_core)
[?25l  Downloading https://files.pythonhosted.org/packages/36/2f/cbe1c967722486ba08eb11fecc6daeb837[...]<jupyter_text>Writing StoriesA good place to start is by writing a few stories. These are example conversations that Rasa Core will learn from. The format works like this:A story starts with `` and you can give it a name. lines that start with `*` are messages sent by the user. Although you don't write the *actual* message, but rather the intent (and the entities) that represent what the user *means*. If you don't know about intents and entities, don't worry! We will talk about them more later. Lines that start with `-` are *actions* taken by your bot. In this case all of our actions are just messages sent back to the user, like `utter_greet`, but in general an action can do anything, including calling an API and interacting with the outside world.<jupyter_code>stories_md = """
## happy path               <!-- name of the story - just for debugging -->
* greet              
  - utter_greet
* mood_great               <!-- user utterance, in format intent[entities] -->
  - utter_happy
* mood_affirm
  - utter_happy
* mood_affirm
  - utter_goodbye
  
  
## sad path 1               <!-- this is already the start of the next story -->
* greet
  - utter_greet             <!-- action the bot should execute -->
* mood_unhappy
  - utter_cheer_up
  - utter_did_that_help
* mood_affirm
  - utter_happy
  
  

## sad path 2
* greet
  - utter_greet
* mood_unhappy
  - utter_cheer_up
  - utter_did_that_help
* mood_deny
  - utter_goodbye
  
## strange user
* mood_affirm
  - utter_happy
* mood_affirm
  - utter_unclear

## say goodbye
* goodbye
  - utter_goodbye
  - utter_goodday
  



"""

%store stories_md > stories.md<jupyter_output>Writing 'stories_md' (str) to file 'stories.md'.<jupyter_text>Defining a DomainThe domain specifies the universe that your bot lives in. You should list all of the intents and actions that show up in your stories. This is also the place to write templates, which contain the messages your bot can send back<jupyter_code>domain_yml = """
intents:
  - greet
  - goodbye
  - mood_affirm
  - mood_deny
  - mood_great
  - mood_unhappy

actions:
- utter_greet
- utter_cheer_up
- utter_did_that_help
- utter_happy
- utter_goodbye
- utter_unclear

templates:
  utter_greet:
  - text: "Hey! How are you?"

  utter_cheer_up:
  - text: "Here is something to cheer you up:"
    image: "https://i.imgur.com/nGF1K8f.jpg"

  utter_did_that_help:
  - text: "Did that help you?"

  utter_unclear:
  - text: "I am not sure what you are aiming for."
  
  utter_happy:
  - text: "Great carry on!"

  utter_goodbye:
  - text: "Bye"
"""

%store domain_yml > domain.yml<jupyter_output>Writing 'domain_yml' (str) to file 'domain.yml'.<jupyter_text>Training your Dialogue ModelNow comes the fun part! We're going to show Rasa Core the stories we wrote above, and train a model on these examples. In this case, the model is a neural network implemented in Keras which learns to predict which action to take next.<jupyter_code>from rasa_core.policies.keras_policy import KerasPolicy
from rasa_core.agent import Agent

agent = Agent('domain.yml', policies=[KerasPolicy()])
agent.train(
        'stories.md',
        validation_split=0.0,
        max_history=3,
        epochs=400
);

agent.persist('models/dialogue');<jupyter_output>INFO:apscheduler.scheduler:Scheduler started
Using TensorFlow backend.
Processed Story Blocks: 100%|██████████| 5/5 [00:00<00:00, 180.11it/s, # trackers=1, samples=33]
Processed Story Blocks: 100%|██████████| 5/5 [00:00<00:00, 135.31it/s, # trackers=5, samples=126]
Processed Story Blocks: 100%|██████████| 5/5 [00:00<00:00, 100.27it/s, # trackers=9, samples=239]
Processed Story Blocks: 100%|██████████| 5/5 [00:00<00:00, 119.84it/s, # trackers=9, samples=352]
INFO:rasa_core.policies.keras_policy:Fitting model with 52 total samples and a validation split of 0.0<jupyter_text>Starting up the botLets start up the bot!<jupyter_code>from rasa_core.agent import Agent
agent = Agent.load('models/dialogue')<jupyter_output><empty_output><jupyter_text>Talking to the bot (no NLU)In this step, we just trained the dialogue model - so basically the conversational flow. So the bot will only understand **structured** input and **no** natural language yet. Go try it out with typing  "/" + one of the intents from your domain before, e.g.:/greet/mood_affirm/mood_deny<jupyter_code>print("Your bot is ready to talk! Type your messages here or send 'stop'")
while True:
  a = input()
  if a == 'stop':
    break
  responses = agent.handle_message(a)
  for response in responses:
    print(response)<jupyter_output>Your bot is ready to talk! Type your messages here or send 'stop'<jupyter_text>Part 2: Add natural language understandingSo far, our bot doesn't understand regular sentences with natural language yet.To do that, we're going to build a language understanding model with Rasa NLU.   Installing NLUWe have to install a couple of more things in this notebook here. **Run the cell below** - in the meantime, you can check out the rest of the code.<jupyter_code>!apt-get -qq install -y graphviz libgraphviz-dev pkg-config && pip install pygraphviz;
!pip install rasa_nlu;
!pip install spacy;
!python -m spacy download en;<jupyter_output>Extracting templates from packages: 100%
Selecting previously unselected package fontconfig.
(Reading database ... 16712 files and directories currently installed.)
Preparing to unpack .../00-fontconfig_2.11.94-0ubuntu2_amd64.deb ...
Unpacking fontconfig (2.11.94-0ubuntu2) ...
Selecting previously unselected package libxinerama1:amd64.
Preparing to unpack .../01-libxinerama1_2%3a1.1.3-1_amd64.deb ...
Unpacking libxinerama1:amd64 (2:1.1.3-1) ...
Selecting previously unselected package libjbig0:amd64.
Preparing to unpack .../02-libjbig0_2.1-3.1_amd64.deb ...
Unpacking libjbig0:amd64 (2.1-3.1) ...
Selecting previously unselected package libapparmor1:amd64.
Preparing to unpack .../03-libapparmor1_2.11.0-2ubuntu17.1_amd64.deb ...
Unpacking libapparmor1:amd64 (2.11.0-2ubuntu17.1) ...
Selecting previously unselected package libdbus-1-3:amd64.
Preparing to unpack .../04-libdbus-1-3_1.10.22-1ubuntu1_amd64.deb ...
Unpacking libdbus-1-3:amd64 (1.10.22-1ubuntu1) ...
Selecting previously unselected[...]<jupyter_text>Language UnderstandingLets create some training data here, grouping user messages by their `intent`s. The intent describes what the messages *mean*.<jupyter_code>nlu_md = """
## intent:greet
- hey
- hello
- hi
- hello there
- good morning
- good evening
- moin
- hey there
- let's go
- hey dude
- goodmorning
- goodevening
- good afternoon

## intent:goodbye
- cu
- good by
- cee you later
- good night
- good afternoon
- bye
- goodbye
- have a nice day
- see you around
- bye bye
- see you later

## intent:mood_affirm
- yes
- indeed
- of course
- that sounds good
- correct

## intent:mood_deny
- no
- never
- I don't think so
- don't like that
- no way
- not really

## intent:mood_great
- perfect
- very good
- great
- amazing
- feeling like a king
- wonderful
- I am feeling very good
- I am great
- I am amazing
- I am going to save the world
- super
- extremely good
- so so perfect
- so good
- so perfect

## intent:mood_unhappy
- my day was horrible
- I am sad
- I don't feel very well
- I am disappointed
- super sad
- I'm so sad
- sad
- very sad
- unhappy
- bad
- very bad
- awful
- terrible
- not so good
- not very good
- extremly sad
- so saad
- so sad
"""

%store nlu_md > nlu.md<jupyter_output>Writing 'nlu_md' (str) to file 'nlu.md'.<jupyter_text>Train the Rasa NLU ModelWe're going to train a model to recognise these intents, so that when we send a message like "hello" to our bot, it will recognise this as a "greet" intent.<jupyter_code>from rasa_nlu.converters import load_data
from rasa_nlu.config import RasaNLUConfig
from rasa_nlu.model import Trainer

training_data = load_data('nlu.md')
pipeline= ["nlp_spacy", "tokenizer_spacy", "intent_featurizer_spacy",  "intent_classifier_sklearn"]
trainer = Trainer(RasaNLUConfig(cmdline_args={"pipeline": pipeline}))
interpreter = trainer.train(training_data)
model_directory = trainer.persist('./projects/default/')<jupyter_output>INFO:rasa_nlu.converters:Training data format at nlu.md is md
INFO:rasa_nlu.training_data:Training data stats: 
	- intent examples: 68 (6 distinct intents)
	- found intents: 'goodbye', 'greet', 'mood_affirm', 'mood_deny', 'mood_great', 'mood_unhappy'
	- entity examples: 0 (0 distinct entities)
	- found entities: 

INFO:rasa_nlu.utils.spacy_utils:Trying to load spacy model with name 'en'
INFO:rasa_nlu.components:Added 'nlp_spacy' to component cache. Key 'nlp_spacy-en'.
INFO:rasa_nlu.model:Starting to train component nlp_spacy
INFO:rasa_nlu.model:Finished training component.
INFO:rasa_nlu.model:Starting to train component tokenizer_spacy
INFO:rasa_nlu.model:Finished training component.
INFO:rasa_nlu.model:Starting to train component intent_featurizer_spacy
INFO:rasa_nlu.model:Finished training component.
INFO:rasa_nlu.model:Starting to train component intent_classifier_sklearn
[Parallel(n_jobs=1)]: Done  12 out of  12 | elapsed:    0.1s finished
INFO:rasa_nlu.model:Finished training comp[...]<jupyter_text>Starting up the bot (with NLU)Now that we've trained the dialogue **and** language understanding models and saved them, we can start up an `Agent` which will handle conversations for us.<jupyter_code>from rasa_core.agent import Agent
agent = Agent.load('models/dialogue', interpreter=model_directory)<jupyter_output>INFO:rasa_nlu.components:Added 'nlp_spacy' to component cache. Key 'nlp_spacy-en'.<jupyter_text>Talking to the Bot (with NLU)We can start talking to the bot in natural language.<jupyter_code>print("Your bot is ready to talk! Type your messages here or send 'stop'")
while True:
  a = input()
  if a == 'stop':
    break
  responses = agent.handle_message(a)
  for response in responses:
    print(response)<jupyter_output>Your bot is ready to talk! Type your messages here or send 'stop'
hey<jupyter_text>Pro Tip: Visualising the Training DataYou can visualise the stories to get a sense of how the conversations go. This is usually a good way to see if there are any stories which don't make sense<jupyter_code>from rasa_core.agent import Agent
from IPython.display import Image

agent = Agent.load('models/dialogue')
agent.visualize("stories.md", "story_graph.png", max_history=2)
Image(filename="story_graph.png")<jupyter_output>Processed Story Blocks: 100%|██████████| 5/5 [00:00<00:00, 253.84it/s, # trackers=1, samples=33]
Processed Story Blocks: 100%|██████████| 5/5 [00:00<00:00, 38908.20it/s]
Processed Story Blocks: 100%|██████████| 5/5 [00:00<00:00, 14716.86it/s]
{'fraction_non_alphanumeric': 0.10741130487071557, 'fraction_numerical': 0.03991280817799158, 'mean_word_length': 4.532224532224532, 'pattern_counts': {'":': 1, '<': 49, '<?xml version=': 0, '>': 46, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '332672', 'license': 'MIT', 'n_tokens_mistral': 4961, 'n_tokens_neox': 4458, 'n_words': 1442, 'path': 'Building_a_Simple_Bot_with_Rasa_Stack_Tutorial.ipynb', 'repo_name': 'villain31/chatbot', 'size': '13304'}