<jupyter_start><jupyter_text>Day 3: Inference with seq2seq model -Outline- 1. Intro (happening now) 2. Today application task: open-domain dialogue bot (chatbot) 3. Dataset: Persona-Chat (convai2 official training data) sections 4, 5 are presented in the second notebook 4. Seq2seq recap: traininng our model 5. Inference: how to extract a response from our model *** 2. Today application task: open-domain dialogue bot (chatbot)<jupyter_code># sanity check for all our data files

! ls -l ../data/convai2*
# lets see how our samples look like

! head -n 20 ../data/convai2_simple_train_wpersona_with_starts.txt<jupyter_output>0	 i like to remodel homes. i like to go hunting. i like to shoot a bow. my favorite holiday is halloween.hi , how are you doing ? i'm getting ready to do some cheetah chasing to stay in shape .	you must be very fast . hunting is one of my favorite hobbies .
0	i am ! for my hobby i like to do canning or some whittling .	i also remodel homes when i am not out bow hunting .
0	that's neat . when i was in high school i placed 6th in 100m dash !	that's awesome . do you have a favorite season or time of year ?
0	i do not . but i do have a favorite meat since that is all i eat exclusively .	what is your favorite meat to eat ?
0	i would have to say its prime rib . do you have any favorite foods ?	i like chicken or macaroni and cheese .
0	do you have anything planned for today ? i think i am going to do some canning .	i am going to watch football . what are you canning ?
0	i think i will can some jam . do you also play footfall for fun ?	if i have time outside of hunting and remodeling homes . [...]<jupyter_text>*discuss* --> any similar task we have discussed so far? Any particular differences?Today we are going to make an open-domain chat bot. We will become more familiar with typical issues arising in the context of dialogue modeling. *** 3. Dataset: Persona-Chat (convai2 official training data) Competition [link](http://convai.io/). There exist many other open-domain datasets. Main feature of convai2 is that it provides additional grounding context for every dialogue which makes it much more interesting.![Example dialogue](nbimgs/personachat.png)This dataset is fully supported by ParlAI framework. Here we are going to avoid large dependencies, so we make our own dataset handler for persona chat.<jupyter_code>from chat_dataset import TextDataset
TextDataset?
train_dataset = TextDataset('../data/convai2_simple_train_wpersona_with_starts.txt', device='cpu')
train_dataset??<jupyter_output><empty_output><jupyter_text>Lets check dataset content<jupyter_code>print('Vocabulary size: {}'.format(train_dataset.get_vocab_size()))

# some words from the dataset
print('\nFirst words: {}'.format([train_dataset.ind2word[i] for i in range(20)]))

# counts of those words
print('\nFirst words counts: {}'.format([train_dataset.counts[train_dataset.ind2word[i]] for i in range(20)]))
# get some sample from our dataset

train_dataset[42]
# turn the input back to the text form

train_dataset.pred2text(train_dataset[42][0])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0740142210730446, 'fraction_numerical': 0.013897866839043309, 'mean_word_length': 4.617059891107078, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '649578', 'license': 'MIT', 'n_tokens_mistral': 918, 'n_tokens_neox': 883, 'n_words': 430, 'path': '03-day-inference/intro_and_data.ipynb', 'repo_name': 'nyu-dl/AMMI-2019-NLP-Part2', 'size': '3094'}