<jupyter_start><jupyter_text>**Deep Learning ChatBot**In this project, I am going to build ChatBot using seq2seq model or encoder-decoder model in TensorFlow. The objective of the model is manuplating given dataset to make a simple chat bot. **What is Seq2Seq model** Seq2Seq is a method of encoder-decoder based machine translation and language processing that maps an input of sequence to an output of sequence with a tag and attention value. The idea is to use 2 RNNs that will work together with a special token and try to predict the next state sequence from the previous sequence. ![enco_deco.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAq4AAAHHCAMAAABJMio7AAABhlBMVEX////c59Ld8PgBBAADBgL9/f3b7vb7+/sFCARbY1e0vqyHiYeksre7vLs6QUGSm4uJi4kRFBA9RER2gYWqq6prbWtFR0S8vbzR0tFERkMgJCG5urnI1MA9PzwhJh5LUUewwMUzNjImKiXN3+ahr7Sir7SoqagMDwvo6OhZW1nz8/O+ybbt7e3d3t2Njo13goZXWVY2ODUtMS+2t7Z4gHPb7fXv7+/i4uLS3sm/wL+anJpsdnlLU1Q4PjRqcmUpLSuep5dmaGZMTkw5OznMzcyChIIvNTTCzbmGiIZocnRCREH5+fnJ2+K/z9aSn6OgoaBYYWNGSUbQ4ejd3d3Y5M/O2sXDw8OGkpb19fXm5uaVo6eGjn8cHxvs7OyrrKups6FfYV/E1dzB0ditt6aRmop9hXdaY2VSVFE+RDoVGRXZ6vK2xctrc2bq6urW19aDj5OPkY9+io3F0b13eXdpcWRPV1hOUE2+ztXT1NPQ3MeZp6uYoZGVnY59fn1hY2FHTURwe35udmkpEwbvAAATX0lEQVR42uzbMYvbQBCG4YVPSwohkCphX+UUNgQZ7JRugkBFOmOuSWOTNs01blLdb488yhE54c6prB3ufQqBd9oXMUhyAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAY6dNpVneBCB9rcziGIDUnTKpmktqYwASt7VQd5Iew7U6zx/D68p8R+C4ryaTjkO1f9dXSg9vBJlpRq64rzrTIg5ttuSKxJUaqjv19ZErEldK5zjcZasY1kWxDqGxa91KeRft6GRb7Mu47q/rItPigadfuKuNlP/OVTGU9qvOLtdMvVm8HG0lLWP4M/4aSxs/cH/FPR3+I9dKvTm5YnKv52q7qx1pWXzsL3GcK7srbpsk13YYkCsmdTvX0dGGXDG127naGtD1cZIrpna+matdA7kiASW5wo9SVp29jCVXJK626oZs39hda3JFAhpbAuwj7ZwnA0jd1hJsfkid3WvbGE7XuY5DtTG5Yio7SW0+H96yrjPpnFfScpyr8p0djcbkimlsZYb3/weZ+TGMcq3s6GrMB4SYRtNagctoPyzIWRfGueaHft7F0TiSK6bSFEU3/tH882C2K+LLuC74yBXJuuTKXRROkCscIVc4Qq5whFzhiP37FQAAAAAAvEf7Pc+w4MXTavUzAC58+yx9+RQAD57V+846AA/2mS5YX+HA00qG9RXp6xdXw/oKB55lWF/hRqYPAXCCXOEIucIRcoUj5ApHyBWOkCscIVc4Qq5whFzhCLnCEXKFI+QKR36xd6+tTQRhFMdn56wSBF+sGMXLmiaCMSLFJJhQElpMiFErBYMtVqIYW2nVFqvUghdQv7mKLih4GYXN5Azn9xGGP8Ozs5ONchUiylWIKFcholyFiHIVIspViChXIaJchYhyFSLKVYgoVyGiXIWIchUiylWIKFcholyFiHIVIspViChXIaJchYhyFSLKVYgoVyGiXIWIcv27VnNQ64+rmGXVcb02aLbMjNCSedKe78dgEffn28a79nyqJfNirRcD2F7c3Bi+iWbZm+HG5uI2gLi3YLxaOygAONZZmZuM7CwbTeZWOse+LVkQ/yu3kAIovbsbsbj77gBA6ivYbMm6b59ZFlffvgaQnqQPdr0GVPZXIy6rixWgtm68WH8ELN08b7mMOktAjXwkOJ6guhMx2qkiuWA8eJig2jls+RzuJEguEG+wy0WgNIw4DUtAcdlMUbZk3ceW0+MuULxnSF06BTyJeD0B0gdmqi6dAF4ybq3fvYyR7hpKu2WcOxIxO1JB46KZot06Ts9ZZs+X0Fg3hFplNFgHgcyrMsotMzWtMq6zDgKZ232ULxs6y7fQOBuxO1vGranNr8uncP22Zfel11t882sR59j31q+G51A8ZKajiMrE8ptUcI3tfOA4wD23Zo4AF8xUPASe2xA8j0F2ntVOqM8EfvQESdtMQTuhPhP40Ys44XpfsIVSFIoStqaxV2yhG0it1r7GFtMdxQU85X/Mypx9iqMmT9mS8T9mZW6PcYZoHEjB+eb11z4hzX/tU1C+ef2Nj3HKs72uoRKFpIIFk7M1LAVUq7UN8Nwn7GE/Csk+enmv/QFuBpVrJ+6xbK8XC2C7MfhnqyhcNHnJloztxuCfjQoFlnexe/gQheUD9kyu9tALanO1tot5kmkgxfsoLO/zfthKcd+G5T7Lw1YrBs8vXdzcRdwyOWoVwPNLFzdXC/EDw6CJ7Sg022iaHDVxLLBZwNo7aFJMAwMsRqHZwcDk6EZQh67Z2cCAItcr2IxCs4krea59DSvB5boSX6EYXuvYiEKzgXqeufbB/RuCX5mL6xS5jhHCRdefvcI4z1xPI4SLrj+bFMYUuVYx299q+R+rqOaZa4LZ/lbL/xgVqhS5AlF4gDxzjRHc6GptgeNDh8pVuSrXf6Jc/VOuzpSrf8rVmXL1T7k6U67+KVdnytU/5epMufqnXJ0pV/+UqzPl6p9ydaZc/VOuzpSrf8rVmXL1T7k6U67+KVdnytU/5epMufqnXJ0pV/+UqzPl6p9ydaZc/VOuzpSrf8rVmXL1T7k6U67+KVdnytU/5epMufqnXJ0pV/+UqzPl6p9ydaZc/VOuzpSrf8rVmXL1T7k6U67+KVdnytU/5epMufqnXJ0pV/+UqzPl6p9ydaZc/VOuzpTrZ/bu77VpKIoDePRbCx2jodpArSUtrc7+gE5mhTLW2afCBtPBYIPtYT4NfPJRX/RvN/ekxyU1uLZkNrf5fhF77s11Tx9uT2+TufmQ69Ih182HXJcOuW4+5Lp0yHXzIdelQ66bD7kuHXLdfMh16ZDr5kOuS4dcNx9yXTGPz7X2sw2v2ndqVU03mO1XqzXHiU2+0opcM5YccX0FSc2pQiNOgcNnTmzS04pcM5b8cO0iSK+9yNWMPHIl140lmesQOL0I9tgLGQG6xtNSJ2Wqxt41i8kP1x5wqXWUaxtAjVzJdRNJ5qoWE0ZdwMMeuZLrinl8rt0Erq/Q9jB8Rq7kuqEkc/WA9wlcD+Ed4pRcyXVTSeZaBdrdv7n2MKwCfXIl1w0lmWu/DfT6f3EF9mpAjVzJdVNRivFcQLzGudaAy+DC3gJXifeMXDOWHHF1XonXONcqcOEYmeTqWJA8cRWvP+Ofqn6a1x7ai1zfeZ53SK5ZS664itf3Ma4vzR46BLrsXR0Lki+uziFwGt1Iu3J/SzVATK6OBckZ134buIjIvAT25jdlkasFyRlXlagyq0DP83qAR66OBckb12GcqwcNuToWJG9ce3GuwOletVo1s+RqQfLD9eJyfjTQv+d6Afyc9wR75GpB8sO1htOhB2B4fzKgTJ1LmY5yHfJZrSwmDa5TN0xDykYw03DdaXitMWqieSPl7bGPkjuWWVcyMLWOzD/RsvEoXCXt6N0s2sl2gdMI18d9+OXqhclVR0dhzmR0djfD7ER+/vVRBeUXHZndlyUtU8tIclXU8oxcV0gJYXakdIOZHaAUSobElCNImmO5HmYkeHVUui+n6XPtDxHE6zoRri+BUG9bin9y/dVPietzSCpvipERXpvRFSSmvIPkwHjdLUByLnh1VC5qiW9Fcl2TazPCdQAo11sA/sQQVaCQQTJXNNLkqmBrta6zdgoHP1LlikprkWurAOV6HZSVuiFaVKAySOaK7+S6CtediNyGcpWhPw3UBmVTbE4j1xsjIR7Zi/WCWZu5X4tRwNvPz1Li+qZ4XTHeZCRS9Uol2CgN45nYvDISBWi5eHYXog5HZrlekLXkuiZXVwHK5nobXhgDGIdqp3+uH8vLIlfZiScZ5Ap86afFNdhJ5SXCVaY+hXUnKDuh2m9FBXoElBO4mp24/oFc1+Lqo/kHoKvszIyZl6lRjPM4gesTAJnkioMfKXEVfCdxrvtA/d7hrBhOnStX4dxJ4Pq0AJDrulzREHXhhZv5BXdO8lZeFSiAncfhiseJaQhS4SoSy3GuzwWwyhWS1+JRgRbMWnJNk2swcFWdD9wqV5ErHv1Yb+vmlWtADXGuFeCTchW5sqiiQGXtC3JNlWsz+BNRpxeOBaZ4xINcB0Apm83Ax5rjPBrXwn15JDDDRR8e4toK/uZHrfW4wgUaoboGgLFeSOCqe+4iV7GdSa4pftQSnZ0o1zMzoWsSuOqeu8hVbJPriueu2q0OAFfUhTQf4KrrAG1jJ6MJ4I8zyFUOslLkKq/PY4eoxQe46jpA29j6eR2odHjuuibXJxM01+cKSWmQvV9HXDCNQPa4mpRb/FZrJa7NUql0M+c6Albm6peCDKRs+sA4g789W7+ETZ/rrFwun6zEtVIO0pLLBxWgw3sG1u5dzVnVdDWu8d51BLgZ5KpJm6v2rstzjfeu52Ypua7N9YmPY+hHLX1Xv0nmmnQyYHbYnHAtLn7Uas3XnCRzTToZeFPAAbmuzbUR0PRLiwdZ7goHWRMgg72r5r8cZO2vcJBVB9i7rsFVywHgC8BJaFTf9vXLWAWqdxAscJXjre3neg3U41zrojH6rZY0pwpU7yBY5LpfwAm5rs3VMA25HotNSWK3OgYwSLgjC/C3n+sJcBTnegTMwjK5W+2YbiHhjqwCKh/IdX2uNwgBTqFXRKZ5HUXf/qficpGr/JTbrecqe2mM6xW0Fpmmuou+/V8FLpPuyHoOfOLXBOtzHcy5jn3AH4RXmoAbzgxU5Y4PjBK4GsbHW861cye3ssa4dipyz7ZkZpzKTEtV7laA8wSuhvERua7CdRQ+YaVyJ9C3fgO3NJGd1KyaAGYgJ61BDX8cjiZuEG1jBXVjm7nOyoB4lNG5Pm+1jyDlct3spDDzdcAM5KQ1qFHphEbr4bNac66dr4F8cl394RfleqP75QgS2V51lQA10a1XR9rUhi3DNnOVnP/9rNYdJKac6bxwjT4uoyNtauctA7muyTV6S5XrG5XSvQrdya0CxeTYHT9J5Cr77XZzLZ+Y/jXONch+xag0ZUfo1j8VFWj9SB6LTeAq++0Hck0lgx1tbMc7Ow+ep+b+vzBu7e5qK7u72/rHQv6egW2JzVyXC7luUciVXC0KuZKrRSFXcrUo5EquFoVcydWikCu5WhRyJVeLQq7kalHIlVwtCrmSq0UhV3K1KORKrhaFXMnVopAruVoUciVXi0Ku5GpRyJVcLQq5kqtFIVdytSjkSq4WhVzJ1aKQK7laFHIlV4tCruRqUciVXC0KuZKrRSFXcrUo5EquFoVcydWikCu5WhRyJVeLQq7kalHIlVwtCrmSq0UhV3K1KL/Zt/uXpsIwjOPPs+sEh36ITjSC5Uk9gZsUsheayGSjjSHlUJItLCrMFzQ1M8mgF6j+82xwAimIoDN33V2fP+Hmy7N7zzlHuSpXIspVuRJRrsqViHJVrkSUq3IlolyVKxHlqlyJKFflSkS5KlciylW5ElGuypWIclWuRJSrciWiXJUrEeWqXIkoV+VKRLkqVyLKVbkSUa7KlYhyVa5ElKtyJaJclSsR5apciShX5UpEuSpXIspVuRJRrsqViHJVrkSUq3IlolyVKxHlqlyJKFflyqOPULn+nf0wpMi1iA/emglEWeYaYS1nze0wosj1GZa8NUsoZJlrAS9z1mwGBYpcK9jy1myhkmWuHUzlrJkKKhS5lrHtrdlGOctcZ7FibnldCcoUuXbR9NYcousy1EXDXK6NoHvJEahhz1uzh5rLUA3XzOX6CDWKXOsB+t6WPoK7LkP1EPs5W+6FwWtHIcZnb8tnxNmeFDGe5mx5GsQUq6tzx/jibfmCY5epY7SNbQOLmKPYBZwrhRh4SwYISy5TpdDYg4K1MHziSLTxzlvyDu2sT4o2Hpg6XhtBm2QXcO4xEm9JggWXscdomcq1igWSXeBMjG/ejm+Isx99bOrq9SvNH60fbuDVhLdi4hVuuiylI+vlrOg9w02ew9W5XUx6KyaxO4rR72LRzPF6il2iw9W5+Qg73oYdRPMuW+nI3hrp9U1wNjIqM8Blb8FlYMaNxHXgIGfBQYAZplXghzySE8/vJEF+VKPPI7GwvvZauM9Wq1t+iGn+v1sTFcTLLnvpyDr8vfauIF53dJ5XMc3+WcHJNKqv3SikI6ts5rj1OqiWHKHSNG5x76+Xk9GNPh1Zwr2/HrRQpXn6el79IajvB3aA+LkblXRk1PcDbwPEJO8N/mo5D0yyLgRLk0B+2Y1QOrJF1oXg5SJwn3Bv/el6hOKhZ3RYRDTjLsCLCBHn89ivESK6G6zznswCSZPtfcJBMwFmL2AHS0fWarC9T7jWaAGzZE8HfmMhBvD+E8/3MP1P7wHEC+5ipCM7/cjzPcy9j6cAOje4j9Z0+u0AwF5ze+vOYJyr7Q/ubG039wAE7QuK9fzIrjVWpjZvj3O1+7c3p1Ya14YjI3pj8A/m5zoBWASduTH4TSMb2dUxGNm/VK91y5VCFGJ8hVGhUu7W6m5MaGT2rK46ERJHxeKRE6Gw3gJazBfT8j/ZwJkNJ0JgFUNaX4XAURFDWl9l/K23MKT1VQhsYEjrq9AAnAgL5SpElKsQUa5CRLkKEeUqRJSrEFGuQkS5ChHlKkSUqxBRrkJEuQoR5SpElKsQUa5CRLkKEeUqRJSrEPnOzh2rRg4DARgemFKoMCzbu1/D1u4CfqN7fzjW2ISADy7VapLvK8wgt38hhJBcKUSuFCJXCpErhciVQuRKIXKlELlSiFwpRK4UIlcKkSuFyJVC5EohcqUQuVKIXClErhQiVwAAAAAA+PmW1taAGnrmPaAGuVLIN3LtrS0B3/DGXFtmD/h/cuU3kyuFHLn23uOxrksc89Jucc4R2+vbPzLbFvA2Z66ZeZsyp37MmTlH7HNE3DP30SkCb3CZ65RHo+e8ypXBfOba4paZsc/P3o50z1ztXRnAZ64RMZ+5LhFTplwZzJdc718T3eTKWC5znSPimdnlyliuct1XmlwZjlwpRK4U8s+9612uDMfJAIVc57rFYzpXlohZrgzhOte5P3PfEnxkzu01y5UBXOZ63hnYf++aXBnAda63KfNPvKzTq97mviuDysx49CUOW3fJlXGlt4ipQ64UIlcKkSuFyJVCendeBQAAAAAAAAAAAAAAAAAAAAAAAAAAAPxtDw4JAAAAAAT9f+0LEwAAAAAAAAAwCnRA85/M0cwCAAAAAElFTkSuQmCC) ![2rnns.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyAAAAGXCAMAAAB8/sS+AAAC61BMVEX////c59Pe8Pj/<KEY>)<jupyter_code><jupyter_output><empty_output><jupyter_text>**Get the Data**I am using Cornel movie data. Availabe [Here](https://www.cs.cornell.edu/~cristian/Cornell_Movie-Dialogs_Corpus.html)<jupyter_code>lines=open('/content/drive/MyDrive/cornelldataset/movie_lines.txt',encoding='utf-8',errors='ignore').read().split('\n')
convers=open('/content/drive/MyDrive/cornelldataset/movie_conversations.txt',encoding='utf-8',errors='ignore').read().split('\n')

import re
import numpy as np
from keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Dense, Embedding, LSTM, Input
from keras.utils.vis_utils import plot_model
import numpy as np<jupyter_output><empty_output><jupyter_text>**About the data**The cornell movie dataset contains two files. The conversation file contains contais the whole converstion of given ID and lines file contains the Id and dialog.<jupyter_code>len(lines)
len(convers)<jupyter_output><empty_output><jupyter_text>**PreProcess data***<jupyter_code>exchn = []
for conver in convers:
  
    exchn.append(conver.split(' +++$+++ ')[-1][1:-1].replace("'", "").replace(",","").split())

diag = {}
for line in lines:
    diag[line.split(' +++$+++ ')[0]] = line.split(' +++$+++ ')[-1]

questions = []
answers = []

for conver in exchn:

  for i in range(len(conver) - 1):

      questions.append(diag[conver[i]])
      answers.append(diag[conver[i+1]])

#max length of 13
sorted_ques = []
sorted_ans = []
for i in range(len(questions)):
    if len(questions[i]) < 13:
        sorted_ques.append(questions[i])
        sorted_ans.append(answers[i])

def clean_text(txt):
    txt = txt.lower()
    txt = re.sub(r"i'm", "i am", txt)
    txt = re.sub(r"he's", "he is", txt)
    txt = re.sub(r"she's", "she is", txt)
    txt = re.sub(r"that's", "that is", txt)
    txt = re.sub(r"what's", "what is", txt)
    txt = re.sub(r"where's", "where is", txt)
    txt = re.sub(r"\'ll", " will", txt)
    txt = re.sub(r"\'ve", " have", txt)
    txt = re.sub(r"\'re", " are", txt)
    txt = re.sub(r"\'d", " would", txt)
    txt = re.sub(r"won't", "will not", txt)
    txt = re.sub(r"can't", "can not", txt)
    txt = re.sub(r"[^\w\s]", "", txt)
    return txt

clean_ques = []
clean_ans = []

for line in sorted_ques:
    clean_ques.append(clean_text(line))
        
for line in sorted_ans:
    clean_ans.append(clean_text(line))

for i in range(len(clean_ans)):
    clean_ans[i] = ' '.join(clean_ans[i].split()[:11])

#since data is big we will only take its portion
## trimming
clean_ans=clean_ans[:80000]
clean_ques=clean_ques[:80000]

#count occurences 
word2count = {}

for line in clean_ques:
    for word in line.split():
        if word not in word2count:
            word2count[word] = 1
        else:
            word2count[word] += 1
for line in clean_ans:
    for word in line.split():
        if word not in word2count:
            word2count[word] = 1
        else:
            word2count[word] += 1

#remove less frequent frequent word
thresh = 5
vocab = {}
word_num = 0
for word, count in word2count.items():
    if count >= thresh:
        vocab[word] = word_num
        word_num += 1
 
 # Add the Special tokens
for i in range(len(clean_ans)):
    clean_ans[i] = '<SOS> ' + clean_ans[i] + ' <EOS>'

tokens = ['<PAD>', '<EOS>', '<UNKN>', '<SOS>']
x = len(vocab)
for token in tokens:
    vocab[token] = x
    x += 1
    
vocab['cameron'] = vocab['<PAD>']
vocab['<PAD>'] = 0

#inv answers dict
inv_vocab = {w:v for v, w in vocab.items()}


encoder_inp = []
for line in clean_ques:
    lst = []
    for word in line.split():
        if word not in vocab:
            lst.append(vocab['<UNKN>'])
        else:
            lst.append(vocab[word])
        
    encoder_inp.append(lst)

decoder_inp = []
for line in clean_ans:
    lst = []
    for word in line.split():
        if word not in vocab:
            lst.append(vocab['<UNKN>'])
        else:
            lst.append(vocab[word])        
    decoder_inp.append(lst)



encoder_inp = pad_sequences(encoder_inp, 13, padding='post', truncating='post')
decoder_inp = pad_sequences(decoder_inp, 13, padding='post', truncating='post')




decoder_final_output = []
for i in decoder_inp:
    decoder_final_output.append(i[1:]) 



decoder_final_output = pad_sequences(decoder_final_output, 13, padding='post', truncating='post')

decoder_final_output = to_categorical(decoder_final_output, len(vocab))

#split data into train and test
encoder_inp_train = encoder_inp[:50000]
decoder_inp_train=decoder_inp[:50000]

encoder_inp_test=encoder_inp[50000:]
decoder_inp_test=decoder_inp[50000:]
x_test=[encoder_inp_test,decoder_inp_test]
decoder_final_output=decoder_final_output[:50000]
y_train=decoder_final_output
y_test=decoder_final_output[50000:]<jupyter_output><empty_output><jupyter_text>**Build the Neural Network**We will build a special kind of model called 'sequence to sequence' (seq2seq in short). You can separate the entire model into 2 small sub-models. The first sub-model is called as [E] Encoder, and the second sub-model is called as [D] Decoder. [E] takes a raw input text data just like any other RNN architectures do. At the end, [E] outputs a neural representation. This is a very typical work, but you need to pay attention what this output really is. The output of [E] is going to be the input data for [D].That is why we call [E] as Encoder and [D] as Decoder. [E] makes an output encoded in neural representational form, and we don't know what it really is. It is somewhat encrypted. [D] has the ability to look inside the [E]'s output, and it will create a totally different output data (Bot answer in this case).<jupyter_code>enc_inp = Input(shape=(13, ))


#Embading layer
VOCAB_SIZE = len(vocab)
embed = Embedding(VOCAB_SIZE+1, output_dim=50, 
                  input_length=13,
                  trainable=True                  
                  )

#encoder LSTM Layer
enc_embed = embed(enc_inp)
enc_lstm = LSTM(400, return_sequences=True, return_state=True)
enc_op, h, c = enc_lstm(enc_embed)
enc_states = [h, c] #enc_states is input for decoder 

dec_inp = Input(shape=(13, ))
#Decoder LSTM Layer
dec_embed = embed(dec_inp)
dec_lstm = LSTM(400, return_sequences=True, return_state=True)
dec_op, _, _ = dec_lstm(dec_embed, initial_state=enc_states)


#dense Layer
dense = Dense(VOCAB_SIZE, activation='softmax')
dense_op = dense(dec_op)

#Create model
model = Model([enc_inp, dec_inp], dense_op)

#compile model
model.compile(loss='categorical_crossentropy',metrics=['acc'],optimizer='adam')

#plot the model
plot_model(model, to_file='model.png', show_shapes=True)


#Train model on our preprocess data
#this model is used to build encoder and decoder
#model.fit([encoder_inp, decoder_inp],decoder_final_output,epochs=20)
model.fit([encoder_inp_train, decoder_inp_train],y_train,epochs=100)

#saving the model
model.save("/content/drive/MyDrive/seq2seq _Model")<jupyter_output>Epoch 1/100
982/982 [==============================] - 47s 14ms/step - loss: 3.5042 - acc: 0.4618
Epoch 2/100
982/982 [==============================] - 13s 13ms/step - loss: 2.7847 - acc: 0.5284
Epoch 3/100
982/982 [==============================] - 13s 13ms/step - loss: 2.6227 - acc: 0.5403
Epoch 4/100
982/982 [==============================] - 13s 13ms/step - loss: 2.5347 - acc: 0.5465
Epoch 5/100
982/982 [==============================] - 13s 13ms/step - loss: 2.4750 - acc: 0.5497
Epoch 6/100
982/982 [==============================] - 13s 14ms/step - loss: 2.4259 - acc: 0.5527
Epoch 7/100
982/982 [==============================] - 13s 13ms/step - loss: 2.3655 - acc: 0.5564
Epoch 8/100
982/982 [==============================] - 13s 13ms/step - loss: 2.3093 - acc: 0.5600
Epoch 9/100
982/982 [==============================] - 13s 14ms/step - loss: 2.2699 - acc: 0.5624
Epoch 10/100
982/982 [==============================] - 13s 13ms/step - loss: 2.2239 - acc: 0.5629
Epoch 11/100
982/98[...]<jupyter_text>**Inference** * The model to make the prediction is called the inference model. * Even though the inference model is a different model from the training model, it would require reference to the features of the training model.* To define the encoder model, we refer to the trained model and take the input layer from the encoder and output the states (cell and hidden state) of the encoder. * To define the decoder model, we define its initial state as the hidden and cell state of the newly created encoder model (the encoder of the inference model).<jupyter_code>#Inference  To talk to chatbot
#loading model
from tensorflow import keras
reconstructed_model = keras.models.load_model("/content/drive/MyDrive/seq2seq _Model")

# The reconstructed model is already compiled and has retained the optimizer
# state, so training can resume:
reconstructed_model.fit([encoder_inp_train, decoder_inp_train],y_train)

#Encoder Model 
enc_model = Model([enc_inp], enc_states)



# decoder Model
decoder_state_input_h = Input(shape=(400,))
decoder_state_input_c = Input(shape=(400,))

decoder_states_inputs = [decoder_state_input_h, decoder_state_input_c]


decoder_outputs, state_h, state_c = dec_lstm(dec_embed , 
                                    initial_state=decoder_states_inputs)


decoder_states = [state_h, state_c]


dec_model = Model([dec_inp]+ decoder_states_inputs,
                                      [decoder_outputs]+ decoder_states)


plot_model(enc_model, to_file='encoder_model.png', show_shapes=True)

plot_model(dec_model, to_file='decoder_model.png', show_shapes=True)<jupyter_output><empty_output><jupyter_text>**Implementation**<jupyter_code>print("##########################################")
print("#       Welcome to Chatbot        #")
print("##########################################")


prepro1 = ""
while prepro1 != 'q':
    prepro1  = input("you : ")
    ## prepro1 = "Hello"

    prepro1 = clean_text(prepro1)
    ## prepro1 = "hello"

    prepro = [prepro1]
    ## prepro1 = ["hello"]

    txt = []
    for x in prepro:
        # x = "hello"
        lst = []
        for y in x.split():
            ## y = "hello"
            try:
                lst.append(vocab[y])
                ## vocab['hello'] = 454
            except:
                lst.append(vocab['<UNKN>'])
        txt.append(lst)

    ## txt = [[454]]
    txt = pad_sequences(txt, 13, padding='post')

    ## txt = [[454,0,0,0,.........13]]

    stat = enc_model.predict( txt )

    empty_target_seq = np.zeros( ( 1 , 1) )
     ##   empty_target_seq = [0]


    empty_target_seq[0, 0] = vocab['<SOS>']
    ##    empty_target_seq = [255]

    flag = False
    decoded_translation = ''

    while not flag :

        dec_outputs , h, c= dec_model.predict([ empty_target_seq] + stat )
        decoder_concat_input = dense(dec_outputs)
        ## decoder_concat_input = [0.1, 0.2, .4, .0, ...............]#these are probability of each word provided by softmax function

        sampled_word_index = np.argmax( decoder_concat_input[0, -1, :] ) #This takes value with max probability
        ## sampled_word_index = [2]

        sampled_word = inv_vocab[sampled_word_index] + ' '

        ## inv_vocab[2] = 'hi'
        ## sampled_word = 'hi '

        if sampled_word != '<EOS> ':
            decoded_translation += sampled_word  

        if sampled_word == '<EOS> ' or len(decoded_translation.split()) > 13:
            flag = True 
        #resetting variables
        empty_target_seq = np.zeros( ( 1 , 1 ) )  
        empty_target_seq[ 0 , 0 ] = sampled_word_index
        ## <SOS> - > hi
        ## hi --> <EOS>
        stat = [h, c]  

    print("Bot : ", decoded_translation )
    print("==============================================")<jupyter_output>##########################################
#       Welcome to Chatbot        #
##########################################
you : hi
WARNING:tensorflow:Model was constructed with shape (None, 13) for input KerasTensor(type_spec=TensorSpec(shape=(None, 13), dtype=tf.float32, name='input_2'), name='input_2', description="created by layer 'input_2'"), but it was called on an input with incompatible shape (None, 1).
{'fraction_non_alphanumeric': 0.11557069510268562, 'fraction_numerical': 0.07597748815165876, 'mean_word_length': 6.239814152966405, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 48, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 3, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '81960', 'license': 'MIT', 'n_tokens_mistral': 10513, 'n_tokens_neox': 9580, 'n_words': 1166, 'path': 'Final/chatbot_seq2seq.ipynb', 'repo_name': 'neupane11/AI-sneupane', 'size': '46259'}