<jupyter_start><jupyter_text>Mengenali IntensiIntensi atau maksud adalah sesuatu yang menjadi tujuan yang ingin dicapai. Sebenarnya, memahami intensi merupakan hal yang rumit karena sifatnya multimodal atau terungkap dari banyak sisi serta ada sisi yang kemungkinan tersembunyi. Mengenali intensi di NLP tidak dimaksudkan sampai ke tingkat seperti seorang manusia memahami maksud dari manusia lain, tetapi hanya pada tingkatan mengetahui maksud dari apa yang terucap / tertulis.Pada dasarnya, ada beberapa teknik untuk mengenali intensi:1. Ekstraksi transitive verb (kata kerja transitif - kata kerja yang memerlukan obyek) dan direct object.2. Rangkaian kalimat - Konjungsi3. Mengenali sinonim untuk kemungkinan intensi yang berbeda-beda4. Menggunakan Semantic Similarity.5. Mengenali intensi dari urutan kalimat Ekstraksi transitive verb (kata kerja transitif - kata kerja yang memerlukan obyek) dan direct object.<jupyter_code>import spacy

nlp = spacy.load('en_core_web_sm')

#doc = nlp(u'show me the best hotel in berlin') #showHotel
#doc = nlp ("get me to the best hotel in town") #getMe
#doc = nlp ("Yodi is the handsome man in all the town, He makes woman like crazy") #makes woman
doc = nlp ("Joni cooked baso every day")

for token in doc:
    if token.dep_ == 'dobj':
        print(token.head.text + token.text.capitalize())
spacy.explain("dobj")<jupyter_output><empty_output><jupyter_text>Rangkaian Kalimat - KonjungsiDilakukan dengan mengenali konjungsi, yaitu kata yang berfungsi untuk mengkoneksikan 2 suatu kata / sekumpulan kata (atau kalimat) dengan kata / sekumpulan kata (atau kalimat) sebelumnya. Contoh:- Pizza and Cola- Take two of these and call me in the morning- He has no money. In addition, he has no means of getting any<jupyter_code>import spacy

nlp = spacy.load('en_core_web_sm')

doc = nlp(u'David repaired his car ')
#doc = nlp(u'He has no money. In addition, he has no means of getting any')

#extract the direct object and the conjunct associated with it
for token in doc:
    if token.dep_ == 'dobj':
        dobj = [token.text]
        conj = [t.text for t in token.conjuncts]

#compose the list of the extracted elements
dobj_conj = dobj + conj
print(dobj_conj)
spacy.explain("conj") 
#Bisa juga dengan menggunakan children. Contoh penggunaan children dari token:
doc = nlp("Give it back! He pleaded.")
give_children = doc[0].children
print ([t.text for t in give_children])
import spacy

nlp = spacy.load('en_core_web_sm')

#doc = nlp(u'I want a pizza and cola.')
doc = nlp(u'Marjoko kicked the Ball')

#extract the direct object and the conjunct associated with it
for token in doc:
    if token.dep_ == 'dobj':
        dobj = [token.text]
        conj = [t.text for t in token.children if t.dep_ == 'conj']

#compose the list of the extracted elements
dobj_conj = dobj + conj
print(dobj_conj)<jupyter_output>['Ball']<jupyter_text>Pada kasus-kasus tertentu, token selain transitive verb dan direct object. Token-token tersebut biasanya terkait dengan transitive verb dan direct object, sehingga harus dieksplorasi relasi sintaksis dengan transitive verb dan direct object. Kode sumber berikut melakukan proses analisis tersebut dan melakukan perbandingan dengan daftar kata yang sudah dibuat.<jupyter_code>import spacy

nlp = spacy.load('en_core_web_sm')

#apply the pipeline to the sample sentence
doc = nlp(u'I want to place an order for a pizza.')


# extract the direct object and its transitive verb
dobj = ''
tverb = ''
for token in doc:
    if token.dep_ == 'dobj':
        dobj = token
        tverb = token.head

# extract the verb for the intent's definition
intentVerb = ''
verbList = ['want', 'like', 'need', 'order']
if tverb.text in verbList:
    intentVerb = tverb
else:
    if tverb.head.dep_ == 'ROOT':
        intentVerb = tverb.head

# extract the object for the intent's definition
intentObj = ''
objList = ['pizza', 'cola']
if dobj.text in objList:
    intentObj = dobj
else:
    for child in dobj.children:
        if child.dep_ == 'prep':
            intentObj = list(child.children)[0]
            break
        elif child.dep_ == 'compound':
            intentObj = child
            break

# print the intent expressed in the sample sentence
print(intentVerb.text + intentObj.text.capitalize())<jupyter_output>wantPizza<jupyter_text>Mengenali SinonimSering terjadi, ekspresi dilakukan dengan menggunakan kata-kata dengan intensi yang sama. Contoh, lihat pada 3 kalimat pada aplikasi chatbot pemesanan pizza:1. I want a dish2. I'd like to order a pizza3. Give me a pieKetiganya mengandung intensi yang sama, yaitu **orderPizza**<jupyter_code>import spacy

nlp = spacy.load('en_core_web_sm')

doc = nlp(u'I want a dish.')

#extract the transitive verb and its direct object from the dependency tree
for token in doc:
    if token.dep_ == 'dobj':
        verb = token.head.text
        dobj = token.text

#create a list of tuples for possible verb synonyms
verbList = [('order','want','give','make'),('show','find')]

#find the tuple containing the transitive verb extracted from the sample
verbSyns = [item for item in verbList if verb in item]

#create a list of tuples for possible direct object synonyms
dobjList = [('pizza','pie','dish'),('cola','soda')]

#find the tuple containing the direct object extracted from the sample
dobjSyns = [item for item in dobjList if dobj in item]

#replace the transitive verb and the direct object with synonyms supported by the application
#and compose the string that represents the intent
intent = verbSyns[0][0] + dobjSyns[0][0].capitalize()
print(intent)<jupyter_output>orderPizza<jupyter_text>Bagaimana jika kata tidak terdapat pada list?<jupyter_code>import spacy

nlp = spacy.load('en_core_web_sm')

doc = nlp(u'I want an apple.')

#extract the transitive verb and its direct object from the dependency tree
for token in doc:
    if token.dep_ == 'dobj':
        verb = token.head.text
        dobj = token.text

#create a list of tuples for possible verb synonyms
verbList = [('order','want','give','make'),('show','find')]

#find the tuple containing the transitive verb extracted from the sample
verbSyns = [item for item in verbList if verb in item]

#create a list of tuples for possible direct object synonyms
dobjList = [('pizza','pie','dish'),('cola','soda')]

#find the tuple containing the direct object extracted from the sample
dobjSyns = [item for item in dobjList if dobj in item]

print(verbSyns)
print(dobjSyns)<jupyter_output>[('order', 'want', 'give', 'make')]
[]<jupyter_text>Semantic Similarity<jupyter_code>import spacy

nlp = spacy.load('en_core_web_md')

doc = nlp(u'I feel like eating a pie')
doc2 = nlp(u'food')

for token in doc:
    if token.dep_ == 'dobj':
        dobj = token

if dobj.similarity(doc2[0]) > 0.4:
    print("Would you like to look at our menu?")

print(dobj.similarity(doc2[0]))<jupyter_output>Would you like to look at our menu?
0.45949376<jupyter_text>Mengenali Intensi dari Urutan KalimatContoh: "I have finished my pizza. I want another one".<jupyter_code>import spacy

nlp = spacy.load('en_core_web_sm')

doc = nlp(u'I have finished my pizza. I want another one.')

verbList = [('order','want','give','make'),('show','find')]
dobjList = [('pizza','pie','pizzaz'),('cola','soda')]
substitutes = ('one','it','same','more')
intent = {'verb': '', 'dobj': ''}

for sent in doc.sents:
    for token in sent:
        if token.dep_ == 'dobj':
            verbSyns = [item for item in verbList if token.head.text in item]
            dobjSyns = [item for item in dobjList if token.text in item]
            substitute = [item for item in substitutes if token.text in item]
            if (dobjSyns != [] or substitute != []) and verbSyns != []:
                intent['verb'] = verbSyns[0][0]
            if dobjSyns != []:
                intent['dobj'] = dobjSyns[0][0]

intentStr = intent['verb'] + intent['dobj'].capitalize()
print(intentStr)<jupyter_output>orderPizza
{'fraction_non_alphanumeric': 0.10017663386323493, 'fraction_numerical': 0.004542013626040878, 'mean_word_length': 4.263612217795485, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '164266', 'license': 'MIT', 'n_tokens_mistral': 2818, 'n_tokens_neox': 2711, 'n_words': 923, 'path': 'novice/03-08/03-08.ipynb', 'repo_name': 'fakihAlim/zimera', 'size': '7926'}