<jupyter_start><jupyter_text>SUMMARYlook into training set raw captions text:1. check how many captions have emoji **(emoji)**2. how many have username mentioned3. how many mixed with hashtag i.e. word prefixed with 4. see if name entity detection work **(huggingface)**5. catch mis-spelled word **(spacy + hunspell)** REFERENCE1. Name entity extraction (huggingface): https://huggingface.co/transformers/usage.html2. Handle mis-spelled words: https://github.com/tokestermw/spacy_hunspell3. Training a sentiment model: https://github.com/bentrevett/pytorch-sentiment-analysis4. List of available models for all tasks: https://huggingface.co/models?filter=text-classification5. How to use huggingface's pipeline for NER/ sentiment analysis: https://huggingface.co/transformers/main_classes/pipelines.html6. NER:     - available arguments: https://huggingface.co/transformers/main_classes/pipelines.htmltransformers.TokenClassificationPipeline7. sentiment classifier:     - available arguments: https://huggingface.co/transformers/main_classes/pipelines.htmltextclassificationpipeline    - Kaggle: pretrained sentiment model pretrained on 1.6 million Twitter dataset: https://www.kaggle.com/paoloripamonti/twitter-sentiment-analysis    - DeepMoji from huggingface: https://medium.com/huggingface/understanding-emotions-from-keras-to-pytorch-3ccb61d5a983<jupyter_code>import os
import json
import re
from collections import defaultdict, Counter
from multiprocessing import Pool

from tqdm import tqdm
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import emoji

%matplotlib inline
%load_ext autoreload
%autoreload 2
import spacy
from spacy_hunspell import spaCyHunSpell

nlp = spacy.load('en_core_web_sm')
hunspell = spaCyHunSpell(nlp, 'linux')
nlp.add_pipe(hunspell)
JSON_PATH = '../../../data/instagram/json/insta-caption-train.json'

with open(JSON_PATH, 'r') as f:
    data = json.load(f)
    
captions = dict()
for usr, usr_dict in data.items():
    for post, post_dict in usr_dict.items():
        id_key = f'{usr}_@_{post}'
        caption = post_dict['caption'].strip()
        captions[id_key] = caption
        
len(captions)<jupyter_output><empty_output><jupyter_text>1. How many captions have emoji?Caveat:- treat :))))) --> :)<jupyter_code>def is_emoji(text):
    boo = False
    for c in text:
        if c in emoji.UNICODE_EMOJI: boo = True
    return boo
emoji_cnt = 0
instance_n = 5

for _, caption in captions.items():
    if is_emoji(caption): 
        emoji_cnt += 1
        if instance_n != 0:
            print(caption)
            instance_n -= 1

print(f'\ncaption with emoji: {emoji_cnt} ({len(captions)})')
def is_target_face(text):
    boo = False
    if (":)" in text) or (":(" in text):
        boo = True
    return boo
face_cnt = 0
instance_n = 5

for _, caption in captions.items():
    if is_target_face(caption): 
        face_cnt += 1
        if instance_n != 0:
            print(caption)
            instance_n -= 1

print(f'\ncaption with emoji: {face_cnt} ({len(captions)})')<jupyter_output>Why does she give me those puppy eyes :( 👀🐶
Going to see ze babies:)
Why do my TWO FAVORITE shows have their season opening at the same time:(
I get to make these all day at work:)))
Melissa's gender reveal was a success!! So glad everything turned out nicely:) #ItsAGirl

caption with emoji: 18755 (648761)<jupyter_text>2a. How many have username mention?<jupyter_code>re.findall(r'@[\S]+', '@riven_hong.asdas-314 hi world')
def is_username(text):
    pattern = re.findall(r'@[\S]+', text)
    if len(pattern) != 0:
        return True
    else:
        return False
username_cnt = 0
instance_n = 5

for _, caption in captions.items():
    if is_username(caption): 
        username_cnt += 1
        if instance_n != 0:
            print(caption)
            instance_n -= 1

print(f'\ncaption with user name: {username_cnt} ({len(captions)})')<jupyter_output>Bob life and loving it @crimsonsfi9est
Shop @blissboutiquear for this top also come in white
You killed this look @mz_marvelous_dr
Beauty @7jessicasally rocking Bliss very well.
My crew tonight minus @kinkykurlydiva she was swiping those cards!  Everything Blis

caption with user name: 36034 (648761)<jupyter_text>2b. A Special Username<jupyter_code>def is_double_at(text):
    pattern = '@@'
    if pattern in text:
        return True
    else:
        return False
special_cnt = 0
instance_n = 10

for _, caption in captions.items():
    if is_double_at(caption):
        special_cnt += 1
        if instance_n != 0:
            print(caption)
            instance_n -= 1

print(f'\ncaption with author name: {special_cnt} ({len(captions)})')
re.sub(r"([!?,\.\"])", r" ", 'I am loving it!!!')<jupyter_output><empty_output><jupyter_text>3a. How many mixed with hashtag?<jupyter_code>def capture_hashtag(text):
    pattern = re.findall(r'#[\S]+', text)
    if len(pattern) != 0:
        return pattern
    else:
        return None
hashtag_cnt = 0
hashtag_set = Counter()
instance_n = 100

for _, caption in captions.items():
    hashtags = capture_hashtag(caption)
    
    if hashtags is not None:    
        hashtag_set.update(hashtags)
        
        hashtag_cnt += 1
        if instance_n != 0:
            print(caption)
            instance_n -= 1

print(f'\ncaption with hashtag: {hashtag_cnt} ({len(captions)})')<jupyter_output>I'm going to get it one day. #selfie game
Mood: Feeling so grateful today and wanted to put some cuteness on your timeline #Journee'
My new friend at vista! #treelizard
My beautiful niece a year ago and then yesterday. #loveher
This is how I like to sit at work. #relaxed
It was that kind of day. #tired
Listen to the saying! Do it. #texasgirl
The face of a natural born killer. #problems #longmorning
Ladies cross their legs, even at 1800+ feet high. #classy
Thanks to my Aunt Eleanor in Boston for this awesome #cowboys bracelet!!
Literally forgot I asked her to call me that haha. #night
The man who made this trip happen and made me happen lol. #mydad
For real! #repost
Had to #repost! This made me and @meliarlene laugh!
20 years ago at the Oscars. #crazy.
#SafeSets ready for domination this season!
Got use out of my puffer jacket today. #cold #sick #work
All these years and I thought that space was meant for motorcycles. #sillyme
You live by your own perception. #quoteoftheday
#tbt my baby[...]<jupyter_text>3b. How many caption have residual word after hashtag? e.g. "I like chocolate! teatime coffe like"<jupyter_code>test1 = "I'm going to get it one day. #selfie game"
test2 = "I'm going to get it one day. #selfie game"
re.split(r'(#[\S]+)', test1, 0)<jupyter_output><empty_output><jupyter_text>3c. Distribution of hashtags?<jupyter_code>all_hashtag_tup = sorted(list(hashtag_set.items()), key = lambda x: x[1], reverse = True)
sum([cnt for tag, cnt in all_hashtag_tup if cnt <= 5])
sorted_tup = sorted(list(hashtag_set.items()), key = lambda x: x[1], reverse = True)[:50]

hashtag_ls, cnt_ls = map(list, zip(*sorted_tup))

fig, ax = plt.subplots(1, figsize = (16, 4))
ax.bar(hashtag_ls, cnt_ls)
ax.set_xticklabels(labels = hashtag_ls, rotation = 90);<jupyter_output><empty_output><jupyter_text>3d. Distribution of Position where hashtag occur<jupyter_code>def hashtag_positions(text):
    positions = [pos for pos, char in enumerate(text) if char == '#']
    if len(positions) == 0:
        return None
    else:
        return positions
position_counter = Counter()

for _, caption in captions.items():
    positions = hashtag_positions(caption)
    
    if positions is not None:    
        caption_len = len(caption)
        positions = [idx/ caption_len for idx in positions]
        position_counter.update(positions)
sorted_tup = sorted(list(position_counter.items()), key = lambda x: x[0], reverse = False)

position_ls, cnt_ls = map(list, zip(*sorted_tup))

fig, ax = plt.subplots(1, figsize = (16, 4))
ax.bar(position_ls, cnt_ls, width = 0.005)
ax.set_title('Hashtag Relative Position Distribution');
#ax.set_xticklabels(labels = position_ls, rotation = 45);
first = sum([cnt for pos, cnt in sorted_tup if pos <= 1/3])
mid = sum([cnt for pos, cnt in sorted_tup if (pos <= 2/3) and (pos > 1/3)])
end = sum([cnt for pos, cnt in sorted_tup if (pos > 2/3)])

print(first)
print(mid)
print(end)<jupyter_output>20907
35627
113628<jupyter_text>3e. Among Those Hashtag, How many are mispelled?<jupyter_code>mispell_cnt = 0
mispell_set = dict()
hashtag_ls = list(hashtag_set.keys())

clean_hashtag_ls = [hashtag.replace('#', '') for hashtag in hashtag_ls]
docs = nlp.pipe(clean_hashtag_ls)

for i, doc in tqdm(enumerate(docs)):
    hashtag = hashtag_ls[i]
    cnt = hashtag_set[hashtag]
    if not doc[0]._.hunspell_spell:
        mispell_set[hashtag] = cnt
        mispell_cnt += cnt

total_tags = sum([cnt for tag, cnt in hashtag_set.items()])
print(f'mispelled hashtag: {mispell_cnt} ({total_tags}) ({(mispell_cnt/total_tags):.3f})')
mispell_set
test_mispell_hashtags = [
    'treelizard', 'letu!', 'wannabeinmygang', '#HowLuckyIAm', 'birdwatching'
]

for mispell_hashtag in test_mispell_hashtags:
    print(nlp(mispell_hashtag)[0]._.hunspell_suggest)<jupyter_output>['tree lizard', 'tree-lizard', 'treeline']
['lech', 'lute', 'let', 'letup', 'lets']
[]
['e', 's', 'i', 'a', 'n', 'r', 't', 'o', 'l', 'c', 'd', 'u', 'g', 'm', 'p']
['bird watching', 'bird-watching', 'birdwatcher']<jupyter_text>4. Name Entity Detection (Person Name)?select a good tokenizer and pretrained model<jupyter_code>from transformers import pipeline
nlp = pipeline("ner", device = 0)
def capture_person_name(caption_tup):
    id_key, caption = caption_tup
    print(f'processing: {id_key}')
    out = nlp(caption)
    if len(out) != 0 and (out[0]['entity'] == 'I-PER' or out[0]['entity'] == 'B-PER'):
        ner_ls = [d['word'] for d in out]
        return (id_key, ner_ls)
    else:
        return None
caption_key, caption_txt = zip(*captions.items())

assert len(caption_key) == len(caption_txt)
test_docs = list(caption_txt)[:100]
%timeit ner_outs = nlp(test_docs)
#ner_tup = [(id_key, ner_ls) for id_key, ner_ls in zip(caption_key, ner_outputs)]
((1.51 / 100) * 600000) / 3600<jupyter_output><empty_output><jupyter_text>5. Sentiment Detectionselect a good tokenizer and good pretrained model**Candidate**- https://huggingface.co/vicd/sentiment?text=Fix+it+please%21- https://huggingface.co/amine/multilingual_sentiment_vocab20k?text=Fix+it+please%21<jupyter_code>senti_nlp = pipeline('sentiment-analysis', device = 0)
caption_txt[:100]
test_docs = 'Fix it please!'
senti_nlp(test_docs)<jupyter_output><empty_output><jupyter_text>Appendix 1: Other Text Processing<jupyter_code>import spacy

nlp = spacy.load('en_core_web_sm')
test = "i'm loving it!@username"
doc = nlp(test)

for i in doc:
    print(i.text)<jupyter_output>i
'm
loving
it!@username<jupyter_text>Appendix 2: Mispell Correction<jupyter_code>samples = [
    "Slit joggers added to site. Killin it top still available",
    "Thanks to my Aunt Eleanor in Boston for this awesome #cowboys bracelet!!",
    "Had to #repost! This made me and @meliarlene laugh!",
    "Available for purchase. Size S M L",
    "My baby girl rocking her Bermuda from Bliss!! My Minnie me"
]
docs = nlp.pipe(samples)

for doc in docs:
    print(f'tokens: {[w.text for w in doc]}')
    print(f'mispell: {[w.text for w in doc if not w._.hunspell_spell]}\n')
doc = nlp(samples[1])
doc[11].text
doc[11]._.hunspell_spell
doc = nlp('@username this is amazing! @username')
print([i.text for i in doc])
doc.retokenize()
doc = nlp('cheeze')
doc[0]._.hunspell_spell, doc[0]._.hunspell_suggest
doc = nlp("Going to see ze babies:)")
doc[3]._.hunspell_spell, doc[3]._.hunspell_suggest<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12264796442011631, 'fraction_numerical': 0.016592541908997605, 'mean_word_length': 4.302947845804988, 'pattern_counts': {'":': 2, '<': 50, '<?xml version=': 0, '>': 50, 'https://': 11, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '570717', 'license': 'MIT', 'n_tokens_mistral': 4333, 'n_tokens_neox': 4127, 'n_words': 1165, 'path': 'data/notebooks/06_analyse_raw_captions_stats.ipynb', 'repo_name': 'riven314/Stylised-Controllable-Image-Captioning', 'size': '11692'}