<jupyter_start><jupyter_text>As we are not developing a chit chat but a virtual conversation between a user and an expert, we will make some assumptions:•	The user will not use emoji or abbreviations in the same sentence with an important request.•	The user might use emoji/abbreviations as a reaction/reply or in a follow up discussion.•	The user might use emoji/abbreviations in a more complex input and a reply might be regarded as emotional and thus might help the conversation going forward.Taking this into consideration, we will implement a secondary flow that will have specific answers/policies and can exist together with the main flow, or standalone if no other input is provided by the user. Important: Abbreviations that are not considered as specific to chit chat needs will be treated in the main flow, together with the sentence-intent. For example: abbreviation for institutes that are NER.<jupyter_code># IN: inputs from Auto-correctand Procesing I
# OUT: send identified emoji/abbreviations to the following pipelines: Untrained NIU/Reaction Analysis<jupyter_output><empty_output><jupyter_text>Objectives:•	Gathering the emoji/abbreviations from auto-correct, processing and mark them for Untrained NIU/Reactions;Language specificities: no;Dependencies: Auto-correct /Processing I/Untrained NIU/ Reaction analysis; Database/ Vocabularies needed: Emoji/Abbreviations vocabularies;<jupyter_code># To dos:
# 1.	Mark sentences with emoji/abbreviations for Untrained answers.
# 2.	Classify all the emoji/abbreviations in 6 categories (sad, blink, kiss, smile, cool, laughing out loud).
# 3.	Sad and Smile will be sent to the Reaction analysis pipeline. 
# 4.	The rest will be sent to Untrained NIU.<jupyter_output><empty_output><jupyter_text>Use adapted codeCode example, but please adapt to the objectives<jupyter_code># https://github.com/amanchadha/coursera-deep-learning-specialization/blob/master/C5%20-%20Sequence%20Models/Week%202/Emojify/Emojify%20-%20v2.ipynb<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06113320079522863, 'fraction_numerical': 0.009940357852882704, 'mean_word_length': 5.75503355704698, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '64118', 'license': 'Apache-2.0', 'n_tokens_mistral': 592, 'n_tokens_neox': 553, 'n_words': 242, 'path': 'NIU-NLU dummy codes/5_Emoji_Abbreviations_(EER)_dummy_code.ipynb', 'repo_name': 'Cezanne-ai/project-2021', 'size': '2012'}