<jupyter_start><jupyter_text>Named Entity Recognition (NER)<jupyter_code>import xml.etree.ElementTree as ET
import re
import spacy
import nltk
from wordcloud import WordCloud
from spacy import displacy
from matplotlib import pyplot as plt<jupyter_output><empty_output><jupyter_text>Reading data<jupyter_code>def readXML(dataPath):
    sb = []
    root = ET.parse(dataPath).getroot()    
    for child in root[0]: 
        if child.tag == 'item': 
            s = child.find('description').text
            s = re.findall("<img.*/><p>(.+)</p>",s)
            sb.append(s[0])
    return sb
xml = readXML("./Data/European Investment Bank Press Releases/data/English_Press-releases.xml")
xml<jupyter_output><empty_output><jupyter_text>Named Entity Recognition  Ex. 1.1: Load the small spaCy model for English which has been trained on web documents  Hint: This model is called *en_core_web_sm*  Ex 1.2: Create a document that has been annotated with named entities  Hint: simply call the Language object *nlp* with your text. This will trigger the whole processing pipeline to run  Hint 2: You cannot give it the array of strings directly, but either have to process each item separately or have to join the separate strings into 1 large one.   Ex 1.3: Highlight the name entities in the text using spaCy's package "displacy" Hint: use method *render* of displacy and set the style to named entities (*ent*)  Ex. 1.4: Are there some abbreviations for the named entities that you don't know? You can get an explanation for them using spaCy's ``explain`` method<jupyter_code><jupyter_output><empty_output><jupyter_text>Visualization  Ex 2.1: Next, we are going to visualize the text as a word cloud. Run the code below. Does it help you to get an idea of the content of the documents?<jupyter_code>wordcloud = WordCloud(width = 2000, height = 2000, background_color = "white").generate(" ".join(xml))
plt.figure(figsize = (10,10))
plt.imshow(wordcloud, interpolation = "bilinear")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08982035928143713, 'fraction_numerical': 0.012974051896207584, 'mean_word_length': 4.616246498599439, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 5}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '662952', 'license': 'MIT', 'n_tokens_mistral': 638, 'n_tokens_neox': 589, 'n_words': 235, 'path': 'Day_5_NLP/Exercises/Exercise 2 - Named Entity Recognition.ipynb', 'repo_name': 'keuperj/ki-sommer', 'size': '2004'}