<jupyter_start><jupyter_text>By looking at noun phrases, you can get information about your text. For example, `a developer conference` indicates that the text mentions a conference, while the date `21 July` lets you know that conference is scheduled for 21 July. You can figure out whether the conference is in the past or the future. `London` tells you that the conference is in London.<jupyter_code># Noun Phrase Detection
conference_text = ('There is a developer conference'
    ' happening on 21 July 2019 in London.')
conference_doc = nlp(conference_text)
# Extract Noun Phrases
for chunk in conference_doc.noun_chunks:
    print (chunk)
import textacy
import textacy.ke
about_talk_text = ('The talk will introduce reader about Use'
                   ' cases of Natural Language Processing in'
                   ' Fintech')
# pattern = r'(<VERB>?<ADV>*<VERB>+)'
about_talk_doc = textacy.make_spacy_doc(about_talk_text,
                                        lang='en_core_web_sm')
# 通过多种算法识别文档中的关键术语
textacy.ke.textrank(about_talk_doc, normalize="lemma", topn=10)
textacy.ke.sgrank(about_talk_doc, ngrams=(1, 2, 3, 4), normalize="lower", topn=0.1)
from textacy.extract import subject_verb_object_triples
def desc(t):
    return t.text, t.root, t.start
def print_triples(text):
    doc=textacy.make_spacy_doc(text, lang='en_core_web_sm')
    triples=subject_verb_object_triples(doc)
    print(doc.text)
    print([(desc(span)) for t in triples for span in t])

about_talk_text = ('The talk will introduce reader about Use'
                   ' cases of Natural Language Processing in'
                   ' Fintech')
print_triples(about_talk_text)
# print_triples('This butterfly is blue, and it flies very fast.')
# print_triples('This butterfly flies very fast.')
print_triples('Brooks likes apples, but she likes oranges')<jupyter_output>Brooks likes apples, but she likes oranges
[('Brooks', Brooks, 0), ('likes', likes, 1), ('apples', apples, 2), ('she', she, 5), ('likes', likes, 6), ('oranges', oranges, 7)]<jupyter_text>You can use NER to redact people’s names from a text. For example, you might want to do this in order to hide personal information collected in a survey. You can use spaCy to do that:<jupyter_code>survey_text = ('Out of 5 people surveyed, <NAME>,'
               ' <NAME> and <NAME> like'
               ' apples. <NAME> and <NAME>'
               ' like oranges.')

def replace_person_names(token):
    if token.ent_iob != 0 and token.ent_type_ == 'PERSON':
        return '[REDACTED] '
    return token.string

def redact_names(nlp_doc):
    for ent in nlp_doc.ents:
        ent.merge()
    tokens = map(replace_person_names, nlp_doc)
    return ''.join(tokens)

survey_doc = nlp(survey_text)
redact_names(survey_doc)
text = "Mesdames et messieurs, nous allons bientôt atterrir, merci d'attacher vos ceintures."
doc = textacy.make_spacy_doc(text, lang='fr_core_news_sm')
triples=subject_verb_object_triples(doc)
[(desc(span)) for t in triples for span in t]
from sagas.nlu.spacy_helper import spacy_mgr, vis
from sagas.tracker_jupyter import enable_jupyter_tracker
enable_jupyter_tracker()

nlp_f = spacy_mgr.get_model('fr')
doc_f = nlp_f(text)
vis(doc_f, text)
nlp_f = spacy_mgr.get_model('en')
# doc_f = nlp_f('This butterfly flies very fast.')
doc_f=nlp_f('<NAME> likes apples, but she likes oranges')
vis(doc_f, doc_f.text)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10341779611078374, 'fraction_numerical': 0.0076605774896876845, 'mean_word_length': 4.239197530864198, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '117575', 'license': 'Apache-2.0', 'n_tokens_mistral': 1176, 'n_tokens_neox': 1136, 'n_words': 346, 'path': 'spacy-2.2/procs-spacy-relation-triple.ipynb', 'repo_name': 'samlet/stack', 'size': '3434'}