<jupyter_start><jupyter_text>Use the Brown Corpus reader nltk.corpus.brown.words() or the Web Text Corpus reader nltk.corpus.webtext.words() to access some sample text in two different genres.<jupyter_code>import nltk
nltk.corpus.brown.sents(categories=['romance'])<jupyter_output><empty_output><jupyter_text>**Read in the texts of the State of the Union addresses, using the state_union corpus reader. Count occurrences of men, women and people in each document. What has hapened to the usages of these words over time?**<jupyter_code>from nltk.corpus import state_union 
cfd=nltk.ConditionalFreqDist( (target,fileid[:4]) 
                             for fileid in state_union.fileids() 
                             for w in state_union.words(fileid) 
                             for target in ['men','women','people'] if w.lower().startswith(target))

cfd.plot()<jupyter_output><empty_output><jupyter_text>**Define a conditional frequency distribution over the Names Corpus that allowsyou to see which initial letters are more frequent for males versus females (see Figure 2-7).**<jupyter_code>from nltk.corpus import names
name_types = names.fileids()
male_names = names.words('male.txt')
female_names = names.words('female.txt')
[w for w in male_names if w in female_names]
#['Abbey', 'Abbie', 'Abby', 'Addie', 'Adrian', 'Adrien', 'Ajay', 'Alex', 'Alexis', 'Alfie', 'Ali', 'Alix',...]
cfd = nltk.ConditionalFreqDist(
(fileid, name[0])
for fileid in names.fileids()
for name in names.words(fileid)
)

cfd.plot()<jupyter_output><empty_output><jupyter_text>What percentage of noun synsets have no hyponyms? You can get all noun syn-  sets using wn.all_synsets('n').<jupyter_code>from nltk.corpus import wordnet as wn
nb_syn =  sum(1 for _ in wn.all_synsets('n'))
nb_syn
fac = len([s for s in wn.all_synsets('n') if len(s.hyponyms()) == 0]) / nb_syn
fac<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11876988335100742, 'fraction_numerical': 0.003181336161187699, 'mean_word_length': 4.666666666666667, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '350268', 'license': 'MIT', 'n_tokens_mistral': 661, 'n_tokens_neox': 633, 'n_words': 186, 'path': 'NLP with Python/Chapter02/Exercise 2.ipynb', 'repo_name': 'Stella-Sirius/NLP-Learning', 'size': '1886'}