<jupyter_start><jupyter_text>CharNet Model<jupyter_code>from lltk.model.charnet import *
ner_txt2names("""
Mr. Bennet was among the earliest of those who waited on Mr. Bingley. He had always intended to visit him, though to the last always assuring his wife that he should not go; and till the evening after the visit was paid she had no knowledge of it. It was then disclosed in the following manner. Observing his second daughter employed in trimming a hat, he suddenly addressed her with:

“I hope Mr. Bingley will like it, Lizzy.”
""")
# big text
import lltk
C=lltk.load('CanonFiction')
t=[t for t in C.texts() if t.title.startswith('Pride and Prej')][0]
t
t.title
# t.txt
len(t.words)
# ppnames = ner_txt2names(t.txt)
# ppnames.most_common()
# t2=[t for t in C.texts() if t.title=='Don Quixote'][0]
# dqnames=ner_txt2names(t2.txt)
# dqnames.most_common()
t3=[t for t in C.texts() if t.author=='Heliodorus'][0]
t3
ner_txt2names(t3.txt)<jupyter_output>2021-02-07 18:27:55 INFO: Loading these models for language: en (English):
=========================
| Processor | Package   |
-------------------------
| tokenize  | ewt       |
| ner       | ontonotes |
=========================

2021-02-07 18:27:55 INFO: Use device: cpu
2021-02-07 18:27:55 INFO: Loading: tokenize
2021-02-07 18:27:55 INFO: Loading: ner
2021-02-07 18:27:55 INFO: Done loading processors!
Parsing paragraphs: 100%|██████████| 338/338 [10:10<00:00,  1.81s/it]
{'fraction_non_alphanumeric': 0.17318435754189945, 'fraction_numerical': 0.0712290502793296, 'mean_word_length': 4.327137546468402, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '31642', 'license': 'MIT', 'n_tokens_mistral': 580, 'n_tokens_neox': 507, 'n_words': 166, 'path': 'notebooks/charnet.ipynb', 'repo_name': 'literarylab/lltk', 'size': '1432'}