<jupyter_start><jupyter_text>---To get started: consult [start](start.ipynb)--- DisplayWe show the ins and outs of displaying NENA texts.<jupyter_code>%load_ext autoreload
%autoreload 2
from tf.app import use<jupyter_output><empty_output><jupyter_text>If you want to use a version of the NENA texts ahead of a release, use the incantation with `hot` in it.That will take time, not only for the download itself, but also for the one-time preprocessing of the data.If you are content with the latest stable release, use the line without the `hot`.<jupyter_code>A = use("nena:clone", checkout="clone", hoist=globals())
# A = use('nena', hoist=globals())<jupyter_output><empty_output><jupyter_text>We pick an example fragment with which we illustrate many ways to represent the DSS material.<jupyter_code>exampleText = ("Barwar", "Gozali and Nozali")
f = T.nodeFromSection(exampleText)
lines = L.d(f, otype="line")[0:10]
sentences = L.d(f, otype="sentence")[0:10]
words = L.d(f, otype="word")[0:100]
letters = L.d(f, otype="letters")[0:200]<jupyter_output><empty_output><jupyter_text>Text formatsThe TF API supports *text formats*. Text formats make selections and apply templates and styles basedon the analysed features of the text. For example: a text-format may ignore flags or clusters, orformat numerals in special ways.Text formats are not baked into TF, but they are defined in the feature `otext` of the corpus.Moreover, for this corpus a TF app has been build that defines additional text-formats.Whereas the formats defined in `otext` are strictly plain text formats, the formatsdefined in the app are able to use typographic styles to shape the text, such as bold, italic, colors, etc.Here is the list of all formats.<jupyter_code>T.formats<jupyter_output><empty_output><jupyter_text>Plain text formatsThe formats whose names start with `text-` are the plain text formats. `text-orig-full`This is the plain Hebrew text of the transcriptions, without text-critical signs and brackets.This is the default format. We do not have to specify it.<jupyter_code>for s in sentences:
    A.plain(s)<jupyter_output><empty_output><jupyter_text>The `plain()` function focuses on the *contents*, and instead of the line number, it gives a full specificationof the location in the GitHub repo for NENA.But we can omit the locations:<jupyter_code>for s in sentences:
    A.plain(s, withPassage=False)<jupyter_output><empty_output><jupyter_text>`text-trans-full`This format gives a full transcription:<jupyter_code>for s in sentences:
    A.plain(s, fmt="text-trans-full")<jupyter_output><empty_output><jupyter_text>`text-trans-fuzzy`This format gives a fuzzy transcription:<jupyter_code>for s in sentences:
    A.plain(s, fmt="text-trans-fuzzy")<jupyter_output><empty_output><jupyter_text>`text-trans-lite`This format gives a light transcription:<jupyter_code>for s in sentences:
    A.plain(s, fmt="text-trans-lite")<jupyter_output><empty_output><jupyter_text>Styled text formatsThe formats whose names start with `layout-` are the styled text formats.They differ from their corresponding plain text formats in that speaker indicators use HTML layout. `layout-orig-full`<jupyter_code>for s in sentences:
    A.plain(s, fmt="layout-orig-full")<jupyter_output><empty_output><jupyter_text>Plain<jupyter_code>words[0]
A.plainTuple((words[0], words[1]), 3)<jupyter_output><empty_output><jupyter_text>PrettyThe ultimate of graphical display is by means of the `pretty()` function.This display is less useful for reading, but instead optimized for showing all information that you mightwish for.It shows a base representation according to a text format of your choice(here we choose `text-orig-full`), and it shows the valuesof a standard set of features.<jupyter_code>A.pretty(sentences[0], withNodes=False)<jupyter_output><empty_output><jupyter_text>Later on, in the [search](search.ipynb) tutorial we see that `pretty()` can also display other features,even features that you or other people have created and added later.Here we call for the feature `lite`, defined letters, words and stresses, which shows light transcriptions.We show it with node numbers, but you could leave them out in an obvious way.<jupyter_code>A.pretty(sentences[0], extraFeatures="lite", withNodes=True)<jupyter_output><empty_output><jupyter_text>But wait, we do not see the `lite` feature anywhere. Because it is defined on words and letters, but not on stresses.Let's show the words:<jupyter_code>A.pretty(sentences[0], extraFeatures="lite", withNodes=True, baseTypes="word")<jupyter_output><empty_output><jupyter_text>We can even show the characters (and now we suppress the node numbers again):<jupyter_code>A.pretty(sentences[0], extraFeatures="lite", baseTypes="letter")<jupyter_output><empty_output><jupyter_text>We can skip intermediate levels, such as `inton` and stress:<jupyter_code>A.pretty(
    sentences[0], hiddenTypes="inton stress", extraFeatures="lite", baseTypes="letter"
)<jupyter_output><empty_output><jupyter_text>Overview of all display options with their current settings:<jupyter_code>A.displayShow()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0974609375, 'fraction_numerical': 0.0046875, 'mean_word_length': 6.830275229357798, 'pattern_counts': {'":': 0, '<': 69, '<?xml version=': 0, '>': 69, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '317208', 'license': 'MIT', 'n_tokens_mistral': 1615, 'n_tokens_neox': 1540, 'n_words': 536, 'path': 'nena/display.ipynb', 'repo_name': 'sethbam9/tutorials', 'size': '5120'}