<jupyter_start><jupyter_text>Here I was looking at dependecies, true orders and mine.<jupyter_code>from tree import Tree
with open("dev.parses") as f:
    pars = f.readlines()
with open("dev.sentences") as f:
    sent = f.readlines()
with open("dev.reorderings") as f:
    reord = f.readlines()
with open("dev.my") as f:
    my = f.readlines()
num_sentences = 1
for p, s, r, m in zip(pars, sent, reord, my)[:num_sentences]:
    print Tree(p)
    print s[:-1]
    print r[:-1]
    print m[:-1]
    print<jupyter_output>(is:VERB  (shinshu:nsubj one:attr ,:p and:cc succeeded:conj .:p))
(shinshu:NOUN  (jodo:nn sect:dep))
(one:NUM  (of:prep))
(succeeded:VERB  (community:nsubj and:cc developed:conj))
(sect:NOUN  ((:p true:amod pure:amod land:nn ):p))
(of:ADP  (sects:pobj))
(community:NOUN  (a:det religious:amod shinran:rcmod ,:p apprentice:appos ,:p))
(developed:VERB  (which:nsubj doctrine:dobj in:prep))
(true:ADJ  (shin-buddhism:amod /:p))
(sects:NOUN  (the:det of:prep))
(shinran:VERB  (that:nsubj))
(apprentice:NOUN  (an:det of:prep))
(doctrine:NOUN  (honen:poss (:p buddhism:dep ):p))
(in:ADP  (period:pobj))
(of:ADP  (buddhism:pobj))
(of:ADP  (honen:pobj))
(honen:NOUN  ('s:ps))
(buddhism:NOUN  (jodo:nn shu:nn /:p pure:amod land:nn))
(period:NOUN  (the:det early:amod kamakura:nn))
(buddhism:NOUN  (japanese:amod))

jodo shinshu ( shin-buddhism / true pure land sect ) is one of the sects of japanese buddhism , and a religious community that shinran , an apprentice of honen , succeeded and which developed ho[...]
{'fraction_non_alphanumeric': 0.1654629021667761, 'fraction_numerical': 0.0026263952724885093, 'mean_word_length': 5.17004048582996, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '622183', 'license': 'MIT', 'n_tokens_mistral': 670, 'n_tokens_neox': 650, 'n_words': 158, 'path': 'data_driven/assignment2/comparison.ipynb', 'repo_name': 'ilivans/machine-translation', 'size': '1523'}