hypfin = open('hyp.txt', 'r')
reffin = open('ref.txt', 'r')
hyp = (my_split(h) for h in hypfin)
hyp
ref = (my_split(r) for r in reffin)
ref
z = zip(hyp, ref)
z
hypline, refline = next(z)
hypline, refline = next(z)
hypline, refline = next(z)
How to workwith generators from file for tokenization rather than materializing a list of strings?