I suspect there might be a much more efficient way to accomplish this task.  I would appreciate a more clever hint for performance increase.  I am also looking for feedback on style and simplification where appropriate.  I assume that the set of what counts as English words is a given and that I don't need to do something clever like an actual dictionary or use Google n-grams.  That being said, the results could differ depending on the domain and also what is accepted as a word.  The original problem set M=3 (3-gram characters), which makes the results not so interesting (ans=[(u'the', 191279), (u'and', 92292), (u'ing', 57323), (u'her', 44627), (u'hat', 38609)]).  #gets the top N most common substrings of M characters in English wordsfrom nltk import FreqDistfrom nltk.corpus import gutenbergdef get_words():    for fileid in gutenberg.fileids():        for word in gutenberg.words(fileid):            yield worddef ngrams(N, which_list, strict=False):    list_size = len(which_list)    stop = list_size    if strict:        stop -= (N - 1)    for i in xrange(0, stop):        element = which_list[i]        ngram = [element]        j = 1        index = j + i        while j < N and index < list_size:            ngram.append(which_list[index])            j += 1            index += 1        yield ''.join(ngram)def m_most_common_ngram_chars(M=5, N=3):    n_grams = []    for word in get_words():        for ngram in ngrams(N, word, strict=True):            n_grams.append(ngram)    f = FreqDist(n_grams)    return f.most_common(M)l = m_most_common_ngram_chars(M=5, N=3)print l
Finding top most N common character grams in English
python;performance;natural language processing