Q:

Reducing computation time of counting word frequency in a corpus (python)

def foo(self, n):
    wordFreqDict = defaultdict(int)
    resultList = []

    for sentence in self.allSentences:
        for word in self.wordSet:
            if word in sentence:
                wordFreqDict[word] += 1

    for candidateWord in wordFreqDict:
        if wordFreqDict[candidateWord] >= n:
            resultList.append(candidateWord)

    return resultList

My function basically return the list of words where a word is in the list iff it occurs in at least n sentences. What I am trying to do is just iterate through the self.allSentences is a list of list of words(sentence) and iterate through the self.wordSet(all unique words in the corpus) and add the frequency of each word to the dictionary (wordFreqDict).
Then loop through the wordFreqDict and add a word with frequency >= n into the resultList.
I am assuming that this will work, but it's taking too long time to check the result.
Is there a way to make it for efficient and make the computation time shorter?
EDIT:
Here is how the self.allSentences is computed
def storeAllSentences(self, corpus):
    for sentence in corpus:
        self.allSentences.append(sentence)

and the self.wordSet:
def getUniqueWordSet(self, corpus):
    for sentence in self.allSentences:
        for word in sentence:
            self.wordSet.append(word)

    self.wordSet = set(self.wordSet)

A:

An assumption here is that a word is chosen if it uniquely appears in at least n sentences. So, even if it occurs 10 times in a single sentence, that's still once. 
A couple of glaring issues are your nested loops and your in check on a string. This is effectively cubic in complexity. It should be possible to reduce this drastically using a set + Counter.
from collections import Counter
from itertools import takewhile

def foo(self, n):
    c = Counter()
    for sent in self.allSentences:
        c.update(set(sent))

    resultSet = list(itertools.takewhile(lambda x: x[1] >= n, c.most_common()))

    return resultSet

Here, I use regex to remove special characters and punctuation, following which I split and retrieve all unique words in the sentence. The Counter is then updated.
Finally, extract all words of the desired frequency (or more) using itertools.takewhile and return.

If sent is a string sentence, you might use re based filtering to remove punctuation and then split:
import re

tempWordSet = set(re.sub('[^\w\s]+', '', sent).split())  
c.update(tempWordSet)

This does not take into consideration whether a word belongs to self.wordSet or not. So if you want that as well, you might modify the first loop a bit to include a filter step:
 c.update(set(filter(lambda x: x in self.wordSet, sent)))

Or, using the second technique:
 tempWordSet = set(filter(lambda x: x in self.wordSet, 
                         re.sub('[^\w\s]+', '', sent).split()))

On an unrelated note, are you trying to perform text mining? You might be interested in looking into TFIDF.

