I've been attempting to teach myself Python over the past couple of months and have been setting myself practical challenges to learn different aspects of the language.I have a deep structure of subdirectories containing .rtf files. The objectives of the following code are as follows:Examine every file in every subdirectory to identify flagged words defined in expression. For each instance a flagged word is identified in a file, capture the file name and the line within that file in which the flagged word appears.Construct a dictionary, in which flagged words are keys and filenames and lines are stored as values.Insert each key/value pair in the dictionary into a MongoDB collectionThe code is as follows:from pymongo import MongoClientimport osimport reimport json# connect to mongodbclient = MongoClient('localhost', 27017)db = client['test-database']restrictions = db['restrictions']# create empty dictionaryd = {}# setup regular expressionsexpression = sexual offences|reporting restriction|reporting restrictions|anonymous|anonymously|secretpattern = re.compile(expression)# read the source filesfor dname, dirs, files in os.walk(path/to/top/level/directory):    for fname in files:        fpath = os.path.join(dname, fname)        with open(fpath, 'r') as f:            source = f.readlines()            for i, line in enumerate(source):                for match in re.finditer(pattern, line):                # set the matched flagword as the key and the filename and line as the value                    d.setdefault(match.group(0), [])                    file_instance = fname, line                    print file_instance                    d[match.group(0)].append(file_instance)# Update the restrictions databasefor key, value in d.iteritems():    xref_id = db.restrictions.insert_one({'flagged_word': key, 'instance': value})An example document in MongoDB generated by this code:    {        _id : ObjectId(5925e0d94fb263cb1417bb73),        flagged_word : restriction,        instance : [             [                 Family3.txt ,                 ii)\tAn application for a reporting restriction order (\reporting restriction order\) to restrict or prohibit the publication of:\n            ]     }The code works as expected, but I have a nagging feeling that the same objectives could be achieved with a far less clunky approach in the code (particularly where all of the for loops are concerned).In the spirit of wishing to learn how to write clearer, more Pythonic code, I wonder if anyone could offer some advice on how this code could be improved. Many thanks in advance for taking the time to read my code.
Python script to identify regex matches in all subdirs and write dict of matches to MongoDB
python;regex;mongodb
f.readlines() is redundant. Just iterate over the file: for i, line in enumerate(f):, or, as you are not using i at all: for line in f: d.setdefault becomes redundant if you use d = collections.defaultdict(list)