Q:

Remove specific rows from CSV file in python

I am trying to remove rows with a specific ID within particular dates from a large CSV file. 
The CSV file contains a column [3] with dates formatted like "1962-05-23" and a column with identifiers [2]: "ddd:011232700:mpeg21:a00191". 
Within the following date range:

01-01-1951 to 12-31-1951
07-01-1962 to 12-31-1962
01-01 to 09-30-1963
7-01 to 07-31-1965
10-01 to 10-31-1965
04-01-1966 to 11-30-1966
01-01-1969 to 12-31-1969
01-01-1970 to 12-31-1989 

I want to remove rows that contain the ID ddd:11*
I think I have to create a variable that contains both the date range and the ID. And look for these in every row, but I'm very new to python so I'm not sure what would be an eloquent way to do this. 
This is what I have now. -CODE UPDATED
import csv
import collections
import sys
import re
from datetime import datetime

csv.field_size_limit(sys.maxsize)

dateranges = [("01-01-1951","12-31-1951"),("07-01-1962","12-31-1962")]
dateranges = list(map(lambda dr: tuple(map(lambda x: datetime.strptime(x,"%m-%d-%Y"),dr)),dateranges))

def datefilter(x):
    x = datetime.strptime(x,"%Y-%m-%d")
    for r in dateranges:
        if r[0]<=x and r[1]>=x: return True
    return False

writer = csv.writer(open('filtered.csv', 'wb'))
for row in csv.reader('my_file.csv', delimiter='\t'):
    if datefilter(row[3]):
        if not row[2].startswith("dd:111"):
            writer.writerow(row) 
    else: 
        writer.writerow(row)
writer.close()

A:

Here is how I would approach that, but it may not be the best method.
from datetime import datetime
dateranges = [("01-01-1951","12-31-1951"),("07-01-1962","12-31-1962")]
dateranges = list(map(lambda dr: tuple(map(lambda x: datetime.strptime(x,"%m-%d-%Y"),dr)),dateranges))

def datefilter(x):
    # The date format is different here to match the format of the csv
    x = datetime.strptime(x,"%Y-%m-%d")
    for r in dateranges:
        if r[0]<=x and r[1]>=x: return True
    return False

with open(main_file, "rb") as fp:
    root = csv.reader(fp, delimiter='\t')
    result = collections.defaultdict(list)
    for row in root:
        if datefilter(row[3]):
            # use a regular expression or any other means to filter on id here
            if row[2].startswith("dd:111"): #code to remove item

What I have done is create a list of tuples of your date ranges (for brevity, I only put 2 ranges in it), and then I convert those into datetime objects.
I have used maps for doing that in one line: first loop over all tuples in that list, applying a function which loops over all entries in that tuple and converts to a date time, using the tuple and list functions to get back to the original structure.  Doing it the long way would look like:
dateranges2=[]
for dr in dateranges:
    dateranges2.append((datetime.strptime(dr[0],"%m-%d-%Y"),datetime.strptime(dr[1],"%m-%d-%Y"))
dateranges = dateranges2

Notice that I just convert each item in the tuple into a datetime, and add the tuples to the new list, replacing the original (which I don't need anymore).
Next, I create a datefilter function which takes a datestring, converts it to a datetime, and then loops over all the ranges, checking if the value is in the range.  If it is, we return True (indicating this item should be filtered), otherwise return False if we have checking all ranges with no match (indicating that we don't filter this item).
Now you can check out the id using any method that you want once the date has matched, and remove the item if desired.  As your example is constant in the first few characters, we can just use the string startswith function to check the id.  If it is more complex, we could use a regex.

