I'm writing a program to process log files.  Currently, I have a script that will extract the desired information from each log and put it into a list - I have about nine different lists all containing different information.Currently, I'm loading the file, extracting the desired infromation, and storing it in a list, finally I write the lists to a CSV.The program roughly follows this outline.#load data#create listslist1 = []list2 = []list3 = []list4 = []list5 = []#extract information to populate lists.#write CSV'swith open('output1.csv', 'wb') as csvfile:        writer = csv.writer(out)    for line in list1:        writer.writerow(line)with open('output2.csv', 'wb') as csvfile:    writer = csv.writer(out)    for line in list2:        writer.writerow(line)with open('output3.csv', 'wb') as csvfile:    writer = csv.writer(out)    for line in list3:        writer.writerow(line)with open('output4.csv', 'wb') as csvfile:    writer = csv.writer(out)    for line in list4:        writer.writerow(line)with open('output5.csv', 'wb') as csvfile:    writer = csv.writer(out)    for line in list5:        writer.writerow(line)with open('output6.csv', 'wb') as csvfile:    writer = csv.writer(out)    for line in list6:        writer.writerow(line)I'm wondering if there is a better/faster way to do this.  Considering these lists can get pretty large. I feel like it might be quicker/faster to load the data and immediately write is to a file as it gets processed.  Thoughts?
Fastest Way to Write Multiple CSV's
python;performance;csv