need advice on how to write lots of data fast
fauigerzigerk: Put the data in a message queue asynchronously. A second process can take the data from there and put it in a database or wherever it's easiest for you to analyse.