Q:

Multiprocessing: Pass file handle to Process

My program spawns multiple processes to do some time consuming calculations. The results are then collected in a queue and a writer process writes them into an output file.
Below is a simplified version of my code which should illustrate my issue. If I comment out the flush statement in the Writer class, test.out is empty at the end of the program.
What exactly is happening here? Is test.out not closed properly? Was it naive to assume that passing the file handle to an autonomous process should work in the first place?
from multiprocessing import JoinableQueue, Process

def main():
    queue = JoinableQueue()
    queue.put("hello world!")

    with open("test.out", "w") as outhandle:
        wproc = Writer(queue, outhandle)
        wproc.start()
        queue.join()

    with open("test.out") as handle:
        for line in handle:
            print(line.strip())

class Writer(Process):

    def __init__(self, queue, handle):
        Process.__init__(self)
        self.daemon = True
        self.queue = queue
        self.handle = handle

    def run(self):
        while True:
            msg = self.queue.get()
            print(msg, file=self.handle)
            #self.handle.flush()
            self.queue.task_done()

if __name__ == '__main__':
    main()

A:

The writer is a separate process. The data it writes to the file might be buffered, and because the process keeps running, it doesn't know that it should flush the buffer (write it to the file). Flushing manually is the right thing to do. 
Normally, the file would be closed when you exit the with block, and this would flush the buffer. But the parent process doesn't know anything about its children's buffers, so the child has to flush it's own buffer (closing the file should work too - that doesn't close the file for the parent, at least on Unix systems). 
Also, check out the Pool class from multiprocessing (https://docs.python.org/3/library/multiprocessing.html#module-multiprocessing.pool) - it might save you some work. 

