I am writing an application that works with satellite images, and my boss asked me to look at some of the commercial applications, and see how they behave. I found a strange behavior and then as I was looking, I found it in other standard applications as well.These programs first write to the temp folder, and then copy it to the intended destination.Example: 7zip first extracts to the temp folder, and then copies the extracted data to the location that you had asked it to extract the data to.I see several problems with this approach:The temp folder might not have enough space, while the intendedlocation might have that much space.If it is a large file, it can take a non-negligible amount of timefor the copy operation.I thought about it a lot, but I couldn't see one single positive point to doing this. Am I missing something, or is there a real benefit to doing this?
What is the benefit of writing to a temp location, And then copying it to the intended destination?
file handling;file systems;file storage
A few reasons I can think of:On most platforms, file moves are atomic, but file writes are not (especially if you can't write all the data in one go). So if you have the typical producer/consumer pattern (one process produces files, the other watches a directory and picks up everything it finds), writing to a temp folder first and only then moving to the real location means the consumer can never see an unfinished file.If the process that writes the file dies halfway through, you have a broken file on your disk. If it's in a real location, you have to take care of cleaning it up yourself, but if it's in a temp location, the OS will take care of it.If the file happens to be created while a backup job is running, the job may pick up an incomplete file; temp directories are generally excluded from backups, so the file will only be included once moved to the final destination.The temp directory may be on a fast-but-volatile filesystem (e.g. a ramdisk), which can be beneficial for things like downloading several chunks of the same file in parallel, or doing in-place processing on the file with lots of seeks. Also, temp directories tend to cause more fragmentation than directories with less frequent reads, writes, and deletes, and keeping the temp directory on a separate partition can help keep fragmentation of the other partitions down.TL;DR - it mostly boils down to atomicity, that is, you want to make it so that (at the final location) the file is either complete or not there at all at any given time.