I've been getting this strange error message in one of my build scripts - cp fails, returning the error File exists. I've even tried using cp -f, which should overwrite the file if it exists, but the error still shows up. Running cp to overwrite existing files works perfectly when I do it manually. What could cause this error?
cannot create regular file 'filename': File exists
cp;concurrency
This turned out to be caused by a race condition. cp checks if the destination file already exists, and if not - overwrites it. The problem was happening because this cp command was being run twice in parallel, which caused the file in question to sometimes appear after checking whether it exists, but before the attempt to create the file. The strace output looks like this:# Command was cp a bstat(b, 0x7fff89510620)               = -1 ENOENT (No such file or directory)stat(a, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0stat(b, 0x7fff895103a0)               = -1 ENOENT (No such file or directory)# File b will be created at this point in timeopen(a, O_RDONLY)                     = 3fstat(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0open(b, O_WRONLY|O_CREAT|O_EXCL, 0644) = -1 EEXIST (File exists)Here's some bash code used to catch this:#!/bin/bashtouch af() {  while true; do    rm -f b    strace -o /tmp/cp${BASHPID}.trace cp a b || break  done}cleanup() {  kill -9 %1 %2}f &f &trap cleanup exitwaitThis same error can occur with mkdir -p or any other action that attempts to overwrite a file. Using flock can help avoid race conditions in cases like this.