I have a friend who is a photographer and she has a lot of large files as a result. Our current solution is where some files are delivered to my house. I don't have the storage capability to store all of her files.Because we are talking Terabytes of data, a full backup can't be kept at her place. We do currently have backups of the post-processed files.What I would like to do is keep a rolling, x-month backup of these files at her place. Ive got space on a drive there, so I just need a script. This script needs to run at least daily and make sure that the 250GB of available space on this disk is filled with the most recent raw files such that the disk is nearly full.Ive been trying tar, as it has a built-in --newer option. However, then Im just creating a massive tarfile every day. When I run the same job tomorrow, Im creating another massive tarfile, which may well be exactly the same as yesterday's. And so on. It seems very inefficient.My initial thought was rsync, but it doesnt seem to have its own built in time options. There are ways you can faff about using a find command and piping in the results, but apparently this doesnt preserve the directory structure at the target.How is this not just a thing in Unix? Am I missing something? Ironically, this would be trivial in Windows: robocopy <source> <destination> /mir /maxage:<date>.To summarise:We have a source tree full of large filesAt any time we may get a new folder of large filesWe have a hard drive which is not as big as the tree of files, but is big enough to contain the last 2 months of filesI want those files copied, as frequently as I choose, to the driveThe folder structure needs to be retainedWhen a file becomes 2 months + 1 day old, delete itNet result: I always have the last 2 months of files on the drive, no matter whats added to the source
Copy a rolling last 2 months of files
rsync;backup;tar;file copy