I have a grep commandgrep -Fvf cleaned1 cleanedR > cleaned2that runs and kills my PC too much ram usagecleanedR is a list of files (14 million of them) that I need to run some operation thru (dowork.sh cleanedR), everything that has been completed is printed into cleaned1 (in a different sort order, so diff wont work)cleaned1 is a list of files (10 million)I had to cancel the dowork.sh operation, to do something else, but I can resume it later thru another list (dowork.sh cleaned2). cleaned2 doesnt exist yetcleaned2 will be a list of 4 million files which I have yet to run dowork.sh thru.Essentially I need to do this mathematical operation (its a subtract operation):list of files cleanedR - list of files cleaned1 = list of files cleaned2cleaned1 and cleanedR are files containing absolute file structure, with millions of files, these are big files. cleaned1 is 1.3G and cleanedR is 1.5G.I have like 30 G of ram available but it used up all of that and crashedI was thinking why does grep use ram on this, can I make ram use some other temp directory. Sort has that option with -T. So I was looking for a similar way for grep.I am open to other ideas.-f runs thru cleaned1 which contains millions of expressions (file names) instead of string regular expression. 1 file per line-F does full match on the line. filenames can be complex and grep can mistake some chars for regular expression chars, we dont want that so we do a full line match.-v is the subtract / exclude operation
grep crashed with too much ram usage as it operated thru gig sized files
grep;ram