simply put, I have a file with lines of text that are unknown to me, something likeabaaddddbbbbccccabaaaaaaabaathe result I'd like to get isddddbbbbccccaaaawhere all the duplicates are completely removed. if one line of text is duplicated anywhere else, remove this line and any repetitions that follow. is this possible to do? all of my searches show output with duplicates removed, leaving one instance, which I would like removed.
remove all duplicates from a text file without sort
linux;text processing
this approach takes 2 passes through the file: one to count the number of times each line occurs, one to print the lines that only appear once:awk 'NR == FNR {count[$0]++; next}; count[$0] == 1' file file