I have two text files containing a list of URLs. I want to compare the two files and print out the differences. I know that we can use 'diff' but here's the problem. The urls on one file are full e.g. http://google.com while on the other file they don't have the http://, e.g. google.comFile 1: http://google.comFile 2: google.comI want these to be ignored and only actually different URLs to be printed. Is it possible?
How to print the difference of two text files using shell scripting?
text processing
Take, for example, these two files:$ cat file1http://google.comexample.comhttp://foobar.org$ cat file2example.comgoogle.comfoobar.orgunique.urlI would just use a tool like sed to remove everything up to the last occurrence of //. Also, in order to use diff, both files need to be sorted. Combining the two gives us:$ diff <(sed 's#.*//##' file1 | sort) <(sort file2) 3a4> unique.urlAlternatively, use comm -3:$ comm -3 <(sed 's#.*//##' file1 | sort) <(sort file2)     unique.urlYou can remove the leading whitespace with sed again:$ comm -3 <(sed 's#.*//##' file1 | sort) <(sort file2) | sed 's/^\s*//'unique.urlAnother approach would be to remove the http:// from both files and print what remains, then pass it through uniq -u which will only print unique lines. Since those URLs which are present in both files won't be unique, this will only print the ones that are present on one of the two files:$ sed 's#.*//##' file1 file2 | sort | uniq -uunique.url