I have a file that contains a list of URLs of the form EDIThttp://www.google.com/absd/siidfs/kfd837382$%^$&,www.google.com,google.comyahoo.com/list/page/jhfjkshdjf...I want to write a script that will display the following output  google.com : 2http://www.google.com: 1yahoo.com : 1 I am stuck with the part that I have to read the URLs from the file and check the whole file again. I am new to bash scripting and hence I don't know how to do this.
How to find frequency of occurrences of strings contained in a file?
bash;shell script;files;text processing;patterns
Original input fileAssuming the following input format:http://www.google.com,www.google.com,google.comyahoo.comWith a result looking like this:google.com : 3 yahoo.com : 1 It's hard to determine the entire situation you're in but given the output you're showing us I'd be inclined to convert the input file first so that all the lines are of the form:google.comgoogle.comgoogle.comyahoo.comAnd then run this file through the following set of commands:$ grep -v ^$ data.txt | \      sed -e 's/,$//' -e 's/.*\.\(.*\)\.\(.*\)$/\1.\2/' | \      sort | uniq -c      3 google.com      1 yahoo.comYou can clean up the format of the output so it matches what you want like this:$ grep -v ^$ data.txt | \      sed -e 's/,$//' -e 's/.*\.\(.*\)\.\(.*\)$/\1.\2/' | \      sort | uniq -c | \      awk '{printf %s : %s\n, $1, $2}'      google.com : 3      yahoo.com : 1EDIT #1The OP had a follow-up question where he changed the inputs in the example. So to count this type of input:http://www.google.com/absd/siidfs/kfd837382$%^$&,www.google.com,google.comyahoo.com/list/page/jhfjkshdjf...You could use this adapted one-liner from the first example:$ grep -v ^$ data2.txt | \      sed -e 's/,$//' \          -e 's#\(http://[^/]\+\).*#\1#' \          -e '/^[^http]/ s/^www\.//' \          -e '/^[^http]/ s#\([^/]\+\).*$#\1#' | \          sort | uniq -c | \          awk '{printf %s : %s\n, $1, $2}'2 : google.com1 : http://www.google.com1 : yahoo.com