sudo tshark -i ppp0 'tcp port 80   \and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'   \-R'http.request.method == GET && http.request.uri contains /ABC/XYZ' \-T fields -e http.host -e http.request.uriI am using the above lines of tshark filtering the output based on /ABC/XYZ.it outputs dynamically random data for example like where i find duplicate lines example :1bcdJOSHhijklmnopqrstuvwxyz1bcdefghijklmnopqrstuvwxyz1bcdefghijklmnopqrstuvwxyz3bcdefghijklmnopqrstuvwxyz2bcdefghijklmnopqrstuvwxyz1bcdJOSHhijklmnopqrstuvwxyz3bcdefghijklmnopqrstuvwxyzIs there any way to remove the duplicates without creating any filemay be using some kind of buffer or pipe.or can tshark itself do itEDIT : I am not sure why but i get what i want after i prefix tshark with stdbuf -o L as suggested by meuh
How can i remove duplicates from the output of tshark output
linux;bash;sed;awk;posix
If you have a stop condition on your tshark you can simply pipe the output into |sort -u. Alternatively, pipe continuous output intoawk '{if(!seen[$0]++)print}'You may need to have tshark not buffer its stdout: try prefixing the tshark with stdbuf -o L.