I have two text files. File 2 has logs over 1,000,000. File 1 has IP addresses line by line. I want to read file 2 lines and search these lines in file 1, I mean:file 1:34.123.21.3245.231.43.2121.34.67.98file 2 :34.123.21.32 0.326 - [30/Oct/2013:06:00:06 +0200]45.231.43.21 6.334 - [30/Oct/2013:06:00:06 +0200]45.231.43.21  3.673 - [30/Oct/2013:06:00:06 +0200]34.123.21.32 4.754 - [30/Oct/2013:06:00:06 +0200]21.34.67.98 1.765 - [30/Oct/2013:06:00:06 +0200]...I want to search for the IP from file 1 line by line in file 2 and print time arguments (example: 0.326) to a new file.How can I do this?
compare files line by line and create new one bash programming
shell script;text processing;diff;join
Join + sortIf you're trying to find IP's that are present in both, you can use the join command but you'll need to use sort to pre-sort the files prior to joining them.$ join -o 2.2 <(sort file1) <(sort file2)Example$ join -o 2.2 <(sort file1) <(sort file2)1.7650.3264.7543.6736.334Another examplefile 1a:$ cat file1a34.123.21.3245.231.43.2121.34.67.981.2.3.45.6.7.89.10.11.12file 2a:$ cat file2a34.123.21.32 0.326 - [30/Oct/2013:06:00:06 +0200]45.231.43.21 6.334 - [30/Oct/2013:06:00:06 +0200]45.231.43.21  3.673 - [30/Oct/2013:06:00:06 +0200]34.123.21.32 4.754 - [30/Oct/2013:06:00:06 +0200]21.34.67.98 1.765 - [30/Oct/2013:06:00:06 +0200]1.2.3.4 1.234 - [30/Oct/2013:06:00:06 +0200]4.3.2.1 4.321 - [30/Oct/2013:06:00:06 +0200]Running the join command:$ join -o 2.2 <(sort file1) <(sort file2)1.2341.7650.3264.7543.6736.334NOTE: The original order of file2 is lost with this method, due to the fact that we sorted it first. However this method only needs to scan file2 a single time now, as a result.grepYou can use grep to search for matches in file2 using lines that are in file1, but this method isn't as efficient as the first method I showed you. It's scanning file2 looking for each line in file1. $ grep -f file1 file2 | awk '{print $2}'Example$ grep -f file1 file2 | awk '{print $2}'0.3266.3343.6734.7541.7651.234Improving grep's performanceYou can speed up the grep's performance by using this form:$ LC_ALL=C grep -f file1 file2 | awk '{print $2}'You can also tell grep that the stings in file1 are fixed length (-F) which will also help in getting better performance.$ LC_ALL=C grep -Ff file1 file2 | awk '{print $2}'Generally in software, you try to avoid having to do this approach though, since it's basically a loop within a loop type of solution. But there are times when it's the best that can be achieved using a computer + software.ReferencesLinux tools to treat files as sets and perform set operations on them