From time to time I need to dig through huge log files (several GB unpacked) to debug a specific error.Now, vim is OK for browsing through the file, but when I need to find something in the file it's completely useless. Is there some tool that could index the log and allow me to search the file faster?Ideally a command line tool.EDIT: Just to clarify, tools like ack or grep aren't suitable since I need to examine the context of the matches and -C,-A,-B just aren't good enough. Plus ack and grep are still unusably slow.
Digging through huge gziped log files
linux;command line;logs
There are really good log indexers that are a bit bigger than command line tool level. Commercially, splunk is the big one and hard to beat. Graylog2 is a nice open source implementation. Elasticsearch+Logstash are quite good too.There is a fair bit of overhead to both setting them up to collect and running the collection server but once you have central log collection as a part of your system, they're great. Querying and reporting logs from multiple sources is the biggest benefit for multi tiered apps.As for command line, if you can plug your logging into syslog, the systemd journald implementation allows for indexing on a number of fields as items are logged. Not sure if that would be useful in text processing though, more for the metadata.Not sure about any ad hoc command line tools, they would still have an extract/load/parse time penalty each time you load the file. That being said, less does most of what I need in a single log file. &pattern  grep pattern/pattern  search forward?pattern  search backwardshift-f   tail -fthe tail mode can take the previous pattern into consideration as well which is great for watching live logs 