<story><title>Grepping logs is still terrible</title><url>http://asylum.madhouse-project.org/blog/2015/05/07/grepping-logs-is-still-terrible/</url></story><parent_chain><item><author>ghshephard</author><text>It&amp;#x27;s beyond me how he doesn&amp;#x27;t understand that text logs are a universal format, easily accessible, that can be instantly turned into whatever binary format you desire with a highly efficient insertion process (Splunk is just one of those that does a great job).&lt;p&gt;Here is the thing he doesn&amp;#x27;t seem to understand - all of us who are sysadmins absolutely understand the value of placing complex and large log files into database so that we can query them efficiently.   We also understand why having multi-terabyte text log files is not useful.&lt;p&gt;But what we find totally unacceptable is log files being shoved into binary repositories as the primary storage location.   Because you know what &lt;i&gt;everyone&lt;/i&gt; has their own idea of what that primary storage location should be, and they are mostly incompatible with each other.&lt;p&gt;The nice thing about text - for the last 40 years it&amp;#x27;s been universally readable, and will be for the next 40 years.  Many of these binary repositories will be unreadable within a short period, and will be immediately unreadable to those people who don&amp;#x27;t know the magic tool to open them.</text></item></parent_chain><comment><author>scrollaway</author><text>&amp;gt; text logs are a universal format&lt;p&gt;Uh, I don&amp;#x27;t know what world you live in but I&amp;#x27;d like the address because mine sucks in comparison.&lt;p&gt;Text logs are definitely not a &amp;quot;universal format&amp;quot;. Easily accessible, sure. Human readable most of the time? Okay. Universal? Ten times nope.&lt;p&gt;Give you an example: uwsgi logs don&amp;#x27;t even have timestamps, and contain whatever crap the program&amp;#x27;s stdout outputs, so you often end up with three different types of your &amp;quot;universal format&amp;quot; in there. I&amp;#x27;m not giving this example because it&amp;#x27;s contrived, but because I was dealing with it the very moment I read your comment.</text></comment>
<story><title>Grepping logs is still terrible</title><url>http://asylum.madhouse-project.org/blog/2015/05/07/grepping-logs-is-still-terrible/</url></story><parent_chain><item><author>ghshephard</author><text>It&amp;#x27;s beyond me how he doesn&amp;#x27;t understand that text logs are a universal format, easily accessible, that can be instantly turned into whatever binary format you desire with a highly efficient insertion process (Splunk is just one of those that does a great job).&lt;p&gt;Here is the thing he doesn&amp;#x27;t seem to understand - all of us who are sysadmins absolutely understand the value of placing complex and large log files into database so that we can query them efficiently.   We also understand why having multi-terabyte text log files is not useful.&lt;p&gt;But what we find totally unacceptable is log files being shoved into binary repositories as the primary storage location.   Because you know what &lt;i&gt;everyone&lt;/i&gt; has their own idea of what that primary storage location should be, and they are mostly incompatible with each other.&lt;p&gt;The nice thing about text - for the last 40 years it&amp;#x27;s been universally readable, and will be for the next 40 years.  Many of these binary repositories will be unreadable within a short period, and will be immediately unreadable to those people who don&amp;#x27;t know the magic tool to open them.</text></item></parent_chain><comment><author>magnifyingglass</author><text>Agreed. Logs are for when everything and anything is broken. They aren&amp;#x27;t supposed to be pretty or highly functional, they are just meant as a starting point for gathering data.</text></comment>