<commit_before>Apache HTTPD logparser
===
This is a Logparsing framework intended to make parsing Apache HTTPD logfiles much easier.

The basic idea is that you should be able to have a parser that you can construct by simply
telling it with what configuration options the line was written.

So we are using the LogFormat that wrote the file as the input parameter for the parser that reads the same file.
In addition to the config options specified in the Apache HTTPD manual under
[Custom Log Formats](http://httpd.apache.org/docs/2.2/mod/mod_log_config.html) the following are also recognized:

* common
* combined
* combinedio
* referer
* agent

Currently *not yet supported features*:

* %{format}t : The time, in the form given by format, which should be in strftime(3) format. (potentially localized)

A simple workaround for this limitation for all logparser versions before 2.5: replace the **%{...}t** with **%{timestamp}i** .
You will then get this timestamp field as if it was a request header: HTTP.HEADER:request.header.timestamp

Analyze almost anything
===
I wrote this parser for practical reallife situations. In reality a lot happens that is not allowed when looking at the<commit_msg>Added travis and license badges to readme<commit_after>Apache HTTPD logparser
===
[![Travis Build status](https://api.travis-ci.org/nielsbasjes/logparser.png)](https://travis-ci.org/nielsbasjes/logparser) [![License](http://img.shields.io/:license-apache-blue.svg)](http://www.apache.org/licenses/LICENSE-2.0.html)

This is a Logparsing framework intended to make parsing Apache HTTPD logfiles much easier.

The basic idea is that you should be able to have a parser that you can construct by simply
telling it with what configuration options the line was written.

So we are using the LogFormat that wrote the file as the input parameter for the parser that reads the same file.
In addition to the config options specified in the Apache HTTPD manual under
[Custom Log Formats](http://httpd.apache.org/docs/2.2/mod/mod_log_config.html) the following are also recognized:

* common
* combined
* combinedio
* referer
* agent

Currently *not yet supported features*:

* %{format}t : The time, in the form given by format, which should be in strftime(3) format. (potentially localized)

A simple workaround for this limitation for all logparser versions before 2.5: replace the **%{...}t** with **%{timestamp}i** .
You will then get this timestamp field as if it was a request header: HTTP.HEADER:request.header.timestamp

Analyze almost anything
===
I wrote this parser for practical reallife situations. In reality a lot happens that is not allowed when looking at the
{'fraction_non_alphanumeric': 0.07565543071161049, 'fraction_numerical': 0.0044943820224719105, 'mean_word_length': 4.744086021505376, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3412756', 'n_tokens_mistral': 723, 'n_tokens_neox': 711, 'n_words': 353}