<commit_before>powerstrip-logfiles
===================

A [Powerstrip](https://github.com/ClusterHQ/powerstrip) adapter that allows collection of in-container log files location by a regular log file collection agent, such as the [Sumo Logic File Collector](https://www.sumologic.com).

Docker and the use of containers is spreading like wildfire. In a Docker-ized environment, certain legacy practices and approaches are being challenged. Centralized logging is the one of them. The most popular way of capturing logs coming from a container is to setup the containerized process such that it logs to stdout. Docker then spools this to disk, from where it can be collected. This is great for many use cases.

At the same time, at work at Sumo Logic our customers are telling us that the stdout approach doesn't always work. Not all containers are setup to follow the process-per-container model. This is sometimes referred to as "fat" containers. There are tons of opinions about whether this is the right thing to do or not. Pragmatically speaking, it is a reality for some users.

Even some programs that are otherwise easily containerized as single processes pose some challenges to the stdout model. For example, popular web servers write at least two log files: access and error logs. There are of course workarounds to map this back to a single stdout stream. But ultimately there's only so much multiplexing that can be done before the demuxing operation becomes too painful.

powerstrip-logfiles presents a proof of concept towards easily centralizing log files from within a container. Simply setting `LOGS=/var/logs/nginx` in the container environment, for example, will use a bind mount to make the Nginx access and error logs available on the host under `/var/logs/container-logfiles/containers/[ID of the Nginx container]/var/log/nginx`. A file-based log collector can now simply be configured to recursively collect from `/var/logs/container-logfiles/containers` and will pick up logs from any container configured with the LOGS environment.

This implementation is based on [powerstrip-weave](https://github.com/binocarlos/powerstrip-weave) by @binocarlos (<NAME> <<EMAIL>>). All mistakes and bugs are entirely mine.


### Install

```bash
$ docker build -t raychaser/powerstrip-logfiles .
```

### Run the adapter

```bash
$ sudo docker build -t raychaser/powerstrip-logfiles:latest . && \<commit_msg>Readme fix<commit_after>powerstrip-logfiles
===================

A [Powerstrip](https://github.com/ClusterHQ/powerstrip) adapter that allows collection of in-container log files location by a regular log file collection agent, such as the [Sumo Logic File Collector](https://www.sumologic.com).

Docker and the use of containers is spreading like wildfire. In a Docker-ized environment, certain legacy practices and approaches are being challenged. Centralized logging is the one of them. The most popular way of capturing logs coming from a container is to setup the containerized process such that it logs to stdout. Docker then spools this to disk, from where it can be collected. This is great for many use cases.

At the same time, at work at Sumo Logic our customers are telling us that the stdout approach doesn't always work. Not all containers are setup to follow the process-per-container model. This is sometimes referred to as "fat" containers. There are tons of opinions about whether this is the right thing to do or not. Pragmatically speaking, it is a reality for some users.

Even some programs that are otherwise easily containerized as single processes pose some challenges to the stdout model. For example, popular web servers write at least two log files: access and error logs. There are of course workarounds to map this back to a single stdout stream. But ultimately there's only so much multiplexing that can be done before the demuxing operation becomes too painful.

powerstrip-logfiles presents a proof of concept towards easily centralizing log files from within a container. Simply setting `LOGS=/var/logs/nginx` in the container environment, for example, will use a bind mount to make the Nginx access and error logs available on the host under `/var/logs/container-logfiles/containers/[ID of the Nginx container]/var/log/nginx`. A file-based log collector can now simply be configured to recursively collect from `/var/logs/container-logfiles/containers` and will pick up logs from any container configured with the LOGS environment.

This implementation is based on [powerstrip-weave](https://github.com/binocarlos/powerstrip-weave) by <NAME> <<EMAIL>>. All mistakes and bugs are entirely mine.


### Install

```bash
$ docker build -t raychaser/powerstrip-logfiles .
```

### Run the adapter

```bash
$ sudo docker build -t raychaser/powerstrip-logfiles:latest . && \
{'fraction_non_alphanumeric': 0.06495123469599502, 'fraction_numerical': 0.0, 'mean_word_length': 4.704142011834319, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 6, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '697119', 'n_tokens_mistral': 1201, 'n_tokens_neox': 1144, 'n_words': 664}