<commit_before># Introduction

mtail is intended to run one per machine, and serve as monitoring glue for multiple applications running on that machine.  It runs one or more programs in a 1:1 mapping to those client applications.

## Configuration Overview

mtail is configured through commandline flags.

The `--help` flag will print a list of flags for configuring `mtail`.

(Flags may be prefixed with either `-` or `--`)

Basic flags necessary to start `mtail`:

  * `--logs` is a comma separated list of filenames to extract from, but can also be used multiple times, and each filename can be a [glob pattern](http://godoc.org/path/filepath#Match).  Named pipes can be read from when passed as a filename to this flag.
  * `--progs` is a directory path containing [mtail programs](Language.md). Programs must have the `.mtail` suffix.

mtail runs an HTTP server on port 3903, which can be changed with the `--port` flag.

# Details

## Launching mtail

```
mtail --progs /etc/mtail --logs /var/log/syslog --logs /var/log/ntp/peerstats
```

`mtail` will start to read the specified logs from their current end-of-file,
and read new updates appended to these logs as they arrive.  It will attempt to
correctly handle log files that have been rotated by renaming or symlink
changes.

### Getting the logs in

Use `--logs` multiple times to pass in glob patterns that match the logs you
want to tail.  This includes named pipes.

### Launching under Docker

`mtail` can be run as a sidecar process if you expose an application container's logs with a volume.

`docker run -d --name myapp -v /var/log/myapp myapp`

for example exports a volume called `/var/log/myapp` (named the same as the
hypothetical path where `myapp`s logs are written.

Then launch the `mtail` docker image and pass in the volume:

    docker run -dP \
       --name myapp-mtail \
       --volumes-from myapp \
       -v examples:/etc/mtail \
       mtail --logs /var/log/myapp --progs /etc/mtail

This example fetches the volumes from the `myapp` container, and mounts them in
the mtail container (which we've called `myapp-mtail`).  We also mount the
`examples` directory as `/etc/mtail` in the container.  We launch `mtail` with
the `logs` and `progs` flags to point to our two mounted volumes.

The `-P` flag ensures `mtail-myapp`'s port 3903 is exposed for collection,
refer to `docker ps` to find out where it's mapped to on the host.

## Writing the programme<commit_msg>Document how to use the poll_notify flag.<commit_after># Introduction

mtail is intended to run one per machine, and serve as monitoring glue for multiple applications running on that machine.  It runs one or more programs in a 1:1 mapping to those client applications.

## Configuration Overview

mtail is configured through commandline flags.

The `--help` flag will print a list of flags for configuring `mtail`.

(Flags may be prefixed with either `-` or `--`)

Basic flags necessary to start `mtail`:

  * `--logs` is a comma separated list of filenames to extract from, but can also be used multiple times, and each filename can be a [glob pattern](http://godoc.org/path/filepath#Match).  Named pipes can be read from when passed as a filename to this flag.
  * `--progs` is a directory path containing [mtail programs](Language.md). Programs must have the `.mtail` suffix.

mtail runs an HTTP server on port 3903, which can be changed with the `--port` flag.

# Details

## Launching mtail

```
mtail --progs /etc/mtail --logs /var/log/syslog --logs /var/log/ntp/peerstats
```

`mtail` will start to read the specified logs from their current end-of-file,
and read new updates appended to these logs as they arrive.  It will attempt to
correctly handle log files that have been rotated by renaming or symlink
changes.

### Getting the logs in

Use `--logs` multiple times to pass in glob patterns that match the logs you
want to tail.  This includes named pipes.

### Polling the file system

If your system is not supported by `fsnotify` then mtail will fall back to polling mode.  You can also specify this explicitly with the `--poll_interval` flag, for example

```
mtail --progs /etc/mtail --logs /var/log/syslog --poll_interval 250ms
```

*N.B.* `mtail` will only discover new files if using `fsnotify`.  If you rely on polling mode, then only the files that mtail sees at program startup will be watched.

### Launching under Docker

`mtail` can be run as a sidecar process if you expose an application container's logs with a volume.

`docker run -d --name myapp -v /var/log/myapp myapp`

for example exports a volume called `/var/log/myapp` (named the same as the
hypothetical path where `myapp`s logs are written.

Then launch the `mtail` docker image and pass in the volume:

    docker run -dP \
       --name myapp-mtail \
       --volumes-from myapp \
       -v examples:/etc/mtail \
       mtail --logs /var/log/myapp --progs /etc/mtail

This example fetches the volumes from the `myapp` container, and mounts them in
the mtail container (which we've called `myapp-mtail`).  We also mount the
`examples` directory as `/etc/mtail` in the container.  We launch `mtail` with
the `logs` and `progs` flags to point to our two mounted volumes.

The `-P` flag ensures `mtail-myapp`'s port 3903 is exposed for collection,
refer to `docker ps` to find out where it's mapped to on the host.

## Writing the programme
{'fraction_non_alphanumeric': 0.08217140732478156, 'fraction_numerical': 0.0042758877114705334, 'mean_word_length': 3.578723404255319, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '184340', 'n_tokens_mistral': 1608, 'n_tokens_neox': 1530, 'n_words': 797}