<commit_before># visual-log

## 実行に必要なソフトウェア

- Docker
- Java

## Elasticsearch - Kibana 起動

```
cd docker
docker-compose up -d
```

## ログ取り込み

- `batch/logs/access.log` にファイルを配置

## nginxのログ形式

```
  log_format ltsv "time:$time_local"
                  "\thost:$remote_addr"
                  "\tforwardedfor:$http_x_forwarded_for"
                  "\tuser:$remote_user"
                  "\treq:$request"<commit_msg>readme追記<commit_after># visual-log

## 実行に必要なソフトウェア

- Docker
- Java

## Elasticsearch - Kibana 起動

```
cd docker
docker-compose up -d
```

## ログ取り込み

- `batch/logs/access.log` にファイルを配置
- `./gradlew batch:bootRun` を実行

## nginxのログ形式

```
  log_format ltsv "time:$time_local"
                  "\thost:$remote_addr"
                  "\tforwardedfor:$http_x_forwarded_for"
                  "\tuser:$remote_user"
                  "\treq:$request"
{'fraction_non_alphanumeric': 0.13872832369942195, 'fraction_numerical': 0.0, 'mean_word_length': 2.2313432835820897, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3885675', 'n_tokens_mistral': 386, 'n_tokens_neox': 361, 'n_words': 50}