<commit_before># magellan-log-collector

## Run local server

```
goapp serve
```

## Deploy

Specify GCP project id and api tokens (comma separated).

```
appcfg.py -A YOUR-PROJECT-ID -E GCLOUD_PROJECT:YOUR-PROJECT-ID -E API_TOKEN:<PASSWORD> update .
```

## Upload package to deploy via Google App Engine Admin API

Run the following command To gather source files and make manifest file. `v1` stands for the version of application.

```
./makepkg.sh v1
```

Upload source files and a manifest file to gcs.

```
gsutil cp -R pkg/v1 gs://your-gae-repository/magellan-log-collector/v1
```<commit_msg>Use `gcloud app deploy` instead of `appcfg.py`<commit_after># magellan-log-collector

## Run local server

```
goapp serve
```

## Deploy

Specify GCP project id and api tokens (comma separated).

```
gcloud --project=YOUR-PROJECT-ID app deploy app.yaml
```

## Upload package to deploy via Google App Engine Admin API

Run the following command To gather source files and make manifest file. `v1` stands for the version of application.

```
./makepkg.sh v1
```

Upload source files and a manifest file to gcs.

```
gsutil cp -R pkg/v1 gs://your-gae-repository/magellan-log-collector/v1
```
{'fraction_non_alphanumeric': 0.11764705882352941, 'fraction_numerical': 0.0067226890756302525, 'mean_word_length': 4.004201680672269, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '693612', 'n_tokens_mistral': 410, 'n_tokens_neox': 393, 'n_words': 143}