<commit_before>Chyle [![Build Status](https://travis-ci.org/antham/chyle.svg?branch=master)](https://travis-ci.org/antham/chyle) [![codecov](https://codecov.io/gh/antham/chyle/branch/master/graph/badge.svg)](https://codecov.io/gh/antham/chyle) [![codebeat badge](https://codebeat.co/badges/c3867610-2741-4ae3-a195-d5e9711c7fcd)](https://codebeat.co/projects/github-com-antham-chyle-master) [![Go Report Card](https://goreportcard.com/badge/github.com/antham/chyle)](https://goreportcard.com/report/github.com/antham/chyle)
=====

Chyle produces a changelog from a git repository.

---

* [Usage](#usage)
* [How it works ?](#how-it-works-)
* [Setup](#setup)
* [Examples](#examples)
* [Documentation](#documentation)

---

## Usage

```
Create a new changelog according to what is defined as config.

Changelog creation follows this process :

1 - fetch commits
2 - filter relevant commits
3 - extract informations from commits fields and publish them to new fields
4 - enrich extracted datas with external apps
5 - publish datas

Usage:
  chyle create [flags]

Global Flags:
      --debug   enable debugging
```

## How it works ?

Chyle fetch a range of commits using given criterias from a git repository. From those commits you can extract relevant datas from commit message, author, and so on, and add it to original payload. You can afterwards if needed, enrich your payload with various useful datas contacting external apps (shell command, apis, ....) and finally, you can publish what you harvested (to an external api, stdout, ....). You can mix all steps together, avoid some, combine some, it's up to you.

## Setup

Download from release page according to your architecture chyle binary : https://github.com/antham/chyle/releases

You need afterwards to configure each module through environments variables : there are activated when you configure at least one environment variable they need to work.

## Examples

We will use this repository : [https://github.com/antham/test-git](https://github.com/antham/test-git), created for chyle testing purpose only, you can try examples on it. Don't forget to clone repository and adapt some environment variables to your configuration.

---

* [Get a JSON ouput of all merge commits](#get-a-json-ouput-of-all-merge-commits)
* [Get a markdown ouput of merge and regular commits](#get-a-markdown-ouput-of-merge-and-regular-commits)
* [Get a JSON ouput of all merge commits and contact github issue api to enrich payload](#get-a-json-ouput-of-all-merge-commits-and-contact-github-issue-api-to-enrich-payload)

---

### Get a JSON ouput of all merge commits

commands :

```bash
export CHYLE_GIT_REFERENCE_FROM=a00ee81c109c8787f0ea161a776d2c9795f816cd
export CHYLE_GIT_REFERENCE_TO=f617fb708dfa6fa290205615ea98c53a860e499d
export CHYLE_GIT_REPOSITORY_PATH=/your-local-path/test-git
export CHYLE_MATCHERS_TYPE=merge
export CHYLE_SENDERS_STDOUT_FORMAT="json"

chyle create
```

output :

```json
{
  "datas": [
    {
      "authorDate": "2017-05-10 22:24:40 +0200 +0200",
      "authorEmail": "<EMAIL>",
      "authorName": "<NAME>",
      "committerDate": "2017-05-10 22:24:40 +0200 +0200",
      "committerEmail": "<EMAIL>",
      "committerName": "GitHub",
      "id": "f617fb708dfa6fa290205615ea98c53a860e499d",
      "message": "Merge pull request #3 from antham/test2\n\nTest2",
      "type": "merge"
    },
    {
      "authorDate": "2017-05-10 22:22:03 +0200 +0200",
      "authorEmail": "<EMAIL>",
      "authorName": "<NAME>",
      "committerDate": "2017-05-10 22:22:03 +0200 +0200",
      "committerEmail": "<EMAIL>",
      "committerName": "GitHub",
      "id": "8fdfae00cbcc66936113a60f5146d110f2ba3c28",
      "message": "Merge pull request #1 from antham/test\n\nTest",
      "type": "merge"
    }
  ],
  "metadatas": {}
}
```

### Get a markdown ouput of merge and regular commits

commands :

```bash
export CHYLE_GIT_REFERENCE_FROM=a00ee81c109c8787f0ea161a776d2c9795f816cd
export CHYLE_GIT_REFERENCE_TO=f617fb708dfa6fa290205615ea98c53a860e499d
export CHYLE_GIT_REPOSITORY_PATH=/your-local-path/test-git
export CHYLE_SENDERS_STDOUT_FORMAT="template"
export CHYLE_SENDERS_STDOUT_TEMPLATE='{{ range $key, $value := .Datas }}
{{ $value.id }} => **{{ regexFind ".*?\n" $value.message | trim }}** *({{ $value.authorName }} - {{ $value.authorDate | date "2006-01-02 15:04:05" }})*
{{ end }}'

chyle create
```
output :

```markdown
f617fb708dfa6fa290205615ea98c53a860e499d => **Merge pull request #3 from antham/test2** *(<NAME> - 2017-05-29 02:08:37)*

d8106fffee242f5b6394a103059b4064a83fcf3b => **Whatever** *(antham - 2017-05-29 02:08:37)*

e0a746c906fba7e2462f5717322b9eb55aca3943 => **Whatever** *(antham - 2017-05-29 02:08:37)*

118ad33a1d4ffc66bbeb74a1aba7524ef192ae62 => **Whatever** *(antham - 2017-05-29 02:08:37)*

78dcf412cc21d4054e06c534876200a89c04622e => **Whatever** *(antham - 2017-05-29 02:08:37)*

44fb3316ea67298df5a2b6fbb43795990575ec32 => **Whatever** *(antham - 2017-05-29 02:08:37)*

8fdfae00cbcc66936113a60f5146d110f2ba3c28 => **Merge pull request #1 from antham/test** *(<NAME> - 2017-05-29 02:08:37)*
```

### Get a JSON ouput of all merge commits and contact github issue api to enrich payload

commands :

```bash
export CHYLE_GIT_REFERENCE_FROM=a00ee81c109c8787f0ea161a776d2c9795f816cd
export CHYLE_GIT_REFERENCE_TO=f617fb708dfa6fa290205615ea98c53a860e499d
export CHYLE_GIT_REPOSITORY_PATH=/your-local-path/test-git
export CHYLE_MATCHERS_TYPE=merge
export CHYLE_EXTRACTORS_GITHUBISSUEID_ORIGKEY=message
export CHYLE_EXTRACTORS_GITHUBISSUEID_DESTKEY=githubIssueId
export CHYLE_EXTRACTORS_GITHUBISSUEID_REG="\#(\d+)"
export CHYLE_DECORATORS_GITHUBISSUE_REPOSITORY_NAME=test-git
export CHYLE_DECORATORS_GITHUBISSUE_CREDENTIALS_OAUTHTOKEN=token
export CHYLE_DECORATORS_GITHUBISSUE_CREDENTIALS_OWNER=antham
export CHYLE_DECORATORS_GITHUBISSUE_KEYS_NUMBER_DESTKEY=ticketNumber
export CHYLE_DECORATORS_GITHUBISSUE_KEYS_NUMBER_FIELD=number
export CHYLE_DECORATORS_GITHUBISSUE_KEYS_COMMENTNUMBER_DESTKEY=commentNumber
export CHYLE_DECORATORS_GITHUBISSUE_KEYS_COMMENTNUMBER_FIELD=comments
export CHYLE_SENDERS_STDOUT_FORMAT="json"

chyle create
```
output :

```json
{
  "datas": [
    {
      "authorDate": "2017-05-10 22:24:40 +0200 +0200",
      "authorEmail": "<EMAIL>",
      "authorName": "<NAME>",
      "commentNumber": 0,
      "committerDate": "2017-05-10 22:24:40 +0200 +0200",
      "committerEmail": "<EMAIL>",
      "committerName": "GitHub",
      "githubIssueId": 3,
      "id": "f617fb708dfa6fa290205615ea98c53a860e499d",
      "message": "Merge pull request #3 from antham/test2\n\nTest2",
      "ticketNumber": 3,
      "type": "merge"
    },
    {
      "authorDate": "2017-05-10 22:22:03 +0200 +0200",
      "authorEmail": "<EMAIL>",
      "authorName": "<NAME>",
      "commentNumber": 0,
      "committerDate": "2017-05-10 22:22:03 +0200 +0200",
      "committerEmail": "<EMAIL>",
      "committerName": "GitHub",
      "githubIssueId": 1,
      "id": "8fdfae00cbcc66936113a60f5146d110f2ba3c28",
      "message": "Merge pull request #1 from antham/test\n\nTest",
      "ticketNumber": 1,
      "type": "merge"
    }
  ],
  "metadatas": {}
}
```

## Documentation

* [Default fields](#default-fields)
* [General config](#general-config)
* [Matchers](#matchers)
* [Extractors](#extractors)
* [Decorators](#decorators)<commit_msg>doc(README) : add circle example<commit_after>Chyle [![Build Status](https://travis-ci.org/antham/chyle.svg?branch=master)](https://travis-ci.org/antham/chyle) [![codecov](https://codecov.io/gh/antham/chyle/branch/master/graph/badge.svg)](https://codecov.io/gh/antham/chyle) [![codebeat badge](https://codebeat.co/badges/c3867610-2741-4ae3-a195-d5e9711c7fcd)](https://codebeat.co/projects/github-com-antham-chyle-master) [![Go Report Card](https://goreportcard.com/badge/github.com/antham/chyle)](https://goreportcard.com/report/github.com/antham/chyle)
=====

Chyle produces a changelog from a git repository.

---

* [Usage](#usage)
* [How it works ?](#how-it-works-)
* [Setup](#setup)
* [Examples](#examples)
* [Documentation](#documentation)

---

## Usage

```
Create a new changelog according to what is defined as config.

Changelog creation follows this process :

1 - fetch commits
2 - filter relevant commits
3 - extract informations from commits fields and publish them to new fields
4 - enrich extracted datas with external apps
5 - publish datas

Usage:
  chyle create [flags]

Global Flags:
      --debug   enable debugging
```

## How it works ?

Chyle fetch a range of commits using given criterias from a git repository. From those commits you can extract relevant datas from commit message, author, and so on, and add it to original payload. You can afterwards if needed, enrich your payload with various useful datas contacting external apps (shell command, apis, ....) and finally, you can publish what you harvested (to an external api, stdout, ....). You can mix all steps together, avoid some, combine some, it's up to you.

## Setup

Download from release page according to your architecture chyle binary : https://github.com/antham/chyle/releases

You need afterwards to configure each module through environments variables : there are activated when you configure at least one environment variable they need to work.

## Examples

We will use this repository : [https://github.com/antham/test-git](https://github.com/antham/test-git), created for chyle testing purpose only, you can try examples on it. Don't forget to clone repository and adapt some environment variables to your configuration.

---

* [Get a JSON ouput of all merge commits](#get-a-json-ouput-of-all-merge-commits)
* [Get a markdown ouput of merge and regular commits](#get-a-markdown-ouput-of-merge-and-regular-commits)
* [Get a JSON ouput of all merge commits and contact github issue api to enrich payload](#get-a-json-ouput-of-all-merge-commits-and-contact-github-issue-api-to-enrich-payload)
* [Populate a release in github from CircleCI](#populate-a-release-in-github-from-circleci)

---

### Get a JSON ouput of all merge commits

commands :

```bash
export CHYLE_GIT_REFERENCE_FROM=a00ee81c109c8787f0ea161a776d2c9795f816cd
export CHYLE_GIT_REFERENCE_TO=f617fb708dfa6fa290205615ea98c53a860e499d
export CHYLE_GIT_REPOSITORY_PATH=/your-local-path/test-git
export CHYLE_MATCHERS_TYPE=merge
export CHYLE_SENDERS_STDOUT_FORMAT="json"

chyle create
```

output :

```json
{
  "datas": [
    {
      "authorDate": "2017-05-10 22:24:40 +0200 +0200",
      "authorEmail": "<EMAIL>",
      "authorName": "<NAME>",
      "committerDate": "2017-05-10 22:24:40 +0200 +0200",
      "committerEmail": "<EMAIL>",
      "committerName": "GitHub",
      "id": "f617fb708dfa6fa290205615ea98c53a860e499d",
      "message": "Merge pull request #3 from antham/test2\n\nTest2",
      "type": "merge"
    },
    {
      "authorDate": "2017-05-10 22:22:03 +0200 +0200",
      "authorEmail": "<EMAIL>",
      "authorName": "<NAME>",
      "committerDate": "2017-05-10 22:22:03 +0200 +0200",
      "committerEmail": "<EMAIL>",
      "committerName": "GitHub",
      "id": "8fdfae00cbcc66936113a60f5146d110f2ba3c28",
      "message": "Merge pull request #1 from antham/test\n\nTest",
      "type": "merge"
    }
  ],
  "metadatas": {}
}
```

### Get a markdown ouput of merge and regular commits

commands :

```bash
export CHYLE_GIT_REFERENCE_FROM=a00ee81c109c8787f0ea161a776d2c9795f816cd
export CHYLE_GIT_REFERENCE_TO=f617fb708dfa6fa290205615ea98c53a860e499d
export CHYLE_GIT_REPOSITORY_PATH=/your-local-path/test-git
export CHYLE_SENDERS_STDOUT_FORMAT="template"
export CHYLE_SENDERS_STDOUT_TEMPLATE='{{ range $key, $value := .Datas }}
{{ $value.id }} => **{{ regexFind ".*?\n" $value.message | trim }}** *({{ $value.authorName }} - {{ $value.authorDate | date "2006-01-02 15:04:05" }})*
{{ end }}'

chyle create
```
output :

```markdown
f617fb708dfa6fa290205615ea98c53a860e499d => **Merge pull request #3 from antham/test2** *(<NAME> - 2017-05-29 02:08:37)*

d8106fffee242f5b6394a103059b4064a83fcf3b => **Whatever** *(antham - 2017-05-29 02:08:37)*

e0a746c906fba7e2462f5717322b9eb55aca3943 => **Whatever** *(antham - 2017-05-29 02:08:37)*

118ad33a1d4ffc66bbeb74a1aba7524ef192ae62 => **Whatever** *(antham - 2017-05-29 02:08:37)*

78dcf412cc21d4054e06c534876200a89c04622e => **Whatever** *(antham - 2017-05-29 02:08:37)*

44fb3316ea67298df5a2b6fbb43795990575ec32 => **Whatever** *(antham - 2017-05-29 02:08:37)*

8fdfae00cbcc66936113a60f5146d110f2ba3c28 => **Merge pull request #1 from antham/test** *(<NAME> - 2017-05-29 02:08:37)*
```

### Get a JSON ouput of all merge commits and contact github issue api to enrich payload

commands :

```bash
export CHYLE_GIT_REFERENCE_FROM=a00ee81c109c8787f0ea161a776d2c9795f816cd
export CHYLE_GIT_REFERENCE_TO=f617fb708dfa6fa290205615ea98c53a860e499d
export CHYLE_GIT_REPOSITORY_PATH=/your-local-path/test-git
export CHYLE_MATCHERS_TYPE=merge
export CHYLE_EXTRACTORS_GITHUBISSUEID_ORIGKEY=message
export CHYLE_EXTRACTORS_GITHUBISSUEID_DESTKEY=githubIssueId
export CHYLE_EXTRACTORS_GITHUBISSUEID_REG="\#(\d+)"
export CHYLE_DECORATORS_GITHUBISSUE_REPOSITORY_NAME=test-git
export CHYLE_DECORATORS_GITHUBISSUE_CREDENTIALS_OAUTHTOKEN=token
export CHYLE_DECORATORS_GITHUBISSUE_CREDENTIALS_OWNER=antham
export CHYLE_DECORATORS_GITHUBISSUE_KEYS_NUMBER_DESTKEY=ticketNumber
export CHYLE_DECORATORS_GITHUBISSUE_KEYS_NUMBER_FIELD=number
export CHYLE_DECORATORS_GITHUBISSUE_KEYS_COMMENTNUMBER_DESTKEY=commentNumber
export CHYLE_DECORATORS_GITHUBISSUE_KEYS_COMMENTNUMBER_FIELD=comments
export CHYLE_SENDERS_STDOUT_FORMAT="json"

chyle create
```
output :

```json
{
  "datas": [
    {
      "authorDate": "2017-05-10 22:24:40 +0200 +0200",
      "authorEmail": "<EMAIL>",
      "authorName": "<NAME>",
      "commentNumber": 0,
      "committerDate": "2017-05-10 22:24:40 +0200 +0200",
      "committerEmail": "<EMAIL>",
      "committerName": "GitHub",
      "githubIssueId": 3,
      "id": "f617fb708dfa6fa290205615ea98c53a860e499d",
      "message": "Merge pull request #3 from antham/test2\n\nTest2",
      "ticketNumber": 3,
      "type": "merge"
    },
    {
      "authorDate": "2017-05-10 22:22:03 +0200 +0200",
      "authorEmail": "<EMAIL>",
      "authorName": "<NAME>",
      "commentNumber": 0,
      "committerDate": "2017-05-10 22:22:03 +0200 +0200",
      "committerEmail": "<EMAIL>",
      "committerName": "GitHub",
      "githubIssueId": 1,
      "id": "8fdfae00cbcc66936113a60f5146d110f2ba3c28",
      "message": "Merge pull request #1 from antham/test\n\nTest",
      "ticketNumber": 1,
      "type": "merge"
    }
  ],
  "metadatas": {}
}
```

### Populate a release in github from CircleCI

Let's create a script release.sh

```bash
#!/usr/bin/env bash

set -e

### Functions

function setPreviousTag {
    if [ ! -n "$1" ]; then
        PREVIOUS_TAG="$(git describe --abbrev=0 --always --tags $2^)"
    fi

    if [[ $PREVIOUS_TAG =~ ^[0-9a-f]{40} ]];then
        PREVIOUS_TAG_SHA="$(git rev-list --max-parents=0 HEAD|head -n 1)"
        PREVIOUS_TAG="first commit"

        return
    fi

    PREVIOUS_TAG_SHA="$(git rev-parse $PREVIOUS_TAG^{commit})"
}

function setCurrentTag {
    local tag=""

    if [ -n "$1" ]; then
        tag="$1"
    fi

    if [ -n "$2" ]; then
        tag="$2"
    fi

    if [ ! -n "$tag" ]; then
        echo "You must declare CIRCLE_TAG or CURRENT_TAG variable"
        exit 1
    fi

    CURRENT_TAG_SHA="$(git rev-parse $tag^{commit})"
    CURRENT_TAG="$tag"
}

echo "-> Setup environment variables"

cd "$REPOSITORY_PATH"

setCurrentTag "$CURRENT_TAG" "$CIRCLE_TAG"
setPreviousTag "$PREVIOUS_TAG" "$CURRENT_TAG"

export CHYLE_GIT_REPOSITORY_PATH=$REPOSITORY_PATH
export CHYLE_GIT_REFERENCE_FROM=$PREVIOUS_TAG_SHA
export CHYLE_GIT_REFERENCE_TO=$CURRENT_TAG_SHA

# Setup matchers

## Pick only merge commits
export CHYLE_MATCHERS_TYPE=merge

# Setup decorators

## Setup github issue decorator

### Github issue id extractor
export CHYLE_EXTRACTORS_GITHUBISSUEID_ORIGKEY=message
export CHYLE_EXTRACTORS_GITHUBISSUEID_DESTKEY=githubIssueId
export CHYLE_EXTRACTORS_GITHUBISSUEID_REG="\#(\d+)"

### Github credentials
export CHYLE_DECORATORS_GITHUBISSUE_CREDENTIALS_OAUTHTOKEN=$GITHUB_TOKEN
export CHYLE_DECORATORS_GITHUBISSUE_CREDENTIALS_OWNER=antham

### Git path
export CHYLE_DECORATORS_GITHUBISSUE_REPOSITORY_NAME=$CIRCLE_PROJECT_REPONAME

### Extract title field
export CHYLE_DECORATORS_GITHUBISSUE_KEYS_TITLE_DESTKEY=title
export CHYLE_DECORATORS_GITHUBISSUE_KEYS_TITLE_FIELD=title

# Setup senders

## Setup github release

### Github credentials
export CHYLE_SENDERS_GITHUBRELEASE_CREDENTIALS_OAUTHTOKEN=$GITHUB_TOKEN
export CHYLE_SENDERS_GITHUBRELEASE_CREDENTIALS_OWNER=antham

### Github release config
export CHYLE_SENDERS_GITHUBRELEASE_RELEASE_TAGNAME=$CIRCLE_TAG
export CHYLE_SENDERS_GITHUBRELEASE_RELEASE_UPDATE=true
export CHYLE_SENDERS_GITHUBRELEASE_RELEASE_TEMPLATE='{{ range $key, $value := .Datas }}
{{ $value.id }} => **{{ $value.title | trim }}** *({{ $value.authorName }} - {{ $value.authorDate | date "2006-01-02 15:04:05" }})*
{{ end }}'

### Git path
export CHYLE_SENDERS_GITHUBRELEASE_REPOSITORY_NAME=$CIRCLE_PROJECT_REPONAME

# Generate changelog
echo "-> Generating changelog between $PREVIOUS_TAG ($PREVIOUS_TAG_SHA) and $CURRENT_TAG ($CURRENT_TAG_SHA)"
```

In CircleCI (v1) we add a deployment section to trigger a build when a tag is created (we need to declare credentials envs inside circle settings project) :

```yaml
machine:
  environment:
    REPOSITORY_PATH: $HOME/$CIRCLE_PROJECT_REPONAME

deployment:
  release:
    tag: /v[0-9]+(\.[0-9]+)*/
    owner: antham
    commands:
      - ./release.sh
```

Now we can create a release in github (we create a tag in the same time), it triggers a circle build and at the end, chyle generates a diff between two tags and populate the release, check [v1.0.0 release of test-git](https://github.com/antham/test-git/releases/tag/v1.0.0).

## Documentation

* [Default fields](#default-fields)
* [General config](#general-config)
* [Matchers](#matchers)
* [Extractors](#extractors)
* [Decorators](#decorators)
{'fraction_non_alphanumeric': 0.12935515982745271, 'fraction_numerical': 0.08688198208162815, 'mean_word_length': 4.871103896103896, 'pattern_counts': {'":': 92, '<': 31, '<?xml version=': 0, '>': 50, 'https://': 23, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2511803', 'n_tokens_mistral': 7807, 'n_tokens_neox': 6541, 'n_words': 1410}