<commit_before># ![Repo Linter](docs/images/P_RepoLinter01_logo_only.png) [![Build Status](https://travis-ci.org/todogroup/repolinter.svg?branch=master)](https://travis-ci.org/todogroup/repolinter)

Lint open source repositories for common issues.

## Usage

To run against a directory, add it to the command line `bin/repolinter.js /my/code/dir`.

To run against a git repository, use the `--git` option: `bin/repolinter.js --git https://my.git.code/awesome`.

## Examples

To quickly get started, checkout this repository and run repolinter against itself.

```
git clone https://github.com/todogroup/repolinter
bin/repolinter.js
✔ license-file-exists: found (LICENSE)
✔ readme-file-exists: found (README.md)
✔ contributing-file-exists: found (CONTRIBUTING)
✔ readme-references-license: File README.md contains license
✔ binaries-not-present: Excluded file type doesn't exist (**/*.exe,**/*.dll)
✔ license-detectable-by-licensee: Licensee identified the license for project: Apache License 2.0
✔ test-directory-exists: found (tests)
✔ source-license-headers-exist: exist
```

## Command line dependencies

Repo Linter will use https://github.com/benbalter/licensee and https://github.com/github/linguist when installed.

Licensee will lead to a test being done to see if the project's licensee is identified by Licensee.

Linguist allows per-language tests to be performed.

Run `bundle install` to get Lincensee and Linguist support.

## Default ruleset
The default ruleset (```rulesets/default.json```) enables the following rules:

All languages:
* [binaries-not-present](#binaries-not-present)
* [contributing-file-exist](#contributing-file-exists)
* [license-detectable-by-licensee](#license-detectable-by-licensee)
* [license-file-exists](#license-file-exists)
* [readme-file-exists](#readme-file-exists)
* [readme-references-license](#readme-references-license)
* [source-license-headers-exist]([#source-license-headers-exist)
* [test-directory-exists](#test-directory-exists)

## Configuring rules
Currently you need to create a new ruleset to add, remove, or configure rules. We'll be adding the ability to inherit from an existing ruleset to simplify this in the future.

### Overriding the ruleset globally or for a project
To override the default ruleset copy ```rulesets/default.json``` to ```repolint.json``` in the target directory, any ancenstor directory of the target directory, or your user directory.

### Disabling rules
To disable a rule change it's value to ```false```, for example:
```
{
  "rules": {
    "all": {
      "license-file-exists:file-existence": false
    }
  }
}
```

### Changing a rule's level
To change the level when a rule returns a failure change the first argument of the rule to ```error```, ```warning```, or ```info```, for example:
```
{
  "rules": {
    "all": {
      "license-detectable-by-licensee": ["info"]
    }
  }
}
```

### Configuring a rule's options
To configure a rule's options change the second argument of the rule to an object specifying the rule's options, see [rules](#rules) for details about each rule's options. For example:
```
{
  "rules": {
    "all": {
      "source-license-headers-exist:file-starts-with": ["warning", {"files": ["**/*.java"], "lineCount": 2, "patterns": ["Copyright", "All rights reserved", "Licensed under"]}]
    }
  }
}
```

### Language-specific rules
Rules can be configured to only run if the repository contains a specific language. Languages are detected using Linguist which must be in your path, see [command line dependencies](#command-line-dependencies) for details.



## Rules
### binaries-not-present
Fails if ```*.dll``` or ```*.exe``` files are in the target directory.

### contributing-file-exists
Fails if there isn't a file matching ```CONTRIB*``` in the root of the target directory.

### directory-existence
Fails if none of the directories specified in the ```directories``` option exist.

### file-contents
Fails if the content of any of the files specified in the ```files``` option doesn't match the regular expression specified in the ```content``` option. 

### file-existence
Fails if none of the files specified in the ```files``` option exist.

### file-starts-with
Produces a failure for each file matching the ```files``` option if the first ```lineCount``` lines don't match all of the regular expressions specified in the ```patterns``` option.

### file-type-exclusion
Fails if any files match the ```type``` option.

### license-detectable-by-licensee
Fails if Licensee doesn't detect the repository's license.

This rule requires ```licensee``` in the path, see (command line dependencies)[#command-line-dependencies] for details.

### license-file-exists
Fails if there isn't a file matching ```LICENSE*``` or ```COPYING*``` in the root of the target directory.

### readme-file-exists
Fails if there isn't a file matching ```README*``` in the root of the target directory.

### readme-references-license
Fails if the files matching ```README*``` doesn't match the regular expression ```license```.

### code-of-conduct-file-exists
Fails if there isn't a file matching ```CODEOFCONDUCT*```, ```CODE-OF-CONDUCT*``` or ```CODE_OF_CONDUCT*``` in the root of the target directory.

### source-license-headers-exist
Produces a failure for each file matching ```**/*.js,!node_modules/**``` option if the first 5 lines don't match all the patterns ```copyright```, ```all rights reserved```, and ```licensed under```.

### test-directory-exists
Fails if there isn't a directory matching ```test*``` or ```specs``` in the root of the target directory.

## License

This project is licensed under the [Apache 2.0](LICENSE) license.
<commit_msg>Update README to mention log-symbols installation<commit_after># ![Repo Linter](docs/images/P_RepoLinter01_logo_only.png) [![Build Status](https://travis-ci.org/todogroup/repolinter.svg?branch=master)](https://travis-ci.org/todogroup/repolinter)

Lint open source repositories for common issues.

## Usage

To run against a directory, add it to the command line `bin/repolinter.js /my/code/dir`.

To run against a git repository, use the `--git` option: `bin/repolinter.js --git https://my.git.code/awesome`.

## Examples

To quickly get started, checkout this repository and run repolinter against itself.

```
git clone https://github.com/todogroup/repolinter
bin/repolinter.js
✔ license-file-exists: found (LICENSE)
✔ readme-file-exists: found (README.md)
✔ contributing-file-exists: found (CONTRIBUTING)
✔ readme-references-license: File README.md contains license
✔ binaries-not-present: Excluded file type doesn't exist (**/*.exe,**/*.dll)
✔ license-detectable-by-licensee: Licensee identified the license for project: Apache License 2.0
✔ test-directory-exists: found (tests)
✔ source-license-headers-exist: exist
```

## Command line dependencies

The npm log-symbols package must be installed to run repolinter.
```
npm install log-symbols
```

Repo Linter will use https://github.com/benbalter/licensee and https://github.com/github/linguist when installed.

Licensee will lead to a test being done to see if the project's licensee is identified by Licensee.

Linguist allows per-language tests to be performed.

Run `bundle install` to get Lincensee and Linguist support.

## Default ruleset
The default ruleset (```rulesets/default.json```) enables the following rules:

All languages:
* [binaries-not-present](#binaries-not-present)
* [contributing-file-exist](#contributing-file-exists)
* [license-detectable-by-licensee](#license-detectable-by-licensee)
* [license-file-exists](#license-file-exists)
* [readme-file-exists](#readme-file-exists)
* [readme-references-license](#readme-references-license)
* [source-license-headers-exist]([#source-license-headers-exist)
* [test-directory-exists](#test-directory-exists)

## Configuring rules
Currently you need to create a new ruleset to add, remove, or configure rules. We'll be adding the ability to inherit from an existing ruleset to simplify this in the future.

### Overriding the ruleset globally or for a project
To override the default ruleset copy ```rulesets/default.json``` to ```repolint.json``` in the target directory, any ancenstor directory of the target directory, or your user directory.

### Disabling rules
To disable a rule change it's value to ```false```, for example:
```
{
  "rules": {
    "all": {
      "license-file-exists:file-existence": false
    }
  }
}
```

### Changing a rule's level
To change the level when a rule returns a failure change the first argument of the rule to ```error```, ```warning```, or ```info```, for example:
```
{
  "rules": {
    "all": {
      "license-detectable-by-licensee": ["info"]
    }
  }
}
```

### Configuring a rule's options
To configure a rule's options change the second argument of the rule to an object specifying the rule's options, see [rules](#rules) for details about each rule's options. For example:
```
{
  "rules": {
    "all": {
      "source-license-headers-exist:file-starts-with": ["warning", {"files": ["**/*.java"], "lineCount": 2, "patterns": ["Copyright", "All rights reserved", "Licensed under"]}]
    }
  }
}
```

### Language-specific rules
Rules can be configured to only run if the repository contains a specific language. Languages are detected using Linguist which must be in your path, see [command line dependencies](#command-line-dependencies) for details.



## Rules
### binaries-not-present
Fails if ```*.dll``` or ```*.exe``` files are in the target directory.

### contributing-file-exists
Fails if there isn't a file matching ```CONTRIB*``` in the root of the target directory.

### directory-existence
Fails if none of the directories specified in the ```directories``` option exist.

### file-contents
Fails if the content of any of the files specified in the ```files``` option doesn't match the regular expression specified in the ```content``` option. 

### file-existence
Fails if none of the files specified in the ```files``` option exist.

### file-starts-with
Produces a failure for each file matching the ```files``` option if the first ```lineCount``` lines don't match all of the regular expressions specified in the ```patterns``` option.

### file-type-exclusion
Fails if any files match the ```type``` option.

### license-detectable-by-licensee
Fails if Licensee doesn't detect the repository's license.

This rule requires ```licensee``` in the path, see (command line dependencies)[#command-line-dependencies] for details.

### license-file-exists
Fails if there isn't a file matching ```LICENSE*``` or ```COPYING*``` in the root of the target directory.

### readme-file-exists
Fails if there isn't a file matching ```README*``` in the root of the target directory.

### readme-references-license
Fails if the files matching ```README*``` doesn't match the regular expression ```license```.

### code-of-conduct-file-exists
Fails if there isn't a file matching ```CODEOFCONDUCT*```, ```CODE-OF-CONDUCT*``` or ```CODE_OF_CONDUCT*``` in the root of the target directory.

### source-license-headers-exist
Produces a failure for each file matching ```**/*.js,!node_modules/**``` option if the first 5 lines don't match all the patterns ```copyright```, ```all rights reserved```, and ```licensed under```.

### test-directory-exists
Fails if there isn't a directory matching ```test*``` or ```specs``` in the root of the target directory.

## License

This project is licensed under the [Apache 2.0](LICENSE) license.

{'fraction_non_alphanumeric': 0.1366091653598188, 'fraction_numerical': 0.0013939710751001916, 'mean_word_length': 4.803336703741152, 'pattern_counts': {'":': 24, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 12, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4275222', 'n_tokens_mistral': 3437, 'n_tokens_neox': 3352, 'n_words': 1260}