<commit_before>#### rd_filters.py

This script provides a simple means of applying the 
functional group filters from the ChEMBL database, as well as a number of 
property  filters from the RDKit, to a set 
of compounds.  As of ChEMBL 23,  the database table **structural_alerts**
contains 8 sets of alerts.  ChEMBL doesn't apper to have much in the way of 
documentation on the different alert sets. 

|Rule Set |Number of Alerts|
|---|---:|
|BMS|	180|
|Dundee|	105|
|Glaxo|	55|
|Inpharmatica|	91|
|LINT|	57|
|MLSMR|	116|
|[PAINS](https://pubs.acs.org/doi/abs/10.1021/jm901137j)|	479|
|SureChEMBL|	166|

The SMARTS patterns in a number of these alerts were not compatible with the RDKit
so I edited them.  A complete list of the changes I made is in the file **Notes.txt**. 

#### Installation

This script has a few requirements
* At least Python 3.6
* The RDKit, you can find installation instructions [here](https://www.rdkit.org/docs/Install.html).  I'd recommend the conda route. 
* The pandas and docopt libraries
```python
pip install pandas docopt
```
The script needs 3 files to operate. 
* rd_filters.py - the main Python script. 
* alert_collection.csv - the set of structural alerts
* rules.json - the configuration file

The script uses the following logic to find alert_collection.csv and 
rules.json. 
1.  Use locations specified by the "--alert" (for alerts.csv) and 
"--rules" (for rules.json) command line arguments.
2.  Look in the current directory.
3.  Look in the directory pointed to by the FILTER_RULES_DATA environment variable.


I'll provide some examples below to illustrate.  


That's it, at this point you should be good to go. 

#### Configuration files

The file **alert_collection.csv** contains alerts.  You shouldn't have to mess with this unless you
want to add your own structural alerts. I think the format is pretty obvious. 

The file **rules.json** controls which filters and alerts are used.  You can use the command 
below to generate a **rules.json** with the default settings. 
```
rd_filters.py template --out rules.json
```

The **rules.json** file looks like this. The values for the properties are the maximum and minimum 
allowed (inclusive).  To set which structural alerts are used, set **true** and **false**. You can 
use multiple alert sets. 
Just edit the file with your [favorite text editor](https://www.gnu.org/software/emacs/).

```javascript
{
    "HBA": [
        0,
        10
    ],
    "HBD": [
        0,
        5
    ],
    "LogP": [
        -5,
        5
    ],
    "MW": [
        0,
        500
    ],
    "Rule_BMS": false,
    "Rule_Dundee": false,
    "Rule_Glaxo": false,
    "Rule_Inpharmatica": true,
    "Rule_LINT": false,
    "Rule_MLSMR": false,
    "Rule_PAINS": false,
    "Rule_SureChEMBL": false,
    "TPSA": [
        0,
        200
    ]
}
``` 

#### Examples

First off, you're going to want to copy **alert_collection.csv** and 
**rules.json** to a directory and set the FILTER_RULES_DATA environment
variable to point to that directory.  If your using a bash-ish shell
and the files are in /home/elvis/data that would be: 
```
export FILTER_RULES_DATA=/home/elvis/data

```

If you type
```<commit_msg>Fixing typos<commit_after>#### rd_filters.py

This script provides a simple means of applying the 
functional group filters from the ChEMBL database, as well as a number of 
property  filters from the RDKit, to a set 
of compounds.  As of ChEMBL 23,  the database table **structural_alerts**
contains 8 sets of alerts.  ChEMBL doesn't apper to have much in the way of 
documentation on the different alert sets. 

|Rule Set |Number of Alerts|
|---|---:|
|BMS|	180|
|Dundee|	105|
|Glaxo|	55|
|Inpharmatica|	91|
|LINT|	57|
|MLSMR|	116|
|[PAINS](https://pubs.acs.org/doi/abs/10.1021/jm901137j)|	479|
|SureChEMBL|	166|

The SMARTS patterns in a number of these alerts were not compatible with the RDKit
so I edited them.  A complete list of the changes I made is in the file **Notes.txt**. 

#### Installation

This script has a few requirements
* At least Python 3.6
* The RDKit, you can find installation instructions [here](https://www.rdkit.org/docs/Install.html).  I'd recommend the conda route. 
* The pandas and docopt libraries
```python
pip install pandas docopt
```
The script needs 3 files to operate. 
* rd_filters.py - the main Python script. 
* alert_collection.csv - the set of structural alerts
* rules.json - the configuration file

The script uses the following logic to find alert_collection.csv and 
rules.json. 
1.  Use locations specified by the "--alert" (for alerts.csv) and 
"--rules" (for rules.json) command line arguments.
2.  Look in the current directory.
3.  Look in the directory pointed to by the FILTER_RULES_DATA environment variable.


I'll provide some examples below to illustrate.  


That's it, at this point you should be good to go. 

#### Configuration files

The file **alert_collection.csv** contains alerts.  You shouldn't have to mess with this unless you
want to add your own structural alerts. I think the format is pretty obvious. 

The file **rules.json** controls which filters and alerts are used.  You can use the command 
below to generate a **rules.json** with the default settings. 
```
rd_filters.py template --out rules.json
```

The **rules.json** file looks like this. The values for the properties are the maximum and minimum 
allowed (inclusive).  To set which structural alerts are used, set **true** and **false**. You can 
use multiple alert sets. 
Just edit the file with your [favorite text editor](https://www.gnu.org/software/emacs/).

```javascript
{
    "HBA": [
        0,
        10
    ],
    "HBD": [
        0,
        5
    ],
    "LogP": [
        -5,
        5
    ],
    "MW": [
        0,
        500
    ],
    "Rule_BMS": false,
    "Rule_Dundee": false,
    "Rule_Glaxo": false,
    "Rule_Inpharmatica": true,
    "Rule_LINT": false,
    "Rule_MLSMR": false,
    "Rule_PAINS": false,
    "Rule_SureChEMBL": false,
    "TPSA": [
        0,
        200
    ]
}
``` 

#### Examples

First off, you're going to want to copy **alert_collection.csv** and 
**rules.json** to a directory and set the FILTER_RULES_DATA environment
variable to point to that directory.  If you are using a bash-ish shell
and the files are in /home/elvis/data that would be: 
```
export FILTER_RULES_DATA=/home/elvis/data

```

If you type
```
{'fraction_non_alphanumeric': 0.10140845070422536, 'fraction_numerical': 0.01784037558685446, 'mean_word_length': 3.2692050768203074, 'pattern_counts': {'":': 26, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 6, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2782397', 'n_tokens_mistral': 2163, 'n_tokens_neox': 1978, 'n_words': 832}