<commit_before>## DPR-API

[![Build Status](https://travis-ci.org/frictionlessdata/dpr-api.svg?branch=master)](https://travis-ci.org/frictionlessdata/dpr-api)
[![Coverage Status](https://coveralls.io/repos/github/frictionlessdata/dpr-api/badge.svg?branch=master)](https://coveralls.io/github/frictionlessdata/dpr-api?branch=master)


Requirement python 2.7

```
$ virtualenv env
$ source env/bin/activate
For test use 
$ pip install -r requirements.txt
$ pip install -r requirements.test.txt
else
$ pip install -r requirements.txt
```

## Environment Setting:
Plz put .env file in root directory and add environment variables to that
```
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_REGION=
GITHUB_CLIENT_ID=
GITHUB_CLIENT_SECRET=
S3_BUCKET_NAME=
FLASKS3_BUCKET_NAME=
SQLALCHEMY_DATABASE_URI=
```
Rename the env.template file to .env file.


### Set up postgres database

```
$ psql -U postgres -c "create user dpr_user password '<PASSWORD>' createdb;"
$ psql -U postgres -c "create database dpr_db owner=dpr_user;"

# create tables
python manager.py db upgrade

# you may need to install psysopg2 manually if comand throws errors
$ sudo apt-get install libpq-dev python-dev
$ pip install psycopg2
```

### DPR API project. 
This is flask based project. We can also deploy this in AWS lambda using zappa.

#### Local Deployment Process: 
For local testing we can start the project using:
    
```
$ python dpr.py
```

#### Lambda Deployment Process:
The lambda configuration is in zappa_settings.json
There are different environment to deploy.

    # Right now only dev env config is there.
    $ zappa deploy stage [This is for first time deployment]
    
    For further deployment:
    $  zappa update stage

##### Zappa Configuration:
```
s3_bucket: Which bucket the zip will be deployed
aws_region: aws region
environment_variables.FLASK_CONFIGURATION what config class the app will take from 
   app.config.py 
```
    
## Api Doc:
All api documentation is maintained by [flasgger](https://github.com/rochacbruno/flasgger)

The swagger UI Url path is {host}/apidocs/index.html
    
## Testing:
Before running tests plz run:
```
$ export FLASK_CONFIGURATION=test
```
. So that tests are not dependent on any env variable. By default it is dependent on
local postgresql instance.

All tests are in tests directory. We use pytest for testing

To run all tests please run this in the base directory:

```
pytest tests
```

* Some times pytest got cached by bash so tests may fail as it points to /usr/local/pytest
 not the virtual env pytest
 then plz run ```./env/bin/pytest tests```

## Submodules:

This project has a sub module dpr-js
To run locally
```
$ cd dpr-js
$ npm install
$ npm run build
```
This will create two directories:
- /app/static/scripts
- /app/templates

When run ```python dpr.py``` it will take these html and scripts.<commit_msg>[xs] update README.md about continuous deployment process<commit_after>## DPR-API

[![Build Status](https://travis-ci.org/frictionlessdata/dpr-api.svg?branch=master)](https://travis-ci.org/frictionlessdata/dpr-api)
[![Coverage Status](https://coveralls.io/repos/github/frictionlessdata/dpr-api/badge.svg?branch=master)](https://coveralls.io/github/frictionlessdata/dpr-api?branch=master)


Requirement python 2.7

```
$ virtualenv env
$ source env/bin/activate
For test use 
$ pip install -r requirements.txt
$ pip install -r requirements.test.txt
else
$ pip install -r requirements.txt
```

## Environment Setting:
Plz put .env file in root directory and add environment variables to that
```
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_REGION=
GITHUB_CLIENT_ID=
GITHUB_CLIENT_SECRET=
S3_BUCKET_NAME=
FLASKS3_BUCKET_NAME=
SQLALCHEMY_DATABASE_URI=
```
Rename the env.template file to .env file.


### Set up postgres database

```
$ psql -U postgres -c "create user dpr_user password '<PASSWORD>' createdb;"
$ psql -U postgres -c "create database dpr_db owner=dpr_user;"

# create tables
python manager.py db upgrade

# you may need to install psysopg2 manually if comand throws errors
$ sudo apt-get install libpq-dev python-dev
$ pip install psycopg2
```

### DPR API project. 
This is flask based project. We can also deploy this in AWS lambda using zappa.

#### Local Deployment Process: 
For local testing we can start the project using:
    
```
$ python dpr.py
```

#### Lambda Deployment Process:
The lambda configuration is in zappa_settings.json
There are different environment to deploy.

    # Right now only dev env config is there.
    $ zappa deploy stage [This is for first time deployment]
    
    For further deployment:
    $  zappa update stage

##### Zappa Configuration:
```
s3_bucket: Which bucket the zip will be deployed
aws_region: aws region
environment_variables.FLASK_CONFIGURATION what config class the app will take from 
   app.config.py 
```
    
## Api Doc:
All api documentation is maintained by [flasgger](https://github.com/rochacbruno/flasgger)

The swagger UI Url path is {host}/apidocs/index.html
    
## Testing:
Before running tests plz run:
```
$ export FLASK_CONFIGURATION=test
```
. So that tests are not dependent on any env variable. By default it is dependent on
local postgresql instance.

All tests are in tests directory. We use pytest for testing

To run all tests please run this in the base directory:

```
pytest tests
```

* Some times pytest got cached by bash so tests may fail as it points to /usr/local/pytest
 not the virtual env pytest
 then plz run ```./env/bin/pytest tests```

## Submodules:

This project has a sub module dpr-js
To run locally
```
$ cd dpr-js
$ npm install
$ npm run build
```
This will create two directories:
- /app/static/scripts
- /app/templates

When run ```python dpr.py``` it will take these html and scripts.

## DPR continuous deployment process

- git pull origin master
- git checkout deploy
  - git checkout -b deploy [if not exists loaclly]
- git merge master
- git push origin deploy

That's it looks at https://travis-ci.org/frictionlessdata/dpr-api
{'fraction_non_alphanumeric': 0.09620505992010653, 'fraction_numerical': 0.002330226364846871, 'mean_word_length': 4.257217847769029, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 11, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '548413', 'n_tokens_mistral': 1944, 'n_tokens_neox': 1892, 'n_words': 692}