<commit_before># Table of content:
- [Project description](#project-description)
- [Results](#results)
- [Deployment steps](#deployment-steps)
    - [Setup](#setup)
    - [Deploy demo](#deploy-demo)
- [Code highlights](#code-highlights)

# Project description

This sample project shows off how to prepare and deploy to [Azure Web Apps](https://azure.microsoft.com/en-gb/services/app-service/web/) a simple Python web service with an image classifying model produced in [CNTK (Cognitive Toolkit)](https://github.com/Microsoft/CNTK) using [FasterRCNN](https://github.com/Microsoft/CNTK/tree/master/Examples/Image/Detection/FasterRCNN)

- [x] Web Service written in Python using Flask module
- [x] Python scripts that allow to evaluate images using CNTK and pretrained model
- [x] Custom deployment scripts for Azure Web Apps
- [x] Automatic setup of custom Python 3.5.x environment with all the required dependencies
- [ ] HTML UI for testing image classification
- [ ] Virtual Python environments for each application deployed to Azure Web Apps

# Results

[Website Demo](http://cntkpywebapptest1.azurewebsites.net/):  
![Demo](doc/iexplore_2017-09-26_23-09-42.jpg)

Sample request and response in Postman:
![Demo](doc/Postman_2017-09-26_22-50-06.jpg)

# Deployment steps

## Setup


1. **Download content of this repo**

    You can either clone this repo or just download it and unzip to some folder

2. **Setup Python environment**

    In order for scripts to work you should have a proper Python environment. If you don't already have it setup then you should follow one of the online tutorials. To setup Python environment and all the dependencies required by CNTK on my local Windows machine I used [this tutorial](https://docs.microsoft.com/en-us/cognitive-toolkit/setup-windows-binary-script) 

3. **Download CNTK model and class map file**

    Go to `/CNTKModels` folder in the location were you unzipped this repo and run `download_model.py`. It will automatically download the pretrained model and class map file required for our evaluation to run properly.

4. **Install Azure CLI tools**

    If you don't have it then you can easily do it by openning Windows Command Prompt and running this command:
    ```Batchfile
    pip install azure-cli
    ```

5.  **Get Azure subscription**

    If you don't own any Azure subscriptions you can always create a [new free trial](https://azure.microsoft.com/en-us/free/) with $200 credits to spend
    
## Deploy demo
1.  **Set variables**

    Open Command Prompt to the location where you unzipped the contents of this repository (for example: `cd C:\Poligon\WebService`) and type in as follows (but make sure to replace the `[]` with a proper value):

    ```
    set uname=[username]
    set pass=[password]
    set appn=[web_app_name]
    set rgname=[resource_group_name]
    ```

2.  **Login to Azure**

    In the same CMD type in:

    ```
    az login
    ```
    You should see something like this:

    ![AZ login](doc/cmd_2017-09-26_22-15-45.jpg)

    Now go to the https://aka.ms/devicelogin website and type in the code:

    ![Device login website](doc/iexplore_2017-09-26_22-17-20.jpg)

    You will then be asked to login with an email connected to your Azure subscription

    If everything goes ok you should see the verification message on the website and in console you should see a list of your Azure subscriptions


3.  **Setup deployment credentials**

    We're setting this up to later be able to remotely deploy code to our Azure Web App
    ```
    az webapp deployment user set --user-name %uname% --password %pass%
    ```

4.  **Create resource group**

    Resource groups help you to better manage your stuff in subscription and it's a basic method of deploying services to Azure. [Read more here](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-overview)

    ```
    az group create --location westeurope --name %rgname%
    ```

5.  **Create new [Azure App Service](https://azure.microsoft.com/en-us/services/app-service/) Plan and new [Azure Web App](https://azure.microsoft.com/en-us/services/app-service/web/)**

    ```
    az appservice plan create --name %appn% --resource-group %rgname% --sku S1
    az webapp create --name %appn% --resource-group %rgname% --plan %appn%
    ```

6.  **Configure Azure Web App and add Python extension**

    Azure Web Apps by default support only Python 2.7 and 3.4. Because I used Python 3.5 I had to use special [extension](https://azure.microsoft.com/en-us/blog/azure-web-sites-extensions/) to setup the environment

    First you need to change some [Application Settings](https://docs.microsoft.com/en-us/azure/app-service/web-sites-configure) on your Web App (the pink ones):
    ![Web App preferences](doc/iexplore_2017-09-26_22-22-19.jpg)
    Changing `Platform` is required and changing `Always On` is optional but I recommend to use it so that our web service stays awake even if not used.

    After we properly save Application Settings we can now add Python 3.5.x extension. In order to this, just type in `extensions` into the search box
    ![Extensions](doc/iexplore_2017-09-26_22-23-19.jpg)

    And then simply add new extension
    ![Extensions](doc/iexplore_2017-09-26_22-23-59.jpg)

    It should take around a minute or two to properly install the extension
    ![Extensions](doc/iexplore_2017-09-26_22-25-04.jpg)


7.  **Setup deployment source for newly created Azure Web App**

    This code will not only setup the deployment source for your app but will also retrive the URL you will need in next steps
    ```
    az webapp deployment source config-local-git --name %appn% --resource-group %rgname% --query url --output tsv
    ```
    It should return something like this:
    ![Remote repo 1](doc/cmd_2017-09-26_22-18-52.jpg)
    

8. **Initialize git and add remote repository**

    Make sure to replace `[remote_repo_address]` with the URL returned in step number 7.
    ```
    git init
    git remote add azure [remote_repo_address]
    ```
    Command with URL should look like this:
    ![Remote repo 2](doc/cmd_2017-09-26_22-20-23.jpg)

9.  **Push application to Azure Web App remote repository** 
    
    Last step is to simply push our applications code to Azure Web App 
    ```
    git add -A
    git commit -m "init"
    git push azure master
    ```

    This will trigger our [custom deployment script](deploy.cmd), copy all the files, setup Python environment and install all the required dependencies from [requirements.txt](requirements.txt) file

10. **Test the application**

    If everything went smooth you should now have a running Python application and you should be able to test it. I used Postman to test HTTP requests and responses

    [Website Demo](http://cntkpywebapptest1.azurewebsites.net/):  
    ![Demo](doc/iexplore_2017-09-26_23-09-42.jpg)

    Sample request and response in Postman:
    ![Demo](doc/Postman_2017-09-26_22-50-06.jpg)

# Code highlights


- [config.py](config.py) - most variables for scripts are set in this file

    
    Variables used by web service to point out directories for temp images and CNTK models:
    ```Python
    [..]
    # directories for web service:
    __C.CNTK.TEMP_PATH = "./Temp"<commit_msg>added back to the top link<commit_after># Table of contents
- [Project description](#project-description)
- [Results](#results)
- [Deployment steps](#deployment-steps)
    - [Setup](#setup)
    - [Deploy demo](#deploy-demo)
- [Code highlights](#code-highlights)

# Project description
[[back to the top]](#table-of-contents)

This sample project shows off how to prepare and deploy to [Azure Web Apps](https://azure.microsoft.com/en-gb/services/app-service/web/) a simple Python web service with an image classifying model produced in [CNTK (Cognitive Toolkit)](https://github.com/Microsoft/CNTK) using [FasterRCNN](https://github.com/Microsoft/CNTK/tree/master/Examples/Image/Detection/FasterRCNN)

- [x] Web Service written in Python using Flask module
- [x] Python scripts that allow to evaluate images using CNTK and pretrained model
- [x] Custom deployment scripts for Azure Web Apps
- [x] Automatic setup of custom Python 3.5.x environment with all the required dependencies
- [ ] HTML UI for testing image classification
- [ ] Virtual Python environments for each application deployed to Azure Web Apps

# Results
[[back to the top]](#table-of-contents)

[Website Demo](http://cntkpywebapptest1.azurewebsites.net/):  
![Demo](doc/iexplore_2017-09-26_23-09-42.jpg)

Sample request and response in Postman:
![Demo](doc/Postman_2017-09-26_22-50-06.jpg)

# Deployment steps

## Setup
[[back to the top]](#table-of-contents)


1. **Download content of this repo**

    You can either clone this repo or just download it and unzip to some folder

2. **Setup Python environment**

    In order for scripts to work you should have a proper Python environment. If you don't already have it setup then you should follow one of the online tutorials. To setup Python environment and all the dependencies required by CNTK on my local Windows machine I used [this tutorial](https://docs.microsoft.com/en-us/cognitive-toolkit/setup-windows-binary-script) 

3. **Download CNTK model and class map file**

    Go to `/CNTKModels` folder in the location were you unzipped this repo and run `download_model.py`. It will automatically download the pretrained model and class map file required for our evaluation to run properly.

4. **Install Azure CLI tools**

    If you don't have it then you can easily do it by openning Windows Command Prompt and running this command:
    ```Batchfile
    pip install azure-cli
    ```

5.  **Get Azure subscription**

    If you don't own any Azure subscriptions you can always create a [new free trial](https://azure.microsoft.com/en-us/free/) with $200 credits to spend
    
## Deploy demo
[[back to the top]](#table-of-contents)
1.  **Set variables**

    Open Command Prompt to the location where you unzipped the contents of this repository (for example: `cd C:\Poligon\WebService`) and type in as follows (but make sure to replace the `[]` with a proper value):

    ```
    set uname=[username]
    set pass=[password]
    set appn=[web_app_name]
    set rgname=[resource_group_name]
    ```

2.  **Login to Azure**

    In the same CMD type in:

    ```
    az login
    ```
    You should see something like this:

    ![AZ login](doc/cmd_2017-09-26_22-15-45.jpg)

    Now go to the https://aka.ms/devicelogin website and type in the code:

    ![Device login website](doc/iexplore_2017-09-26_22-17-20.jpg)

    You will then be asked to login with an email connected to your Azure subscription

    If everything goes ok you should see the verification message on the website and in console you should see a list of your Azure subscriptions


3.  **Setup deployment credentials**

    We're setting this up to later be able to remotely deploy code to our Azure Web App
    ```
    az webapp deployment user set --user-name %uname% --password %pass%
    ```

4.  **Create resource group**

    Resource groups help you to better manage your stuff in subscription and it's a basic method of deploying services to Azure. [Read more here](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-overview)

    ```
    az group create --location westeurope --name %rgname%
    ```

5.  **Create new [Azure App Service](https://azure.microsoft.com/en-us/services/app-service/) Plan and new [Azure Web App](https://azure.microsoft.com/en-us/services/app-service/web/)**

    ```
    az appservice plan create --name %appn% --resource-group %rgname% --sku S1
    az webapp create --name %appn% --resource-group %rgname% --plan %appn%
    ```

6.  **Configure Azure Web App and add Python extension**

    Azure Web Apps by default support only Python 2.7 and 3.4. Because I used Python 3.5 I had to use special [extension](https://azure.microsoft.com/en-us/blog/azure-web-sites-extensions/) to setup the environment

    First you need to change some [Application Settings](https://docs.microsoft.com/en-us/azure/app-service/web-sites-configure) on your Web App (the pink ones):
    ![Web App preferences](doc/iexplore_2017-09-26_22-22-19.jpg)
    Changing `Platform` is required and changing `Always On` is optional but I recommend to use it so that our web service stays awake even if not used.

    After we properly save Application Settings we can now add Python 3.5.x extension. In order to this, just type in `extensions` into the search box
    ![Extensions](doc/iexplore_2017-09-26_22-23-19.jpg)

    And then simply add new extension
    ![Extensions](doc/iexplore_2017-09-26_22-23-59.jpg)

    It should take around a minute or two to properly install the extension
    ![Extensions](doc/iexplore_2017-09-26_22-25-04.jpg)


7.  **Setup deployment source for newly created Azure Web App**

    This code will not only setup the deployment source for your app but will also retrive the URL you will need in next steps
    ```
    az webapp deployment source config-local-git --name %appn% --resource-group %rgname% --query url --output tsv
    ```
    It should return something like this:
    ![Remote repo 1](doc/cmd_2017-09-26_22-18-52.jpg)
    

8. **Initialize git and add remote repository**

    Make sure to replace `[remote_repo_address]` with the URL returned in step number 7.
    ```
    git init
    git remote add azure [remote_repo_address]
    ```
    Command with URL should look like this:
    ![Remote repo 2](doc/cmd_2017-09-26_22-20-23.jpg)

9.  **Push application to Azure Web App remote repository** 
    
    Last step is to simply push our applications code to Azure Web App 
    ```
    git add -A
    git commit -m "init"
    git push azure master
    ```

    This will trigger our [custom deployment script](deploy.cmd), copy all the files, setup Python environment and install all the required dependencies from [requirements.txt](requirements.txt) file

10. **Test the application**

    If everything went smooth you should now have a running Python application and you should be able to test it. I used Postman to test HTTP requests and responses

    [Website Demo](http://cntkpywebapptest1.azurewebsites.net/):  
    ![Demo](doc/iexplore_2017-09-26_23-09-42.jpg)

    Sample request and response in Postman:
    ![Demo](doc/Postman_2017-09-26_22-50-06.jpg)

# Code highlights
[[back to the top]](#table-of-contents)


- [config.py](config.py) - most variables for scripts are set in this file

    
    Variables used by web service to point out directories for temp images and CNTK models:
    ```Python
    [..]
    # directories for web service:
    __C.CNTK.TEMP_PATH = "./Temp"
{'fraction_non_alphanumeric': 0.09702150756496371, 'fraction_numerical': 0.027545966483479206, 'mean_word_length': 3.540973505853358, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 22, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '846459', 'n_tokens_mistral': 4590, 'n_tokens_neox': 4148, 'n_words': 1740}