<commit_before># Quick Start

Before you can use **EasyML Studio**, you must configure the environment which include your ***development environment*** and ***runtime server environment*** which contains *Preparation, Run Docker containers and Start services* three steps to run it successfully.

## Development Environment
### Step 1: Import project to IDEA or Eclipse 

- Get code from Git repository <https://github.com/ICT-BDA/EasyML>
- Import the code to your IDE via maven project 

<div align=center>
<img src="./img/import_eml.png" height ="500px" margin-top = "10px" alt="import project to IDEA"/>
</div>

- Only support **java version 1.7**

<div align=center>
<img src="./img/import_jdk.png" height ="400px"  alt="configure JDK path"/>
</div>

  
### Step 2: Configure GWT Lib Path

* Make sure your IDE have install ***Maven***
* Use Maven to down load all related dependency package
<div align=center>
<img src="./img/import_maven.png" height ="400px" width = "550" alt="configure maven"/>
</div>

* Make sure you can see the whole **External Libraries** on the *Project* list
<div align=center>
<img src="./img/import_maven_suc.png"  alt="External libraries"/>
</div>

* Install GWT plugin in your IDE (IDEA user can skip this step)
### Step 3: Create and run our EML web application
  After you have get all dependencies, you can start building your *EasyML Studio* web app for the following steps:

* Edit run configurations

<div align=center>
<img src="./img/import_config.png" alt="Edit configuration"/>
</div>

* Add GWT web app via `+` at upper left corner, if you do not see `GWT Configuration` in the list, you might have something wrong in the step of *Configure GWT Lib Path*. Go back to the last step, resolve the problem.

<div align=center>
<img src="./img/import_rundetail.png" width = "90%" alt="Edit Easy ML"/>
</div>

* `Use Super Dev Mode` can allow you debugging your web app at the browser side. It is make debugging more effient, for which remember choosing it. 
* When you have finished all the steps above, you can click the green run button to make and debug the **EasyML**. After a while, you can browse EasyML in your *Chrome* to accesss it.
<div align=center>
<img src="./img/home_page.png" width = "90%" alt="Easy ML home page"/>
</div>

## Preparation for virtual server cluster
Our server cluster is based on ***Docker***, thus you can build run time environment on your own computer. It is convenient for you to develop project without any remote connections. Furthermore, you can also contribute to the server environments. The docker version server cluster is not stable and efficient, for which we can do a series of things on it. However, you first step to access it is installing Docker.
### Step 1: Install Docker 
* Just follow the [official guide](https://www.docker.com/) to install Docker. If you install the `docker in windows`, at least `Minimum Memory Requirement (RAM) = 8GB` for standalone computer. Hard drive must contain `enough space (10GB)` in which you install Docker. Otherwise it will be very very slow.
* Make sure your docker service runs correctly via `Docker info` and `Docker version`
* No matter which system your computer is, stop the **Firewall** of your system
* If you are using *centos 7*, you also should stop the **selinux**, in order to avoiding run Docker container error. 
* Run ```sudo docker run hello-world``` to see if we have install docker successfully.


### Step 2: Pull mysql server images from docker hub
* Pull our mysql server images from our [docker hub](https://hub.docker.com/u/nkxujun/):


    ```docker pull nkxujun/mysql_eml```
### Step 3: Pull ubuntu_eml images from docker hub
* Our Eml server images is based on ubuntu, so pull it first:   


    ```docker pull nkxujun/ubuntu_eml```
* You can use `docker images` to see if you have pulled these two images successfully:

	<img src="./img/origin_images.png" width = "90%" alt="eml_images"/>
 
### Step 4: Download install dependent package
Every single server in our cluster is created by one *docker image*, and this *image* can be built via a **Dockerfile** which has defined by us and includes all utilities we need such as hadoop. Thus we need to download the **Dockerfile** and all dependent files and configuration files from our [google drive disk](https://drive.google.com/open?id=0B5Lj6qkCMBbFWW5uYlJwb2drb1k) or [Baidu Cloud](https://pan.baidu.com/s/1c2jGcJe).


### Step 5: Build Eml server images  
* Enter the path of the files which you have download last step
* Use build.sh to build our image, this process will last for a few minute  
     ```sh build.sh``` 
* You can use `docker images` to see if you have built successfully:
<commit_msg>Update baidu cloud url to fix mysql container restart script.<commit_after># Quick Start

Before you can use **EasyML Studio**, you must configure the environment which include your ***development environment*** and ***runtime server environment*** which contains *Preparation, Run Docker containers and Start services* three steps to run it successfully.

## Development Environment
### Step 1: Import project to IDEA or Eclipse 

- Get code from Git repository <https://github.com/ICT-BDA/EasyML>
- Import the code to your IDE via maven project 

<div align=center>
<img src="./img/import_eml.png" height ="500px" margin-top = "10px" alt="import project to IDEA"/>
</div>

- Only support **java version 1.7**

<div align=center>
<img src="./img/import_jdk.png" height ="400px"  alt="configure JDK path"/>
</div>

  
### Step 2: Configure GWT Lib Path

* Make sure your IDE have install ***Maven***
* Use Maven to down load all related dependency package
<div align=center>
<img src="./img/import_maven.png" height ="400px" width = "550" alt="configure maven"/>
</div>

* Make sure you can see the whole **External Libraries** on the *Project* list
<div align=center>
<img src="./img/import_maven_suc.png"  alt="External libraries"/>
</div>

* Install GWT plugin in your IDE (IDEA user can skip this step)
### Step 3: Create and run our EML web application
  After you have get all dependencies, you can start building your *EasyML Studio* web app for the following steps:

* Edit run configurations

<div align=center>
<img src="./img/import_config.png" alt="Edit configuration"/>
</div>

* Add GWT web app via `+` at upper left corner, if you do not see `GWT Configuration` in the list, you might have something wrong in the step of *Configure GWT Lib Path*. Go back to the last step, resolve the problem.

<div align=center>
<img src="./img/import_rundetail.png" width = "90%" alt="Edit Easy ML"/>
</div>

* `Use Super Dev Mode` can allow you debugging your web app at the browser side. It is make debugging more effient, for which remember choosing it. 
* When you have finished all the steps above, you can click the green run button to make and debug the **EasyML**. After a while, you can browse EasyML in your *Chrome* to accesss it.
<div align=center>
<img src="./img/home_page.png" width = "90%" alt="Easy ML home page"/>
</div>

## Preparation for virtual server cluster
Our server cluster is based on ***Docker***, thus you can build run time environment on your own computer. It is convenient for you to develop project without any remote connections. Furthermore, you can also contribute to the server environments. The docker version server cluster is not stable and efficient, for which we can do a series of things on it. However, you first step to access it is installing Docker.
### Step 1: Install Docker 
* Just follow the [official guide](https://www.docker.com/) to install Docker. If you install the `docker in windows`, at least `Minimum Memory Requirement (RAM) = 8GB` for standalone computer. Hard drive must contain `enough space (10GB)` in which you install Docker. Otherwise it will be very very slow.
* Make sure your docker service runs correctly via `Docker info` and `Docker version`
* No matter which system your computer is, stop the **Firewall** of your system
* If you are using *centos 7*, you also should stop the **selinux**, in order to avoiding run Docker container error. 
* Run ```sudo docker run hello-world``` to see if we have install docker successfully.


### Step 2: Pull mysql server images from docker hub
* Pull our mysql server images from our [docker hub](https://hub.docker.com/u/nkxujun/):


    ```docker pull nkxujun/mysql_eml```
### Step 3: Pull ubuntu_eml images from docker hub
* Our Eml server images is based on ubuntu, so pull it first:   


    ```docker pull nkxujun/ubuntu_eml```
* You can use `docker images` to see if you have pulled these two images successfully:

	<img src="./img/origin_images.png" width = "90%" alt="eml_images"/>
 
### Step 4: Download install dependent package
Every single server in our cluster is created by one *docker image*, and this *image* can be built via a **Dockerfile** which has defined by us and includes all utilities we need such as hadoop. Thus we need to download the **Dockerfile** and all dependent files and configuration files from our [google drive disk](https://drive.google.com/open?id=0B5Lj6qkCMBbFWW5uYlJwb2drb1k) or [Baidu Cloud](https://pan.baidu.com/s/1miFXzVq).


### Step 5: Build Eml server images  
* Enter the path of the files which you have download last step
* Use build.sh to build our image, this process will last for a few minute  
     ```sh build.sh``` 
* You can use `docker images` to see if you have built successfully:

{'fraction_non_alphanumeric': 0.0982689465906692, 'fraction_numerical': 0.008760819083808317, 'mean_word_length': 4.368271954674221, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 49, 'https://': 10, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '598275', 'n_tokens_mistral': 2805, 'n_tokens_neox': 2602, 'n_words': 1351}