<commit_before>In this tutorial, we will install all of the needed requirements for a LEMP stack (Linux, Nginx, MySQL, and PHP). This will all take place on a virtual machine created for us by IITS.  They can be contacted at [Prodesk](mailto:<EMAIL>). In this tutorial we will: 

#### Part I
1. Install Nginx
2. Install mySQL
3. Install PHP
4. Install and configure uWsgi

#### Part II
5. Install virtualenv
6. Create an virtual environment for the project
7. Install requirements in the virtualenv
8. Step X: Install Git and Clone Your Project

## Step 1: Connect to the Virtual Machine
You can connect to the virtual machine using [SSH](https://en.wikipedia.org/wiki/Secure_Shell).

    $ ssh yourusername@virtualmachinename

If you do not have a username on the server, one can be created by the DS librarians using `$ adduser <username>` and `$ sudo usermod -a -G sudo <username>` Use `control + D` to disconnect from SSH and return to your local machine. 

### Common Unix commands
- `ls` list files in current directory
- `pwd` print the name of the current directory (stands for "print working directory")
- `locate` show the location of a file or directory
- `cd <directory-name>` change directory (you can supply a relative or absolute path)
- `mv <file-path> <destination>` move a file
- `vi <file-path>` open a file in the vi text editor (within vi press the `i` key to edit text, and `esc` then `:w` to save changes.  Type `esc` then `:q` to quit)
- `scp` copy from one machine/server to another (ex. `$ scp /source/path/image.jpg user@server:/destination/path)` Note: when moving files onto a server, you'll typically need to copy to /tmp/ first then to the desired destination.
- `sudo <command>` run a command that requires security privileges.

## Step 2: Install virtualenv

Since this is our first interaction with the apt packaging system in this session, we should update our local package index before we begin so that we are using the most up-to-date information:

    $ sudo apt-get update

Enter the following commands to install virtualenv:

    $ sudo apt-get install python-virtualenv
    $ cd
    $ sudo mkdir /usr/local/lib/python-virtualenv
    $ cd /usr/local/lib/python-virtualenv

If you are using Python 3, make sure to specify the `--python=python3.X` flag (where X is the proper version number) in the following command. Otherwise, the virtual environment will be created with the default version of Python, which is typically Python 2.

    $ sudo virtualenv <project name>
    $ cd <project name>

Now you can activate your virtual environment with this command: `$ source bin/activate`

Note how the command line has changed to show that you are running in a virtual environment: `(bridge) ajanco@bridge:/usr/local/lib/python-virtualenv/bridge$`

## Step 3: Install Git and Clone Your Project
[Git](https://en.wikipedia.org/wiki/Git) is version-control software that is used to keep track of changes on collaborative projects. It allows you to save successive revisions of your work and then view them at a later time (for example, if you made a horrible mistake and just want to return to a working version), and it has strong support for multiple developers working on a single project. You can install Git like this:

    $ sudo apt-get install git

All of our projects live on [Github](https://github.com/HCDigitalScholarship). Find the project that you're working on and click on the title.  This will open the repository (usually abbreviated as "repo"). Check the name of the branch and be sure that you're in the branch that you need.  Next, click on the "Clone or download" button.  A URL will appear.  Select and copy that address.

Now, switch to a terminal window open on the server and type in the commands below. You can also `cd` into `/srv/www/`, `/srv/html/`, or `/var/www/` if you'd like. Whatever you choose, Git is going to create a folder there with the name of the repo and its project files.

    $ cd /srv/
    $ sudo git clone https://github.com/HCDigitalScholarship/project_name.git

The URL above comes from the "Clone or download" button on Github. You will then be prompted to enter your username and password.

### Common Git commands
- `$ git status` view the status of the current git repo
- `$ git add <file-path>` add the file or directory to the git working index (e.g., after you've edited some files)
- `$ git commit -m "<commit-message>"` commit the changes in the working index (that you added with the `git add` command). Think of this as a "save progress" command.
- `$ git log` view metadata about previous commits
- `$ git checkout -b <branch>` create a new branch and switch to it
- `$ git checkout <branch>` switch to an already-existing branch
- `$ git checkout <commit-hash>` look at another commit (a previous "save"). The commit hash can be found with the `git log` command.
- `$ git freeze`
- `$ git stash`

## Step X: Install Project Requirements

    $ sudo pip install -r requirements.txt

Test the app to make sure all requirements are installed by running 

    $ python manage.py runserver

If you don't get any errors, then the app is properly configured and you can move on to the next step!

## Step X: Install uWsgi
What is an application server and why are we using it? 
Sockets: uWSGI can create multiple types of sockets, and we will use two of them here--HTTP and Unix. We'll start with an HTTP socket so that we can reqeust HTML from the app without having to also configure a web server. Once we are sure that uWSGI is properly configured, we will switch to a Unix file socket for performance and security reasons.

    $ pip install uwsgi

Create a new file /etc/uwsgi/apps-available/project_name.ini and paste in the following contents:
`
[uwsgi]

# Environmental settings: 
uid = www-data
gid = www-data
#socket = /run/uwsgi/app/duchemin/duchemin.socket
`

First we are setting the user and group that can access the file socket, and specifying the location of the file socket. We specify the /run/uwsgi/app folder because we know the uWSGI service has permission to access it. The file socket location line should be commented out initially for reasons that will be explained in a moment.

`# For testing: curl -L http://localhost:5000
socket = localhost:5000
protocol = http
`
These lines are only there for testing purposes, and they create the HTTP socket. They will eventually be commented out. We can test the socket by requesting HTML using the "curl" command from the command line. 
`
# Application settings
plugin = python
chdir = /srv/DuChemin
#virtualenv = /usr/local/lib/python-virtualenv/ticha-django
module=duchemin.wsgi:application
`
These lines contain specifics for the application server hooking into the app.
`plugin` specifies the type of application. In most cases, e.g. a Django app, this will be `python`, but it could also be ruby, python3, mongoDB, PHP, and so on. See the uWSGI docs for all of the available plugins.

`chdir` specifies the project root directory. For a Flask or Django project, this is the folder that contains `manage.py`

`virtualenv` specifies the location of the virtual environment.

`module` defines the application for uWSGI. The name of the application should be identical to to the name specified in the `WSGI_APPLICATION` line of `settings.py`

`# Performance tuning
processes = 4
threads = 2`

The python plugin by default does not have threads enabled, so we enable threading support here. We can adjust these lines based on performance once the project is deployed.

Save and quit.

Create a symbolic link in `/apps-enabled` to the `.ini` file in `/apps-available` to activate the configuration:
    `$ ln -s /etc/uwsgi/apps-enabled/config.ini /etc/uwsgi/apps-available/config.ini`
    
Restart the service
    $ sudo service uwsgi restart
    
Now let's test the socket to see if we can get an HTML response:
    $ curl -L http://localhost:5000

The `-L` option follows redirects, which is necessary if your project redirects from the index page.

If you get an HTML response, you've successfully configured uWSGI! If you get an error or no response, check the logs at `/var/log/uwsgi/app/nameofapp.log`

Once you've successfully configured uWSGI, comment out the two lines related to the HTTP socket and uncomment the line for the Unix file socket. Save, quit, and restart uWSGI.
<commit_msg>use tab for code blocks, else # in code is BOLD<commit_after>In this tutorial, we will install all of the needed requirements for a LEMP stack (Linux, Nginx, MySQL, and PHP). This will all take place on a virtual machine created for us by IITS.  They can be contacted at [Prodesk](mailto:<EMAIL>). In this tutorial we will: 

#### Part I
1. Install Nginx
2. Install mySQL
3. Install PHP
4. Install and configure uWsgi

#### Part II
5. Install virtualenv
6. Create an virtual environment for the project
7. Install requirements in the virtualenv
8. Step X: Install Git and Clone Your Project

## Step 1: Connect to the Virtual Machine
You can connect to the virtual machine using [SSH](https://en.wikipedia.org/wiki/Secure_Shell).

    $ ssh yourusername@virtualmachinename

If you do not have a username on the server, one can be created by the DS librarians using `$ adduser <username>` and `$ sudo usermod -a -G sudo <username>` Use `control + D` to disconnect from SSH and return to your local machine. 

### Common Unix commands
- `ls` list files in current directory
- `pwd` print the name of the current directory (stands for "print working directory")
- `locate` show the location of a file or directory
- `cd <directory-name>` change directory (you can supply a relative or absolute path)
- `mv <file-path> <destination>` move a file
- `vi <file-path>` open a file in the vi text editor (within vi press the `i` key to edit text, and `esc` then `:w` to save changes.  Type `esc` then `:q` to quit)
- `scp` copy from one machine/server to another (ex. `$ scp /source/path/image.jpg user@server:/destination/path)` Note: when moving files onto a server, you'll typically need to copy to /tmp/ first then to the desired destination.
- `sudo <command>` run a command that requires security privileges.

## Step 2: Install virtualenv

Since this is our first interaction with the apt packaging system in this session, we should update our local package index before we begin so that we are using the most up-to-date information:

    $ sudo apt-get update

Enter the following commands to install virtualenv:

    $ sudo apt-get install python-virtualenv
    $ cd
    $ sudo mkdir /usr/local/lib/python-virtualenv
    $ cd /usr/local/lib/python-virtualenv

If you are using Python 3, make sure to specify the `--python=python3.X` flag (where X is the proper version number) in the following command. Otherwise, the virtual environment will be created with the default version of Python, which is typically Python 2.

    $ sudo virtualenv <project name>
    $ cd <project name>

Now you can activate your virtual environment with this command: `$ source bin/activate`

Note how the command line has changed to show that you are running in a virtual environment: `(bridge) ajanco@bridge:/usr/local/lib/python-virtualenv/bridge$`

## Step 3: Install Git and Clone Your Project
[Git](https://en.wikipedia.org/wiki/Git) is version-control software that is used to keep track of changes on collaborative projects. It allows you to save successive revisions of your work and then view them at a later time (for example, if you made a horrible mistake and just want to return to a working version), and it has strong support for multiple developers working on a single project. You can install Git like this:

    $ sudo apt-get install git

All of our projects live on [Github](https://github.com/HCDigitalScholarship). Find the project that you're working on and click on the title.  This will open the repository (usually abbreviated as "repo"). Check the name of the branch and be sure that you're in the branch that you need.  Next, click on the "Clone or download" button.  A URL will appear.  Select and copy that address.

Now, switch to a terminal window open on the server and type in the commands below. You can also `cd` into `/srv/www/`, `/srv/html/`, or `/var/www/` if you'd like. Whatever you choose, Git is going to create a folder there with the name of the repo and its project files.

    $ cd /srv/
    $ sudo git clone https://github.com/HCDigitalScholarship/project_name.git

The URL above comes from the "Clone or download" button on Github. You will then be prompted to enter your username and password.

### Common Git commands
- `$ git status` view the status of the current git repo
- `$ git add <file-path>` add the file or directory to the git working index (e.g., after you've edited some files)
- `$ git commit -m "<commit-message>"` commit the changes in the working index (that you added with the `git add` command). Think of this as a "save progress" command.
- `$ git log` view metadata about previous commits
- `$ git checkout -b <branch>` create a new branch and switch to it
- `$ git checkout <branch>` switch to an already-existing branch
- `$ git checkout <commit-hash>` look at another commit (a previous "save"). The commit hash can be found with the `git log` command.
- `$ git freeze`
- `$ git stash`

## Step X: Install Project Requirements

    $ sudo pip install -r requirements.txt

Test the app to make sure all requirements are installed by running 

    $ python manage.py runserver

If you don't get any errors, then the app is properly configured and you can move on to the next step!

## Step X: Install uWsgi
What is an application server and why are we using it? 
Sockets: uWSGI can create multiple types of sockets, and we will use two of them here--HTTP and Unix. We'll start with an HTTP socket so that we can reqeust HTML from the app without having to also configure a web server. Once we are sure that uWSGI is properly configured, we will switch to a Unix file socket for performance and security reasons.

    $ pip install uwsgi

Create a new file /etc/uwsgi/apps-available/project_name.ini and paste in the following contents:

    [uwsgi]

    #Environmental settings: 
    uid = www-data
    gid = www-data
    #socket = /run/uwsgi/app/duchemin/duchemin.socket
    
First we are setting the user and group that can access the file socket, and specifying the location of the file socket. We specify the /run/uwsgi/app folder because we know the uWSGI service has permission to access it. The file socket location line should be commented out initially for reasons that will be explained in a moment.

    # For testing: curl -L http://localhost:5000
    socket = localhost:5000
    protocol = http

These lines are only there for testing purposes, and they create the HTTP socket. They will eventually be commented out. We can test the socket by requesting HTML using the "curl" command from the command line. 

    # Application settings
    plugin = python
    chdir = /srv/DuChemin
    #virtualenv = /usr/local/lib/python-virtualenv/ticha-django
    module=duchemin.wsgi:application

These lines contain specifics for the application server hooking into the app.
`plugin` specifies the type of application. In most cases, e.g. a Django app, this will be `python`, but it could also be ruby, python3, mongoDB, PHP, and so on. See the uWSGI docs for all of the available plugins.

`chdir` specifies the project root directory. For a Flask or Django project, this is the folder that contains `manage.py`

`virtualenv` specifies the location of the virtual environment.

`module` defines the application for uWSGI. The name of the application should be identical to to the name specified in the `WSGI_APPLICATION` line of `settings.py`

    # Performance tuning
    processes = 4
    threads = 2`

The python plugin by default does not have threads enabled, so we enable threading support here. We can adjust these lines based on performance once the project is deployed.

Save and quit.

Create a symbolic link in `/apps-enabled` to the `.ini` file in `/apps-available` to activate the configuration:
    `$ ln -s /etc/uwsgi/apps-enabled/config.ini /etc/uwsgi/apps-available/config.ini`
    
Restart the service
    $ sudo service uwsgi restart
    
Now let's test the socket to see if we can get an HTML response:
    $ curl -L http://localhost:5000

The `-L` option follows redirects, which is necessary if your project redirects from the index page.

If you get an HTML response, you've successfully configured uWSGI! If you get an error or no response, check the logs at `/var/log/uwsgi/app/nameofapp.log`

Once you've successfully configured uWSGI, comment out the two lines related to the HTTP socket and uncomment the line for the Unix file socket. Save, quit, and restart uWSGI.

{'fraction_non_alphanumeric': 0.06931635628012138, 'fraction_numerical': 0.0034509430594395194, 'mean_word_length': 3.8817891373801916, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 8, 'lorem ipsum': 0, 'www.': 8, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2793195', 'n_tokens_mistral': 4654, 'n_tokens_neox': 4427, 'n_words': 2507}