<commit_before>#Silk

*Silk has now moved to the django-silk organization and is looking for contributors - if you think you can help out, please get in touch!*

[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/django-silk/silk?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[![TravisCI Build](https://img.shields.io/travis/django-silk/silk/master.svg)](https://travis-ci.org/django-silk/silk)
[![PyPI Download](https://img.shields.io/pypi/v/django-silk.svg)](https://pypi.python.org/pypi/django-silk)
[![PyPI Python Versions](https://img.shields.io/pypi/pyversions/django-silk.svg)](https://pypi.python.org/pypi/django-silk)

Silk is a live profiling and inspection tool for the Django framework. Silk intercepts and stores HTTP requests and database queries before presenting them in a user interface for further inspection:

<img src="https://raw.githubusercontent.com/django-silk/silk/master/screenshots/1.png" width="720px"/>

## Contents

* [Requirements](#requirements)
* [Installation](#installation)
* [Features](#features)
* [Configuration](#configuration)
    * [Authentication/Authorisation](#authentication-authorisation)
    * [Request/Response bodies](#request-response-bodies)
    * [Meta-Profiling](#meta-profiling)
    * [Recording a fraction of requests](#recording-a-fraction-of-requests)
    * [Clearing logged data](#clearing-logged-data)
* [Development](#development)

## Requirements

Silk has been tested with:

* Django: 1.7, 1.8, 1.9, 1.10[dev]
* Python: 2.7, 3.3, 3.4, 3.5


## Installation

Via pip into a virtualenv:

```bash
pip install django-silk
```

In `settings.py` add the following:

```python
MIDDLEWARE_CLASSES = (
    ...
    'silk.middleware.SilkyMiddleware',
    ...
)

INSTALLED_APPS = (
    ...
    'silk'
)
```

**Note:** The middleware is placement sensitive. If the middleware before `silk.middleware.SilkyMiddleware` returns from `process_request` then `SilkyMiddleware` will never get the chance to execute. Therefore you must ensure that any middleware placed before never returns anything from `process_request`. See the [django docs](https://docs.djangoproject.com/en/dev/topics/http/middleware/#process-request) for more information on this.

**Note:** If you're using `django.middleware.gzip.GZipMiddleware`, place that **before** `silk.middleware.SilkyMiddleware`, otherwise you'll get an encoding error.

To enable access to the user interface add the following to your `urls.py`:

```python
urlpatterns += [url(r'^silk/', include('silk.urls', namespace='silk'))]
```

before running migrate:

```bash
python manage.py makemigrations

python manage.py migrate
```


Silk will automatically begin interception of requests and you can proceed to add profiling
if required. The UI can be reached at `/silk/`

### Alternative Installation

Via [github tags](https://github.com/django-silk/silk/releases):

```bash
pip install django-silk-<version>.tar.gz
```

You can install from master using the following, but please be aware that the version in master
may not be working for all versions specified in [requirements](#requirements)

```bash
pip install -e git+https://github.com/django-silk/silk.git#egg=silk
```

## Features

Silk primarily consists of:

* Middleware for intercepting Requests/Responses
* A wrapper around SQL execution for profiling of database queries
* A context manager/decorator for profiling blocks of code and functions either manually or dynamically.
* A user interface for inspection and visualisation of the above.

### Request Inspection

The Silk middleware intercepts and stores requests and responses and stores them in the configured database.
These requests can then be filtered and inspecting using Silk's UI through the request overview:

<img src="https://raw.githubusercontent.com/django-silk/silk/master/screenshots/1.png" width="720px"/>

It records things like:

* Time taken
* Num. queries
* Time spent on queries
* Request/Response headers
* Request/Response bodies

and so on.

Further details on each request are also available by clicking the relevant request:

<img src="https://raw.githubusercontent.com/django-silk/silk/master/screenshots/2.png" width="720px"/>

### SQL Inspection

Silk also intercepts SQL queries that are generated by each request. We can get a summary on things like
the tables involved, number of joins and execution time:

<img src="https://raw.githubusercontent.com/django-silk/silk/master/screenshots/3.png" width="720px"/>

Before diving into the stack trace to figure out where this request is coming from:

<img src="https://raw.githubusercontent.com/django-silk/silk/master/screenshots/5.png" width="720px"/>

### Profiling

Turn on the SILKY_PYTHON_PROFILER setting to use Python's built-in cProfile profiler. Each request will be separately profiled and the profiler's output will be available on the request's Profiling page in the Silk UI.

```python
SILKY_PYTHON_PROFILER = True
```

If you'd like to also generate a binary `.prof` file that works with `snakeviz` or other cProfile tools set the following:

```python
SILKY_PYTHON_PROFILER_BINARY = True
```

A download button will become available with a binary `.prof` file for every request.

Silk can also be used to profile specific blocks of code/functions. It provides a decorator and a context
manager for this purpose.

For example:

```python
@silk_profile(name='View Blog Post')
def post(request, post_id):
    p = Post.objects.get(pk=post_id)
    return render_to_response('post.html', {
        'post': p
    })
```

Whenever a blog post is viewed we get an entry within the Silk UI:

<img src="https://raw.githubusercontent.com/django-silk/silk/master/screenshots/7.png" width="720px"/>

Silk profiling not only provides execution time, but also collects SQL queries executed within the block in the same fashion as with requests:<commit_msg>Updates readme for highlight new setting<commit_after>#Silk

*Silk has now moved to the django-silk organization and is looking for contributors - if you think you can help out, please get in touch!*

[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/django-silk/silk?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[![TravisCI Build](https://img.shields.io/travis/django-silk/silk/master.svg)](https://travis-ci.org/django-silk/silk)
[![PyPI Download](https://img.shields.io/pypi/v/django-silk.svg)](https://pypi.python.org/pypi/django-silk)
[![PyPI Python Versions](https://img.shields.io/pypi/pyversions/django-silk.svg)](https://pypi.python.org/pypi/django-silk)

Silk is a live profiling and inspection tool for the Django framework. Silk intercepts and stores HTTP requests and database queries before presenting them in a user interface for further inspection:

<img src="https://raw.githubusercontent.com/django-silk/silk/master/screenshots/1.png" width="720px"/>

## Contents

* [Requirements](#requirements)
* [Installation](#installation)
* [Features](#features)
* [Configuration](#configuration)
    * [Authentication/Authorisation](#authentication-authorisation)
    * [Request/Response bodies](#request-response-bodies)
    * [Meta-Profiling](#meta-profiling)
    * [Recording a fraction of requests](#recording-a-fraction-of-requests)
    * [Clearing logged data](#clearing-logged-data)
* [Development](#development)

## Requirements

Silk has been tested with:

* Django: 1.7, 1.8, 1.9, 1.10[dev]
* Python: 2.7, 3.3, 3.4, 3.5


## Installation

Via pip into a virtualenv:

```bash
pip install django-silk
```

In `settings.py` add the following:

```python
MIDDLEWARE_CLASSES = (
    ...
    'silk.middleware.SilkyMiddleware',
    ...
)

INSTALLED_APPS = (
    ...
    'silk'
)
```

**Note:** The middleware is placement sensitive. If the middleware before `silk.middleware.SilkyMiddleware` returns from `process_request` then `SilkyMiddleware` will never get the chance to execute. Therefore you must ensure that any middleware placed before never returns anything from `process_request`. See the [django docs](https://docs.djangoproject.com/en/dev/topics/http/middleware/#process-request) for more information on this.

**Note:** If you're using `django.middleware.gzip.GZipMiddleware`, place that **before** `silk.middleware.SilkyMiddleware`, otherwise you'll get an encoding error.

To enable access to the user interface add the following to your `urls.py`:

```python
urlpatterns += [url(r'^silk/', include('silk.urls', namespace='silk'))]
```

before running migrate:

```bash
python manage.py makemigrations

python manage.py migrate
```


Silk will automatically begin interception of requests and you can proceed to add profiling
if required. The UI can be reached at `/silk/`

### Alternative Installation

Via [github tags](https://github.com/django-silk/silk/releases):

```bash
pip install django-silk-<version>.tar.gz
```

You can install from master using the following, but please be aware that the version in master
may not be working for all versions specified in [requirements](#requirements)

```bash
pip install -e git+https://github.com/django-silk/silk.git#egg=silk
```

## Features

Silk primarily consists of:

* Middleware for intercepting Requests/Responses
* A wrapper around SQL execution for profiling of database queries
* A context manager/decorator for profiling blocks of code and functions either manually or dynamically.
* A user interface for inspection and visualisation of the above.

### Request Inspection

The Silk middleware intercepts and stores requests and responses and stores them in the configured database.
These requests can then be filtered and inspecting using Silk's UI through the request overview:

<img src="https://raw.githubusercontent.com/django-silk/silk/master/screenshots/1.png" width="720px"/>

It records things like:

* Time taken
* Num. queries
* Time spent on queries
* Request/Response headers
* Request/Response bodies

and so on.

Further details on each request are also available by clicking the relevant request:

<img src="https://raw.githubusercontent.com/django-silk/silk/master/screenshots/2.png" width="720px"/>

### SQL Inspection

Silk also intercepts SQL queries that are generated by each request. We can get a summary on things like
the tables involved, number of joins and execution time:

<img src="https://raw.githubusercontent.com/django-silk/silk/master/screenshots/3.png" width="720px"/>

Before diving into the stack trace to figure out where this request is coming from:

<img src="https://raw.githubusercontent.com/django-silk/silk/master/screenshots/5.png" width="720px"/>

### Profiling

Turn on the SILKY_PYTHON_PROFILER setting to use Python's built-in cProfile profiler. Each request will be separately profiled and the profiler's output will be available on the request's Profiling page in the Silk UI.

```python
SILKY_PYTHON_PROFILER = True
```

If you'd like to also generate a binary `.prof` file that works with `snakeviz` or other cProfile tools set the following:

```python
SILKY_PYTHON_PROFILER_BINARY = True
```
You can specify where to store the generated binary `.prof` files to a path of your choosing.
```python
# Default path value is ''
SILKY_PYTHON_PROFILER_RESULT_PATH = '/path/to/profiles/'
```

A download button will become available with a binary `.prof` file for every request.

Silk can also be used to profile specific blocks of code/functions. It provides a decorator and a context
manager for this purpose.

For example:

```python
@silk_profile(name='View Blog Post')
def post(request, post_id):
    p = Post.objects.get(pk=post_id)
    return render_to_response('post.html', {
        'post': p
    })
```

Whenever a blog post is viewed we get an entry within the Silk UI:

<img src="https://raw.githubusercontent.com/django-silk/silk/master/screenshots/7.png" width="720px"/>

Silk profiling not only provides execution time, but also collects SQL queries executed within the block in the same fashion as with requests:
{'fraction_non_alphanumeric': 0.1110648572142203, 'fraction_numerical': 0.00716010323869786, 'mean_word_length': 4.859512195121951, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 34, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6010398', 'n_tokens_mistral': 3840, 'n_tokens_neox': 3591, 'n_words': 1210}