<commit_before>---
layout: post
title: "Profiling a Django App hosted on Google App Engine"
date:   2016-03-19
tags:
  - app engine
  - django
  - djangae
---

Ever used Django with App Engine? How about the [Datastore](https://cloud.google.com/appengine/docs/python/datastore/)? Well, if you didn't
[Potato](https://p.ota.to/) actually developed an in-house solution called [Djangae](https://github.com/potatolondon/djangae) that makes
Django play nicely and almost seamlessly with the Datastore via its ORM. As if
you are writing an app that'll work with a SQL database, except you don't.

If you go down that road, how do you figure out why a piece of Django code is slow?
This brings us to Appstats.

## What is Appstats

[Appstats](https://cloud.google.com/appengine/docs/python/tools/appstats) is a
profiling library that comes together with the App Engine SDK. Its
only purpose is profiling RPC performance of apps. The App Engine Remote
Procedure Calls include:

* Datastore calls (`Datastore.Get`, `Datastore.Put`, etc)
* Memcache calls (`memcache.get`, `memcache.set`, etc)
* URL Fetches
* Mail sends

## Why do you need it

Django offers a lovely ORM to abstract our code from a specific database. It
has tremendous benefits and (at least) one big drawback. It hides the complexity of
database queries. It's easier to make a very expensive query (or make it even
multiple times during a request) without realizing it. That's why profilers exists:
find out why an app is slow.

There is a popular library called [django-debug-toolbar](https://github.com/django-debug-toolbar/django-debug-toolbar).
It shows developers the raw SQL queries and the time it took to
perform them. Unfortunately it's rather useless when it comes to profiling an App Engine
app - remember there is no SQL. But thanks to App Engine developers, we have Appstats.

## How to use it

It's actually incredibly simple. To enable Appstats you need to update only 2 files:

{% highlight python %}
# settings.py
MIDDLEWARE_CLASSES += (
  'google.appengine.ext.appstats.recording.AppStatsDjangoMiddleware'
)
{% endhighlight %}

{% highlight yaml %}
# app.yaml
builtins:<commit_msg>Fix the example with a broken tuple<commit_after>---
layout: post
title: "Profiling a Django App hosted on Google App Engine"
date:   2016-03-19
tags:
  - app engine
  - django
  - djangae
---

Ever used Django with App Engine? How about the [Datastore](https://cloud.google.com/appengine/docs/python/datastore/)? Well, if you didn't
[Potato](https://p.ota.to/) actually developed an in-house solution called [Djangae](https://github.com/potatolondon/djangae) that makes
Django play nicely and almost seamlessly with the Datastore via its ORM. As if
you are writing an app that'll work with a SQL database, except you don't.

If you go down that road, how do you figure out why a piece of Django code is slow?
This brings us to Appstats.

## What is Appstats

[Appstats](https://cloud.google.com/appengine/docs/python/tools/appstats) is a
profiling library that comes together with the App Engine SDK. Its
only purpose is profiling RPC performance of apps. The App Engine Remote
Procedure Calls include:

* Datastore calls (`Datastore.Get`, `Datastore.Put`, etc)
* Memcache calls (`memcache.get`, `memcache.set`, etc)
* URL Fetches
* Mail sends

## Why do you need it

Django offers a lovely ORM to abstract our code from a specific database. It
has tremendous benefits and (at least) one big drawback. It hides the complexity of
database queries. It's easier to make a very expensive query (or make it even
multiple times during a request) without realizing it. That's why profilers exists:
find out why an app is slow.

There is a popular library called [django-debug-toolbar](https://github.com/django-debug-toolbar/django-debug-toolbar).
It shows developers the raw SQL queries and the time it took to
perform them. Unfortunately it's rather useless when it comes to profiling an App Engine
app - remember there is no SQL. But thanks to App Engine developers, we have Appstats.

## How to use it

It's actually incredibly simple. To enable Appstats you need to update only 2 files:

{% highlight python %}
# settings.py
MIDDLEWARE_CLASSES += (
  'google.appengine.ext.appstats.recording.AppStatsDjangoMiddleware',
)
{% endhighlight %}

{% highlight yaml %}
# app.yaml
builtins:
{'fraction_non_alphanumeric': 0.08319889375432127, 'fraction_numerical': 0.004148421295229315, 'mean_word_length': 4.286236297198538, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7388648', 'n_tokens_mistral': 1297, 'n_tokens_neox': 1222, 'n_words': 575}