In Part 1 of this two-part series, I looked at Forward and Reverse Geocoding using FourSquare's TwoFishes. I provided some background on TwoFishes, and shared some lessons that I learned on building the geospatial database/indices for forward and reverse geocoding.
Building the TwoFishes database/indices takes some significant time; its not something you want to do on startup. Once you have the database/indices built, you want to get it up and running. There's no better way than to use a Docker container to package up the Foursquare code, and our newly built database. So, in this blog, I will dive into the best way (IMHO) to containerize TwoFishes to account for its idosyncracies.
For example, after the container is built, and TwoFishes is starts up for the first time, it performs some secondary processesing of the database as it prepares to handle requests. This results in the creation of some additional files, which are ephemeral if you run this inside a docker container that doesn't have a volume. This processing takes some time (more than 5 minutes), so it's not something I want to happen every time the container starts. If you don't attach a volume to your container, that's exactly what will happen. For this purpose, I'm using an EFS volume to provide persistent storage for my TwoFishes container.
Here is a working Docker implementation. When TwoFishes starts up the first time, it looks at the database/index, and does some pre-processing prior to being completely ready to handle requests. This takes a while, and I don't want to incur this cost on every startup. Therefore, I use a volume to avoid long start times on the container.
I have provided this code in https://github.com/CorkHounds/twofishes-docker.git.
Let's step through the dockerfile. I have provided comments inline below to explain what the various segments are doing.
RUN bash -l -c "/var/lib/dpkg/info/ca-certificates-java.postinst configure"
Notice that you will need a data folder (e.g. us-data) that has your geo database/indices that we created in Part 1 of this series. At a minimum, this would provide forward geocoding, but could also include the data/indices for reverse geocoding depending on the steps you followed in Part 1. Just to be clear, the folder doesn't need to be named 'us-data' ... but whatever you call it, you will want to (1) upload it to your docker container, and then (2) reference it from your run.sh script below to start the service (e.g. serve.py -p 8080 us-data/).
Now, lets look at the run.sh file. First, we'll want to make sure that run.sh is executable by running chmod +x run.sh. Now, we can step through the file.
# exit the shell if any action returns a non-zero status.
This script copies the FSQIO contents to disk, and serves the application from disk. To build the container, you simply run docker build -t twofishes:latest .. You can replace the tag:version as you see fit.
Before you run this container, you'll need to change the vm.max_map_count that determines the maximum number of memory map areas a process may have in virtual memory. TwoFishes requires a larger memory map ceiling.
Running that command only temporarily incrases the virtual mem map areas; we should set this permenantly by editing the /etc/sysctl.conf file by adding vm.max_map_count=131072.
In the above command, I am only exposing port 8081, but there are other ports listed in the TwoFishes docs you may want to expose. Also, in the above docker run command, I detach the container so that it runs in the background. To see the logs, simply run docker ps, get the Container ID, and then run docker logs -f <CONTAINER ID>. You will see that the TwoFishes service traverses several phases. First, it installs all dependencies beginning with Installing setuptools, pip, wheel...done followed by dozens of others.
INFO: Admin HTTP interface started on port 8082.
You can read more about request parameters in the TwoFishes docs.
If we do not increase the virtual memory map areas, we see the following error during the first run.
or reduce the number required by passing --vm_map_count MAP_COUNT, but expect OOMS!
In my setup, I'm running this container on AWS Elastic Container Service (ECS), with an underlying Elastic File System (EFS) volume. If anyone is interested in that configuration, just ask and I can provide it.