<commit_before>---
title: "No Redis Needed: building a Postgres-backed recommendation engine for Rails"
date: 2017-03-19 20:33 UTC
tags:
---

I recently became interested in how online recommendation engines work, and decided to try implementing one myself as a Rails gem. I ended up building a gem that runs recommendations on your relational database, instead of using Redis like other gems. Read on to learn more about how I got there.

## Existing Rails recommendation gems

Looking at the existing options for Rails recommendation engines, I quickly realized two things.

### Realization #1: Basic recommendation algorithms are surprisingly simple!

If you've heard of the Netflix challenge or all the fancy algorithms that big companies use to recommend things to you, you might think that recommendation is a really hard problem. And you're not wrong. Producing high-quality recommendations is a deep subject that whole companies specialize in.

But producing basic, workable recommendations to users is actually a pretty simple problem! To take an example, let's see how we could provide recommendations to users on a movie website.

On our website, users can "like" movies, so we have a record of all the movies that each user liked.

First, we take each pair of movies in our database and compute a similarity between the two movies. To find this similarity, we compare the lists of users who liked the two movies. If many people liked both of the movies, chances are they're fairly similar. Conversely, if not many people liked both movies, they're probably less similar. A table of similarities might look something like this:

<h4>Movie similarities</h4>
<table>
  <tr>
    <th></th>
    <th>Batman</th><commit_msg>add readmore break<commit_after>---
title: "No Redis Needed: building a Postgres-backed recommendation engine for Rails"
date: 2017-03-19 20:33 UTC
tags:
---

I recently became interested in how online recommendation engines work, and decided to try implementing one myself as a Rails gem. I ended up building a gem that runs recommendations on your relational database, instead of using Redis like other gems. Read on to learn more about how I got there.

READMORE

## Existing Rails recommendation gems

Looking at the existing options for Rails recommendation engines, I quickly realized two things.

### Realization #1: Basic recommendation algorithms are surprisingly simple!

If you've heard of the Netflix challenge or all the fancy algorithms that big companies use to recommend things to you, you might think that recommendation is a really hard problem. And you're not wrong. Producing high-quality recommendations is a deep subject that whole companies specialize in.

But producing basic, workable recommendations to users is actually a pretty simple problem! To take an example, let's see how we could provide recommendations to users on a movie website.

On our website, users can "like" movies, so we have a record of all the movies that each user liked.

First, we take each pair of movies in our database and compute a similarity between the two movies. To find this similarity, we compare the lists of users who liked the two movies. If many people liked both of the movies, chances are they're fairly similar. Conversely, if not many people liked both movies, they're probably less similar. A table of similarities might look something like this:

<h4>Movie similarities</h4>
<table>
  <tr>
    <th></th>
    <th>Batman</th>
{'fraction_non_alphanumeric': 0.04724863151829444, 'fraction_numerical': 0.00864304235090752, 'mean_word_length': 4.5374800637958534, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 19, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '535258', 'n_tokens_mistral': 850, 'n_tokens_neox': 786, 'n_words': 530}