<commit_before>---
layout: article
title: Compiled queries in Scala
categories: scala
comments: true
toc: true
image:
  feature: compiled-queries-feature.jpg
  teaser: compiled-queries-teaser.jpg
  caption: Northwest San Francisco and the Golden Gate strait from the Hamon Observation Tower at de Young Museum. February 2017
---

Why resort to the complexities of dynamic code generation and compilation at
runtime?

**Speed**.

To demonstrate, lets build an interpreter that runs projection and filtering on
a simulated database, then build a compiled version and look at some performance
numbers.

To be clear, this is our goal: **compile a data structure representing a query
into native code to speed up a query loop**.

We'll look at two specific code-generation tools to achieve our goal: ASM and
Scala Quasiquotes.

ASM has been used by Java developers for years for all sorts of codegen
purposes. It's is very fast and has low-memory requirements, but it's also very
low-level, making it time-consuming and tedious to use and very difficult to
debug.

Quasiquotes are a new Scala tool for code generation. They are similar to
macros, except they can be used at runtime whereas macros are compile-time only.<commit_msg>Simplify<commit_after>---
layout: article
title: Compiled queries in Scala
categories: scala
comments: true
toc: true
image:
  feature: compiled-queries-feature.jpg
  teaser: compiled-queries-teaser.jpg
  caption: Northwest San Francisco and the Golden Gate strait from the Hamon Observation Tower at de Young Museum. February 2017
---

Why resort to the complexities of dynamic code generation and compilation at
runtime?

**Speed**.

To demonstrate, lets build an interpreter that runs projection and filtering on
a simulated database, then build a compiled version and look at some performance
numbers.

This is our goal: **compile a data structure representing a query into native
code to speed up a query loop**. We'll look at two specific code-generation
tools to achieve our goal: ASM and Scala Quasiquotes.

ASM has been used by Java developers for years for all sorts of codegen
purposes. It's is very fast and has low-memory requirements, but it's also very
low-level, making it time-consuming and tedious to use and very difficult to
debug.

Quasiquotes are a new Scala tool for code generation. They are similar to
macros, except they can be used at runtime whereas macros are compile-time only.
{'fraction_non_alphanumeric': 0.04726674886970818, 'fraction_numerical': 0.0032881216605014385, 'mean_word_length': 4.519274376417234, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2393383', 'n_tokens_mistral': 646, 'n_tokens_neox': 601, 'n_words': 362}