<commit_before>---
layout: default
title: Writing Applications in DDlog
---

# Writing Applications in DDlog

DDlog is a higher-level language for writing DeepDive applications in succinct Datalog-like syntax.
We are gradually extending the language to allow expression of advanced SQL queries used by complex extractors as well as a variety of factors with Markov Logic-like syntax.
A reference for ddlog lanugage features can be found [here](https://github.com/HazyResearch/ddlog/wiki/DDlog-Language-Features).

## Writing a DDlog Program

A DDlog program consists of the following parts:

1. Schema declarations (eg. relational tables)
2. Data transformation rules
  a. UDFs
  b. += operator TODO : How do you call this?


All DDlog code should be placed in a file named `app.ddlog` under the DeepDive application.
A complete example written in DDlog can be found from [examples/spouse_example/postgres/ddlog](https://github.com/HazyResearch/deepdive/blob/master/examples/spouse_example/postgres/ddlog)


### Basic Syntax

Each declaration and rule ends with a period (`.`).
The order of the rules have no meaning.
Comments in DDlog begin with a hash (`#`) character.

### Schema Declaration

First of all, we declare the relations we use throughout the program.  These relations refer to tables that will appear in your database.  See [deepdive sql TODO](fill in with real link) for details on how to interact with the database.
The order doesn't matter, but it's a good idea to place them at the beginning because that makes it easier to understand.

#### Relations Syntax
    relation_name(
      column1_name  column1_type,
      column2_name  column2_type,
      ...).

Similar to a SQL table definition, a schema declaration is just the name of a relation followed by a comma separated list of the columns and their types.

Below is a real example from [the spouse example TODO] (add real link here).

<script defer src="https://gist-it.appspot.com/github.com/HazyResearch/deepdive/blob/master/examples/spouse_example/postgres/ddlog/app.ddlog?footer=minimal&slice=36:38">
</script>

Here we are defining a relation named "has_spouse_features" with two columns, named "relation_id" and "feature" respectively.  In this case, they are both of type "text" but ddlog supports other types as well, including int, text[] and others.  For more details, see the [reference guide TODO](add real link here).

#### Variable Relations
Finally, we declare a variable relation that we want DeepDive to predict the marginal probability for us.

<script defer src="https://gist-it.appspot.com/github.com/HazyResearch/deepdive/blob/master/examples/spouse_example/postgres/ddlog/app.ddlog?footer=minimal&slice=39:41">
</script>

### Candidate Mapping and Supervision Rules
A user-defined function for mapping candidates and supervision them is written in Python and declared as below with a name `ext_people`.
The function is supposed to take as input a triple of sentence id, array of words, and array of NER tags of the words and output rows like the `people_mentions` relation.<commit_msg>Fixed relation syntax<commit_after>---
layout: default
title: Writing Applications in DDlog
---

# Writing Applications in DDlog

DDlog is a higher-level language for writing DeepDive applications in succinct Datalog-like syntax.
We are gradually extending the language to allow expression of advanced SQL queries used by complex extractors as well as a variety of factors with Markov Logic-like syntax.
A reference for ddlog lanugage features can be found [here](https://github.com/HazyResearch/ddlog/wiki/DDlog-Language-Features).

## Writing a DDlog Program

A DDlog program consists of the following parts:

1. Schema declarations (eg. relational tables)
2. Data transformation rules
  a. UDFs
  b. += operator TODO : How do you call this?


All DDlog code should be placed in a file named `app.ddlog` under the DeepDive application.
A complete example written in DDlog can be found from [examples/spouse_example/postgres/ddlog](https://github.com/HazyResearch/deepdive/blob/master/examples/spouse_example/postgres/ddlog)


### Basic Syntax

Each declaration and rule ends with a period (`.`).
The order of the rules have no meaning.
Comments in DDlog begin with a hash (`#`) character.

### Schema Declaration

First of all, we declare the relations we use throughout the program.  These relations refer to tables that will appear in your database.  See [deepdive sql TODO](fill in with real link) for details on how to interact with the database.
The order doesn't matter, but it's a good idea to place them at the beginning because that makes it easier to understand.

#### Relations Syntax
```
relation_name(
  column1_name  column1_type,
  column2_name  column2_type,
  ...).
```
Similar to a SQL table definition, a schema declaration is just the name of a relation followed by a comma separated list of the columns and their types.

Below is a real example from [the spouse example TODO] (add real link here).
```
article(
  id int,
  length int,
  author text,
  words text[]).
```
Here we are defining a relation named "article" with four columns, named "id", "length", "author" and "words" respectively. Each column has it's own type, here utilizing `int`, `text` and `text[]`.
[//]:  Comment?

#### Variable Relations
Finally, we declare a variable relation that we want DeepDive to predict the marginal probability for us.

<script defer src="https://gist-it.appspot.com/github.com/HazyResearch/deepdive/blob/master/examples/spouse_example/postgres/ddlog/app.ddlog?footer=minimal&slice=39:41">
</script>

### Candidate Mapping and Supervision Rules
A user-defined function for mapping candidates and supervision them is written in Python and declared as below with a name `ext_people`.
The function is supposed to take as input a triple of sentence id, array of words, and array of NER tags of the words and output rows like the `people_mentions` relation.
{'fraction_non_alphanumeric': 0.07405541561712846, 'fraction_numerical': 0.004030226700251889, 'mean_word_length': 4.525046382189239, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 7, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4795423', 'n_tokens_mistral': 1656, 'n_tokens_neox': 1569, 'n_words': 773}