task_categories:translation size_categories:1K<n<10K license:mit region:us
---
license: mit
task_categories:
- translation
pretty_name: 'BabbelPhish: Natural Language to Message Query Language'
size_categories:
- 1K<n<10K
---


---
# BabbelPhish
BabbelPhish is a dataset based on the [Sublime Security Message Query Language (MQL)](https://docs.sublimesecurity.com/docs/message-query-language) used for email security detection engineering. This dataset is specially created for the BabbelPhish project, which focuses on leveraging large language models to facilitate the work of detection engineers.
This dataset comprises around 3,000 examples drawn from various sources. We've utilized the following:
- [Sublime Security Documentation](https://docs.sublimesecurity.com/docs/message-query-language)
- [Message Data Model (Schema)](https://docs.sublimesecurity.com/docs/message-query-language)
- [Sublime Rules Repo](https://github.com/sublime-security/sublime-rules/)
- [Sublime Community Slack](https://join.slack.com/t/sublimecommunity/shared_invite/zt-1hhwosroy-LvflKNVE3HEtgIcbHdB1sw)


Additionally, we employed additional human-in-the-loop annotation to generate the prompts in this dataset. Each example involves a natural language description paired with an MQL query. 

The BabbelPhish-dataset does not have a natural online source like Stack Overflow. Therefore, we've made a significant effort to generate a unique dataset that closely mirrors the real-world challenges detection engineers face.
We hope this data provides a detailed view of translating natural language prompts into MQL, serving as a valuable resource for similar tasks and research.

## Dataset description
The BabbelPhish dataset contains several fields of interest, and their descriptions are as follows:

- *id*: A unique identifier for each record in the dataset.
- *prompt*: A natural language description or question that outlines the intended task or the specific information to be queried. This forms the input for our language model.
- *completion*: An MQL code snippet corresponding to the prompt. This is the target output generated by the language model.
- *prompt_size*: The character length of the prompt.
- *completion_size*: The character length of the MQL completion.
- *min_line_size*: The minimum line size in the MQL completion.
- *max_line_size*: The maximum line size in the MQL completion.
- *mean_line_size*: The average line size in the MQL completion.
- *ratio*: The record's computed character/token ratio, generated using the tokenizer.


## Usage
```python
from datasets import load_dataset
dataset = load_dataset("sublime-security/babbelphish")
dataset
DatasetDict({
    train: Dataset({
        features: ['id', 'prompt', 'completion', 'prompt_size', 'completion_size', 'min_line_size', 'max_line_size', 'mean_line_size', 'ratio'],
        num_rows: 2857
    })
    test: Dataset({
        features: ['id', 'prompt', 'completion', 'prompt_size', 'completion_size', 'min_line_size', 'max_line_size', 'mean_line_size', 'ratio'],
        num_rows: 50
    })
})
```

## Additional resources

- [Sublime Security Homepage](https://www.sublime.security).
- [BabbelPhish Github Repo](https://github.com/bfilar/babbelphish)
