task_categories:text-retrieval task_categories:text-classification task_ids:document-retrieval annotations_creators:expert-generated language_creators:found multilinguality:monolingual size_categories:10K<n<100K source_datasets:original language:fr license:cc-by-nc-sa-4.0 legal arxiv:2108.11792 region:us
The Belgian Statutory Article Retrieval Dataset (BSARD) is a French native dataset for studying legal information retrieval. 
BSARD consists of more than 22,600 statutory articles from Belgian law and about 1,100 legal questions posed by Belgian citizens 
and labeled by experienced jurists with relevant articles from the corpus.
@inproceedings{louis-spanakis-2022-statutory,
    title = "A Statutory Article Retrieval Dataset in {F}rench",
    author = "Louis, Antoine  and Spanakis, Gerasimos",
    booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.acl-long.468",
    doi = "10.18653/v1/2022.acl-long.468",
    pages = "6789--6803",
}
---
annotations_creators:
- expert-generated
language_creators:
- found
language:
- fr
license:
- cc-by-nc-sa-4.0
multilinguality:
- monolingual
pretty_name: BSARD
size_categories:
- 10K<n<100K
source_datasets:
- original
task_categories:
- text-retrieval
- text-classification
task_ids:
- document-retrieval
paperswithcode_id: bsard
tags:
- legal
---

# Dataset Card for BSARD

## Table of Contents
- [Table of Contents](#table-of-contents)
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Repository:** [maastrichtlawtech/bsard](https://github.com/maastrichtlawtech/bsard)
- **Paper:** [A Statutory Article Retrieval Dataset in French](https://arxiv.org/abs/2108.11792)
- **Point of Contact:** [Maastricht Law & Tech Lab](law-techlab@maastrichtuniversity.nl)

### Dataset Summary

The Belgian Statutory Article Retrieval Dataset (BSARD) is a French native dataset for studying legal information retrieval. BSARD consists of more than 22,600 statutory articles from Belgian law and about 1,100 legal questions posed by Belgian citizens and labeled by experienced jurists with relevant articles from the corpus.

### Supported Tasks and Leaderboards

- `document-retrieval`: The dataset can be used to train models for ad-hoc legal information retrieval. Such model is presented with a short user query written in natural language and asked to retrieve relevant legal information from a knowledge source (such as statutory articles).

### Languages

The text in the dataset is in French, as spoken in Wallonia and Brussels-Capital region. The associated BCP-47 code is `fr-BE`.

## Dataset Structure

### Data Instances

A typical data point comprises a question, with additional `category`, `subcategory`, and `extra_description` fields that elaborate on it, and a list of `article_ids` from the corpus of statutory articles that are relevant to the question.

An example from the BSARD test set looks as follows:
```
{
 'id': '724',
 'question': 'La police peut-elle me fouiller pour chercher du cannabis ?',
 'category': 'Justice',
 'subcategory': 'Petite délinquance',
 'extra_description': 'Détenir, acheter et vendre du cannabis',
 'article_ids': '13348'
}
```

### Data Fields

- In **"questions_fr_train.csv"** and **"questions_fr_test.csv"**:
  - `id`: an *int32* feature corresponding to a unique ID number for the question.
  - `question`: a *string* feature corresponding to the question.
  - `category`: a *string* feature corresponding to the general topic of the question.
  - `subcategory`: a *string* feature corresponding to the sub-topic of the question.
  - `extra_description`: a *string* feature corresponding to the extra categorization tags of the question.
  - `article_ids`: a *string* feature of comma-separated article IDs relevant to the question.

- In **"articles_fr.csv"**:
  - `id`: an *int32* feature corresponding to a unique ID number for the article.
  - `article`: a *string* feature corresponding to the full article.
  - `code`: a *string* feature corresponding to the law code to which the article belongs.
  - `article_no`: a *string* feature corresponding to the article number in the code.
  - `description`: a *string* feature corresponding to the concatenated headings of the article.
  - `law_type`: a *string* feature whose value is either *"regional"* or *"national"*.

### Data Splits

This dataset is split into train/test set. Number of questions in each set is given below:

|                            | Train  |  Test |
| -----                      | ------ |  ---- |
| BSARD                      |    886 |   222 |

## Dataset Creation

### Curation Rationale

The dataset is intended to be used by researchers to build and evaluate models on retrieving law articles relevant to an input legal question. It should not be regarded as a reliable source of legal information at this point in time, as both the questions and articles correspond to an outdated version of the Belgian law from May 2021 (time of dataset collection). In the latter case, the user is advised to consult daily updated official legal resources (e.g., the Belgian Official Gazette).

### Source Data

#### Initial Data Collection and Normalization

BSARD was created in four stages: (i) compiling a large corpus of Belgian law articles, (ii) gathering legal questions with references to relevant law articles, (iii) refining these questions, and (iv) matching the references to the corresponding articles from the corpus.

#### Who are the source language producers?

Speakers were not directly approached for inclusion in this dataset and thus could not be asked for demographic information. Questions were collected, anonimyzed, and reformulated by [Droits Quotidiens](https://www.droitsquotidiens.be/fr/equipe). Therefore, no direct information about the speakers’ age and gender distribution, or socioeconomic status is available. However, it is expected that most, but not all, of the speakers are adults (18+ years), speak French as a native language, and live in Wallonia or Brussels-Capital region.

### Annotations

#### Annotation process

Each year, [Droits Quotidiens](https://www.droitsquotidiens.be/fr/equipe), a Belgian organization whose mission is to clarify the law for laypeople, receives and collects around 4,000 emails from Belgian citizens asking for advice on a personal legal issue. In practice, their legal clarification process consists of four steps. First, they identify the most frequently asked questions on a common legal issue. Then, they define a new anonymized "model" question on that issue expressed in natural language terms, i.e., as close as possible as if a layperson had asked it. Next, they search the Belgian law for articles that help answer the model question and reference them.

#### Who are the annotators?

A total of six Belgian jurists from [Droits Quotidiens](https://www.droitsquotidiens.be/fr/equipe) contributed to annotating the questions. All have a law degree from a Belgian university and years of experience in providing legal advice and clarifications of the law. They range in age from 30-60 years, including one man and five women, gave their ethnicity as white European, speak French as a native language, and represent upper middle class based on income levels.

### Personal and Sensitive Information

The questions represent informal, asynchronous, edited, written language that does not exceed 44 words. None of them contained hateful, aggressive, or inappropriate language as they were all reviewed and reworded by Droits Quotidiens to be neutral, anonymous, and comprehensive. The legal articles represent strong, formal, written language that can contain up to 5,790 words.

## Considerations for Using the Data

### Social Impact of Dataset

In addition to helping advance the state-of-the-art in retrieving statutes relevant to a legal question, BSARD-based models could improve the efficiency of the legal information retrieval process in the context of legal research, therefore enabling researchers to devote themselves to more thoughtful parts of their research. Furthermore, BSARD can become a starting point of new open-source legal information search tools so that the socially weaker parties to disputes can benefit from a free professional assisting service.

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

First, the corpus of articles is limited to those collected from 32 Belgian codes, which obviously does not cover the entire Belgian law as thousands of articles from decrees, directives, and ordinances are missing. During the dataset construction, all references to these uncollected articles are ignored, which causes some questions to end up with only a fraction of their initial number of relevant articles. This information loss implies that the answer contained in the remaining relevant articles might be incomplete, although it is still appropriate.

Additionally, it is essential to note that not all legal questions can be answered with statutes alone. For instance, the question “Can I evict my tenants if they make too much noise?” might not have a detailed answer within the statutory law that quantifies a specific noise threshold at which eviction is allowed. Instead, the landlord should probably rely more on case law and find precedents similar to their current situation (e.g., the tenant makes two parties a week until 2 am). Hence, some questions are better suited than others to the statutory article retrieval task, and the domain of the less suitable ones remains to be determined.

## Additional Information

### Dataset Curators

The dataset was created by Antoine Louis during work done at the Law & Tech lab of Maastricht University, with the help of jurists from [Droits Quotidiens](https://www.droitsquotidiens.be/fr/equipe).

### Licensing Information

BSARD is licensed under the [CC BY-NC-SA 4.0 license](https://creativecommons.org/licenses/by-nc-sa/4.0/).

### Citation Information

```latex
@inproceedings{louis2022statutory,
  title = {A Statutory Article Retrieval Dataset in French},
  author = {Louis, Antoine and Spanakis, Gerasimos},
  booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics},
  month = may,
  year = {2022},
  address = {Dublin, Ireland},
  publisher = {Association for Computational Linguistics},
  url = {https://aclanthology.org/2022.acl-long.468/},
  doi = {10.18653/v1/2022.acl-long.468},
  pages = {6789–6803},
}
```

### Contributions

Thanks to [@antoiloui](https://github.com/antoiloui) for adding this dataset.