size_categories:10K<n<100K language:de government bundestag region:us
---
language:
- de
tags:
- government
- bundestag
size_categories:
- 10K<n<100K
---
# Dataset Card for Dataset Name

## Dataset Description

- **Homepage:**  Quelle: https://dip.bundestag.de/

### Dataset Summary

Extracted the pdf documents from the DIP Service. 

In the current version, the documents are between the following dates:

START_DATE = "2015-05-07"
END_DATE = "2023-07-09"

Distribution over the document types:
![Document Type Distribution](https://huggingface.co/datasets/wirthual/dip-bundestag/resolve/main/distribution.png)

### Languages

German

## Dataset Structure

Each row of the dataset consists of the following fields: doc_id and text

The document id can be used to retrieve the metadata for the underlying PDF file by sending a request to the follwoing endpoint:

https://search.dip.bundestag.de/api/v1/swagger-ui/#/Drucksachen/getDrucksache



### Data Fields

index

doc_id

text

### Data Splits

No split

## Dataset Creation

Download PDF, extract text, [dehyphenize](https://github.com/pd3f/dehyphen) text

Text got extracted within the boarders of the following rectangle to avoid header/footer text:

<img src="https://huggingface.co/datasets/wirthual/dip-bundestag/resolve/main/rect_for_extraction.png" alt="Rectangle of extraction" style="width:400px;"/>


## Dataset Curation

At this point, no complex curation of the dataset was performed.


#### Who are the source language producers?

https://dip.bundestag.de/


### Licensing Information

Quelle: Deutscher Bundestag/Bundesrat – DIP / "Bundestags-Drucksache"

For further detail see:

https://dip.bundestag.de/documents/nutzungsbedingungen_dip.pdf