license:mit region:us
Super-EURLEX dataset containing legal documents from multiple languages.
                The datasets are build/scrapped from the EURLEX Website [https://eur-lex.europa.eu/homepage.html]
                With one split per language and sector, because the available features (metadata) differs for each 
                sector. Therefore, each sample contains the content of a full legal document in up to 3 different 
                formats. Those are raw HTML and cleaned HTML (if the HTML format was available on the EURLEX website 
                during the scrapping process) and cleaned text.
                The cleaned text should be available for each sample and was extracted from HTML or PDF.
                'Cleaned' HTML stands here for minor cleaning that was done to preserve to a large extent the necessary 
                HTML information like table structures while removing unnecessary complexity which was introduced to the 
                original documents due to actions like writing each sentence into a new object. 
                Additionally, each sample contains metadata which was scrapped on the fly, this implies the following 
                2 things. First, not every sector contains the same metadata. Second, most metadata might be 
                irrelevant for most use cases. 
                In our minds the most interesting metadata is the celex-id which is used to identify the legal 
                document at hand, but also contains a lot of information about the document 
                see [https://eur-lex.europa.eu/content/tools/eur-lex-celex-infographic-A3.pdf] as well as eurovoc-
                concepts, which are labels that define the content of the documents. 
                Eurovoc-Concepts are, for example, only available for the sectors 1, 2, 3, 4, 5, 6, 9, C, and E.
                The Naming of most metadata is kept like it was on the eurlex website, except for converting 
                it to lower case and replacing whitespaces with '_'.
---
license: mit
---
