OSCAR from Inria group
{'login': 'jchwenger', 'id': 34098722, 'node_id': 'MDQ6VXNlcjM0MDk4NzIy', 'avatar_url': 'https://avatars.githubusercontent.com/u/34098722?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/jchwenger', 'html_url': 'https://github.com/jchwenger', 'followers_url': 'https://api.github.com/users/jchwenger/followers', 'following_url': 'https://api.github.com/users/jchwenger/following{/other_user}', 'gists_url': 'https://api.github.com/users/jchwenger/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/jchwenger/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/jchwenger/subscriptions', 'organizations_url': 'https://api.github.com/users/jchwenger/orgs', 'repos_url': 'https://api.github.com/users/jchwenger/repos', 'events_url': 'https://api.github.com/users/jchwenger/events{/privacy}', 'received_events_url': 'https://api.github.com/users/jchwenger/received_events', 'type': 'User', 'site_admin': False}
[{'id': 2067376369, 'node_id': 'MDU6TGFiZWwyMDY3Mzc2MzY5', 'url': 'https://api.github.com/repos/huggingface/datasets/labels/dataset%20request', 'name': 'dataset request', 'color': 'e99695', 'default': False, 'description': 'Requesting to add a new dataset'}]
[]
PR is already open here : #348 
The only thing remaining is to compute the metadata of each subdataset (one per language + shuffled/unshuffled).
As soon as #863 is merged we can start computing them. This will take a bit of time though Grand, thanks for this!
## Adding a Dataset
- **Name:** *OSCAR* (Open Super-large Crawled ALMAnaCH coRpus), multilingual parsing of Common Crawl (separate crawls for many different languages), [here](https://oscar-corpus.com/).
- **Description:** *OSCAR or Open Super-large Crawled ALMAnaCH coRpus is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the goclassy architecture.*
- **Paper:** *[here](https://hal.inria.fr/hal-02148693)*
- **Data:** *[here](https://oscar-corpus.com/)*
- **Motivation:** *useful for unsupervised tasks in separate languages. In an ideal world, your team would be able to obtain the unshuffled version, that could be used to train GPT-2-like models (the shuffled version, I suppose, could be used for translation).*

I am aware that you do offer the "colossal" Common Crawl dataset already, but this has the advantage to be available in many subcorpora for different languages.

{'url': 'https://api.github.com/repos/huggingface/datasets/issues/866/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}