Add script csv datasets
{'login': 'jplu', 'id': 959590, 'node_id': 'MDQ6VXNlcjk1OTU5MA==', 'avatar_url': 'https://avatars.githubusercontent.com/u/959590?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/jplu', 'html_url': 'https://github.com/jplu', 'followers_url': 'https://api.github.com/users/jplu/followers', 'following_url': 'https://api.github.com/users/jplu/following{/other_user}', 'gists_url': 'https://api.github.com/users/jplu/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/jplu/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/jplu/subscriptions', 'organizations_url': 'https://api.github.com/users/jplu/orgs', 'repos_url': 'https://api.github.com/users/jplu/repos', 'events_url': 'https://api.github.com/users/jplu/events{/privacy}', 'received_events_url': 'https://api.github.com/users/jplu/received_events', 'type': 'User', 'site_admin': False}
[]
[]
['Very interesting thoughts, we should think deeper about all what you raised indeed.'
 'Ok here is a proposal for a more general API and workflow.\r\n\r\n# New `ArrowBasedBuilder`\r\n\r\nFor all the formats that can be directly and efficiently loaded by Arrow (CSV, JSON, Parquet, Arrow), we don\'t really want to have to go through a conversion to python and back to Arrow. This new builder has a `_generate_tables` method to yield `Arrow.Tables` instead of single examples.\r\nThe tables can be directly casted in Arrow so it\'s not necessary to supply  `Features`, they can be deduced from the `Table` column.\r\n\r\n# Central role of the `BuilderConfig` to store all the arguments necessary for the Dataset creation.\r\n \r\n`BuilderConfig` provide a few defaults fields `name`, `version`,  `description`,  `data_files` and `data_dir` which can be used to store values necessary for the creation of the dataset. It can be freely extended to store additional information (see the example for `CsvConfig`).\r\n\r\nOn the contrary, `DatasetInfo` is designed as an organized and delimited information storage class with predefined fields.\r\n\r\n`DatasetInfo` now store two names:\r\n- `builder_name`: Name of the builder script used to create the dataset\r\n- `config_name`: Name of the configuration used to create the dataset.\r\n\r\n# Refactoring `load()` arguments and all the chain of processing including the `DownloadManager`\r\n\r\n`load()` now accept a selection of arguments which are used to update the `BuilderConfig` and some kwargs which are used to handle the download process.\r\n\r\nSupplying a `BuilderConfig` as `config` will override the config provided in the dataset. Supplying a `str` will get the associated config from the dataset. Default is to fetch the first config of the dataset.\r\n\r\nGiving additional arguments to `load()` will override the arguments in the `BuilderConfig`.\r\n\r\n# CSV script\r\n\r\nThe `csv.py` script is provided as an example, usage is:\r\n```python\r\nbbc = nlp.load(\'/Users/thomwolf/Documents/GitHub/datasets/datasets/nlp/csv\',\r\n               name=\'bbc\',\r\n               version="1.0.1",\r\n               split=\'train\',\r\n               data_files={\'train\': [\'/Users/thomwolf/Documents/GitHub/datasets/datasets/dummy_data/csv/test.csv\']},\r\n               skip_rows=10,\r\n               download_mode=\'force_redownload\')\r\n```\r\n\r\n# Checksums\r\n\r\nWe now don\'t raise an error if the checksum file is not found.\r\n\r\n# `DownloadConfig`\r\n\r\nWe now have a download configuration class to handle all the specific arguments for file caching like proxies, using only local files or user-agents.'
 "Ok merging this for now.\r\n\r\nOne general note is that it's a bit hard to handle the `ClassLabel` generally in both `nlp` and `Arrow` since a class label typically need some metadata for the class names. For now, I raise a `NotImplementedError` when an `ArrowBuilder` output a table with a `DictionaryType` is encountered (which could be a simple equivalent for a `ClassLabel` Feature in Arrow tables).\r\n\r\nIn general and if we need this in the future for some Beam Datasets for instance, I think we should use one of the `metadata` fields in the `Arrow` type or table's schema to store the relation with indices and class names.\r\n\r\nSo ping me if you meet Beam datasets which uses `ClassLabels` (cc @lhoestq @patrickvonplaten @mariamabarham)."]
{'url': 'https://api.github.com/repos/huggingface/datasets/pulls/25', 'html_url': 'https://github.com/huggingface/datasets/pull/25', 'diff_url': 'https://github.com/huggingface/datasets/pull/25.diff', 'patch_url': 'https://github.com/huggingface/datasets/pull/25.patch', 'merged_at': datetime.datetime(2020, 5, 7, 21, 14, 49)}
This is a PR allowing to create datasets from local CSV files. A usage might be:

```python
import nlp
ds = nlp.load(
    path="csv",
    name="bbc",
    dataset_files={
        nlp.Split.TRAIN: ["datasets/dummy_data/csv/train.csv"],
        nlp.Split.TEST: [""datasets/dummy_data/csv/test.csv""]
    },
    csv_kwargs={
        "skip_rows": 0,
        "delimiter": ",",
        "quote_char": "\"",
        "header_as_column_names": True
    }
)
```

```
Downloading and preparing dataset bbc/1.0.0 (download: Unknown size, generated: Unknown size, total: Unknown size) to /home/jplu/.cache/huggingface/datasets/bbc/1.0.0...
Dataset bbc downloaded and prepared to /home/jplu/.cache/huggingface/datasets/bbc/1.0.0. Subsequent calls will reuse this data.
{'test': Dataset(schema: {'category': 'string', 'text': 'string'}, num_rows: 49), 'train': Dataset(schema: {'category': 'string', 'text': 'string'}, num_rows: 99), 'validation': Dataset(schema: {'category': 'string', 'text': 'string'}, num_rows: 0)}
```

How it is read:

- `path`: the `csv` word means "I want to create a CSV dataset"
- `name`: the name of this dataset is `bbc`
- `dataset_files`: this is a dictionary where each key is the list of files corresponding to the key split.
- `csv_kwargs`: this is the keywords arguments to "explain" how to read the CSV files
  * `skip_rows`: number of rows have to be skipped, starting from the beginning of the file
  * `delimiter`: which delimiter is used to separate the columns
  * `quote_char`: which quote char is used to represent a column where the delimiter appears in one of them
  * `header_as_column_names`: will use the first row (header) of the file as name for the features. Otherwise the names will be automatically generated as `f1`, `f2`, etc... Will be applied after the `skip_rows` parameter.

**TODO**:  for now the `csv.py` is copied each time we create a new dataset as `ds_name.py`, this behavior will be modified to have only the `csv.py` script copied only once and not for all the CSV datasets.
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/25/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}