Extend text to support yielding lines, paragraphs or documents
{'login': 'albertvillanova', 'id': 8515462, 'node_id': 'MDQ6VXNlcjg1MTU0NjI=', 'avatar_url': 'https://avatars.githubusercontent.com/u/8515462?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/albertvillanova', 'html_url': 'https://github.com/albertvillanova', 'followers_url': 'https://api.github.com/users/albertvillanova/followers', 'following_url': 'https://api.github.com/users/albertvillanova/following{/other_user}', 'gists_url': 'https://api.github.com/users/albertvillanova/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/albertvillanova/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/albertvillanova/subscriptions', 'organizations_url': 'https://api.github.com/users/albertvillanova/orgs', 'repos_url': 'https://api.github.com/users/albertvillanova/repos', 'events_url': 'https://api.github.com/users/albertvillanova/events{/privacy}', 'received_events_url': 'https://api.github.com/users/albertvillanova/received_events', 'type': 'User', 'site_admin': False}
[]
[]
['The parameter can also be named `split_by` with values "line", "paragraph" or "document" (no \'s\' at the end)'
 '> The parameter can also be named `split_by` with values "line", "paragraph" or "document" (no \'s\' at the end)\r\n\r\n@lhoestq @mariosasko I would avoid the term `split` in this context and keep it only for "train", "validation" and "test" splits.\r\n- https://huggingface.co/docs/datasets/process.html#split\r\n  > datasets.Dataset.train_test_split() creates train and test splits, if your dataset doesn’t already have them.\r\n- https://huggingface.co/docs/datasets/process.html#process-multiple-splits\r\n  > Many datasets have splits that you can process simultaneously with datasets.DatasetDict.map().\r\n\r\nPlease note that in the documentation, one of the terms more frequently used in this context is **"row"**:\r\n- https://huggingface.co/docs/datasets/access.html#features-and-columns\r\n  > A dataset is a table of rows and typed columns.\r\n\r\n  > Return the number of rows and columns with the following standard attributes:\r\n  > dataset.num_columns\r\n  > 4\r\n  > dataset.num_rows\r\n  > 3668\r\n\r\n- https://huggingface.co/docs/datasets/access.html#rows-slices-batches-and-columns\r\n  > Get several rows of your dataset at a time with slice notation or a list of indices:\r\n- https://huggingface.co/docs/datasets/process.html#map\r\n  > This function can even create new rows and columns.\r\n\r\nOther of the terms more frequently used in the docs (in the code as well) is **"example"**:\r\n- https://huggingface.co/docs/datasets/process.html#map\r\n  > It allows you to apply a processing function to each example in a dataset, independently or in batches.\r\n- https://huggingface.co/docs/datasets/process.html#batch-processing\r\n  > datasets.Dataset.map() also supports working with batches of examples.\r\n- https://huggingface.co/docs/datasets/process.html#split-long-examples\r\n  > When your examples are too long, you may want to split them\r\n- https://huggingface.co/docs/datasets/process.html#data-augmentation\r\n  > With batch processing, you can even augment your dataset with additional examples.\r\n\r\nLess frequently used: **"item"**:\r\n- https://huggingface.co/docs/datasets/package_reference/main_classes.html#datasets.Dataset.add_item\r\n  > Add item to Dataset.\r\n\r\nOther term used in the docs (although less frequently) is **"sample"**. The advantage of this word is that it is also a verb, so we can use the parameter: "sample_by" (if you insist on using a verb instead of a noun).\r\n\r\nIn summary, these proposals:\r\n- config.row\r\n- config.example\r\n- config.item\r\n- config.sample\r\n- config.sample_by'
 "I like `sample_by`. Another idea I had was `separate_by`.\r\n\r\nIt could also be `sampling`, `sampling_method`, `separation_method`.\r\n\r\nNot a big fan of the proposed nouns alone since they are very generic, that's why I tried to have something more specific.\r\n\r\nI also agree that we actually should avoid `split` to avoid any confusion"
 'Thanks for the analysis of the used terms. I also like `sample_by` (`separate_by` is good too).'
 'Thank you !! :D ']
{'url': 'https://api.github.com/repos/huggingface/datasets/pulls/3442', 'html_url': 'https://github.com/huggingface/datasets/pull/3442', 'diff_url': 'https://github.com/huggingface/datasets/pull/3442.diff', 'patch_url': 'https://github.com/huggingface/datasets/pull/3442.patch', 'merged_at': datetime.datetime(2021, 12, 20, 16, 39, 18)}
Add `config.row` option to `text` module to allow yielding lines (default, current case), paragraphs or documents.

Feel free to comment on the name of the config parameter `row`:
- Currently, the docs state datasets are made of rows and columns
- Other names I considered: `example`, `item`
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/3442/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}