documentation missing how to split a dataset
We added a way to shuffle datasets (shuffle the indices and then reorder to make a new dataset).
You can do `shuffled_dset = dataset.shuffle(seed=my_seed)`. It shuffles the whole dataset.
There is also `dataset.train_test_split()` which if very handy (with the same signature as sklearn).

Closing this issue as we added the docs for splits and tools to split datasets. Thanks again for your feedback !
I am trying to understand how to split a dataset ( as arrow_dataset). 
I know I can do something like this to access a split which is already in the original dataset : 

`ds_test = nlp.load_dataset('imdb, split='test') `

But how can I split ds_test into a test and a validation set (without reading the data into memory and keeping the arrow_dataset as container)?
I guess it has something to do with the module split :-) but there is no real documentation in the code but only a reference to a longer description: 

> See the  [guide on splits](https://github.com/huggingface/nlp/tree/master/docs/splits.md)  for more information.

But the guide seems to be missing.

To clarify: I know that this has been modelled after the dataset of tensorflow and that some of the documentation there can be used [like this one](https://www.tensorflow.org/datasets/splits). But to come back to the example above: I cannot simply split the testset doing this: 
`ds_test = nlp.load_dataset('imdb, split='test'[:5000]) `
`ds_val = nlp.load_dataset('imdb, split='test'[5000:])`

because the imdb test data is sorted by class (probably not a good idea anyway)

documentation missing how to split a dataset 
 I am trying to understand how to split a dataset ( as arrow_dataset). 
I know I can do something like this to access a split which is already in the original dataset : 

`ds_test = nlp.load_dataset('imdb, split='test') `

But how can I split ds_test into a test and a validation set (without reading the data into memory and keeping the arrow_dataset as container)?
I guess it has something to do with the module split :-) but there is no real documentation in the code but only a reference to a longer description: 

> See the  [guide on splits](https://github.com/huggingface/nlp/tree/master/docs/splits.md)  for more information.

But the guide seems to be missing.

To clarify: I know that this has been modelled after the dataset of tensorflow and that some of the documentation there can be used [like this one](https://www.tensorflow.org/datasets/splits). But to come back to the example above: I cannot simply split the testset doing this: 
`ds_test = nlp.load_dataset('imdb, split='test'[:5000]) `
`ds_val = nlp.load_dataset('imdb, split='test'[5000:])`

because the imdb test data is sorted by class (probably not a good idea anyway)
 
 We added a way to shuffle datasets (shuffle the indices and then reorder to make a new dataset).
You can do `shuffled_dset = dataset.shuffle(seed=my_seed)`. It shuffles the whole dataset.
There is also `dataset.train_test_split()` which if very handy (with the same signature as sklearn).

Closing this issue as we added the docs for splits and tools to split datasets. Thanks again for your feedback !