Add `remove_columns` to `IterableDataset`
{'login': 'cccntu', 'id': 31893406, 'node_id': 'MDQ6VXNlcjMxODkzNDA2', 'avatar_url': 'https://avatars.githubusercontent.com/u/31893406?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/cccntu', 'html_url': 'https://github.com/cccntu', 'followers_url': 'https://api.github.com/users/cccntu/followers', 'following_url': 'https://api.github.com/users/cccntu/following{/other_user}', 'gists_url': 'https://api.github.com/users/cccntu/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/cccntu/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/cccntu/subscriptions', 'organizations_url': 'https://api.github.com/users/cccntu/orgs', 'repos_url': 'https://api.github.com/users/cccntu/repos', 'events_url': 'https://api.github.com/users/cccntu/events{/privacy}', 'received_events_url': 'https://api.github.com/users/cccntu/received_events', 'type': 'User', 'site_admin': False}
[]
[]
Thanks ! That looks all good :)

I don't think that batching would help. Indeed we're dealing with python iterators that yield elements one by one, so batched `map` needs to accumulate a batch, apply the function, and then yield examples from the batch.

Though once we have parallel processing in `map`, we can reconsider it

Also feel free to check the CI failure - apparently the import of `Union` is missing Thanks for the review and explaining that! 
On top of what you said, I think `remove_columns` is very unlikely to be a bottleneck, so it doesn't matter anyways. Thank you for reviewing! @mariosasko 

I wonder how the checking would work. Is there any checking present in `IterableDataset ` now? What if `.remove_columns()` is applied after some arbitrary `.map()`? > I wonder how the checking would work. Is there any checking present in IterableDataset now? What if .remove_columns() is applied after some arbitrary .map()?

That's the challenge here indeed ^^ In this case it's not trivial to know the names of the columns. Feel free to open an issue so we can discuss this
Fixes #2944

WIP
* Not tested yet.
* We might want to allow batched remove for efficiency.

@lhoestq Do you think it should have `batched=` and `batch_size=`?
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/3030/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}
{'url': 'https://api.github.com/repos/huggingface/datasets/pulls/3030', 'html_url': 'https://github.com/huggingface/datasets/pull/3030', 'diff_url': 'https://github.com/huggingface/datasets/pull/3030.diff', 'patch_url': 'https://github.com/huggingface/datasets/pull/3030.patch', 'merged_at': '2021-10-08T15:31:53Z'}