[firewalled env] OFFLINE mode
I'm on master, so using all the available bells and whistles already.

If you look at the common issues - it for example tries to look up files if they appear in `_PACKAGED_DATASETS_MODULES` which it shouldn't do.

--------------

Yes, there is a nuance to it. As I mentioned it's firewalled - that is it has a network but making any calls outside - it just hangs in:

```
sin_addr=inet_addr("xx.xx.xx.xx")}, [28->16]) = 0
close(5)                                = 0
socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_TCP) = 5
connect(5, {sa_family=AF_INET, sin_port=htons(3128), sin_addr=inet_addr("yy.yy.yy.yy")}, 16^C) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
```
until it times out.

That's why we need to be able to tell the software that there is no network to rely on even if there is one (good for testing too).

So what I'm thinking is that this is a simple matter of pre-ambling any network call wrappers with:

```
if HF_DATASETS_OFFLINE:
    assert "Attempting to make a network call under Offline mode"
```

and then fixing up if there is anything else to fix to make it work.

--------------

Otherwise I think the only other problem I encountered is that we need to find a way to pre-cache metrics, for some reason it's not caching it and wanting to fetch it from online.

Which is extra strange since it already has those files in the `datasets` repo itself that is on the filesystem.

The workaround I had to do is to copy `rouge/rouge.py` (with the parent folder) from the datasets repo to the current dir - and then it proceeded.
This issue comes from a need to be able to run `datasets` in a firewalled env, which currently makes the software hang until it times out, as it's unable to complete the network calls.

I propose the following approach to solving this problem, using the example of `run_seq2seq.py` as a sample program. There are 2 possible ways to going about it.

## 1. Manual

manually prepare data and metrics files, that is transfer to the firewalled instance the dataset and the metrics and run:

```
DATASETS_OFFLINE=1 run_seq2seq.py  --train_file xyz.csv --validation_file xyz.csv ...
```

`datasets` must not make any network calls and if there is a logic to do that and something is missing it should assert that this or that action requires network and therefore it can't proceed.

## 2. Automatic

In some clouds one can prepare a datastorage ahead of time with a normal networked environment but which doesn't have gpus and then one switches to the gpu instance which is firewalled, but it can access all the cached data. This is the ideal situation, since in this scenario we don't have to do anything manually, but simply run the same application twice:

1. on the non-firewalled instance:
```
run_seq2seq.py  --dataset_name wmt16 --dataset_config ro-en ...
```

which should download and cached everything.

2. and then immediately after on the firewalled instance, which shares the same filesystem
```
DATASETS_OFFLINE=1 run_seq2seq.py  --dataset_name wmt16 --dataset_config ro-en ...
```

and the metrics and datasets should be cached by the invocation number 1 and any network calls be skipped and if the logic is missing data it should assert and not try to fetch any data from online.

## Common Issues

1. for example currently `datasets` tries to look up online datasets if the files contain json or csv, despite the paths already provided

```
     if dataset and path in _PACKAGED_DATASETS_MODULES:
```

2. it has an issue with metrics. e.g. I had to manually copy `rouge/rouge.py` from the `datasets` repo to the current dir - or it was hanging.

I had to comment out `head_hf_s3(...)` calls to make things work. So all those `try: head_hf_s3(...)` shouldn't be tried with `DATASETS_OFFLINE=1`

Here is the corresponding issue for `transformers`: https://github.com/huggingface/transformers/issues/10379

Thanks.
[firewalled env] OFFLINE mode 
 This issue comes from a need to be able to run `datasets` in a firewalled env, which currently makes the software hang until it times out, as it's unable to complete the network calls.

I propose the following approach to solving this problem, using the example of `run_seq2seq.py` as a sample program. There are 2 possible ways to going about it.

## 1. Manual

manually prepare data and metrics files, that is transfer to the firewalled instance the dataset and the metrics and run:

```
DATASETS_OFFLINE=1 run_seq2seq.py  --train_file xyz.csv --validation_file xyz.csv ...
```

`datasets` must not make any network calls and if there is a logic to do that and something is missing it should assert that this or that action requires network and therefore it can't proceed.

## 2. Automatic

In some clouds one can prepare a datastorage ahead of time with a normal networked environment but which doesn't have gpus and then one switches to the gpu instance which is firewalled, but it can access all the cached data. This is the ideal situation, since in this scenario we don't have to do anything manually, but simply run the same application twice:

1. on the non-firewalled instance:
```
run_seq2seq.py  --dataset_name wmt16 --dataset_config ro-en ...
```

which should download and cached everything.

2. and then immediately after on the firewalled instance, which shares the same filesystem
```
DATASETS_OFFLINE=1 run_seq2seq.py  --dataset_name wmt16 --dataset_config ro-en ...
```

and the metrics and datasets should be cached by the invocation number 1 and any network calls be skipped and if the logic is missing data it should assert and not try to fetch any data from online.

## Common Issues

1. for example currently `datasets` tries to look up online datasets if the files contain json or csv, despite the paths already provided

```
     if dataset and path in _PACKAGED_DATASETS_MODULES:
```

2. it has an issue with metrics. e.g. I had to manually copy `rouge/rouge.py` from the `datasets` repo to the current dir - or it was hanging.

I had to comment out `head_hf_s3(...)` calls to make things work. So all those `try: head_hf_s3(...)` shouldn't be tried with `DATASETS_OFFLINE=1`

Here is the corresponding issue for `transformers`: https://github.com/huggingface/transformers/issues/10379

Thanks. 
 I'm on master, so using all the available bells and whistles already.

If you look at the common issues - it for example tries to look up files if they appear in `_PACKAGED_DATASETS_MODULES` which it shouldn't do.

--------------

Yes, there is a nuance to it. As I mentioned it's firewalled - that is it has a network but making any calls outside - it just hangs in:

```
sin_addr=inet_addr("xx.xx.xx.xx")}, [28->16]) = 0
close(5)                                = 0
socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_TCP) = 5
connect(5, {sa_family=AF_INET, sin_port=htons(3128), sin_addr=inet_addr("yy.yy.yy.yy")}, 16^C) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
```
until it times out.

That's why we need to be able to tell the software that there is no network to rely on even if there is one (good for testing too).

So what I'm thinking is that this is a simple matter of pre-ambling any network call wrappers with:

```
if HF_DATASETS_OFFLINE:
    assert "Attempting to make a network call under Offline mode"
```

and then fixing up if there is anything else to fix to make it work.

--------------

Otherwise I think the only other problem I encountered is that we need to find a way to pre-cache metrics, for some reason it's not caching it and wanting to fetch it from online.

Which is extra strange since it already has those files in the `datasets` repo itself that is on the filesystem.

The workaround I had to do is to copy `rouge/rouge.py` (with the parent folder) from the datasets repo to the current dir - and then it proceeded.