[experiment] missing default_experiment-1-0.arrow
Thank you for explaining that in a great way, @lhoestq 

So the bottom line is that the `transformers` examples are broken since they don't do any of that. At least `run_seq2seq.py` just does `metric = load_metric(metric_name)`

What test would you recommend to reliably reproduce this bug in `examples/seq2seq/run_seq2seq.py`?
the original report was pretty bad and incomplete - my apologies!

Please see the complete version here: https://github.com/huggingface/datasets/issues/1942#issuecomment-786336481

------------

As mentioned here https://github.com/huggingface/datasets/issues/1939 metrics don't get cached, looking at my local `~/.cache/huggingface/metrics` - there are many `*.arrow.lock` files but zero metrics files.

w/o the network I get:
```
FileNotFoundError: [Errno 2] No such file or directory: '~/.cache/huggingface/metrics/sacrebleu/default/default_experiment-1-0.arrow
```
there is just `~/.cache/huggingface/metrics/sacrebleu/default/default_experiment-1-0.arrow.lock`

I did run the same `run_seq2seq.py` script on the instance with network and it worked just fine, but only the lock file was left behind.

this is with master.

Thank you.
[experiment] missing default_experiment-1-0.arrow 
 the original report was pretty bad and incomplete - my apologies!

Please see the complete version here: https://github.com/huggingface/datasets/issues/1942#issuecomment-786336481

------------

As mentioned here https://github.com/huggingface/datasets/issues/1939 metrics don't get cached, looking at my local `~/.cache/huggingface/metrics` - there are many `*.arrow.lock` files but zero metrics files.

w/o the network I get:
```
FileNotFoundError: [Errno 2] No such file or directory: '~/.cache/huggingface/metrics/sacrebleu/default/default_experiment-1-0.arrow
```
there is just `~/.cache/huggingface/metrics/sacrebleu/default/default_experiment-1-0.arrow.lock`

I did run the same `run_seq2seq.py` script on the instance with network and it worked just fine, but only the lock file was left behind.

this is with master.

Thank you. 
 Thank you for explaining that in a great way, @lhoestq 

So the bottom line is that the `transformers` examples are broken since they don't do any of that. At least `run_seq2seq.py` just does `metric = load_metric(metric_name)`

What test would you recommend to reliably reproduce this bug in `examples/seq2seq/run_seq2seq.py`?