[experiment] missing default_experiment-1-0.arrow
When you're using metrics in a distributed setup, there are two cases:
1. you're doing two completely different experiments (two evaluations) and the 2 metrics jobs have nothing to do with each other
2. you're doing one experiment (one evaluation) but use multiple processes to feed the data to the metric.

In case 1. you just need to provide two different `experiment_id` so that the metrics don't collide.
In case 2. they must have the same experiment_id (or use the default one), but in this case you also need to provide the `num_processes` and `process_id`

If understand correctly you're in situation 2.

If so, you make sure that you instantiate the metrics with both the right `num_processes` and `process_id` parameters ?

If they're not set, then the cache files of the two metrics collide it can cause issues. For example if one metric finishes before the other, then the cache file is deleted and the other metric gets a FileNotFoundError
There's more information in the [documentation](https://huggingface.co/docs/datasets/loading_metrics.html#distributed-setups) if you want

Hope that helps !
the original report was pretty bad and incomplete - my apologies!

Please see the complete version here: https://github.com/huggingface/datasets/issues/1942#issuecomment-786336481

------------

As mentioned here https://github.com/huggingface/datasets/issues/1939 metrics don't get cached, looking at my local `~/.cache/huggingface/metrics` - there are many `*.arrow.lock` files but zero metrics files.

w/o the network I get:
```
FileNotFoundError: [Errno 2] No such file or directory: '~/.cache/huggingface/metrics/sacrebleu/default/default_experiment-1-0.arrow
```
there is just `~/.cache/huggingface/metrics/sacrebleu/default/default_experiment-1-0.arrow.lock`

I did run the same `run_seq2seq.py` script on the instance with network and it worked just fine, but only the lock file was left behind.

this is with master.

Thank you.
[experiment] missing default_experiment-1-0.arrow 
 the original report was pretty bad and incomplete - my apologies!

Please see the complete version here: https://github.com/huggingface/datasets/issues/1942#issuecomment-786336481

------------

As mentioned here https://github.com/huggingface/datasets/issues/1939 metrics don't get cached, looking at my local `~/.cache/huggingface/metrics` - there are many `*.arrow.lock` files but zero metrics files.

w/o the network I get:
```
FileNotFoundError: [Errno 2] No such file or directory: '~/.cache/huggingface/metrics/sacrebleu/default/default_experiment-1-0.arrow
```
there is just `~/.cache/huggingface/metrics/sacrebleu/default/default_experiment-1-0.arrow.lock`

I did run the same `run_seq2seq.py` script on the instance with network and it worked just fine, but only the lock file was left behind.

this is with master.

Thank you. 
 When you're using metrics in a distributed setup, there are two cases:
1. you're doing two completely different experiments (two evaluations) and the 2 metrics jobs have nothing to do with each other
2. you're doing one experiment (one evaluation) but use multiple processes to feed the data to the metric.

In case 1. you just need to provide two different `experiment_id` so that the metrics don't collide.
In case 2. they must have the same experiment_id (or use the default one), but in this case you also need to provide the `num_processes` and `process_id`

If understand correctly you're in situation 2.

If so, you make sure that you instantiate the metrics with both the right `num_processes` and `process_id` parameters ?

If they're not set, then the cache files of the two metrics collide it can cause issues. For example if one metric finishes before the other, then the cache file is deleted and the other metric gets a FileNotFoundError
There's more information in the [documentation](https://huggingface.co/docs/datasets/loading_metrics.html#distributed-setups) if you want

Hope that helps !