🐛 [Metrics] ROUGE is non-deterministic
> Hi, can you give a full self-contained example to reproduce this behavior?

There is a notebook in the post ;)
If I run the ROUGE metric 2 times, with same predictions / references, the scores are slightly different.

Refer to [this Colab notebook](https://colab.research.google.com/drive/1wRssNXgb9ldcp4ulwj-hMJn0ywhDOiDy?usp=sharing) for reproducing the problem.

Example of F-score for ROUGE-1, ROUGE-2, ROUGE-L in 2 differents run :

> ['0.3350', '0.1470', '0.2329']
['0.3358', '0.1451', '0.2332']

---

Why ROUGE is not deterministic ?
🐛 [Metrics] ROUGE is non-deterministic 
 If I run the ROUGE metric 2 times, with same predictions / references, the scores are slightly different.

Refer to [this Colab notebook](https://colab.research.google.com/drive/1wRssNXgb9ldcp4ulwj-hMJn0ywhDOiDy?usp=sharing) for reproducing the problem.

Example of F-score for ROUGE-1, ROUGE-2, ROUGE-L in 2 differents run :

> ['0.3350', '0.1470', '0.2329']
['0.3358', '0.1451', '0.2332']

---

Why ROUGE is not deterministic ? 
 > Hi, can you give a full self-contained example to reproduce this behavior?

There is a notebook in the post ;)