Add comet metric
{'login': 'ricardorei', 'id': 17256847, 'node_id': 'MDQ6VXNlcjE3MjU2ODQ3', 'avatar_url': 'https://avatars.githubusercontent.com/u/17256847?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/ricardorei', 'html_url': 'https://github.com/ricardorei', 'followers_url': 'https://api.github.com/users/ricardorei/followers', 'following_url': 'https://api.github.com/users/ricardorei/following{/other_user}', 'gists_url': 'https://api.github.com/users/ricardorei/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/ricardorei/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/ricardorei/subscriptions', 'organizations_url': 'https://api.github.com/users/ricardorei/orgs', 'repos_url': 'https://api.github.com/users/ricardorei/repos', 'events_url': 'https://api.github.com/users/ricardorei/events{/privacy}', 'received_events_url': 'https://api.github.com/users/ricardorei/received_events', 'type': 'User', 'site_admin': False}
[]
[]
['I also thought a bit about the fact that "sources" can\'t be added to the batch.. but changing that would require a lot more changes. And I agree that the idea of adding them as part of the references is not ideal. Conceptually they are not references.\r\n\r\nI would keep it like this for now.. And in the future, work on a more consistent batch interface.']
{'url': 'https://api.github.com/repos/huggingface/datasets/pulls/1577', 'html_url': 'https://github.com/huggingface/datasets/pull/1577', 'diff_url': 'https://github.com/huggingface/datasets/pull/1577.diff', 'patch_url': 'https://github.com/huggingface/datasets/pull/1577.patch', 'merged_at': datetime.datetime(2021, 1, 14, 13, 33, 10)}
Hey! I decided to add our new Crosslingual Optimized Metric for Evaluation of Translation (COMET) to the list of the available metrics.

COMET was [presented at EMNLP20](https://www.aclweb.org/anthology/2020.emnlp-main.213/) and it is the highest performing metric, so far, on the WMT19 benchmark.

We also participated in the [WMT20 Metrics shared task ](http://www.statmt.org/wmt20/pdf/2020.wmt-1.101.pdf) where once again COMET was validated as a top-performing metric. 


I hope that this metric will help researcher's and industry workers to better validate their MT systems in the future 🤗 !

Cheers,
Ricardo




{'url': 'https://api.github.com/repos/huggingface/datasets/issues/1577/reactions', 'total_count': 2, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 2, 'rocket': 0, 'eyes': 0}