Add perplexity to metrics
{'login': 'emibaylor', 'id': 27527747, 'node_id': 'MDQ6VXNlcjI3NTI3NzQ3', 'avatar_url': 'https://avatars.githubusercontent.com/u/27527747?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/emibaylor', 'html_url': 'https://github.com/emibaylor', 'followers_url': 'https://api.github.com/users/emibaylor/followers', 'following_url': 'https://api.github.com/users/emibaylor/following{/other_user}', 'gists_url': 'https://api.github.com/users/emibaylor/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/emibaylor/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/emibaylor/subscriptions', 'organizations_url': 'https://api.github.com/users/emibaylor/orgs', 'repos_url': 'https://api.github.com/users/emibaylor/repos', 'events_url': 'https://api.github.com/users/emibaylor/events{/privacy}', 'received_events_url': 'https://api.github.com/users/emibaylor/received_events', 'type': 'User', 'site_admin': False}
[]
[]
["Awesome thank you ! The implementation of the parent `Metric` class was assuming that all metrics were supposed to have references/predictions pairs - I just changed that so you don't have to override `compute()`. I took the liberty of doing the changes directly inside this PR to make sure it works as expected with perplexity.\r\n\r\nOther than that it looks in pretty good shape :) I just did minor changes like remove a remaining `print` as well as fixing the `Features` defined in `_info()`. I also renamed `input_text` to `input_texts` since it makes it more obvious that it's a list of strings - let me know if it sounds good to you.\r\n\r\nLet me know if you'd like to make other changes or if it's all good for you !"
 'The test with the full test set seems to take too much time in the CI - maybe we can just select `split="test[:few_examples]"` (around 10 maybe ?)']
{'url': 'https://api.github.com/repos/huggingface/datasets/pulls/3757', 'html_url': 'https://github.com/huggingface/datasets/pull/3757', 'diff_url': 'https://github.com/huggingface/datasets/pull/3757.diff', 'patch_url': 'https://github.com/huggingface/datasets/pull/3757.patch', 'merged_at': datetime.datetime(2022, 2, 25, 17, 13, 34)}
Adding perplexity metric

This code differs from the code in [this](https://huggingface.co/docs/transformers/perplexity) HF blog post because the blogpost code fails in at least the following circumstances:
- returns nans whenever the stride = 1
- hits a runtime error when the stride is significantly larger than the max model length (e.g. if max_model_length = 512 and stride = 1024)


Note that:
- As it is, it only works for causal models. Pseudoperplexity can be added later as another metric to work with masked language models.
- It takes in a list of strings so that it can be dataset independent. This does mean that it doesn't currently batch inputs, and is therefore relatively slow.
- It overwrites the metrics compute() function for a specific perplexity compute() function. This is because the current general metrics compute() function requires model-generated predictions, which doesn't make sense in the context of perplexity
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/3757/reactions', 'total_count': 1, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 1, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}