Allow loading community metrics from the hub, just like datasets
{'login': 'eladsegal', 'id': 13485709, 'node_id': 'MDQ6VXNlcjEzNDg1NzA5', 'avatar_url': 'https://avatars.githubusercontent.com/u/13485709?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/eladsegal', 'html_url': 'https://github.com/eladsegal', 'followers_url': 'https://api.github.com/users/eladsegal/followers', 'following_url': 'https://api.github.com/users/eladsegal/following{/other_user}', 'gists_url': 'https://api.github.com/users/eladsegal/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/eladsegal/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/eladsegal/subscriptions', 'organizations_url': 'https://api.github.com/users/eladsegal/orgs', 'repos_url': 'https://api.github.com/users/eladsegal/repos', 'events_url': 'https://api.github.com/users/eladsegal/events{/privacy}', 'received_events_url': 'https://api.github.com/users/eladsegal/received_events', 'type': 'User', 'site_admin': False}
[{'id': 1935892871, 'node_id': 'MDU6TGFiZWwxOTM1ODkyODcx', 'url': 'https://api.github.com/repos/huggingface/datasets/labels/enhancement', 'name': 'enhancement', 'color': 'a2eeef', 'default': True, 'description': 'New feature or request'}, {'id': 2067400324, 'node_id': 'MDU6TGFiZWwyMDY3NDAwMzI0', 'url': 'https://api.github.com/repos/huggingface/datasets/labels/generic%20discussion', 'name': 'generic discussion', 'color': 'c5def5', 'default': False, 'description': 'Generic discussion on the library'}]
[]
Hi ! Thanks for your message :) This is a great idea indeed. We haven't started working on this yet though. For now I guess you can host your metric on the Hub (either with your model or your dataset) and use `hf_hub_download` to download it (docs [here](https://github.com/huggingface/huggingface_hub/blob/main/docs/hub/how-to-downstream.md#cached_download)) This is a great solution in the meantime, thanks! Here's the code I used, in case it can be of help to someone else:
```python
import os, shutil
from huggingface_hub import hf_hub_download
def download_metric(repo_id, file_path):
    # repo_id: for models "username/model_name", for datasets "datasets/username/model_name"
    local_metric_path = hf_hub_download(repo_id=repo_id, filename=file_path)
    updated_local_metric_path = (os.path.dirname(local_metric_path) + os.path.basename(local_metric_path).replace(".", "_") + ".py")
    shutil.copy(local_metric_path, updated_local_metric_path)
    return updated_local_metric_path

metric = load_metric(download_metric(REPO_ID, FILE_PATH))
``` Solved with https://github.com/huggingface/evaluate 🤗  Yay!! cc @lvwerra @sashavor @douwekiela 

Please share your feedback @eladsegal =)
**Is your feature request related to a problem? Please describe.**
Currently, I can load a metric implemented by me by providing the local path to the file in `load_metric`.
However, there is no option to do it with the metric uploaded to the hub.
This means that if I want to allow other users to use it, they must download it first which makes the usage less smooth.

**Describe the solution you'd like**
Load metrics from the hub just like datasets are loaded.
In order to not break stuff, the convention can be to put the metric file in a "metrics" folder in the hub.


{'url': 'https://api.github.com/repos/huggingface/datasets/issues/3543/reactions', 'total_count': 1, '+1': 1, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}