Add raw data files to the Hub with GitHub LFS for canonical dataset
{'login': 'zlucia', 'id': 40370937, 'node_id': 'MDQ6VXNlcjQwMzcwOTM3', 'avatar_url': 'https://avatars.githubusercontent.com/u/40370937?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/zlucia', 'html_url': 'https://github.com/zlucia', 'followers_url': 'https://api.github.com/users/zlucia/followers', 'following_url': 'https://api.github.com/users/zlucia/following{/other_user}', 'gists_url': 'https://api.github.com/users/zlucia/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/zlucia/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/zlucia/subscriptions', 'organizations_url': 'https://api.github.com/users/zlucia/orgs', 'repos_url': 'https://api.github.com/users/zlucia/repos', 'events_url': 'https://api.github.com/users/zlucia/events{/privacy}', 'received_events_url': 'https://api.github.com/users/zlucia/received_events', 'type': 'User', 'site_admin': False}
[]
{'login': 'albertvillanova', 'id': 8515462, 'node_id': 'MDQ6VXNlcjg1MTU0NjI=', 'avatar_url': 'https://avatars.githubusercontent.com/u/8515462?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/albertvillanova', 'html_url': 'https://github.com/albertvillanova', 'followers_url': 'https://api.github.com/users/albertvillanova/followers', 'following_url': 'https://api.github.com/users/albertvillanova/following{/other_user}', 'gists_url': 'https://api.github.com/users/albertvillanova/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/albertvillanova/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/albertvillanova/subscriptions', 'organizations_url': 'https://api.github.com/users/albertvillanova/orgs', 'repos_url': 'https://api.github.com/users/albertvillanova/repos', 'events_url': 'https://api.github.com/users/albertvillanova/events{/privacy}', 'received_events_url': 'https://api.github.com/users/albertvillanova/received_events', 'type': 'User', 'site_admin': False}
[{'login': 'albertvillanova', 'id': 8515462, 'node_id': 'MDQ6VXNlcjg1MTU0NjI=', 'avatar_url': 'https://avatars.githubusercontent.com/u/8515462?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/albertvillanova', 'html_url': 'https://github.com/albertvillanova', 'followers_url': 'https://api.github.com/users/albertvillanova/followers', 'following_url': 'https://api.github.com/users/albertvillanova/following{/other_user}', 'gists_url': 'https://api.github.com/users/albertvillanova/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/albertvillanova/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/albertvillanova/subscriptions', 'organizations_url': 'https://api.github.com/users/albertvillanova/orgs', 'repos_url': 'https://api.github.com/users/albertvillanova/repos', 'events_url': 'https://api.github.com/users/albertvillanova/events{/privacy}', 'received_events_url': 'https://api.github.com/users/albertvillanova/received_events', 'type': 'User', 'site_admin': False}]
['Hi @zlucia, I would actually suggest hosting the dataset as a huggingface.co-hosted dataset.\r\n\r\nThe only difference with a "canonical"/legacy dataset is that it\'s nested under an organization (here `stanford` or `stanfordnlp` for instance – completely up to you) but then you can upload your data using git-lfs (unlike "canonical" datasets where we don\'t host the data)\r\n\r\nLet me know if this fits your use case!\r\n\r\ncc\'ing @osanseviero @lhoestq and rest of the team 🤗'
 'Hi @zlucia,\r\n\r\nAs @julien-c pointed out, the way to store/host raw data files in our Hub is by using what we call "community" datasets:\r\n- either at your personal namespace: `load_dataset("zlucia/casehold")`\r\n- or at an organization namespace: for example, if you create the organization `reglab`, then `load_dataset("reglab/casehold")`\r\n\r\nPlease note that "canonical" datasets do not normally store/host their raw data at our Hub, but in a third-party server. For "canonical" datasets, we just host the "loading script", that is, a Python script that downloads the raw data from a third-party server, creates the HuggingFace dataset from it and caches it locally.\r\n\r\nIn order to create an organization namespace in our Hub, please follow this link: https://huggingface.co/organizations/new\r\n\r\nThere are already many organizations at our Hub (complete list here: https://huggingface.co/organizations), such as:\r\n- Stanford CRFM: https://huggingface.co/stanford-crfm\r\n- Stanford NLP: https://huggingface.co/stanfordnlp\r\n- Stanford CS329S: Machine Learning Systems Design: https://huggingface.co/stanford-cs329s\r\n\r\nAlso note that you in your organization namespace:\r\n- you can add any number of members\r\n- you can store both raw datasets and models, and those can be immediately accessed using `datasets` and `transformers`\r\n\r\nOnce you have created an organization, these are the steps to upload/host a raw dataset: \r\n- The no-code procedure: https://huggingface.co/docs/datasets/upload_dataset.html\r\n- Using the command line (terminal): https://huggingface.co/docs/datasets/share.html#add-a-community-dataset\r\n\r\nPlease, feel free to ping me if you have any further questions or need help.\r\n'
 "Ah I see, I think I was unclear whether there were benefits to uploading a canonical dataset vs. a community provided dataset. Thanks for clarifying. I'll see if we want to create an organization namespace and otherwise, will upload the dataset under my personal namespace."]
I'm interested in sharing the CaseHOLD dataset (https://arxiv.org/abs/2104.08671) as a canonical dataset on the HuggingFace Hub and would like to add the raw data files to the Hub with GitHub LFS, since it seems like a more sustainable long term storage solution, compared to other storage solutions available to my team. From what I can tell, this option is not immediately supported if one follows the sharing steps detailed here: [https://huggingface.co/docs/datasets/share_dataset.html#sharing-a-canonical-dataset](https://huggingface.co/docs/datasets/share_dataset.html#sharing-a-canonical-dataset), since GitHub LFS is not supported for public forks. Is there a way to request this? Thanks!
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/3164/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}