The Beginners Guide to Cleaning a Dataset
โข
21
Caleb Fahlgren PRO
cfahlgren1
AI & ML interests
None yet
Recent Activity
New activity
about 16 hours ago
open-acc/README
liked
a dataset
about 23 hours ago
duckdb-nsql-hub/duckdb-docs
updated
a dataset
about 23 hours ago
duckdb-nsql-hub/duckdb-docs
Articles
Organizations
cfahlgren1's activity
replied to
their
post
1 day ago
or shakespeare ๐คฃ @not-lain
posted
an
update
1 day ago
Post
605
You can create charts, leaderboards, and filters on top of any Hugging Face dataset in less than a minute
โข ASCII Bar Charts ๐
โข Powered by DuckDB WASM โก
โข Download results to Parquet ๐ฝ
โข Embed and Share results with friends ๐ฌ
Do you have any interesting queries?
โข ASCII Bar Charts ๐
โข Powered by DuckDB WASM โก
โข Download results to Parquet ๐ฝ
โข Embed and Share results with friends ๐ฌ
Do you have any interesting queries?
reacted to
davanstrien's
post with โค๏ธ
1 day ago
Post
1016
huggingface.co/DIBT is dead! Long live https://huggingface.co/data-is-better-together!
We're working on some very cool projects so we're doing a bit of tidying of the Data is Better Together Hub org ๐ค
replied to
their
post
1 day ago
Heavy is the head that wears the crown
reacted to
fracapuano's
post with โค๏ธ
1 day ago
Post
927
Sharing what we have built over the course of the weekend at the
@llamameta
hackathon, by Cerebral Valley in London ๐ฌ๐ง ๐
@gabrycina @calebgcc and I competed with 200+ participants and 50+ teams for a 24-hrs sprint centered around hacking for impact! We focused on applications of robotics to those in need of assisted living, moving our focus to enable greater autonomy and accessibility of robotics in everyday life.
complete list of assets ๐
๐ค trained robotics policies
v1:
- fracapuano/moss-pills
- fracapuano/moss-cup
v2:
- fracapuano/meta-grasp
๐ค datasets
v1:
- fracapuano/pills
- fracapuano/cup
v2:
- fracapuano/cupim
You can find a live demo of our submission at: https://x.com/_fracapuano/status/1858102728691458554
If you want to know more about how we collected 100GB+ of data, trained multiple RL-policies using @lerobot and used Llama-3.2 models to handle user interactions and switch between tasks, go ahead and have a look! Also, don't be a stranger, and reach out ๐ฆพ
Our project is fully open-source, for the community (and ourselves, ๐จโ๐ณ) to build! A huge thank you to @cadene for the help (and the robot ๐คญ) - truly feeling these hugs-vibes ๐ค , and to @thomwolf and @clem for sharing our work across
Little extra:
โก๏ธ Our ๐ง EEG waves๐ง -based control of the ๐ฆพrobotic arm๐ฆพ
@gabrycina @calebgcc and I competed with 200+ participants and 50+ teams for a 24-hrs sprint centered around hacking for impact! We focused on applications of robotics to those in need of assisted living, moving our focus to enable greater autonomy and accessibility of robotics in everyday life.
complete list of assets ๐
๐ค trained robotics policies
v1:
- fracapuano/moss-pills
- fracapuano/moss-cup
v2:
- fracapuano/meta-grasp
๐ค datasets
v1:
- fracapuano/pills
- fracapuano/cup
v2:
- fracapuano/cupim
You can find a live demo of our submission at: https://x.com/_fracapuano/status/1858102728691458554
If you want to know more about how we collected 100GB+ of data, trained multiple RL-policies using @lerobot and used Llama-3.2 models to handle user interactions and switch between tasks, go ahead and have a look! Also, don't be a stranger, and reach out ๐ฆพ
Our project is fully open-source, for the community (and ourselves, ๐จโ๐ณ) to build! A huge thank you to @cadene for the help (and the robot ๐คญ) - truly feeling these hugs-vibes ๐ค , and to @thomwolf and @clem for sharing our work across
Little extra:
โก๏ธ Our ๐ง EEG waves๐ง -based control of the ๐ฆพrobotic arm๐ฆพ
reacted to
LukeNeumann's
post with ๐คฏ
1 day ago
Post
1005
Nine years ago, I uploaded the first 8K resolution video to YouTube and I've been stockpiling 8K footage ever since: https://www.youtube.com/watch?v=sLprVF6d7Ug&t
Should @Overlaiapp release the first open-source 8K video dataset?
Could anyone even fine tune a model with this?๐
Should @Overlaiapp release the first open-source 8K video dataset?
Could anyone even fine tune a model with this?๐
-
- 6 replies
replied to
LukeNeumann's
post
1 day ago
Would be massive! Let us know if you need any help ๐ค
reacted to
dvilasuero's
post with ๐๐ค
1 day ago
Post
962
@Jesse-marqo
and the Marqo team are killing it on the Hub: top embedding models and datasets!
Here's how to start using their new evaluation dataset for curation and labelling:
1. Deploy Argilla on Spaces: https://huggingface.co/new-space?template=argilla%2Fargilla-template-space
2. Load Marqo/amazon-products-eval with the UI wizard.
3. Start curating!
Here's how to start using their new evaluation dataset for curation and labelling:
1. Deploy Argilla on Spaces: https://huggingface.co/new-space?template=argilla%2Fargilla-template-space
2. Load Marqo/amazon-products-eval with the UI wizard.
3. Start curating!
posted
an
update
1 day ago
Post
557
What rank are you on Hugging Face Top Yappers? ๐ฃ๏ธ
Find your rank here with this link: cfahlgren1/hub-stats
The Top 3:
- @fdaudens
- @singhsidhukuldeep
- @akhaliq
I am at #71 and need to get my numbers up! ๐
Find your rank here with this link: cfahlgren1/hub-stats
The Top 3:
- @fdaudens
- @singhsidhukuldeep
- @akhaliq
I am at #71 and need to get my numbers up! ๐
-
- 4 replies
reacted to
nyuuzyou's
post with ๐ฅ
3 days ago
Post
880
๐ผ๏ธ Introducing Public Domain Pictures Dataset -
nyuuzyou/publicdomainpictures
Dataset highlights:
- 644,412 public domain images with comprehensive metadata from publicdomainpictures.net
- English language metadata including titles, descriptions, and keywords
- Each entry contains rich metadata including:
- Unique image ID and full-size image URLs
- Detailed titles and descriptions
- Keyword/tag collections
- Creator attribution
- Released to the public domain under Creative Commons Zero (CC0) license
Dataset highlights:
- 644,412 public domain images with comprehensive metadata from publicdomainpictures.net
- English language metadata including titles, descriptions, and keywords
- Each entry contains rich metadata including:
- Unique image ID and full-size image URLs
- Detailed titles and descriptions
- Keyword/tag collections
- Creator attribution
- Released to the public domain under Creative Commons Zero (CC0) license
- 2 replies
posted
an
update
3 days ago
Post
2030
You can clean and format datasets entirely in the browser with a few lines of SQL.
In this post, I replicate the process @mlabonne used to clean the new microsoft/orca-agentinstruct-1M-v1 dataset.
The cleaning process consists of:
- Joining the separate splits together / add split column
- Converting string messages into list of structs
- Removing empty system prompts
https://huggingface.co/blog/cfahlgren1/the-beginners-guide-to-cleaning-a-dataset
Here's his new cleaned dataset: mlabonne/orca-agentinstruct-1M-v1-cleaned
In this post, I replicate the process @mlabonne used to clean the new microsoft/orca-agentinstruct-1M-v1 dataset.
The cleaning process consists of:
- Joining the separate splits together / add split column
- Converting string messages into list of structs
- Removing empty system prompts
https://huggingface.co/blog/cfahlgren1/the-beginners-guide-to-cleaning-a-dataset
Here's his new cleaned dataset: mlabonne/orca-agentinstruct-1M-v1-cleaned
- 1 reply
There is no harm, but if I turn the notification on or off for an unapproved gated repo, an error message will appear.
Hey @John6666 , this should be fixed very soon.
reacted to
erikkaum's
post with ๐ฅ
3 days ago
Post
1611
A while ago I started experimenting with compiling the Python interpreter to WASM.
To build a secure, fast, and lightweight sandbox for code execution โ ideal for running LLM-generated Python code.
- Send code simply as a POST request
- 1-2ms startup times
Hack away:
https://github.com/ErikKaum/runner
To build a secure, fast, and lightweight sandbox for code execution โ ideal for running LLM-generated Python code.
- Send code simply as a POST request
- 1-2ms startup times
Hack away:
https://github.com/ErikKaum/runner
posted
an
update
6 days ago
Post
2190
Why use Google Drive when you can have:
โข Free storage with generous limits๐
โข Dataset Viewer (Sorting, Filtering, FTS) ๐
โข Third Party Library Support
โข SQL Console ๐ง
โข Security ๐
โข Community, Reach, and Visibility ๐
It's a no brainer!
Check out our post on what you get instantly out of the box when you create a dataset.
https://huggingface.co/blog/researcher-dataset-sharing
โข Free storage with generous limits๐
โข Dataset Viewer (Sorting, Filtering, FTS) ๐
โข Third Party Library Support
โข SQL Console ๐ง
โข Security ๐
โข Community, Reach, and Visibility ๐
It's a no brainer!
Check out our post on what you get instantly out of the box when you create a dataset.
https://huggingface.co/blog/researcher-dataset-sharing
- 1 reply
Yeah for sure! Would be cool to see links to the leaderboards of these to see more than top 5 and see where most of the community is ๐ @maxiw
Maybe like top 100 or top 500 with sql console saved link
This is cool, here's a list of the top most commented posts!
https://huggingface.co/datasets/maxiw/hf-posts/embed/sql-console/_ppKBx4
Qwen2.5 Coder + SQL Console ๐๐ค
wdyt? @victor
reacted to
m-ric's
post with ๐๐ฅ
7 days ago
Post
3624
๐ง๐ต๐ฒ ๐ป๐ฒ๐
๐ ๐ฏ๐ถ๐ด ๐๐ผ๐ฐ๐ถ๐ฎ๐น ๐ป๐ฒ๐๐๐ผ๐ฟ๐ธ ๐ถ๐ ๐ป๐ผ๐ ๐ฆ, ๐ถ๐'๐ ๐๐๐ฏ ๐ฃ๐ผ๐๐๐! [INSERT STONKS MEME WITH LASER EYES]
See below: I got 105k impressions since regularly posting Hub Posts, coming close to my 275k on Twitter!
โ๏ธ Computed with the great dataset maxiw/hf-posts
โ๏ธ Thanks to Qwen2.5-Coder-32B for showing me how to access dict attributes in a SQL request!
cc @merve who's far in front of me
See below: I got 105k impressions since regularly posting Hub Posts, coming close to my 275k on Twitter!
โ๏ธ Computed with the great dataset maxiw/hf-posts
โ๏ธ Thanks to Qwen2.5-Coder-32B for showing me how to access dict attributes in a SQL request!
cc @merve who's far in front of me
-
- 9 replies