Running
112
๐
GPU Poor LLM Arena
Compact LLM Battle Arena: Frugal AI Face-Off!
Michael Pieler
MicPie
AI & ML interests
ML
Recent Activity
liked
a Space
about 1 month ago
k-mktr/gpu-poor-llm-arena
Organizations
MicPie's activity
liked
a
Space
about 1 month ago
upvoted
a
collection
3 months ago
liked
2
Spaces
4 months ago
reacted to
m-ric's
post with ๐
4 months ago
Post
2252
๐ง๐ต๐ฒ ๐ต๐๐ด๐ฒ ๐ฐ๐ผ๐๐ ๐ผ๐ณ ๐ฟ๐ฒ๐๐ฒ๐ฎ๐ฟ๐ฐ๐ต ๐ผ๐ป ๐ณ๐ฟ๐ผ๐ป๐๐ถ๐ฒ๐ฟ ๐๐๐ ๐ ๐ธ
Google DeepMind recently released a great paper that shows optimal hyperparameters to train across different regimes: Scaling Exponents Across Parameterizations and Optimizers, with data from 10,000 training runs.
One engineer decided to quantify the price of such a large-scale experiment.
๐ฌ And the bill is hefty: ~13M USD
This exact number is to take with a grain of salt because many approximations were necessary to get the final result.
โ๏ธ But still this ballpark means that for this sole experiment, the price is way over what most startups or research labs could afford.
This means that open-sourcing research is more important than ever, to put everyone in the ecosystem on a roughly equal footing. Don't let OpenAI run first, they'll keep everything for themselves!
Read the full post that quantifies the paper's cost ๐ https://152334h.github.io/blog/scaling-exponents/
Google DeepMind recently released a great paper that shows optimal hyperparameters to train across different regimes: Scaling Exponents Across Parameterizations and Optimizers, with data from 10,000 training runs.
One engineer decided to quantify the price of such a large-scale experiment.
๐ฌ And the bill is hefty: ~13M USD
This exact number is to take with a grain of salt because many approximations were necessary to get the final result.
โ๏ธ But still this ballpark means that for this sole experiment, the price is way over what most startups or research labs could afford.
This means that open-sourcing research is more important than ever, to put everyone in the ecosystem on a roughly equal footing. Don't let OpenAI run first, they'll keep everything for themselves!
Read the full post that quantifies the paper's cost ๐ https://152334h.github.io/blog/scaling-exponents/
- 1 reply
100 word justifications from Llama3 annotations
2
#3 opened 6 months ago
by
MicPie
MicPie
liked
a
dataset
6 months ago
liked
a
Space
6 months ago
upvoted
a
collection
7 months ago
liked
a
model
7 months ago
Training details?
1
#2 opened 7 months ago
by
MicPie
MicPie
authored
4
papers
7 months ago
Stable LM 2 1.6B Technical Report
Paper
โข
2402.17834
โข
Published
โข
3
Are large language models superhuman chemists?
Paper
โข
2404.01475
โข
Published
โข
16
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
Paper
โข
2204.06745
โข
Published
โข
1
Few-shot Adaptation Works with UnpredicTable Data
Paper
โข
2208.01009
โข
Published
Domain and provenance annotation
9
#1 opened about 1 year ago
by
haukur
haukur
Semantic Scholar API metadata for this dataset?
2
#1 opened over 1 year ago
by
MicPie
MicPie
authored
a
paper
over 1 year ago
liked
a
dataset
over 1 year ago
autoevaluator