All LLMs Write Great Code, But Some Make (A Lot) Fewer Mistakes
•
3
Yi Cui
onekq
AI & ML interests
Benchmark, Code Generation,
Articles
Organizations
onekq's activity
posted
an
update
2 days ago
Post
295
Announce 🎉 WebApp1K-Duo 🎉
onekq-ai/WebApp1K-Duo-React
This is to keep up the challenge after OpenAI o1 models saturated the WebApp1K benchmark. The new benchmark brings SOTA to 67%. Let the hill climbing commence!
onekq-ai/WebApp1K-models-leaderboard
PS: I will publish more findings soon.
onekq-ai/WebApp1K-Duo-React
This is to keep up the challenge after OpenAI o1 models saturated the WebApp1K benchmark. The new benchmark brings SOTA to 67%. Let the hill climbing commence!
onekq-ai/WebApp1K-models-leaderboard
PS: I will publish more findings soon.
replied to
zhabotorabi's
post
2 days ago
the Mistral API? the model name is probably diffrent. I used mistral-large-2 but had to use the name mistral-large-latest. The team will help you via chat.
posted
an
update
4 days ago
Post
498
🐋 DeepSeek 🐋2.5 is hands-down the best open-source model, leaving its peers way behind. It even beats GPT-4o mini.
onekq-ai/WebApp1K-models-leaderboard
The inference of the official API is painfully slow though. I heard the team is short on GPUs (well, who isn't).
onekq-ai/WebApp1K-models-leaderboard
The inference of the official API is painfully slow though. I heard the team is short on GPUs (well, who isn't).
replied to
their
post
6 days ago
pass@1 for 🍓o1-mini🍓: 0.94!!
💸💸💸💸
#gpt #o1 #inference #RL #selfplay #WebApp1K
posted
an
update
6 days ago
Post
1086
If your plan keeps changing it's a sign that you are living the moment.
I just got the pass@1 result of GPT 🍓o1-preview🍓 : 0.95!!!
This means my benchmark is cast into oblivion, I need to up the ante. I am all ears to suggestions. onekq-ai/WebApp1K-models-leaderboard
I just got the pass@1 result of GPT 🍓o1-preview🍓 : 0.95!!!
This means my benchmark is cast into oblivion, I need to up the ante. I am all ears to suggestions. onekq-ai/WebApp1K-models-leaderboard
-
- 1 reply