Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
AkimfromParis 
posted an update 1 day ago
Post
741
🇯🇵 The Open Japanese LLM Leaderboard created by LLM-jp 🌸 in partnership with HuggingFace 🤗 was released today!

Blog: https://huggingface.co/blog/leaderboard-japanese
Space: llm-jp/open-japanese-llm-leaderboard

🌍 The leaderboard is available in both Japanese and English
📚 Based on the evaluation tool, llm-jp-eval with more than 20 datasets for Japanese LLMs
📊 The leaderboard showcases all the metrics for NLP experts, plus averages for NLP beginners
💻 For the comfort of users, we chose a horizontal UI, and implemented it in a light and dark theme on Gradio
🔬 The radar chart provides a very interesting visualization of metrics!
🌱 We are using the Japanese research platform, MDX, so please be patient!
⚡ LLM bigger than +70B will be evaluated soon…

How do you say “GPUs Go Brrr” in Japanese - > GPUがブンブン~! (To pronounce "GPU ga bunbun!") 🔥

Congrats!

·

@kaisugi Thank you! It was a great project and it's barely the beginning.
Hopefully, the open-source community will evaluate more LLMs, and that we will discover more insights.
We have a memory restriction from mdx on Qwen 2.5 72B, but we will found a solution very soon.

This comment has been hidden