metadata

title: GPU Poor LLM Arena
emoji: 🏆
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 5.1.0
app_file: app.py
pinned: false
license: mit
short_description: 'Compact LLM Battle Arena: Frugal AI Face-Off!'

🏆 GPU-Poor LLM Gladiator Arena 🏆

Welcome to the GPU-Poor LLM Gladiator Arena, where frugal meets fabulous in the world of AI! This project pits compact language models (maxing out at 9B parameters) against each other in a battle of wits and words.

🤔 Starting from "Why?"

In the recent months, we've seen a lot of these "Tiny" models released, and some of them are really impressive.

Gradio Exploration: This project serves me as a playground for experimenting with Gradio app development; I am learning how to create interactive AI interfaces with it.
Tiny Model Evaluation: I wanted to develop a personal (and now public) stats system for evaluating tiny language models. It's not too serious, but it provides valuable insights into the capabilities of these compact powerhouses.
Accessibility: Built on Ollama, this arena allows pretty much anyone to experiment with these models themselves. No need for expensive GPUs or cloud services!
Pure Fun: At its core, this project is about having fun with AI. It's a lighthearted way to explore and compare different models. So, haters, feel free to chill – we're just here for a good time!

🌟 Features

Battle Arena: Pit two mystery models against each other and decide which pint-sized powerhouse reigns supreme.
Leaderboard: Track the performance of different models over time.
Performance Chart: Visualize model performance with interactive charts.
Privacy-Focused: Uses local Ollama API, avoiding pricey commercial APIs and keeping data close to home.
Customizable: Easy to add new models and prompts.

🚀 Getting Started

Prerequisites

Python 3.7+
Gradio
Plotly
Ollama (running locally)

Installation

Clone the repository:

git clone https://huggingface.co/spaces/k-mktr/gpu-poor-llm-arena.git
cd gpu-poor-llm-arena

Install the required packages:
```
pip install gradio plotly requests
```
Ensure Ollama is running locally or via a remote server.
Run the application:
```
python app.py
```

🎮 How to Use

Open the application in your web browser (typically at http://localhost:7860).
In the "Battle Arena" tab:
- Enter a prompt or use the random prompt generator (🎲 button).
- Click "Generate Responses" to see outputs from two random models.
- Vote for the better response.
Check the "Leaderboard" tab to see overall model performance.
View the "Performance Chart" tab for a visual representation of model wins and losses.

🛠 Configuration

You can customize the arena by modifying the arena_config.py file:

Add or remove models from the APPROVED_MODELS list.
Adjust the API_URL and API_KEY if needed.
Customize example_prompts for more variety in random prompts.

📊 Leaderboard

The leaderboard data is stored in leaderboard.json. This file is automatically updated after each battle.

🤖 Models

The arena currently supports various compact models, including:

LLaMA 3.2 (1B and 3B versions)
LLaMA 3.1 (8B version)
Gemma 2 (2B and 9B versions)
Qwen 2.5 (0.5B, 1.5B, 3B, and 7B versions)
Mistral 0.3 (7B version)
Phi 3.5 (3.8B version)
Hermes 3 (8B version)
Aya 23 (8B version)

🤝 Contributing

Contributions are welcome! Please feel free to suggest a model that Ollama supports. Some results are already quite surprising.

📜 License

This project is open-source and available under the MIT License

🙏 Acknowledgements

Thanks to the Ollama team for providing that amazing tool.
Shoutout to all the AI researchers and compact language models teams for making this frugal AI arena possible!

Enjoy the battles in the GPU-Poor LLM Gladiator Arena! May the best compact model win! 🏆