About

app.py

@@ -144,19 +144,6 @@ def get_leaderboard_df():
     return df
 
 
-TITLE = """
-🚀 Open RL Leaderboard
-"""
-
-INTRODUCTION_TEXT = """
-Welcome to the Open RL Leaderboard! This is a community-driven benchmark for reinforcement learning models.
-"""
-
-ABOUT_TEXT = """
-The Open RL Leaderboard is a community-driven benchmark for reinforcement learning models.
-"""
-
-
 def select_env(df: pd.DataFrame, env_id: str):
     df = df[df["env_id"] == env_id]
     df = df.sort_values("mean_episodic_return", ascending=False)
@@ -178,6 +165,87 @@ def format_df(df: pd.DataFrame):
     return df.values.tolist()
 
 
+TITLE = """
+🚀 Open RL Leaderboard
+"""
+
+INTRODUCTION_TEXT = """
+Welcome to the Open RL Leaderboard! This is a community-driven benchmark for reinforcement learning models.
+"""
+
+ABOUT_TEXT = r"""
+The Open RL Leaderboard is a community-driven benchmark for reinforcement learning models.
+
+## 🔌 How to have your agent evaluated?
+
+The Open RL Leaderboard constantly scans the 🤗 Hub to detect new models to be evaluated. For your model to be evaluated, it must meet the following criteria.
+
+1. The model must be public on the 🤗 Hub
+2. The model must contain an `agent.pt` file.
+3. The model must be [tagged](https://huggingface.co/docs/hub/model-cards#model-cards) `reinforcement-learning`
+4. The model must be [tagged](https://huggingface.co/docs/hub/model-cards#model-cards) with the name of the environment you want to evaluate (for example `MountainCar-v0`)
+
+Once your model meets these criteria, it will be automatically evaluated on the Open RL Leaderboard. That's it!
+
+## 🏗️ How do I build the `agent.pt`?
+
+The `agent.pt` file is a [TorchScript module](https://pytorch.org/docs/stable/jit.html#). It must be loadable using `torch.jit.load`.
+The module must take batch of observations as input and return batch of actions. To check if your model is compatible with the Open RL Leaderboard, you can run the following code:
+
+```python
+import gymnasium as gym
+import numpy as np
+import torch
+
+agent_path = "path/to/agent.pt"
+env_id = ... # e.g. "MountainCar-v0"
+
+agent = torch.jit.load(agent_path)
+env = gym.make(env_id)
+observations = np.array([env.observation_space.sample()])
+observations = torch.from_numpy(observations)
+actions = agent(observations)
+actions = actions.numpy()[0]
+assert env.action_space.contains(actions)
+```
+
+## 🕵 How are the models evaluated?
+
+The evaluation is done by running the agent on the environment for 100 episodes. 
+
+For further information, please refer to the [Open RL Leaderboard evaulation script](https://huggingface.co/spaces/open-rl-leaderboard/leaderboard/blob/main/src/evaluation.py).
+
+### The particular case of Atari environments
+
+Atari environments are evaluated on the `NoFrameskip-v4` version of the environment. For example, to evaluate an agent on the `Pong` environment, you must tag your model with `PongNoFrameskip-v4`. The environment is then wrapped to match the standard Atari preprocessing pipeline.
+
+- No-op reset with a maximum of 30 no-ops
+- Max and skip with a skip of 4
+- Episodic life (although the reported score is for the full episode, not the life)
+- Fire reset
+- Clip reward (although the reported score is not clipped)
+- Resize observation to 84x84
+- Grayscale observation
+- Frame stack of 4
+
+## 🚑 Troubleshooting
+
+If you encounter any issue, please open an issue on the [Open RL Leaderboard repository](https://huggingface.co/spaces/open-rl-leaderboard/leaderboard/discussions/new).
+
+## 📜 Citation
+
+```bibtex
+@misc{open-rl-leaderboard,
+  author = {Quentin Gallouédec and TODO},
+  title = {Open RL Leaderboard},
+  year = {2024},
+  publisher = {Hugging Face},
+  howpublished = "\url{https://huggingface.co/spaces/open-rl-leaderboard/leaderboard}",
+}
+```
+"""
+
+
 with gr.Blocks() as demo:
     gr.HTML(TITLE)
     gr.Markdown(INTRODUCTION_TEXT, elem_classes="markdown-text")