Terry Zhuo
commited on
Commit
•
889b484
1
Parent(s):
702e87d
update notes
Browse files
app.py
CHANGED
@@ -248,7 +248,7 @@ with demo:
|
|
248 |
- <u>Instruct</u> (🔥Vibe Check🔥): Code Generation based on the (less verbose) NL-oriented instructions. This variant tests if the models are really capable enough to understand human intents to code.
|
249 |
- `complete` and `instruct` represent the calibrated Pass@1 score on the BigCodeBench benchmark variants.
|
250 |
- `elo_mle` represents the task-level Bootstrap of Maximum Likelihood Elo rating on `BigCodeBench-Complete`, which starts from 1000 and is boostrapped 500 times.
|
251 |
-
- `size`
|
252 |
- Model providers have the responsibility to avoid data contamination. Models trained on close data can be affected by contamination.
|
253 |
- For more details check the 📝 About section.
|
254 |
""",
|
|
|
248 |
- <u>Instruct</u> (🔥Vibe Check🔥): Code Generation based on the (less verbose) NL-oriented instructions. This variant tests if the models are really capable enough to understand human intents to code.
|
249 |
- `complete` and `instruct` represent the calibrated Pass@1 score on the BigCodeBench benchmark variants.
|
250 |
- `elo_mle` represents the task-level Bootstrap of Maximum Likelihood Elo rating on `BigCodeBench-Complete`, which starts from 1000 and is boostrapped 500 times.
|
251 |
+
- `size` is the amount of activated model weight during inference.
|
252 |
- Model providers have the responsibility to avoid data contamination. Models trained on close data can be affected by contamination.
|
253 |
- For more details check the 📝 About section.
|
254 |
""",
|