added gsm8k, updated ranking table

dist/index.html

@@ -212,77 +212,62 @@
         
     <h2>New leaderboard, new results!</h2>
         <p>We’ve started with adding and evaluating the models in the “maintainer’s highlights” section (cf. above) and are looking forward to the community submitting their new models to this new version of the leaderboard!!</p>
-        <aside>As the cluster has been extremely full, models of more than 140B parameters (such as Falcon-180B and BLOOM) will be run a bit later. </aside>
+        <aside>As the cluster has been extremely full, you can expect models to keep on appearing in the next days! </aside>
 
         <h3>What do the rankings look like?</h3>
 
-        <p>Taking a look at the top 10 models on the previous version of the Open LLM Leaderboard, and comparing with this updated version, 5 models appear to have a relatively stable ranking: Meta’s Llama3-70B, both instruct and base version, 01-ai’s Yi-1.5-34B, chat version, Cohere’s Command R + model, and lastly Smaug-72B, from AbacusAI.</p>
-        <p>We’ve been particularly impressed by Llama-70B-instruct, ranking top across many evaluations (even though this instruct version loses 15 points to its pretrained version counterpart on GPQA which begs the question whether the particularly extensive instruction fine-tuning done by the Meta team on this model affected some expert/graduate level knowledge?).</p>
-        <p>Also very interesting is the fact that a new challenger climbed the ranks to reach 2nd place despite its smaller size. With only 13B parameters, Microsoft’s Phi-3-medium-4K-instruct model shows a performance equivalent to models 2 to 4 times its size. It would be very interesting to have more information on the training procedure for Phi or an independant reproduction from an external team with open training recipes/datasets.</p>
-        <p>Here is a detail of the changes in rankings:</p>
+        <p>Taking a look at the top 10 models on the previous version of the Open LLM Leaderboard, and comparing with this updated version, some models appear to have a relatively stable ranking (in bold below): Qwen-2-72B instruct, Meta’s Llama3-70B, both instruct and base version, 01-ai’s Yi-1.5-34B, chat version, Cohere’s Command R + model, and lastly Smaug-72B, from AbacusAI.</p>
+        <p>We’ve been particularly impressed by Qwen2-72B-Instruct, one step above other models (notably thanks to its performance in math, long range reasoning, and knowledge)</p> 
+        <p>The current second best model, Llama-3-70B-Instruct, interestingly loses 15 points to its pretrained version counterpart on GPQA, which begs the question whether the particularly extensive instruction fine-tuning done by the Meta team on this model affected some expert/graduate level knowledge.</p>
+        <p>Also very interesting is the fact that a new challenger climbed the ranks to reach 3rd place despite its smaller size. With only 13B parameters, Microsoft’s Phi-3-medium-4K-instruct model shows a performance equivalent to models 2 to 4 times its size. It would be very interesting to have more information on the training procedure for Phi or an independant reproduction from an external team with open training recipes/datasets.</p>
         
         <table>
                 <tr>
                     <th>Rank</th>
-                    <th>Leaderboard v1</th>
-                    <th>Leaderboard v2</th>
+                    <th>New Leaderboard Ranking</th>
                 </tr>
                 <tr>
                     <td>⭐</td>
-                    <td><b>abacusai/Smaug-72B-v0.1</b></td>
-                    <td><b>meta-llama/Meta-Llama-3-70B-Instruct</b></td>
+                    <td><b>Qwen/Qwen2-72B-Instruct</b></td>
                 </tr>
                 <tr>
                     <td>2</td>
                     <td><b>meta-llama/Meta-Llama-3-70B-Instruct</b></td>
-                    <td><em>microsoft/Phi-3-medium-4k-instruct</em></td>
                 </tr>
                 <tr>
                     <td>3</td>
-                    <td><b>abacusai/Smaug-34B-v0.1</b></td>
-                    <td>01-ai/Yi-1.5-34B-Chat</td>
+                    <td><em>microsoft/Phi-3-medium-4k-instruct</em></td>
                 </tr>
                 <tr>
                     <td>4</td>
-                    <td>mlabonne/AlphaMonarch-7B</td>
-                    <td><b>abacusai/Smaug-72B-v0.1</b></td>
+                    <td><b>01-ai/Yi-1.5-34B-Chat</b></td>
                 </tr>
                 <tr>
                     <td>5</td>
-                    <td>mlabonne/Beyonder-4x7B-v3</td>
-                    <td><b>CohereForAI/c4ai-command-r-plus<b></td>
-                </tr>
+                    <td><b>CohereForAI/c4ai-command-r-plus</b></td>
                 <tr>
                     <td>6</td>
-                    <td><b>01-ai/Yi-1.5-34B-Chat</b></td>
-                    <td>Qwen/Qwen1.5-110B-Chat</td>
+                    <td><b>abacusai/Smaug-72B-v0.1</b></td>
                 </tr>
                 <tr>
                     <td>7</td>
-                    <td><b>CohereForAI/c4ai-command-r-plus</b></td>
-                    <td>NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO</td>
+                    <td>Qwen/Qwen1.5-110B</td>
                 </tr>
                 <tr>
                     <td>8</td>
-                    <td>upstage/SOLAR-10.7B-Instruct-v1.0</td>
-                    <td><b>meta-llama/Meta-Llama-3-70B</b></td>
+                    <td>Qwen/Qwen1.5-110B-Chat</td>
                 </tr>
                 <tr>
                     <td>9</td>
-                    <td><b>meta-llama/Meta-Llama-3-70B</b></td>
-                    <td>01-ai/Yi-1.5-9B-Chat</td>
+                    <td>microsoft/Phi-3-small-128k-instruct</td>
                 </tr>
                 <tr>
                     <td>10</td>
-                    <td>01-ai/Yi-1.5-34B</td>
-                    <td>01-ai/Yi-1.5-34B-32K</td>
+                    <td> 01-ai/Yi-1.5-9B-Chat</td>
                 </tr>
             </table>
-            <p>We’ve been particularly impressed by Llama-70B-instruct, who is the best model across many evaluations (though it has 15 points less than it’s base counterpart on GPQA - does instruct tuning remove knowledge?).</p>
-
-            <p>Interestingly, a new challenger climbed the ranks to arrive in 2nd place despite its smaller size: Phi-3-medium-4K-instruct, only 13B parameters but a performance equivalent to models 2 to 4 times its size.</p>
 
-            <p>We also provide the most important top and bottom ranking changes.</p>
+            <p>Here is a detail of the changes in rankings:</p>
 
             <div class="main-plot-container">
                 <figure><img src="assets/images/ranking_top10_bottom10.png"/></figure>

src/index.html

@@ -212,77 +212,62 @@
         
     <h2>New leaderboard, new results!</h2>
         <p>We’ve started with adding and evaluating the models in the “maintainer’s highlights” section (cf. above) and are looking forward to the community submitting their new models to this new version of the leaderboard!!</p>
-        <aside>As the cluster has been extremely full, models of more than 140B parameters (such as Falcon-180B and BLOOM) will be run a bit later. </aside>
+        <aside>As the cluster has been extremely full, you can expect models to keep on appearing in the next days! </aside>
 
         <h3>What do the rankings look like?</h3>
 
-        <p>Taking a look at the top 10 models on the previous version of the Open LLM Leaderboard, and comparing with this updated version, 5 models appear to have a relatively stable ranking: Meta’s Llama3-70B, both instruct and base version, 01-ai’s Yi-1.5-34B, chat version, Cohere’s Command R + model, and lastly Smaug-72B, from AbacusAI.</p>
-        <p>We’ve been particularly impressed by Llama-70B-instruct, ranking top across many evaluations (even though this instruct version loses 15 points to its pretrained version counterpart on GPQA which begs the question whether the particularly extensive instruction fine-tuning done by the Meta team on this model affected some expert/graduate level knowledge?).</p>
-        <p>Also very interesting is the fact that a new challenger climbed the ranks to reach 2nd place despite its smaller size. With only 13B parameters, Microsoft’s Phi-3-medium-4K-instruct model shows a performance equivalent to models 2 to 4 times its size. It would be very interesting to have more information on the training procedure for Phi or an independant reproduction from an external team with open training recipes/datasets.</p>
-        <p>Here is a detail of the changes in rankings:</p>
+        <p>Taking a look at the top 10 models on the previous version of the Open LLM Leaderboard, and comparing with this updated version, some models appear to have a relatively stable ranking (in bold below): Qwen-2-72B instruct, Meta’s Llama3-70B, both instruct and base version, 01-ai’s Yi-1.5-34B, chat version, Cohere’s Command R + model, and lastly Smaug-72B, from AbacusAI.</p>
+        <p>We’ve been particularly impressed by Qwen2-72B-Instruct, one step above other models (notably thanks to its performance in math, long range reasoning, and knowledge)</p> 
+        <p>The current second best model, Llama-3-70B-Instruct, interestingly loses 15 points to its pretrained version counterpart on GPQA, which begs the question whether the particularly extensive instruction fine-tuning done by the Meta team on this model affected some expert/graduate level knowledge.</p>
+        <p>Also very interesting is the fact that a new challenger climbed the ranks to reach 3rd place despite its smaller size. With only 13B parameters, Microsoft’s Phi-3-medium-4K-instruct model shows a performance equivalent to models 2 to 4 times its size. It would be very interesting to have more information on the training procedure for Phi or an independant reproduction from an external team with open training recipes/datasets.</p>
         
         <table>
                 <tr>
                     <th>Rank</th>
-                    <th>Leaderboard v1</th>
-                    <th>Leaderboard v2</th>
+                    <th>New Leaderboard Ranking</th>
                 </tr>
                 <tr>
                     <td>⭐</td>
-                    <td><b>abacusai/Smaug-72B-v0.1</b></td>
-                    <td><b>meta-llama/Meta-Llama-3-70B-Instruct</b></td>
+                    <td><b>Qwen/Qwen2-72B-Instruct</b></td>
                 </tr>
                 <tr>
                     <td>2</td>
                     <td><b>meta-llama/Meta-Llama-3-70B-Instruct</b></td>
-                    <td><em>microsoft/Phi-3-medium-4k-instruct</em></td>
                 </tr>
                 <tr>
                     <td>3</td>
-                    <td><b>abacusai/Smaug-34B-v0.1</b></td>
-                    <td>01-ai/Yi-1.5-34B-Chat</td>
+                    <td><em>microsoft/Phi-3-medium-4k-instruct</em></td>
                 </tr>
                 <tr>
                     <td>4</td>
-                    <td>mlabonne/AlphaMonarch-7B</td>
-                    <td><b>abacusai/Smaug-72B-v0.1</b></td>
+                    <td><b>01-ai/Yi-1.5-34B-Chat</b></td>
                 </tr>
                 <tr>
                     <td>5</td>
-                    <td>mlabonne/Beyonder-4x7B-v3</td>
-                    <td><b>CohereForAI/c4ai-command-r-plus<b></td>
-                </tr>
+                    <td><b>CohereForAI/c4ai-command-r-plus</b></td>
                 <tr>
                     <td>6</td>
-                    <td><b>01-ai/Yi-1.5-34B-Chat</b></td>
-                    <td>Qwen/Qwen1.5-110B-Chat</td>
+                    <td><b>abacusai/Smaug-72B-v0.1</b></td>
                 </tr>
                 <tr>
                     <td>7</td>
-                    <td><b>CohereForAI/c4ai-command-r-plus</b></td>
-                    <td>NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO</td>
+                    <td>Qwen/Qwen1.5-110B</td>
                 </tr>
                 <tr>
                     <td>8</td>
-                    <td>upstage/SOLAR-10.7B-Instruct-v1.0</td>
-                    <td><b>meta-llama/Meta-Llama-3-70B</b></td>
+                    <td>Qwen/Qwen1.5-110B-Chat</td>
                 </tr>
                 <tr>
                     <td>9</td>
-                    <td><b>meta-llama/Meta-Llama-3-70B</b></td>
-                    <td>01-ai/Yi-1.5-9B-Chat</td>
+                    <td>microsoft/Phi-3-small-128k-instruct</td>
                 </tr>
                 <tr>
                     <td>10</td>
-                    <td>01-ai/Yi-1.5-34B</td>
-                    <td>01-ai/Yi-1.5-34B-32K</td>
+                    <td> 01-ai/Yi-1.5-9B-Chat</td>
                 </tr>
             </table>
-            <p>We’ve been particularly impressed by Llama-70B-instruct, who is the best model across many evaluations (though it has 15 points less than it’s base counterpart on GPQA - does instruct tuning remove knowledge?).</p>
-
-            <p>Interestingly, a new challenger climbed the ranks to arrive in 2nd place despite its smaller size: Phi-3-medium-4K-instruct, only 13B parameters but a performance equivalent to models 2 to 4 times its size.</p>
 
-            <p>We also provide the most important top and bottom ranking changes.</p>
+            <p>Here is a detail of the changes in rankings:</p>
 
             <div class="main-plot-container">
                 <figure><img src="assets/images/ranking_top10_bottom10.png"/></figure>