snorkelai
/

Snorkel-Mistral-PairRM-DPO

@@ -44,14 +44,13 @@ to learn more about "Programmatically scale human preferences and alignment in G
 #### Result:
-- This model scored **30.2** on [Alpaca-Eval 2.0](https://tatsu-lab.github.io/alpaca_eval/) - ranked 3rd and the highest for an open source base model at the time of publication.
-- Utilizing the model with PairRM, which involved generating 16 responses and submitting the highest-scoring one by PairRM, we scored **34.86** - ranked 2nd.
 The best model on the leaderboard is "gpt-4-turbo".
-We acknowledge that Alpaca-Eval 2.0 is not the full reflection of LLMs' performances.
-However, in this work, as we are aligning toward general "human preferences", this benchmark serves as a compatible, representative benchmark.
-We expect more word on new alignment axes from the community and perform evaluation on other suitable benchmarks.
 We recognize that the Alpaca-Eval 2.0 benchmark does not entirely capture the full range of capabilities and performances of LLMs.
 However, in our current work, where the goal is to align with general "human preferences," Alpaca-Eval 2.0 serves as a suitable and representative benchmark.
 Moving forward, we anticipate further contributions from the community regarding new alignment axes, and conduct evaluations using other appropriate benchmarks.
@@ -61,7 +60,6 @@ The model is a quick demonstration that the LLMs can be programmatically aligned
 It does not have any moderation mechanisms. We're looking forward to engaging with the community on ways to
 make the model finely respect guardrails, allowing for deployment in environments requiring moderated outputs.
 ## Acknowledgments
 - The Mistral AI Team for developing and releasing the advanced Mistral-7B-Instruct-v0.2 model.
 - The author of the [Direct Preference Optimization paper](https://arxiv.org/abs/2305.18290) for the innovative approach

 #### Result:
+On [**Alpaca-Eval 2.0**](https://tatsu-lab.github.io/alpaca_eval/):
+- The base model: [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) scored **14.72**.
+After applying the above methodology:
+- This model scored **30.2** - ranked 3rd and the highest for an open-source base model at the time of publication.
+- Utilizing the model with PairRM, which involved generating 16 responses and submitting the highest-scoring response by PairRM, we scored **34.86** - ranked 2nd.
 The best model on the leaderboard is "gpt-4-turbo".
 We recognize that the Alpaca-Eval 2.0 benchmark does not entirely capture the full range of capabilities and performances of LLMs.
 However, in our current work, where the goal is to align with general "human preferences," Alpaca-Eval 2.0 serves as a suitable and representative benchmark.
 Moving forward, we anticipate further contributions from the community regarding new alignment axes, and conduct evaluations using other appropriate benchmarks.
 It does not have any moderation mechanisms. We're looking forward to engaging with the community on ways to
 make the model finely respect guardrails, allowing for deployment in environments requiring moderated outputs.
 ## Acknowledgments
 - The Mistral AI Team for developing and releasing the advanced Mistral-7B-Instruct-v0.2 model.
 - The author of the [Direct Preference Optimization paper](https://arxiv.org/abs/2305.18290) for the innovative approach