Create README.md

README.md

@@ -0,0 +1,89 @@
+
+
+# Qwen2.5-Math-RM-72B
+
+## Introduction
+
+Over the past year, we have dedicated significant effort to researching and enhancing the reasoning capabilities of large language models, with a particular focus on their ability to solve arithmetic and mathematical problems. We introduce a serise of math-specific large language models of our Qwen2.5 series,  Qwen2.5-Math, Qwen2.5-Math-Instruct-1.5B/7B/72B and Qwen2.5-Math-RM-72B. Qwen2.5-Math is a series of specialized math language models built upon the Qwen2.5 LLMs, which significantly outperforms the mathematical capabilities of open-source models and even closed-source models (e.g., GPT4o). We hope that Qwen2.5-Math can contribute to the scientific community for solving advanced mathematical problems that require complex, multi-step logical reasoning.
+
+
+## Model Details
+
+
+For more details, please refer to our [blog post](https://qwenlm.github.io/blog/qwen2-math/) and [GitHub repo](https://github.com/QwenLM/Qwen2-Math).
+
+
+## Requirements
+* `transformers>=4.40.0` for Qwen2-Math models. The latest version is recommended.
+
+> [!Warning]
+> <div align="center">
+> <b>
+> 🚨 This is a must because `transformers` integrated Qwen2 codes since `4.37.0`.
+> </b>
+> </div>
+
+For requirements on GPU memory and the respective throughput, see similar results of Qwen2 [here](https://qwen.readthedocs.io/en/latest/benchmark/speed_benchmark.html).
+
+## Quick Start
+
+> [!Important]
+>
+> **Qwen2.5-Math-RM-72B** is a reward model typically used for offering feedback on the quality of reasoning and intermediate steps, serving in RLHF and Rejection Sampling.
+
+### 🤗 Hugging Face Transformers
+
+Here we show a code snippet to show you how to use the Qwen2.5-Math-RM-72B with `transformers`:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "Qwen/Qwen2.5-Math-RM-72B"
+device = "auto" # the device to load the model onto
+
+model = AutoModel.from_pretrained(
+    model_name, 
+    device_map=device, 
+    torch_dtype=torch.bfloat16,
+    trust_remote_code=True,
+).eval()
+
+tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
+
+chat = [
+    {"role": "system", "content": "Please reason step by step, and put your final answer within \\boxed{}."},
+    {"role": "user", "content": "Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?"},
+    {"role": "assistant", "content": "To determine how much Janet makes from selling the duck eggs at the farmers' market, we need to follow these steps:\n\n1. Calculate the total number of eggs laid by the ducks each day.\n2. Determine how many eggs Janet eats and bakes for herself each day.\n3. Find out how many eggs are left to be sold.\n4. Calculate the revenue from selling the remaining eggs at $2 per egg.\n\nLet's start with the first step:\n\n1. Janet's ducks lay 16 eggs per day.\n\nNext, we calculate how many eggs Janet eats and bakes for herself each day:\n\n2. Janet eats 3 eggs for breakfast every morning.\n3. Janet bakes 4 eggs for her friends every day.\n\nSo, the total number of eggs Janet eats and bakes for herself each day is:\n\\[ 3 + 4 = 7 \\text{ eggs} \\]\n\nNow, we find out how many eggs are left to be sold:\n\\[ 16 - 7 = 9 \\text{ eggs} \\]\n\nFinally, we calculate the revenue from selling the remaining eggs at $2 per egg:\n\\[ 9 \\times 2 = 18 \\text{ dollars} \\]\n\nTherefore, Janet makes \\(\\boxed{18}\\) dollars every day at the farmers' market."}
+] # 3.75
+
+conversation_str = tokenizer.apply_chat_template(
+    chat, 
+    tokenize=False, 
+    add_generation_prompt=False
+)
+input_ids = tokenizer.encode(
+    conversation_str, 
+    return_tensors="pt", 
+    add_special_tokens=False
+).to(model.device)
+
+outputs = model(input_ids=input_ids)
+print(outputs[0])
+```
+
+### 🤖 ModelScope
+We strongly advise users, especially those in mainland China, to use ModelScope. `snapshot_download` can help you solve issues concerning downloading checkpoints.
+
+
+## Citation
+
+If you find our work helpful, feel free to give us a citation.
+
+```
+@article{yang2024qwen2,
+  title={Qwen2 technical report},
+  author={Yang, An and Yang, Baosong and Hui, Binyuan and Zheng, Bo and Yu, Bowen and Zhou, Chang and Li, Chengpeng and Li, Chengyuan and Liu, Dayiheng and Huang, Fei and others},
+  journal={arXiv preprint arXiv:2407.10671},
+  year={2024}
+}
+```