|
--- |
|
tags: |
|
- merge |
|
- mergekit |
|
- moe |
|
- frankenmoe |
|
- abacusai/Llama-3-Smaug-8B |
|
- cognitivecomputations/dolphin-2.9-llama3-8b |
|
- Weyaxi/Einstein-v6.1-Llama3-8B |
|
- dreamgen-preview/opus-v1.2-llama-3-8b-base-run3.4-epoch2 |
|
base_model: |
|
- abacusai/Llama-3-Smaug-8B |
|
- cognitivecomputations/dolphin-2.9-llama3-8b |
|
- Weyaxi/Einstein-v6.1-Llama3-8B |
|
- dreamgen-preview/opus-v1.2-llama-3-8b-base-run3.4-epoch2 |
|
license: apache-2.0 |
|
--- |
|
|
|
 |
|
|
|
# 🚀 Skyro-4X8B |
|
Skyro-4X8B is a Mixure of Experts (MoE) made with the following models using [Mergekit](https://github.com/arcee-ai/mergekit): |
|
|
|
* [abacusai/Llama-3-Smaug-8B](https://huggingface.co/abacusai/Llama-3-Smaug-8B) |
|
* [cognitivecomputations/dolphin-2.9-llama3-8b](https://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b) |
|
* [Weyaxi/Einstein-v6.1-Llama3-8B](https://huggingface.co/Weyaxi/Einstein-v6.1-Llama3-8B) |
|
* [dreamgen-preview/opus-v1.2-llama-3-8b-base-run3.4-epoch2](https://huggingface.co/dreamgen-preview/opus-v1.2-llama-3-8b-base-run3.4-epoch2) |
|
|
|
## 🧩 Configuration |
|
|
|
```yamlname: "Skyro-4X8B" |
|
base_model: meta-llama/Meta-Llama-3-8B |
|
gate_mode: hidden |
|
experts: |
|
- source_model: abacusai/Llama-3-Smaug-8B |
|
positive_prompts: |
|
- "chat" |
|
- "assistant" |
|
- "tell me" |
|
- "explain" |
|
- "I want" |
|
- source_model: cognitivecomputations/dolphin-2.9-llama3-8b |
|
positive_prompts: |
|
- "math" |
|
- "mathematics" |
|
- "code" |
|
- "engineering" |
|
- "solve" |
|
- "logic" |
|
- "rationality" |
|
- "puzzle" |
|
- "solve" |
|
- source_model: Weyaxi/Einstein-v6.1-Llama3-8B |
|
positive_prompts: |
|
- "science" |
|
- "medical" |
|
- "physics" |
|
- "engineering" |
|
- "math" |
|
- "logic" |
|
- "rationality" |
|
- "mathematics" |
|
- "solve" |
|
- source_model: dreamgen-preview/opus-v1.2-llama-3-8b-base-run3.4-epoch2 |
|
positive_prompts: |
|
- "story" |
|
- "roleplay" |
|
- "role-play" |
|
- "storywriting" |
|
- "character" |
|
- "narrative" |
|
- "creative" |
|
``` |
|
|
|
## 💻 Usage |
|
|
|
```python |
|
!pip install -qU transformers accelerate |
|
|
|
from transformers import AutoTokenizer |
|
import transformers |
|
import torch |
|
|
|
model = "saucam/Skyro-4X8B" |
|
messages = [{"role": "user", "content": "In a student council election, candidate A got 20% of the votes while candidate B got 50% more than candidate A's votes. The rest of the votes was given to candidate C. If there were 100 voters, how many votes did candidate C get?"}] |
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model) |
|
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
|
pipeline = transformers.pipeline( |
|
"text-generation", |
|
model=model, |
|
torch_dtype=torch.float16, |
|
device_map="auto", |
|
) |
|
|
|
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95) |
|
print(outputs[0]["generated_text"]) |
|
``` |
|
|
|
## Sample output |
|
|
|
``` |
|
config.json: 100%|██████████████████████████████████████████████████████████████| 878/878 [00:00<00:00, 4.18MB/s] |
|
model.safetensors.index.json: 100%|██████████████████████████████████████████| 53.5k/53.5k [00:00<00:00, 101MB/s] |
|
model-00001-of-00006.safetensors: 100%|█████████████████████████████████████| 9.89G/9.89G [03:47<00:00, 43.4MB/s] |
|
model-00002-of-00006.safetensors: 100%|█████████████████████████████████████| 9.98G/9.98G [03:23<00:00, 49.0MB/s] |
|
model-00003-of-00006.safetensors: 100%|█████████████████████████████████████| 9.98G/9.98G [03:44<00:00, 44.5MB/s] |
|
model-00004-of-00006.safetensors: 100%|█████████████████████████████████████| 9.90G/9.90G [03:30<00:00, 46.9MB/s] |
|
model-00005-of-00006.safetensors: 100%|█████████████████████████████████████| 9.08G/9.08G [03:08<00:00, 48.1MB/s] |
|
model-00006-of-00006.safetensors: 100%|█████████████████████████████████████| 1.05G/1.05G [00:20<00:00, 51.3MB/s] |
|
Downloading shards: 100%|█████████████████████████████████████████████████████████| 6/6 [17:58<00:00, 179.78s/it] |
|
Loading checkpoint shards: 100%|███████████████████████████████████████████████████| 6/6 [01:27<00:00, 14.59s/it] |
|
WARNING:root:Some parameters are on the meta device device because they were offloaded to the cpu. |
|
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. |
|
|
|
<|im_start|>user |
|
In a student council election, candidate A got 20% of the votes while candidate B got 50% more than candidate A's votes. The rest of the votes was given to candidate C. If there were 100 voters, how many votes did candidate C get?<|im_end|> |
|
<|im_start|>assistant |
|
Let's denote the number of votes candidate A got as \( A \). |
|
|
|
Candidate B got 50% more votes than candidate A, so candidate B got \( A + 0.5A = 1.5A \) votes. |
|
|
|
Candidate C got the rest of the votes, which means \( C = 100 - (A + 1.5A) \). |
|
|
|
We know that candidate A got 20% of the votes, so \( A = 20\% \times 100 = 20 \). |
|
|
|
Now we can calculate candidate C's votes: |
|
\( C = 100 - (20 + 1.5 \times 20) \) |
|
\( C = 100 - (20 + 30) \) |
|
\( C = 100 - 50 \) |
|
\( C = 50 \). |
|
|
|
Therefore, candidate C got 50 votes.<|im_end|> |
|
``` |
