Steelskull
commited on
Commit
•
58d81a6
1
Parent(s):
0c4850f
Update README.md
Browse files
README.md
CHANGED
@@ -4,7 +4,7 @@ tags:
|
|
4 |
- merge
|
5 |
- mergekit
|
6 |
base_model:
|
7 |
-
-
|
8 |
---
|
9 |
|
10 |
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0">
|
@@ -21,10 +21,10 @@ base_model:
|
|
21 |
pre { background-color: rgba(255, 255, 255, 0.05); padding: 10px; border-radius: 5px; overflow-x: auto; }
|
22 |
code { font-family: 'Courier New', monospace; color: #A3BE8C; } </style> </head> <body> <div class="container">
|
23 |
<div class="header">
|
24 |
-
<h1>Aura-llama-3</h1> </div> <div class="info">
|
25 |
-
<img src="https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/
|
26 |
<p>Now that the cute anime girl has your attention.</p>
|
27 |
-
<p>UPDATE: Model
|
28 |
<p>Aura-llama is using the methodology presented by SOLAR for scaling LLMs called depth up-scaling (DUS), which encompasses architectural modifications with continued pretraining. Using the solar paper as a base, I integrated Llama-3 weights into the upscaled layers, and In the future plan to continue training the model.</p>
|
29 |
<p>Aura-llama is a merge of the following models to create a base model to work from:</p>
|
30 |
<ul>
|
|
|
4 |
- merge
|
5 |
- mergekit
|
6 |
base_model:
|
7 |
+
- failspy/Llama-3-8B-Instruct-abliterated
|
8 |
---
|
9 |
|
10 |
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
|
21 |
pre { background-color: rgba(255, 255, 255, 0.05); padding: 10px; border-radius: 5px; overflow-x: auto; }
|
22 |
code { font-family: 'Courier New', monospace; color: #A3BE8C; } </style> </head> <body> <div class="container">
|
23 |
<div class="header">
|
24 |
+
<h1>Aura-llama-3-Abliterated</h1> </div> <div class="info">
|
25 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/AwLNDVB-GIY7k0wnVV_TX.png" alt="Aura-llama-Abliterated Image">
|
26 |
<p>Now that the cute anime girl has your attention.</p>
|
27 |
+
<p>UPDATE: Model is now using the abliterated version of meta llama 3 8b </p>
|
28 |
<p>Aura-llama is using the methodology presented by SOLAR for scaling LLMs called depth up-scaling (DUS), which encompasses architectural modifications with continued pretraining. Using the solar paper as a base, I integrated Llama-3 weights into the upscaled layers, and In the future plan to continue training the model.</p>
|
29 |
<p>Aura-llama is a merge of the following models to create a base model to work from:</p>
|
30 |
<ul>
|