Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+llama-3.1-70b-instruct-lorablated.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,69 @@
+---
+base_model: meta-llama/Meta-Llama-3.1-70B-Instruct
+library_name: transformers
+license: llama3.1
+tags:
+- abliterated
+- uncensored
+- mergekit
+- autoquant
+- gguf
+---
+
+# 🦙 Llama-3.1-70B-Instruct-lorablated
+
+![](https://i.imgur.com/5Y0Riis.png)
+
+<center>🦙 <a href="https://huggingface.co/mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated"><i>Llama 3.1 8B Instruct abliterated</i></a></center>
+
+This is an uncensored version of [Llama 3.1 70B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct) created with abliteration (see [this article](https://huggingface.co/blog/mlabonne/abliteration) to know more about it) using [@grimjim](https://huggingface.co/grimjim)'s recipe.
+
+More precisely, this is a **LoRA-abliterated** (lorablated) model:
+
+1. **Extraction**: We extract a LoRA adapter by comparing two models: a censored Llama 3 and an abliterated Llama 3
+2. **Merge**: We merge this new LoRA adapter using [task arithmetic](https://arxiv.org/abs/2212.04089) to a censored Llama 3.1 to abliterate it.
+
+I adapted this recipe to Llama 3.1 70B using [failspy/Meta-Llama-3-70B-Instruct-abliterated-v3.5](https://huggingface.co/failspy/Meta-Llama-3-70B-Instruct-abliterated-v3.5) and optimized the LoRA rank.
+
+The model is fully uncensored in my tests and maintains a high level of quality. A more rigorous evaluation is still needed to measure the impact of this process on benchmarks.
+
+Special thanks to [@grimjim](https://huggingface.co/grimjim) for this technique (see his [8B model](https://huggingface.co/grimjim/Llama-3.1-8B-Instruct-abliterated_via_adapter)) and [@FailSpy](https://huggingface.co/failspy) for his [70B abliterated model](https://huggingface.co/failspy/Meta-Llama-3-70B-Instruct-abliterated-v3.5). Please follow them if you're interested in abliterated models.
+
+In addition, thanks to [brev.dev](https://brev.dev/) for providing me with compute!
+
+## ⚡️ Quantization
+
+This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using ./meta-llama/Meta-Llama-3.1-70B-Instruct + Llama-3-70B-Instruct-abliterated-LORA as a base.
+
+## 🧩 Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+base_model: meta-llama/Meta-Llama-3.1-70B-Instruct+Llama-3-70B-Instruct-abliterated-LORA
+dtype: bfloat16
+merge_method: task_arithmetic
+parameters:
+  normalize: false
+slices:
+- sources:
+  - layer_range: [0, 80]
+    model: meta-llama/Meta-Llama-3.1-70B-Instruct+Llama-3-70B-Instruct-abliterated-LORA
+    parameters:
+      weight: 1.0
+```
+
+You can reproduce this model using the following commands:
+
+```bash
+# Setup
+git clone https://github.com/arcee-ai/mergekit.git
+cd mergekit && pip install -e .
+pip install bitsandbytes
+
+# Extraction
+mergekit-extract-lora failspy/Meta-Llama-3-70B-Instruct-abliterated-v3.5 meta-llama/Meta-Llama-3-70B-Instruct Llama-3-70B-Instruct-abliterated-LORA --rank=64
+
+# Merge using previous config
+mergekit-yaml config.yaml Llama-3.1-70B-Instruct-lorablated --allow-crimes --lora-merge-cache=./cache
+```

llama-3.1-70b-instruct-lorablated.Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf30cada02efab5438ea14e1fc055a460cf5c18bfea46b88d03e156f92a31559
+size 26375113216