Duplicate from Lewdiculous/Llama-3-Lumimaid-8B-v0.1-OAS-GGUF-IQ-Imatrix

Co-authored-by: Lewdiculous <[email protected]>

.gitattributes

@@ -0,0 +1,48 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+imatrix.dat filter=lfs diff=lfs merge=lfs -text
+Llama-3-Lumimaid-8B-v0.1-OAS-Q4_K_M-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3-Lumimaid-8B-v0.1-OAS-Q4_K_S-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3-Lumimaid-8B-v0.1-OAS-Q5_K_M-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3-Lumimaid-8B-v0.1-OAS-Q5_K_S-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3-Lumimaid-8B-v0.1-OAS-Q6_K-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3-Lumimaid-8B-v0.1-OAS-Q8_0-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3-Lumimaid-8B-v0.1-OAS-F16.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3-Lumimaid-8B-v0.1-OAS-IQ3_M-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3-Lumimaid-8B-v0.1-OAS-IQ3_S-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3-Lumimaid-8B-v0.1-OAS-IQ3_XXS-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3-Lumimaid-8B-v0.1-OAS-IQ4_NL-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3-Lumimaid-8B-v0.1-OAS-IQ4_XS-imat.gguf filter=lfs diff=lfs merge=lfs -text

Llama-3-Lumimaid-8B-v0.1-OAS-F16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f3c9f10efe36bacf6feed9987bcc7862895148e546dadadd8edf4b5912fb975
+size 16068890432

Llama-3-Lumimaid-8B-v0.1-OAS-IQ3_M-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67ceef34e89ebeafcbd3e326d55503a324788d9723a7854e6a8fbba0ca503cc4
+size 3784822976

Llama-3-Lumimaid-8B-v0.1-OAS-IQ3_S-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6377a1ea708ed9bd5a5fa799e70359f4090164a657adc79109f53a5b37572488
+size 3682324672

Llama-3-Lumimaid-8B-v0.1-OAS-IQ3_XXS-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fec41a66fc849baa9ed963f60c8b45602f3cb6841fc605d1132f1527639f4c53
+size 3274911936

Llama-3-Lumimaid-8B-v0.1-OAS-IQ4_NL-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2ed9af1032ca66320b8d1a6c072efdfa356bdbd9c5682a8966c29d1e20949f0f
+size 4677988544

Llama-3-Lumimaid-8B-v0.1-OAS-IQ4_XS-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a62e790d2593df40910a9535f1bd5ecff208f203ad5fcf86a1e9c91dc8221fbb
+size 4447662272

Llama-3-Lumimaid-8B-v0.1-OAS-Q4_K_M-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1199440aa13c55f5f2cad1cb215535306f21e52a81de23f80a9e3586c8ac1c50
+size 4920733888

Llama-3-Lumimaid-8B-v0.1-OAS-Q4_K_S-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6cec716e1544579dd291bed88e9898c759418caf1bf807125de312376c08cea3
+size 4692668608

Llama-3-Lumimaid-8B-v0.1-OAS-Q5_K_M-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af568492efc49e32da42fd88310ad3a523a3baedc6c3b014f19ff21afe343b72
+size 5732987072

Llama-3-Lumimaid-8B-v0.1-OAS-Q5_K_S-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:85df1a759098ea6d9355fa17a561b7230ad35f465d40fd9c6dc10c38526831ca
+size 5599293632

Llama-3-Lumimaid-8B-v0.1-OAS-Q6_K-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:077a9777880d94aaa1e1cdd7c699eac5fc72f6372c45d218738cf932e287146c
+size 6596006080

Llama-3-Lumimaid-8B-v0.1-OAS-Q8_0-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9306bb3e51905a01a5960c646126a55801f929dad0faebd48876179d6c4d7b9
+size 8540770496

README.md

@@ -0,0 +1,104 @@
+---
+license: cc-by-nc-4.0
+tags:
+- roleplay
+- llama3
+- sillytavern
+---
+
+> [!TIP]
+> **Support:** <br>
+> My upload speeds have been cooked and unstable lately. <br>
+> Realistically I'd need to move to get a better provider. <br>
+> If you **want** and you are able to... <br>
+> You can [**support my various endeavors here (Ko-fi)**](https://ko-fi.com/Lewdiculous). <br>
+> I apologize for disrupting your experience.
+
+GGUF-IQ-Imatrix quants for [NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS](https://huggingface.co/NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS).
+
+**Author:** <br>
+"This model received the Orthogonal Activation Steering treatment, **meaning it will rarely refuse any request.**"
+
+> [!IMPORTANT]  
+> **Relevant:** <br>
+> These quants have been done after the fixes from [**llama.cpp/pull/6920**](https://github.com/ggerganov/llama.cpp/pull/6920) have been merged. <br>
+> Use **KoboldCpp** version **1.64** or higher, make sure you're up-to-date.
+
+> [!WARNING]
+> Compatible SillyTavern presets [here (simple)](https://huggingface.co/ChaoticNeutrals/Poppy_Porpoise-v0.7-L3-8B/tree/main/Official%20Poppy%20Porpoise%20ST%20Presets)) or [here (Virt's Roleplay Presets - recommended)](https://huggingface.co/Virt-io/SillyTavern-Presets). <br>
+> Use the latest version of KoboldCpp. **Use the provided presets for testing.** <br>
+> Feedback and support for the Authors is always welcome. <br>
+> If there are any issues or questions let me know.
+
+> [!NOTE]
+> For **8GB VRAM** GPUs, I recommend the **Q4_K_M-imat** quant for up to 12288 context sizes.
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/65d4cf2693a0a3744a27536c/JUxfdTot7v7LTdIGYyzYM.png)
+
+**Original model information:**
+
+## Lumimaid 0.1
+
+<center><div style="width: 100%;">
+    <img src="https://cdn-uploads.huggingface.co/production/uploads/630dfb008df86f1e5becadc3/d3QMaxy3peFTpSlWdWF-k.png" style="display: block; margin: auto;">
+</div></center>
+
+This model uses the Llama3 **prompting format**
+
+Llama3 trained on our RP datasets, we tried to have a balance between the ERP and the RP, not too horny, but just enough.
+
+We also added some non-RP dataset, making the model less dumb overall. It should look like a 40%/60% ratio for Non-RP/RP+ERP data.
+
+This model includes the new Luminae dataset from Ikari.
+
+This model have received the Orthogonal Activation Steering treatment, meaning it will rarely refuse any request.
+
+If you consider trying this model please give us some feedback either on the Community tab on hf or on our [Discord Server](https://discord.gg/MtCVRWTZXY).
+
+## Credits:
+- Undi
+- IkariDev
+
+## Description
+
+This repo contains FP16 files of Lumimaid-8B-v0.1-OAS.
+
+Switch: [8B](https://huggingface.co/NeverSleep/Llama-3-Lumimaid-8B-v0.1) - [70B](https://huggingface.co/NeverSleep/Llama-3-Lumimaid-70B-v0.1) - [70B-alt](https://huggingface.co/NeverSleep/Llama-3-Lumimaid-70B-v0.1-alt) - [8B-OAS](https://huggingface.co/NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS) - [70B-OAS](https://huggingface.co/NeverSleep/Llama-3-Lumimaid-70B-v0.1-OAS)
+
+## Training data used:
+- [Aesir datasets](https://huggingface.co/MinervaAI)
+- [NoRobots](https://huggingface.co/datasets/Doctor-Shotgun/no-robots-sharegpt)
+- [limarp](https://huggingface.co/datasets/lemonilia/LimaRP) - 8k ctx
+- [toxic-dpo-v0.1-sharegpt](https://huggingface.co/datasets/Undi95/toxic-dpo-v0.1-sharegpt)
+- [ToxicQAFinal](https://huggingface.co/datasets/NobodyExistsOnTheInternet/ToxicQAFinal)
+- Luminae-i1 (70B/70B-alt) (i2 was not existing when the 70b started training) | Luminae-i2 (8B) (this one gave better results on the 8b) - Ikari's Dataset
+- [Squish42/bluemoon-fandom-1-1-rp-cleaned](https://huggingface.co/datasets/Squish42/bluemoon-fandom-1-1-rp-cleaned) - 50% (randomly)
+- [NobodyExistsOnTheInternet/PIPPAsharegptv2test](https://huggingface.co/datasets/NobodyExistsOnTheInternet/PIPPAsharegptv2test) - 5% (randomly)
+- [cgato/SlimOrcaDedupCleaned](https://huggingface.co/datasets/cgato/SlimOrcaDedupCleaned) - 5% (randomly)
+- Airoboros (reduced)
+- [Capybara](https://huggingface.co/datasets/Undi95/Capybara-ShareGPT/) (reduced)
+
+
+## Models used (only for 8B)
+
+- Initial LumiMaid 8B Finetune
+- Undi95/Llama-3-Unholy-8B-e4
+- Undi95/Llama-3-LewdPlay-8B
+
+## Prompt template: Llama3
+
+```
+<|begin_of_text|><|start_header_id|>system<|end_header_id|>
+
+{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
+
+{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
+
+{output}<|eot_id|>
+```
+
+## Others
+
+Undi: If you want to support us, you can [here](https://ko-fi.com/undiai).
+
+IkariDev: Visit my [retro/neocities style website](https://ikaridevgit.github.io/) please kek

imatrix.dat

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bbdaf55a2765eacbf63f9e2a0358f0d97aafe48f7f1228b2c1489f92a4666e23
+size 4988193