Upload folder with huggingface_hub

.gitattributes

@@ -0,0 +1,58 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+Reflection-70B-BF16.template.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.Q4_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.Q8_0_L-00001-of-00004.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.Q8_0_L-00002-of-00004.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.Q8_0_L-00004-of-00004.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.Q8_0_L-00003-of-00004.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.Q2_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.Q5_K_L-00001-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.Q5_K_L-00002-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.Q5_K_L-00003-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.bf16-00001-of-00008.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.bf16-00002-of-00008.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.bf16-00003-of-00008.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.bf16-00004-of-00008.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.bf16-00005-of-00008.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.bf16-00006-of-00008.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.bf16-00007-of-00008.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.bf16-00008-of-00008.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.Q6_K_L-00001-of-00004.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.Q6_K_L-00002-of-00004.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.Q6_K_L-00003-of-00004.gguf filter=lfs diff=lfs merge=lfs -text
+Reflection-Llama-3.1-70B.Q6_K_L-00004-of-00004.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,81 @@
+---
+license: llama3.1
+language:
+- en
+base_model: mattshumer/ref_70_e3
+pipeline_tag: text-generation
+library_name: ggml
+datasets:
+- froggeric/imatrix
+---
+
+# Reflection-Llama-3.1-70B-GGUF
+
+![image/webp](https://cdn-uploads.huggingface.co/production/uploads/6604e5b21eb292d6df393365/lQJH2XICEKaACm9lfH7ZM.webp)
+
+GGUF quantized models of [mattshumer/ref_70_e3](https://huggingface.co/mattshumer/ref_70_e3)
+
+> This is the new, working version of the Reflection Llama 3.1 70B model.
+
+**Reflection Llama-3.1 70B is (currently) the world's top open-source LLM, trained with a new technique called Reflection-Tuning that teaches a LLM to detect mistakes in its reasoning and correct course.**
+
+| Quantization | Size   |
+| ------------ | ------ |
+| FP16         | 141GB  |
+| Q8_0_L       | 73GB   |
+| Q6_K_L       | 56.2GB |
+| Q5_K_L       | 52.6GB |
+| Q5_K_S       | ??.?GB |
+| Q4_K_L       | 42.1GB |
+| Q3_K_L       | 40GB   |
+| Q2_K_L       | 29.4GB |
+
+The `_L` suffix means that the token embeddings and output weight are at fp16 precision.
+
+## Benchmarks
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/60518f3731c5be7f3dd5ebc3/zNs-ZFs0SbnomH7mikiOU.png)
+
+All benchmarks tested have been checked for contamination by running [LMSys's LLM Decontaminator](https://github.com/lm-sys/llm-decontaminator). When benchmarking, we isolate the `<output>` and benchmark on solely that section.
+
+Trained from Llama 3.1 70B Instruct, you can sample from Reflection Llama-3.1 70B using the same code, pipelines, etc. as any other Llama model. It even uses the stock Llama 3.1 chat template format (though, we've trained in a few new special tokens to aid in reasoning and reflection).
+
+During sampling, the model will start by outputting reasoning inside `<thinking>` and `</thinking>` tags, and then once it is satisfied with its reasoning, it will output the final answer inside `<output>` and `</output>` tags. Each of these tags are special tokens, trained into the model.
+
+This enables the model to separate its internal thoughts and reasoning from its final answer, improving the experience for the user.
+
+Inside the `<thinking>` section, the model may output one or more `<reflection>` tags, which signals the model has caught an error in its reasoning and will attempt to correct it before providing a final answer.
+
+## System Prompt
+
+The system prompt used for training this model is:
+
+```
+You are a world-class AI system, capable of complex reasoning and reflection. Reason through the query inside <thinking> tags, and then provide your final response inside <output> tags. If you detect that you made a mistake in your reasoning at any point, correct yourself inside <reflection> tags.
+```
+
+We recommend using this exact system prompt to get the best results from Reflection Llama-3.1 70B. You may also want to experiment combining this system prompt with your own custom instructions to customize the behavior of the model.
+
+## Chat Format
+
+As mentioned above, the model uses the standard Llama 3.1 chat format. Here’s an example:
+
+```
+<|begin_of_text|><|start_header_id|>system<|end_header_id|>
+
+You are a world-class AI system, capable of complex reasoning and reflection. Reason through the query inside <thinking> tags, and then provide your final response inside <output> tags. If you detect that you made a mistake in your reasoning at any point, correct yourself inside <reflection> tags.<|eot_id|><|start_header_id|>user<|end_header_id|>
+
+what is 2+2?<|eot_id|><|start_header_id|>assistant<|end_header_id|>
+```
+
+## Tips for Performance
+
+- We are initially recommending a `temperature` of `.7` and a `top_p` of `.95`.
+- For increased accuracy, append `Think carefully.` at the end of your messages.
+
+## Dataset / Report
+
+Both the dataset and a brief report detailing how we trained this model will be released next week, alongside our Reflection 405B model that we expect will be the top-performing LLM in the world, including closed-source models.
+
+---
+
+Thanks to Jason Kuperberg and Josh Bickett from the [HyperWrite](https://hyperwriteai.com) team for reviewing drafts of the report we'll be releasing next week.

Reflection-Llama-3.1-70B.Q4_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b687928bfef696faf2614b262f7b361139979ea6a3a8ea576278f5cf40874aa7
+size 45270399712

Reflection-Llama-3.1-70B.Q5_K_L-00001-of-00003.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e384d38710b9c64c9d030efa313af29ac0583a5fccf670d2efef999e4602a77
+size 19952734496

Reflection-Llama-3.1-70B.Q5_K_L-00002-of-00003.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:209e133c800ccbde8ce5cadb0b6a8e953c56f0ebd3d0ad7f1b6966e02d20e31f
+size 19994822080

Reflection-Llama-3.1-70B.Q5_K_L-00003-of-00003.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7ae2fff7e3be4306ec93ab45d93fd2aedd88c0f0d0861a3d1e46f7c03bbd08f
+size 12620932448

Reflection-Llama-3.1-70B.Q6_K_L-00001-of-00004.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ab061a0a1051a2e4ac3ffb3f9c9ac7e970eef3872f139412adc241b829e0b76
+size 19850889152

Reflection-Llama-3.1-70B.Q6_K_L-00002-of-00004.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8812e320269690c0662b6ce13b3661234fbf4973baea5779f76088b413ec42a
+size 19847461760

Reflection-Llama-3.1-70B.Q6_K_L-00003-of-00004.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e92c3d992ac368dba2377501ccbbb175ec503959802ac66441ade68d3fe4e6f9
+size 18567477440

Reflection-Llama-3.1-70B.Q6_K_L-00004-of-00004.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1b3683dd55a5819d622cc84b696cea77b0a4ca6909c47b0a91b10292ad0d19f
+size 2101444768

Reflection-Llama-3.1-70B.bf16-00001-of-00008.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c9bab6e636f2e420efc5f980b6837efd564f03d3d80e693a4be28b0d89c106d
+size 19692480544

Reflection-Llama-3.1-70B.bf16-00002-of-00008.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bff08321720ca8717dc96a520e38aead47bb82bdca3163202ef20e6d06df41ce
+size 19932092672

Reflection-Llama-3.1-70B.bf16-00003-of-00008.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0151b17c183545616ac79c2660a217014236de59d0f7fef8f00a5e4cb1ce6c0c
+size 19898538112

Reflection-Llama-3.1-70B.bf16-00004-of-00008.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70fb5345d6a03d21cd730a7cd6dac6f66ebfdc12a894b3ee31c702c5e09f129f
+size 19596581184

Reflection-Llama-3.1-70B.bf16-00005-of-00008.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2234360244a01ad978919af203ea72452ed00f9523015bde9695d84c7dfee3ee
+size 19764287520

Reflection-Llama-3.1-70B.bf16-00006-of-00008.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3bb035f618e1335161ddabb4874c828499840dfddd7a7adf8e10ab6fe97a9ae4
+size 19596548352

Reflection-Llama-3.1-70B.bf16-00007-of-00008.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3b33b5564f1f7518998fdfbed5510b0ede2a12ab8d4e1d0939f2b8fcc04a55b
+size 19764320320

Reflection-Llama-3.1-70B.bf16-00008-of-00008.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e3f0b8f6a8308cf72fe02d8ef1d89d984832d3960dd16288705ed29d8faa9b9c
+size 2873262656